Systematic Literature Review

What actually makes a review systematic

Students frequently describe a thorough literature review as systematic. The terms are not interchangeable, and using the wrong one invites a question you will not be able to answer.

A review is systematic when it has:

• A focused, answerable question — narrow enough that inclusion decisions are unambiguous.
• A protocol written before you start, specifying the question, criteria, search strategy and analysis plan.
• Explicit inclusion and exclusion criteria, decided in advance rather than adjusted as you go.
• A documented, reproducible search — databases, exact terms, filters, dates.
• Screening against those criteria, ideally by more than one person, with disagreements resolved by a stated process.
• A record of every study excluded and why, at full-text stage.
• Formal appraisal of study quality using an established tool.
• Synthesis that goes beyond describing each study in turn.
• Transparent reporting, conventionally including a flow diagram accounting for every record from search to inclusion.

Missing any of those does not make the review worthless — it makes it a different kind of review, and it should be named accordingly. Scoping reviews, rapid reviews, integrative reviews and narrative reviews are all legitimate and have their own expectations.

The protocol, and registering it

Writing the method before you see the results is what prevents the review being shaped by what you find. It is the same logic as pre-registering a trial.

Your protocol should specify the question, the inclusion and exclusion criteria, the databases and search strategy, the screening process, the appraisal tool, and how you intend to synthesise. Writing it forces decisions that are much harder to make defensibly once you have seen which studies would be included or excluded by each choice.

Registration puts the protocol on public record with a date. For health-related reviews, PROSPERO is the established register and registration is widely expected. Other fields have their own repositories, and some have no convention at all — in which case depositing the protocol somewhere with a timestamp still strengthens the review.

If you deviate from the protocol later, that is permitted and common. What is required is reporting the deviation and the reason. Silent deviation is the problem, because it removes the protection the protocol was providing.

Searching and screening

The search

Search several databases — one is not enough, and coverage differs substantially between them. Your discipline has conventional choices; your librarian will know them, and involving a subject librarian in building the strategy is standard practice that most doctoral students do not think to do.

Build the strategy from your question’s concepts, combining synonyms within each concept and then combining concepts. Use each database’s own subject headings alongside free-text terms, since the same idea is indexed differently in different systems. Terminology also varies between fields, and searching only your own vocabulary silently excludes relevant work.

Supplement the database search: check reference lists of included studies, track citations forwards, and consider unpublished sources where publication bias matters — published work skews towards positive findings, and a review of only published studies inherits that skew.

Record the exact search string for each database with the date run. This is required for reporting and cannot be reconstructed afterwards.

Screening

Two stages: titles and abstracts first, then full texts of anything that survives.

Double screening — two people independently applying the criteria — is the standard because it catches errors and inconsistency, and agreement between screeners is conventionally reported. In doctoral work a second screener is not always available; where that is the case, say so, and consider having a supervisor double-screen a subset as a check. Describing the limitation is better than implying a process you did not follow.

At full-text stage, record the reason for every exclusion. This is the part people skip and then have to redo, because the flow diagram requires it.

Expect the numbers to be large. Reviews commonly screen thousands of records to include a few dozen, and the ratio surprises people planning their first one.

Appraisal and synthesis

Quality appraisal assesses whether each included study was well conducted, using an established tool appropriate to the study designs you have. Different tools exist for trials, observational studies, qualitative research and mixed methods — use one designed for what you are appraising, name it, and cite it.

Appraisal is not a filter by default. You can exclude low-quality studies, weight them differently, or include everything and examine whether quality relates to findings. Any is defensible; decide in the protocol rather than after seeing which studies score badly.

Synthesis is where reviews are strongest or weakest. Describing each study in turn is not synthesis, however systematic the search was.

• Meta-analysis pools effect sizes statistically. It requires studies similar enough in question, design and measurement for pooling to mean anything — combining incommensurable studies produces a precise number about nothing.
• Narrative synthesis groups and compares findings systematically without pooling, and is appropriate when studies are too heterogeneous to combine.
• Qualitative synthesis approaches — thematic synthesis, meta-ethnography and others — develop interpretations across qualitative studies rather than aggregating them.

Whatever the method, the synthesis should answer the review question, not merely summarise what exists.

Is a systematic review the right choice?

Worth asking honestly before committing, because the workload is substantial and the format does not suit every question.

It suits a focused question where a body of comparable empirical work exists, where the aim is to establish what the evidence collectively shows, and where you can commit the time — often a year or more if done properly at doctoral standard.

It suits badly broad exploratory questions, emerging areas with few studies, questions requiring conceptual rather than empirical synthesis, and situations where a second screener is impossible and the field expects one.

Consider the alternatives. A scoping review maps what exists in a broad area and is appropriate when the literature is not yet mature enough for a focused question. A rapid review uses systematic methods with acknowledged shortcuts. An integrative review combines empirical and theoretical work. Each has its own reporting expectations, and choosing the right one is stronger than doing a weak version of a systematic review.

One practical warning: if the systematic review is a chapter of your thesis rather than the whole thesis, be realistic about the proportion of your time it will take. Many candidates find it consumes a year they had allocated to something else.