Skip to Main Content

Systematic Reviews

What is a Systematic Review?

A systematic review is a formal research study. It follows a clear, predefined structure to find, assess, and analyze studies that have all tried to answer a similar question. The results of a systematic review can provide a reliable picture of what we know - and what remains uncertain. It usually takes many months to do a systematic review.

Systematic review methods aim to minimize bias in reviewing research. Bias can result from a weakness or flaw in the way a review was designed, the way it was done, or the way it was analyzed.

Source: What is a Systematic Review?. Pubmed Health[Internet]. Bethesda (MD): National Library of Medicine (US)

Characteristics:

  • The scope of the review is identified in advance (eg review question and sub‐questions and/or sub‐group analysis to be undertaken)
  • Comprehensive search to find all relevant studies
  • Use of explicit criteria to include / exclude studies
  • Application of established standards to critically appraise study quality
  • Explicit methods of extracting and synthesizing study findings (qualitative or quantitative)
  • May include a meta-analysis (quantitative synthesis) *optional

Uses:

  • Identifies, appraises and synthesizes all available research that is relevant to a particular review question
  • Collates all that is known on a given topic and identifies the basis of that knowledge  
  • Comprehensive report using explicit processes so that rationale, assumptions and methods are open to scrutiny by external parties
  • Can be replicated / updated

Limitations:

  • Systematic reviews with narrowly defined review questions provide specific answers to specific questions  
  • Alternative questions that have not been answered usually need to be reconstructed by the reader  

Steps of Conducting a Systematic Review

1. Formulating a topic

Clear, structured questions are essential to systematic reviews. An example of a structure is the PICO framework for clinical questions (Population, Intervention, Comparison, Outcome)

“The research question should be precise so that the review team can structure the other components of the systematic review," according to the National Academy report.
 

2. Developing the systematic review protocol

A protocol is a detailed research plan, specifying all the steps the systematic reviewers will take, from formulating their topic to analyzing and reporting the results.

"A protocol should be made publicly available at the start of a systematic review in order to prevent the effects of author bias, allow feedback at an early stage in the systematic review," says the National Academy report, and have “the benefit that other researchers can identify ongoing reviews."
 

3. Finding and assessing individual studies

According to the National Academy, “The search for evidence and critical assessment of the individual studies identified are the core of a systematic review. These steps require meticulous execution and documentation to minimize the risk of a biased synthesis of evidence."

To do this, systematic reviewers:

  • construct a formal search strategy – a structured organization of search terms to capture information, tailored to each database being searched;
  • follow clear criteria for which studies will be eligible for inclusion and which will be rejected;
  • assess the strengths, weaknesses, and risk of bias in any studies they find; and
  • address the potential for unpublished studies to tip the scales.
     

4. Synthesizing the body of evidence

This process, according to the National Academy report, "refers to the collation, combination, and summary of the results." It involves qualitative synthesis to convey understanding of the quality and meaning of the evidence. Quantitative synthesis, or meta-analysis, combines and weighs comparable data from more than one study.
 

5. Providing a detailed and comprehensive report

Systematic reviews should report all 27 items in the PRISMA Statement ( P referred R eporting I tems for Systematic reviews and M eta- A nalyses). This includes providing a diagram that shows the flow of studies found, screened, and kept or rejected.

If a systematic review report does not provide enough information about what was done and how, it will not be possible to know how reliable the review is. According to the National Academy report, “A report should provide enough detail that a knowledgeable reader could reproduce the systematic review."

Source: What is a Systematic Review?. Pubmed Health[Internet]. Bethesda (MD): National Library of Medicine (US)

What You Need To Conduct a Systematic Review

Time (18 months, average)

The average systematic review requires 18 months of work. “…to find out about a healthcare intervention it is worth searching research literature thoroughly to see if the answer is already known. This may require considerable work over many months…” (Cochrane Collaboration)

If that timeline doesn’t meet your needs, you can take a look at other types reviews.

Team Members (at minimum...)

“Expert searchers are an important part of the systematic review team, crucial throughout the review process-from the development of the proposal and research question to publication.” (McGowan & Sampson, 2005)

*Be sure to ask your information professional (librarian) to write a methods section regarding the search methods and to give them co-authorship. You may also want to consider providing a copy of one of the search strategies used in an appendix.

Searching

Before your librarian creates a search strategy and starts searching in earnest you should write a detailed PICOTT question, determine the inclusion and exclusion criteria for your study, run a preliminary search, and have 2-4 articles that already fit the criteria for your review.

What is searched depends on the topic of the review but should include...

  • At least 3 standard medical databases like PubMed/Medline, CINAHL, possibly Scopus, etc..
  • At least 3 grey literature resources like Clinicaltrials.gov, SIGLE, ProQuest Dissertations & Theses, etc...

Systematic Review Standards