Support for systematic reviews

A systematic review has a clearly formulated research question with pre-defined selection criteria. It is characterised by systematic and clear methods for searching, selecting and critically assessing relevant research.  
 
A systematic review has four cornerstones, it should be:

• Comprehensive. The overall goal of a search strategy for a systematic literature review is to be comprehensive and exhaustive.
• Transparent. Accurate documentation is very important, documenting the choices made throughout the process.
• Reproducible. The documentation is also important to be able to reproduce the review.
• Designed to minimise bias. By searching in several databases, including grey literature to avoid publication bias and having at least two reviewers in the screening.

What is the difference between a systematic review and a literature review?

A systematic review explores and maps a topic in depth by using a scientific method with a clearly defined and narrow research question. The ambition is to find all relevant research in a defined area. A literature review aims to provide an overview of the literature written in a field. The idea is to provide a broad understanding of a research area within the topic, without necessarily finding all available research.

Types of systematic reviews

• Systematic review: A review that aims to comprehensively identify all relevant studies to answer a specific question.
• Systematic map: Maps out and categorises existing literature to identify gaps or interesting clusters in the research literature, from which further reviews and/or primary research can be conducted.
• Rapid review: A literature review conducted systematically, but within a limited time frame and with restrictions on the scope.
• Scoping review: Sometimes a scoping review is carried out as a preliminary to a full systematic review, to narrow down the existing literature. It can also be carried out as a mapping exercise to summarise and disseminate research findings.
• Systematised review: An attempt to include elements of the systematic review process, but which does not meet the standard of a full systematic review.

Tip!
Even if you are not going to do a full systematic review, you can still use a systematic and structured approach, especially when searching for information.

The first step is to define a clear and well-defined research question. To make the question searchable, you need to identify the main concepts of the question, and then create different search blocks. Read more on the Search Tips page.

Use a wide range of search terms for each concept in the different search blocks. You need to search with both free text terms and controlled terms, which some databases have in searchable thesauri.

Structure your research question

Start by considering the various components that make up your research question. There are a few different standardised acronyms that can be helpful. A common example is PICO:

• P - Patients, Population or Problem
• I - Intervention or Exposure
• C - Comparison
• O - Outcome

PICO is often used in medicine, but works well for other subjects. Sometimes not all aspects are used, for example if comparison is missing or too vague, PIO can be more useful.

Not all aspects are suitable for inclusion in the search query, geographical areas can for example become problematic. It may be better to include the aspects in your selection criteria instead. See the Screening section below.

Example

• P: Crops of Zea Mays
• I: Cover Crop Treatment
• C: No CCT (comparison with other year or other location)
• O: Yields

Selection criteria:

• Yield records came from corn following a cover crop treatment, and corn following no cover.
• Yields were reported in more than one year or location.
• Enough information was provided to compute study variances.
• The studies were conducted in the United States or Canada.

Example from: Marcillo, GS & Miguez, FE (2017) Corn yield response to winter cover crops: An updated meta-analysis. Journal of Soil and Water Conservation 72(3): 226-239. https://doi.org/10.2489/jswc.72.3.226.

Tip!
There are other acronyms than PICO, read more at Uppsala University Library's guide Systematic reviews.

Test your search

Select one database to test and refine your search in, to ensure that it captures relevant material before proceeding to do searches in all selected databases. An easy way to test whether the search captures what you want to find is to select a number of key articles that should be include in your search. If they appear in the results, you are well on your way, if not, you need to go back and adjust your search.

A common perception when conducting systematic searches is that the number of hits is large, and that much of the material included is outside the scope of the review. It is important to remember that a systematic search needs to be broad enough to capture all relevant studies. There are studies that show that the number of selected articles after screening is usually about 2-4% of the total number of hits from the search, see, for example, the article "Precision of healthcare systematic review searches in a cross-sectional sample". If you still feel that the search gives too extensive a result, you may rather need to go back and adjust your research question.

Tip!
When conducting a systematic review, do not forget that the search and its results are the main part of the data collection.

As in other parts of the research process, it is very important to document everything carefully. Your search process must be described in sufficient detail to be reported, reproduced and updated.

For each database you search, you should document:

• date when the search was carried out
• the name of the database, and the platform or provider supplying it
• your search strategy – include both keywords and how these were combined in the search
• the number of hits for each individual search
• any filters or limitations used, such as dates, languages, publication types etc.

Example

The search strategy should be documented for each individual database and published in an appendix to the published article. If the library's search specialists perform searches, they also compile the documentation for these.

To document the screening, it is very helpful to follow a specific protocol, such as PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) or ROSES (RepOrting standards for Systematic Evidence Syntheses in environmental research). In the protocol you can document in detail which documents were excluded in which part of the screening, and why. PRISMA also has a specific protocol for the search strategy, PRISMA-S.

• PRISMA
• PRISMA-S
• ROSES

Tip!
Document the search process from day one and continue throughout the whole project.  

It is important to search multiple databases as no single database contains all the information. For a complete systematic review, you should search all databases that may contain relevant information, but it is common to start with at least three. By searching both broad, multisciplinary and subject-specific databases, you have a good chance of finding different types of relevant material.

Web of Science Core Collection and Scopus are two recommended multidisciplinary databases.

Here is a selection of subject-specific databases that SLU has access to:

• CAB Abstracts, CABI
• Food Science & Technology Abstracts, FSTA
• GreenFile
• Garden, Landscape & Horticulture Index

SLU provides access to many databases, you can find them all under Databases via the library website.

Tip!
Create an account in the databases you use to easily save and reuse your searches.

Grey literature refers to various types of scientifically produced material that is not published as a scientific article. Examples of grey literature include dissertations, reports, preprints, government publications, standards, annual reports, financial statements and policy documents. Including grey literature in a systematic review could be important to ultimately avoid bias. Depending on the topic, purpose and objectives of the review, you may not need to search for grey literature. The decision whether to do it or not should be discussed and motivated in the documentation.

It is usually more challenging to search for grey publications in as structured manner as articles. Depending on where you search, it may also not be possible to use equally advanced search strings. It is therefore particularly important to be clear in describing and documenting your approach to the search for grey literature.

A selection of resources for searching grey material:

• BioRxiv. Freely available repository of preprints in biology.
• Lens. Various types of scientific publications including patents. Freely available.
• Media archive. News and press material mainly in Sweden and the Nordic countries. Subscription service, access on SLU campus or via VPN.
• Retriever Business. Industry information such as annual reports and financial statements for all Swedish companies. Subscription service, access on SLU campus or via VPN.
• SLUpub. Dissertations, reports and conference contributions from SLU researchers. Freely available.

When searching for information in multiple databases and additional sources, it is important to collect everything in one place. Reference management programmes like EndNote and Zotero are just what you need in this situation.

• Read more about how to get started with a reference management programme

You will have many duplicates, and it is important to remove these before screening and evaluating your references. EndNote has built-in functionality to remove duplicates.

• Karolinska Institutet Library's guide to effective duplicate cleaning in EndNote

Screening refers to the step of evaluating and selecting the studies to be included in the review. Before starting the screening process, it is important to agree in advance on how to assess the material being evaluated. Inclusion and exclusion critera need to be clearly defined in order to decide whether a source should be included in the review or not. It is also important that more than one person screens the search results, not least to avoid bias.

There are different tools to use during the screening process:

• CADIMA (Freely available)
• Covidence
• DistillerSR
• EPPI Reviewer
• Rayyan (Freely available).

Tip!
Library search specialists have experience with Rayyan and can help you get started with the program.