Searching

Databases and sources

Searching databases will allow you to identify published literature on your topic. The databases you search for your review will depend on the research question you are trying to answer. You can find a list of the databases the Library subscribes to on the Library’s Databases by Subject page.

Commonly searched databases for health-related reviews include, but are not limited to:

PubMed: Provides citations with abstracts to worldwide biomedical literature. Covers research, clinical practice, administration, policy issues and healthcare.
PsycINFO: This American Psychological Association (APA) database contains more than 2 million citations and summaries of scholarly journal articles, book chapters, books, technical reports and dissertations, all in psychology and related disciplines, dating as far back as the 1800s.
Web of Science: Covers the sciences, including computer science and engineering, as well as the social sciences, arts and humanities.
Scopus: Abstract and citation database containing both peer-reviewed research literature and quality web sources. Scopus covers scientific, technical, medical and social sciences fields as well as the arts and humanities.
Wiley-Blackwell Cochrane Library: A collection of evidence-based medicine databases, including The Cochrane Database of Systematic Reviews, which examine evidence for and against the effectiveness and appropriateness of treatments (medications, surgery, education, etc) in specific circumstances.

The databases you search may not index all of the articles important to your review. Articles can accidentally get missed, or database coverage may not include specific journals altogether. As a result, hand searching important journals in your field of research can help improve the comprehensiveness of your review. For more information about hand-searching visit Rutgers University Libraries' guide on Systematic Reviews in the Health Sciences, "Hand searching".

Advanced searching and controlled vocabulary

Effective searching involves more than simply putting keywords in a search box. Systematic review search strategies need to be crafted in a manner that aims to ensure reproducibility and that no relevant articles are missed.

Identifying your review's key concepts and their synonyms or related terms should be your first step. Find more information about that process here: How do I identify my search terms? (Library Research Skills Tutorial).

Using search operators like AND and OR can drastically improve the quality of your search results. You can find more information about using search operators here: How do I combine my search terms? (Library Research Skills Tutorial)

Other, more advanced search techniques can be used to search for specific phrases, for keywords at a specific distance from others (adjacency searching), for spelling variants (wildcard searching) and for variant endings (truncation), amongst others. Individual databases often provide information on how to use these techniques in the “Help” sections of their websites. The way these search techniques are applied can vary from database to database. As a result, each database you search will require its own unique search strategy.

Another important aspect of databases to consider is their use of controlled vocabulary. Databases often use a specific set of words and phrases to index their articles. This set of words and phrases is known as a database’s controlled vocabulary. Controlled vocabulary can be extremely helpful for finding all of the articles on a specific topic, even if the articles themselves use different wording to describe a concept. As a result, using a database’s controlled vocabulary can help ensure comprehensiveness in your search, especially in cases where you have not been able to identify all possible synonyms for a key concept.

One good example of controlled vocabulary is the use of Medical Subject Headings (MeSH) to index articles in PubMed. Check out this PubMed help guide to learn how to Search PubMed using the MeSH database.

Both controlled vocabulary terms and keyword searching should be used in systematic review searches. Doing so should help prevent your search from missing articles in the database that have yet to go through the indexing process.

Testing searches

You should be creating and experimenting with several different versions of a search string before running your final search. You can use these trial searches as an opportunity to test your selection criteria as well.

To test that your searches are retrieving appropriate results, you can try these two methods: Sentinel Articles and the NOT operator.

Using sentinel articles

When developing your review topic, you likely came across a few articles that fit your selection criteria. You can use them to test your search, as these are articles that you should expect your systematic search to retrieve.

These are called sentinel articles, and you can build a search that retrieves only these articles by using the articles’ unique identifiers. For example, articles in PubMed have a unique identifier called the PMID. You can build a search consisting of only PMIDs by searching the PMIDs like this: PMID search for 31319564 OR 31226349 OR 34523250.

To test your search, use the database Search History to combine your systematic search and your sentinel article search with AND. The combined search will show you the number of articles common to both searches. If your combined search contains fewer records than your sentinel search, it means that not all of your sentinel articles are being retrieved by your current systematic search.

To determine which sentinel articles are missing, return to the database’s Search History and combine your sentinel search with your systematic search using NOT (so, Sentinel NOT YourSystematicSearch). Once you know which articles are not being retrieved by your systematic search, you can examine their records to determine why they were not being retrieved and make modifications to your search strategy as needed.

Using the search operator "NOT"

As you create your search, you may try different versions of your search using different parameters. For example, you may try limiting your keywords to the title/abstract field and try searching them in All Fields. You may try using truncation with a search term and you may just brainstorm the relevant terms with no truncation.

To test these searches against each other, you can use the NOT operator to see the differences between the searches. For example, you could have a first search that is very broad because you searched all your keywords as an All Fields search and you want to try to narrow it by searching your keywords only in the Title and Abstract fields. Run both of those searches, and then from the database’s Search History page, use the NOT operator to combine the searches: Search 1 (All Fields) NOT Search 2 (Title/Abstract). This will show you all the records that will be removed from your search if you make the changes to your search. Look at the records - are they relevant to your topic? If they are, perhaps you do not want to make these changes.

Filters/HEDGES

Filters are standardized or pre-formulated searches for specific concepts, methodologies, populations or geographic regions. They are also referred to as hedges or clinical queries. Filters will use a combination of keywords and controlled vocabulary and are database and platform specific. For example, a search filter designed to be used in PsycINFO on the EBSCO platform will not work the same way in PsycINFO on a different platform (such as OVID or APA PsycNet).

Useful resources for Search Filters:

Note: If you use a filter in your search, remember to cite it.

Grey literature

To ensure a truly comprehensive search and to prevent publication bias from playing a role in the results of your systematic review, other sources beyond journal literature databases should be considered. This is known as grey literature searching, which aims to find literature beyond what is published in traditional publication venues. Grey literature can include things like government documents, professional association guidelines, theses and dissertations, industry reports, whitepapers, and more.

The type and amount of grey literature searching needed will depend on your research question – there is no one correct way to carry it out. Grey literature searching requires you to think about the kinds of organizations and venues that might be creating documents that are of relevance to your research question.

Once you have identified the organizations that might be producing documents pertinent to your review, you will need to determine where those documents are kept. This may involve a keyword search of the organization’s webpage or navigating to a page on their site that holds their publications.

A Google search is another potential strategy you can use to find grey literature. See here for more information about running Google searches for grey literature (Simon Fraser University).

It is important to document the sources, search strategies, and results of your grey literature searching. This ensures reproducibility in your review methods.

Some sources to consider include (but are not limited to):

Government Documents: see the Concordia Library's guide to finding Government Information
Health Canada's Clinical Trials Database
Canadian Clinical Practice Guidelines: CPG Infobase
CADTH Grey Matters: searchable tool of grey literature sources (health organizations)
New York Academy of Medicine Grey Literature Report: No longer updated, but still accessible
Theses and dissertations: Try searching ProQuest Dissertations & Theses Global, but also, institutional repositories often store these kinds of documents (e.g., Spectrum: Concordia University's open access research repository)
Canada Commons: Canadian public policy papers from think tanks, government and research organizations.

Your subject librarian may also be able to help you identify good sources for your grey literature search. This grey literature guide from the University of Toronto, and these guides from the University of British Columbia (Health Sciences or Non-Health Subjects) provide additional sources for grey literature searching.

Search logs and reporting

Documenting your search process

When conducting systematic reviews, it is important to document the search process you used. It can be helpful to keep a search diary or log. Documenting your process will allow others to analyze the comprehensiveness of your search. This information will also be required if you intend to publish your review, either as part of a thesis or as a peer-reviewed journal article.

Be sure to keep track of the following:

Database you searched, along with the platform provider (e.g. PsycINFO on Ebsco vs PsycINFO on APA PsycNet; ERIC on Ebsco vs ERIC on Proquest, etc.);
The date each search was conducted;
Keywords and subject headings (include information such as if terms were truncated, exploded, etc.);
Search history (include combination of terms you used);
Number of results retrieved from each search and combination;
Total number of records retrieved;
Identify any duplicate records;
Numbers pre-screening and post-screening;
Identify the source (journal name, conference proceedings, etc.) and years for all hand-searched sources.

Reporting guidelines

When conducting a systematic review, you will need to keep track of many details related to the way you have designed and implemented your search strategy (see previous section on “documenting your search process”). For more detailed explanations of what search-related items you should be reporting and how to report them, refer to the reporting guidelines below:

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses): The minimum set of items for reporting in systematic reviews and meta-analyses (consult the flow diagram and checklist).
MOOSE (Meta-Analysis of Observational Studies in Epidemiology): Reporting guidelines for meta-analyses of observational studies.
CONSORT (Consolidated Standards of Reporting Trials) Statement: An evidence-based, minimum set of recommendations for reporting randomized trials.
EQUATOR (Enhancing the QUAlity and Transparency Of health Research): A database of reporting guidelines for different study types and other resources relevant to research reporting.