Should social science researchers rethink their ‘go-to’ databases and search platforms when doing literature reviews?

Should social science researchers rethink their ‘go-to’ databases and search platforms when doing literature reviews?

This post is written by Paul Kellner as part of Good Questions Review, a living literature review about how social science can be useful for making decisions. It was made possible through support from Open Philanthropy.

Note: this post is continuously updated as relevant articles are added to Good Questions Review. An important part of this project is archiving substantive edits to posts. To do this we create a digital object identifier for the original post as well as subsequent versions. Please find the archived version of this post here.

It can be easy to fall into old habits when undertaking complex work. If a tool or approach has worked well for us in the past, it’s often easiest to follow the same, well-known path. Two relatively recent articles questioning education and social work researchers’ assumptions about their “go to” databases in their fields raise some important questions about these habits. Where and how we search for literature can directly shape the way we frame research questions and methods, and therefore have policy and practice impact. This is especially true if the researcher is embarking on a project like a systematic review because the sample of articles identified by the search ultimately dictates the quality of the project.

This post is slightly “under the hood” look at some evidence that can inform how social scientists think about their document search approaches. The included articles all focus on searches for evidence synthesis projects (taking a rigorous, comprehensive, and systematic approach to identifying and combining evidence). It’s not meant to be a full primer on how and why to select databases and search platforms for these types of projects, but the included articles generally provide an empirical basis for several standards of practice in the field, including:

  • documenting and justifying your search strategy
  • searching multiple databases, including at least one meta database like Scopus or Web of Science
  • consider using other techniques like citation tracking to ensure comprehensiveness.

Most of the articles in this post are framed from the perspective that researchers in some fields are not well acquainted with these standards of practice and may often be guided by familiarity or ease of use when choosing a database or search platform. This post seeks to provide some empirical insight into why social scientists may want to consider carefully where and how they search for information. We’ll engage with the following:

  • findings about subject-specific databases in the social sciences
  • findings about a range of databases and platforms that researchers from several disciplines, including the social sciences, may use
  • a tool that may help researches make more informed decisions about databases to search.

'Go to' databases may need to be supplemented by other approaches

Fitzgerald, Weaver, and Droog (2025)1 explored the overlap between four specialised education databases including ERIC, a “go to” database in their field. The article is framed around the idea that the authors perceive many researchers as not having a strong sense of the strengths and weaknesses of the databases available to them. By analysing how many journals were indexed in each of the specialist databases, they found that a large proportion of journals were only included by one database and that several others were only included by two or three (not all four) databases. Moreover, they found that ERIC had the least number of unique journals. Based on this, the authors conclude that researchers need to examine their assumptions about which database(s) they choose when undertaking a literature review. Given this result, it also shows that searching multiple specialist databases in education is likely necessary if the researchers seek to develop the type of comprehensive search that is necessary for a systematic review. Their database decisions should then be justified based on the project parameters and reported so others can assess the quality of their search.

Similarly, Pascoe et al. (2021)2 analysed the performance of a key database in social work, Social Work Abstracts, against a wide range of specialist databases and other meta databases (e.g. Scopus or Web of Science). The authors indicated that database selection is a critical consideration because researchers are often time poor, the volume of publications in the field is constantly growing, and many reviews in this field end up having direct practice implications. They had similar findings to Fitzgerald, Weaver, and Droog (2025); the “go to” database in their field did not perform particularly well and they concluded that social science researchers should use multiple specialist databases and/or a platform like EBSCO that can search multiple databases at once if they are looking to undertake a comprehensive search in their field. The authors of this article acknowledged that researchers often need to strike a balance between how much time they have, their levels of search literacy, and their access to proprietary databases. Ultimately, they also underline that researchers need to examine their knowledge and assumptions about databases, and make justifiable decisions based on both rigour and practicality.

Bias in search inputs and results

Zooming out from subject-specific database discussions, there are other recent studies that underline why and how researchers may want to examine their assumptions when starting a new literature search.

One recent study from Kacaperski et al. (2024)3 considered if confirmation biased searches inputs in platforms like Google Scholar and Semantic Scholar resulted in biased results. The authors framed the study around the idea that if search results increase the degree of confirmation bias in researchers’ work – e.g. their searches serve them outputs aligned with their pre-existing belief rather than the ‘true’ data – that these biases can meaningfully shape the direction of future research and practice. They found that biases from users’ inputs were reflected in the most relevant outputs from searches in Google Scholar and Semantic Scholar. Moreover, they found that some subjects also had inherent biases – for instance, searches related to health skewed towards articles discussing benefits rather than risks. Given the fact that it is often difficult to know how search platforms calculate relevance in their outputs, these results indicate that researchers need to maintain self-awareness about how they interpret the most relevant results as well as potential biases inherent to their search approach (i.e. is their search framed in a way that seeks a particular type of answer). This emphasis on self-awareness and identifying known unknowns is similar to a discussion in another Good Questions review post on generative AI competencies for social scientists.

More general cross-database analyses

Another way that researchers sometimes round out their knowledge on a subject is by citation tracking. This involves one of two practices, either looking backwards in time by reviewing the citations list in a paper that is known to be highly relevant and finding other relevant papers within the list (backwards tracking). Alternatively, one can look forwards in time to find papers that cite a paper known to be highly relevant (forwards tracking). These techniques are often used systematic reviews because they help researchers ensure the comprehensiveness of their search. These practices are particularly useful for finding relevant papers that may not be captured by the main search approach due to using different terminology, being written for a different discipline, among other reasons. A paper by Gusenbauer (2024)4 found that databases perform differently for forward and backwards tracking – Google Scholar, Semantic Scholar, Research Gate, and Lens excelled for forward tracking, whereas databases like Web of Science were found to be better for backwards tracking. These findings imply that if researchers plan to employ tracking techniques, they may need to consider their knowledge and awareness of papers on an issue when selecting a database. If they are only aware of very recent papers and are mostly looking backwards, they may choose a different database than if they are only aware of relatively old papers and are looking forward.

An earlier article from Gusenbauer and Haddaway (2020)5 evaluated the performance of 28 commonly used search systems for the purpose of undertaking a comprehensive search. They used a range of metrics including recall (the proportion of relevant articles returned by a search from all known relevant articles, i.e. sensitivity), precision (the proportion of relevant articles within the total articles returned by the search, i.e. specificity), and reproducibility (the ability to design a search that can be replicated by others). They found substantial differences between the performance of several search systems and could only recommend about half of all of those evaluated for researchers seeking to undertake a comprehensive search. Moreover, their findings indicate very few open access platforms (e.g. Google Scholar) can be recommended and that Google Scholar may not be an ideal primary search platform for a comprehensive search. Rather these resources may be best placed to provide a supplement to searches in other databases. Given how frequently researchers and students use Google Scholar, this again underlines the notion that researchers may want to be self-aware about their understanding of and assumptions about the search tools they are using.

Finally, Haupka et al. (2024)6 find there are trade-offs between the strengths and weakness of proprietary meta databases like Scopus and Web of Science, and open databases like Open Alex or Semantic Scholar. They explored the coverage of several  search platforms and found that open services often had a much broader range of materials included in their databases. However, these databases were found to have less precise meta data than proprietary platforms like Scopus and Web of Science. This means that a skilled researcher may be more likely to easily find the types of documents they seek with a properly-used proprietary tool, even if the open systems may be more likely to have a wider range of relevant documents.

Summing up and a useful tool

To summarise the implications of the included articles’ findings:

  1. We may need to examine our assumptions about our “go to” databases in our respective fields.
  2. We may need to be more self-aware about potential confirmation bias in our search inputs and results.
  3. Different databases may be better for backwards or forward citation tracking.
  4. Open databases may have ease of use and comprehensiveness, but may not perform well-enough to be used on its own or as the primary platform for a comprehensive search.

All of these findings support the general advice provided at the outset – search multiple databases, make documented and justified decisions, and consider how you undertake supplemental activities like citation tracking. In other words, researchers very much should consider their assumptions about “go to” databases and search platforms, because those choices can meaningfully influence the documents they find, how they understand the evidence in their field, and in the end, how they frame questions and undertake research.

However, these findings provide a way forward for researchers that is about a clear as mud! If a researcher is time poor or doesn’t know where to start, several studies saying your approach should depend on your unique circumstances may risk researchers’ falling into old habits. I’d be remiss if I didn’t say that if researchers have access to a academic librarian who can help, engaging with them would likely be time well spent. They’re full-time specialists in this sort of thing. This said, there is also a relatively new tool that can help researchers quickly explore where and how they might undertake their search. Gusenbauer (2024)7 shares useful information about their project Searchsmart.org. It’s designed to help researchers determine which databases might be ideal for their particular needs. It allows the user to compare the databases at a glance to determine their coverage of a range of subjects/disciplines/keywords, which might be best for backward or forward citation tracking, or which service provide non-paywalled access. As a professional in this field, I’ve found it useful to learn about entirely new and useful databases, as well as examine my assumptions about how I might approach certain projects. This might be a great first port of call for researchers looking to navigate the questions raised in this post (either that or reach out to your local librarian!).

Articles cited

  1. Fitzgerald SR, Weaver KD, Droog A. Selecting a specialized education database for literature reviews and evidence synthesis projects. Res synth methods. 2025;16(1):30-41. doi:10.1017/rsm.2024.11
  2. Pascoe KM, Waterhouse-Bradley B, McGinn T. Systematic Literature Searching in Social Work: A Practical Guide With Database Appraisal. Research on Social Work Practice. 2021;31(5):541-551. doi:10.1177/1049731520986857
  3. Kacperski C, Bielig M, Makhortykh M, Sydorova M, Ulloa R. Examining bias perpetuation in academic search engines: An algorithm audit of Google and Semantic Scholar. FM. Published online November 3, 2024. doi:10.5210/fm.v29i11.13730
  4. Gusenbauer M. Beyond Google Scholar, Scopus, and Web of Science: An evaluation of the backward and forward citation coverage of 59 databases’ citation indices. Research Synthesis Methods. 2024;15(5):802-817. doi:10.1002/jrsm.1729
  5. Gusenbauer M, Haddaway NR. Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Research Synthesis Methods. 2020;11(2):181-217. doi:10.1002/jrsm.1378
  6. Haupka N, Culbert JH, Schniedermann A, Jahn N, Mayr P. Analysis of the Publication and Document Types in OpenAlex, Web of Science, Scopus, Pubmed and Semantic Scholar. Published online June 21, 2024. doi:10.48550/arXiv.2406.15154
  7. Gusenbauer M. Searchsmart.org: Guiding researchers to the best databases and search systems for systematic reviews and beyond. Research Synthesis Methods. 2024;15(6):1200-1213. doi:10.1002/jrsm.1746