What competencies do social scientists need to responsibly incorporate AI tools into their research practices?

What competencies do social scientists need to responsibly incorporate AI tools into their research practices?

This post is written by Paul Kellner as part of Good Questions Review, a living literature review about how social science can be useful for making decisions. It was made possible through support from Open Philanthropy.

Note: this post is continuously updated as relevant articles are added to Good Questions Review. An important part of this project is archiving substantive edits to posts. To do this we create a digital object identifier for the original post as well as subsequent versions. Please find the archived version of this post here.

The diversity and capabilities of Generative AI (Gen AI) tools relevant to social science research are changing at a very rapid pace. However, the pace of high-quality research (usually slower than “rapid”) may not be fully aligned with the pace of Gen AI’s development. Part of Good Questions Review’s purpose is to understand how social science researchers can improve their ability to ask useful, timely questions and deploy the latest methods in context of social and technological change. Therefore, this post explores:

  • What do recent articles say about the skills, knowledge, and other capacities that social scientists need to realise the opportunities and address the risks posed by using Gen AI in their research?

I’ve done a thorough search and written an overview of some of the recent reviews on this subject. Then, I’ve attempted to draw out some common insights from across several writers within multiple subdisciplines of social science.

The papers that underpin this post

The papers that underpin this post are the result of a thorough but non-comprehensive search for very recent articles that comment on the implications of (primarily) Gen AI for social science research. Most are literature reviews focused on a subfield or an opinion/perspective piece drawing on recent literature. Here’s a quick profile of the papers that underpin the rest of this post:

  • Bail (2024) is a perspective piece describing the potential and limitations of Gen AI that ultimately argues the need to open-source Gen AI infrastructure in the social sciences 1.
  • Davidson (2024) provides an overview of recent studies in social and computer science to underline how Gen AI can complement sociological research 2.
  • Lorenz et al. (2024) offers practical ideas about how to navigate the “promises and pitfalls” of using Gen AI in management research 3.
  • Sebastian et al. (2025) provides a critical review of the implications of Gen AI in social science research 4.
  • Sivarudran, Pillai, & Matus (2024) explored the potential regulatory solutions to alleviate the risks of large language models in qualitative research through a literature review and qualitative research 5.
  • Victor et al. (2023) describes risks and opportunities presented by Gen AI for social work researchers and journal editors and provides guidance about how to address those matters 6.

This post provides a high-level, point-in-time overview of authors’ views on what social scientists need to know and do in the context of the rapid development Gen AI. This post will surely need to be updated almost as soon as its published!

Note: This post uses a new citations approach with superscripts to indicate which papers supported which point – have a look at the article details at the end of the post.

Overview of some key opportunities and risks of Gen AI for social scientists

This section summarises some key opportunities presented by the reviewed papers, alongside a few closely associated risks. Additional consideration of risks will be also provided in the next section on emerging competencies.

Across the papers that underpin this post, it is clear that there are Gen AI capabilities that can enhance, or impact, the practice of all stages of the research process. This includes, but is not limited to, research idea generation and literature review, data collection, data analysis, and writing about and sharing findings 3.

Idea generation and literature review

More than one paper said that Gen AI has potential to significantly enhance or impact the way that research questions are developed, literature is searched and synthesised, research topics and theoretical insights are identified, and conceptual models our outlined and refined 1,3,6. Lorenz et al. (2024) underlines the key role that Gen AI technologies can play in accelerating and strengthening the ways that researchers identify gaps in literature and refine their approaches to addressing them. Beyond this, the data collection and analysis possibilities addressed in the next section may “…significantly expand the range of research questions that social scientists can study”1(p1).

A range of risks associated with these opportunities were also mentioned. The two most prominent were 1) researchers starting to rely too heavily on Gen AI tools and thereby letting their critical thinking abilities deteriorate, and 2) researchers not maintaining sufficient awareness of how much AI outputs are shaping their research approach 4,6.

Data collection and analysis

There are several emerging ways that Gen AI may potentially reshape data collection and analysis. For instance, the Bail (2024) article says that survey samples could one day be made more diverse and researchers may be able to administer longer surveys through the use of “silicon samples” comprised synthetic, large language model-based survey respondents mimicking behaviours of human respondents 1. Very many risks related to bias are present here including, some large language models currently behaving in a way that exaggerates extreme viewpoints, exhibits bias depending on question type, or otherwise differs from the human respondents they are designed to simulate 1. Based on this, it’s likely that these samples might first be useful and reliable in the process of developing measures and scales, or pretesting a survey tool 1. Bail (2024) also describes the possibility of Gen AI being used to interact directly with human participants of research in an unsupervised manner. The same risks apply as to the previous example, but could be magnified given the involvement of a human participant.

The potential value of Gen AI for analysis was also mentioned by several papers. Most notably, Gen AI can, and already does, allow researcher to  “ask questions of,” summarise, and identify patterns in qualitative data sets 2,6, which can result in create substantial time and effort efficiencies 6. These efficiencies can, in turn, mean that researchers may be able to undertake research that analyses larger volumes of text – thus open up additional avenues for research questions not previously thought to be feasible using only human effort. However, again, there are several risks. Notably, because the models that underpin Gen AI tools are often “black boxes”, it is unclear to researchers how the training data used to create these tools will shape its analytical outputs 2. Thus, researchers may be unaware of, or inaccurately estimate, the biases in Gen AI outputs. Additionally, more than one author said that it is yet to be seen if large language models will able to replicate the “nuanced human understanding and social skills required for rigorous scholarly work” in the social sciences 3(p2). In other words, it is not clear if and/or human content matter expertise, social skills, and other specialist knowledge and skills will be sufficiently replicated in a Gen AI tool.

Writing up and sharing findings

Finally, although it is not addressed in detail here, the authors of the included papers mentioned that Gen AI can be used in a number of potentially useful ways like generating text and translating texts 6. The section on attribution later in this article will touch briefly on this.

A list of emerging Gen AI competencies for social scientists

Based on the above-mentioned opportunities, and other factors, the authors described several recommendations, tips, and other suggestions about how social scientists will need to proceed in the coming years when engaging with Gen AI. These items have been synthesised across the papers and roughly consolidated under the headings of “emerging competencies.” There are few caveats to consider. The term competencies is used loosely to refer to skills, knowledge, and capacities that can be used together to act effectively. Also, it should be noted that these suggested competencies are highly interrelated, not comprehensive, and merely reflect my reading of the cited authors’ work. Much of their organisation reflects the very useful contributions of the 5 “don’ts” listed in Lorenz et al. (2024) and the recommendations made in Victor et al. (2023). Thus, if you’re looking for more detail than is provided in this post, please start by reading those papers.  Lastly, these have not yet been compared to other competency frameworks published for other potentially relevant published frameworks (e.g. those being designed for current students).

Critical assessment of outputs

Most of the papers reviewed touched on the idea that social scientists need to develop skills and knowledge to help them critically assess the outputs from Gen AI. In particular, researchers need to be aware of the potential biases inherent in the models used by various tools 1,3–6. As mentioned earlier, Gen AI tools are likely to reproduce a range of biases in their training – e.g. those related to race or gender 4,6. Several papers also highlighted that many tools hallucinate – AI presenting false information as true 1,3,6. Given these risks, researchers will need to build awareness of these issues, and similar emerging issues. Alongside awareness, they’ll need to develop skills to evaluate and verify the reliability and validity of outputs 3,6.

Most papers also mentioned that commercially available Gen AI tools are not built in a manner that allows researchers to easily understand how and why an output was produced 1,3,5,6. This is due to several factors including lack of transparency[i] about the data on which the model was trained. At least one author called for social scientists to participate in developing open source Gen AI models 1 that allow for researchers to understand, and possibly control, the inputs and systems with which they are working. This will allow these researchers to better understand the strengths and weaknesses of outputs. In addition to this, scholars need to develop their ability to communicate about what they do and don’t know in relation to the outputs of Gen AI tools they’ve used 6.

Based on this, key aspects of this competency include:

  • Knowledge of what they do and don’t know about the tool they are using, and the ability to communicate about these facts;
  • Awareness of the potential biases in the subject matter they are working with;
  • Skills for critically assessing outputs from tools for bias, likely including the ability to engage with credible information about the subject of their work using additional external data sources to validate findings; and
  • Potentially, skills for participating in collaborative efforts to build models that have fewer of the research-related weaknesses inherent in commercially available models.

Refined ethics practices for research including Gen AI

Researchers will clearly need new and/or refined competencies for assessing and responding to a range of ethical considerations.

As mentioned above, Gen AI may allow researchers to deploy entirely new research methods that will have their own unique ethical implications (e.g. AI may not behave as anticipated when used to interact with human participants). Researchers will need to develop their ability to critically assess the ethical implications of these new tools 2,4. For instance, using the example earlier about Gen AI tools interacting directly with experimental participants in lieu of a human researcher. This could put participants at risk of being subjected to abusive language, AI tools not performing the experimental tasks in explainable ways, or providing participants with unwanted self-knowledge. To mitigate this risk, the above competency about critical assessment of outputs is relevant. However, additionally, researchers will need to develop the ability to develop new ethical protocols for preventing or responding to unanticipated or problematic Gen AI behaviours. It is no small task to anticipate, prevent and respond to this risk, and so researchers advocating for and supporting the development of institutional systems, policies, and guidance about responsible use of Gen AI is also merited 5.

Privacy and data management

Within the broader ethics umbrella, most papers also said that many Gen AI tools raise substantial privacy concerns related to research involving human subjects. Because many tools are private-company-owned, cloud-based services, researchers who uncritically or indiscriminately use these tools, may also risk exposing research participants personal or sensitive data 6. Mishandling of private data may result in researchers violating institutional, regulatory, or ethical requirements 2,3,6. These issues mean that researchers may need to refine the ways that they gain informed consent from participants if they use Gen AI systems, including effectively understanding and explaining the implications of AI use in data collection, analysis, and management. Just as with informed consent processes more generally, there may be specific nuances to consider when deploying Gen AI in research with various vulnerable populations.

Based on the above, new and/or refined ethics competencies might include:

  • Knowledge of the data handling policies of tools that they are considering using;
  • Skills and guidelines to adequately informing and gaining consent from human research participants about the use of Gen AI tools;
  • Skills for good decision-making about if, when, and how data should be used in conjunction with Gen AI tools that uphold existing ethical principles like informed consent; and
  • Willingness to advocate for and support the development of systems, policies, and guidance that helps researchers navigate a rapidly changing set of tools.

Attribution and accountability

Several papers also raised the importance of new considerations related to attribution and accountability 1,3,5,6.

The lack of transparency about the training data for several models means that when researchers use Gen AI for analytical purposes, the quality or recency of the data used to support the analysis may be unclear 6. Additionally, Gen AI tools ability to rapidly categorise and synthesise qualitative data may result in researchers uncritically taking on ideas without verification 3. In some cases, this may result in researchers incorporating Gen AI produced  interpretations that may be flawed when compared to analysis rooted in specific content matter expertise and/or recent, high-quality, peer-reviewed sources, as might be present in human analysis.

Additionally, when generating text or developing ideas, the researcher can state if and how they used Gen AI, but it may be difficult to parse the originality of the text or ideas 6. This difficulty in attribution also means that it is difficult to know who is ultimately accountable for the ideas put forward. Based on my reading of the current literature, there is not currently consensus on defining the accountability of popular Gen AI tools 5.

One potential solution for these concerns is again, social science sub-disciplines developing open source tools and guidance that support researchers to navigate these concerns 1 at the pace of technological development.

Based on this, social science researchers need to maintain awareness of these attribution and accountability issues and stay current with the suggested and/or required practices in their field.

Quality control for inputs

A few papers also mentioned that social science researchers need to build their ability to input “…diverse, quality data and specific prompts to ensure unbiased and relevant results” 3(p13). Given the diversity of Gen AI tools available, the emerging writing in this field confirms what a casual user of Gen AI tool might assume, that prompting skills need to be developed in a tool specific way 3. Taking careful consideration of the critical assessment of outputs competencies mentioned earlier, social scientists using Gen AI will need to develop skills for inputting data and prompts in a manner that results in outputs that are reliable and valid. This may include understanding how to develop prompts that ensure that are able to know that subject-specific knowledge is utilised by the model (e.g. data may be interpreted differently depending on a researcher’s theoretical framework and existing knowledge) 3. Additionally, developing input practices that consider if and how interpretations of data may be affected by the Gen AI tools’ current ability (or inability) to consider context and other factors like a interviewees emphasis on particular words, emotional state, or community-specific meanings of words 5.

Continuous learning on an individual and group level

Lastly, it merits noting that individual- or community-level learning on a regular basis was mentioned by several authors 3,5,6. This may involve establishing new methods and modes for ongoing learning as well 5.

Conclusion

Based on this overview of recent review-level articles on social scientists’ use of Gen AI, it is clear that researchers will need to develop a range of new or refined competencies to realise the opportunities and mitigate the risks. As it pertains to the purpose of Good Questions review, new methodological avenues opened by Gen AI will yield new questions that had not been possible to pursue before due to a range of resource constraints. Moreover, it seems likely that in the near term, more research projects may also be asking important meta scientific research questions – research questions about how science itself in undertaken. The same competencies that are required to integrate Gen AI tools into research methods, can also be used to assess whether or not Gen AI’s integration actually makes research more effective or efficient, and whether the benefits outweigh the risks.

[i] There are large and growing literatures on transparency, as well as interpretability and explainability of AI systems, but it is not feasible to consider these issues in this post.

Articles cited

  1. Bail CA. Can Generative AI improve social science? Proc Natl Acad Sci USA. 2024;121(21):e2314021121. doi:10.1073/pnas.2314021121
  2. Davidson T. Start Generating: Harnessing Generative Artificial Intelligence for Sociological Research. Socius: Sociological Research for a Dynamic World. 2024;10:1-17.
  3. Lorenz F, Lorenzen S, Franco M, Velz J, Clauß T. Generative artificial intelligence in management research: a practical guide on mistakes to avoid. Manag Rev Q. Published online December 19, 2024. doi:10.1007/s11301-024-00469-2
  4. Sebastian R, Kottekkadan NN, Thomas TK, Niyas Kk M. Generative AI tools (ChatGPT*) in social science research. JICES. Published online January 20, 2025. doi:10.1108/JICES-10-2024-0145
  5. Sivarudran Pillai V, Matus K. Regulatory solutions to alleviate the risks of generative AI models in qualitative research. Journal of Asian Public Policy. Published online September 8, 2024:1-24. doi:10.1080/17516234.2024.2399098
  6. Victor BG, Sokol RL, Goldkind L, Perron BE. Recommendations for Social Work Researchers and Journal Editors on the Use of Generative AI and Large Language Models. Journal of the Society for Social Work and Research. 2023;14(3):563-577. doi:10.1086/726021