Exploring the role of ChatGPT in rapid intervention text development

Hannah M Bowers*, Cynthia Ochieng, Sarah E Bennett, Sarah Denford, Milly Johnston, Lucy Yardley

*Corresponding author for this work

Research output: Contribution to journalArticle (Academic Journal)

Abstract

Background: There have been successful applications of AI to answering health-related questions, which suggests a potential role for AI in assisting with development of intervention text. This paper explores how ChatGPT might be used to support the rapid development of intervention text. Methods: Three case studies are presented. In the first case study, ChatGPT (using GPT-4) was asked to generate sleep advice for adolescents. In case study two, ChatGPT (using GPT-3) was asked to optimise advice for people experiencing homelessness on staying hydrated in extreme heat. Case study three asked ChatGPT using GPT-3 and GPT-4 to optimise an information sheet for participation in a study developing an intervention for maternal blood pressure. Outputs were evaluated by the researchers who developed the text, and in case studies two and three were shown to public and patient contributors for feedback. Results: ChatGPT was able to generate informative advice about sleep in case study one and was able to accurately summarise information in case studies two and three. In all three cases, elements or aspects were omitted that were included in the researcher-generated text that was based on behaviour change theory, evidence and input from public and patient contributors. However, in case study three, feedback from public contributors suggested ChatGPTs outputs were preferred to the original, although the outputs omitted information and were not at the requested accessible reading level. Conclusions: ChatGPT was able to accurately generate and summarise health information. However, this information typically excluded core behaviour change techniques and was sometimes inappropriate for the target users. There is likely to be a valuable role for generative AI in the intervention development process, but this will need to be combined with detailed scrutiny and input from researchers and public contributors.
Original languageEnglish
JournalF1000Research
DOIs
Publication statusAccepted/In press - 23 Oct 2023

Bibliographical note

Acknowledgements
We would like to thank Mr Joe Shervell, Sabroso Ltd., for providing access to ChatGPT Plus and advising on the use of ChatGPT. We would also like to thank the PPI contributors for their comments in case studies two and three. LY is an NIHR Senior Investigator and her research programme is partly supported by NIHR Applied Research Collaboration (ARC)-West and NIHR Health Protection Research Unit (HPRU) for Behavioural Science and Evaluation.

Structured keywords

  • Health and Wellbeing (Psychological Science)

Keywords

  • Chat GPT
  • Intervention development
  • AI
  • Behaviour Change

Fingerprint

Dive into the research topics of 'Exploring the role of ChatGPT in rapid intervention text development'. Together they form a unique fingerprint.

Cite this