Not everything is online grooming: False risk finding in large language model assessments of human conversations

Ellie  Prosser; Matthew  Edwards

Not everything is online grooming: False risk finding in large language model assessments of human conversations

Ellie Prosser, Matthew Edwards

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

61 Downloads (Pure)

Abstract

Large Language Models (LLMs) have rapidly been adopted by the general public, and as usage of these models becomes commonplace, they naturally will be used for increasingly human-centric tasks, including security advice and risk identifications for personal situations. It is imperative that systems used in such a manner are well-calibrated. In this paper, 6 popular LLMs were evaluated for their propensity towards false or over-cautious risk finding in online interactions between real people, with a focus on the risk of online grooming, the advice generated for such contexts, and the impact of prompt specificity. Through an analysis of 3840 generated answers, it was found that models could find online grooming in even the most harmless of interactions, and that the generated advice could be harmful, judgemental, and controlling. We describe these shortcomings, and identify areas for improvement, including suggestions of future research directions.

Original language	English
Title of host publication	The First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security (NLPAICS’2024) Proceedings
Editors	Ruslan Mitkov, Saad Ezzini, Cengiz Acarturk, Tharindu Ranasinghe, Paul Rayson, Mo El-Haj, Ignatius Ezeani, Matthew Bradbury, Nouran Khallaf
Publisher	ACL Anthology
Pages	219-288
Number of pages	10
Publication status	Published - 30 Jul 2024
Event	First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security - Lancaster University, Lancaster, United Kingdom Duration: 29 Jul 2024 → 30 Jul 2024 https://nlpaics.com/

Conference

Conference	First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security
Abbreviated title	NPLAICS'2024
Country/Territory	United Kingdom
City	Lancaster
Period	29/07/24 → 30/07/24
Internet address	https://nlpaics.com/

Research Groups and Themes

Cyber Security

Access to Document

Full-text PDF (accepted manuscript)
This is the accepted author manuscript (AAM) of the article which has been made Open Access under the University of Bristol's Scholarly Works Policy. The final published version (Version of Record) can be found on the publisher's website. The copyright of any third-party content, such as images, remains with the copyright holder.
Accepted author manuscript, 134 KBLicence: CC BY

https://aclanthology.org/2024.nlpaics-1.0/Licence: CC BY

Handle.net

Persistent link

Cite this

Prosser, E., & Edwards, M. (2024). Not everything is online grooming: False risk finding in large language model assessments of human conversations. In R. Mitkov, S. Ezzini, C. Acarturk, T. Ranasinghe, P. Rayson, M. El-Haj, I. Ezeani, M. Bradbury, & N. Khallaf (Eds.), The First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security (NLPAICS’2024) Proceedings (pp. 219-288). ACL Anthology. https://aclanthology.org/2024.nlpaics-1.0/

Prosser, Ellie ; Edwards, Matthew . / Not everything is online grooming : False risk finding in large language model assessments of human conversations. The First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security (NLPAICS’2024) Proceedings. editor / Ruslan Mitkov ; Saad Ezzini ; Cengiz Acarturk ; Tharindu Ranasinghe ; Paul Rayson ; Mo El-Haj ; Ignatius Ezeani ; Matthew Bradbury ; Nouran Khallaf. ACL Anthology, 2024. pp. 219-288

@inproceedings{ca7fa33ccc794341b6d592d6dfa8ea1f,

title = "Not everything is online grooming: False risk finding in large language model assessments of human conversations",

abstract = "Large Language Models (LLMs) have rapidly been adopted by the general public, and as usage of these models becomes commonplace, they naturally will be used for increasingly human-centric tasks, including security advice and risk identifications for personal situations. It is imperative that systems used in such a manner are well-calibrated. In this paper, 6 popular LLMs were evaluated for their propensity towards false or over-cautious risk finding in online interactions between real people, with a focus on the risk of online grooming, the advice generated for such contexts, and the impact of prompt specificity. Through an analysis of 3840 generated answers, it was found that models could find online grooming in even the most harmless of interactions, and that the generated advice could be harmful, judgemental, and controlling. We describe these shortcomings, and identify areas for improvement, including suggestions of future research directions.",

author = "Ellie Prosser and Matthew Edwards",

year = "2024",

month = jul,

day = "30",

language = "English",

pages = "219--288",

editor = "Ruslan Mitkov and Saad Ezzini and Cengiz Acarturk and Tharindu Ranasinghe and Paul Rayson and Mo El-Haj and Ignatius Ezeani and Bradbury, \{Matthew \} and Nouran Khallaf",

booktitle = "The First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security (NLPAICS{\textquoteright}2024) Proceedings",

publisher = "ACL Anthology",

note = "First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security, NPLAICS'2024 ; Conference date: 29-07-2024 Through 30-07-2024",

url = "https://nlpaics.com/",

}

Prosser, E & Edwards, M 2024, Not everything is online grooming: False risk finding in large language model assessments of human conversations. in R Mitkov, S Ezzini, C Acarturk, T Ranasinghe, P Rayson, M El-Haj, I Ezeani, M Bradbury & N Khallaf (eds), The First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security (NLPAICS’2024) Proceedings. ACL Anthology, pp. 219-288, First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security, Lancaster, United Kingdom, 29/07/24. <https://aclanthology.org/2024.nlpaics-1.0/>

Not everything is online grooming: False risk finding in large language model assessments of human conversations. / Prosser, Ellie ; Edwards, Matthew .
The First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security (NLPAICS’2024) Proceedings. ed. / Ruslan Mitkov; Saad Ezzini; Cengiz Acarturk; Tharindu Ranasinghe; Paul Rayson; Mo El-Haj; Ignatius Ezeani; Matthew Bradbury; Nouran Khallaf. ACL Anthology, 2024. p. 219-288.

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - Not everything is online grooming

T2 - First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security

AU - Prosser, Ellie

AU - Edwards, Matthew

PY - 2024/7/30

Y1 - 2024/7/30

N2 - Large Language Models (LLMs) have rapidly been adopted by the general public, and as usage of these models becomes commonplace, they naturally will be used for increasingly human-centric tasks, including security advice and risk identifications for personal situations. It is imperative that systems used in such a manner are well-calibrated. In this paper, 6 popular LLMs were evaluated for their propensity towards false or over-cautious risk finding in online interactions between real people, with a focus on the risk of online grooming, the advice generated for such contexts, and the impact of prompt specificity. Through an analysis of 3840 generated answers, it was found that models could find online grooming in even the most harmless of interactions, and that the generated advice could be harmful, judgemental, and controlling. We describe these shortcomings, and identify areas for improvement, including suggestions of future research directions.

AB - Large Language Models (LLMs) have rapidly been adopted by the general public, and as usage of these models becomes commonplace, they naturally will be used for increasingly human-centric tasks, including security advice and risk identifications for personal situations. It is imperative that systems used in such a manner are well-calibrated. In this paper, 6 popular LLMs were evaluated for their propensity towards false or over-cautious risk finding in online interactions between real people, with a focus on the risk of online grooming, the advice generated for such contexts, and the impact of prompt specificity. Through an analysis of 3840 generated answers, it was found that models could find online grooming in even the most harmless of interactions, and that the generated advice could be harmful, judgemental, and controlling. We describe these shortcomings, and identify areas for improvement, including suggestions of future research directions.

UR - https://nlpaics-2024.org/

M3 - Conference Contribution (Conference Proceeding)

SP - 219

EP - 288

BT - The First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security (NLPAICS’2024) Proceedings

A2 - Mitkov, Ruslan

A2 - Ezzini, Saad

A2 - Acarturk, Cengiz

A2 - Ranasinghe, Tharindu

A2 - Rayson, Paul

A2 - El-Haj, Mo

A2 - Ezeani, Ignatius

A2 - Bradbury, Matthew

A2 - Khallaf, Nouran

PB - ACL Anthology

Y2 - 29 July 2024 through 30 July 2024

ER -

Prosser E , Edwards M. Not everything is online grooming: False risk finding in large language model assessments of human conversations. In Mitkov R, Ezzini S, Acarturk C, Ranasinghe T, Rayson P, El-Haj M, Ezeani I, Bradbury M, Khallaf N, editors, The First International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security (NLPAICS’2024) Proceedings. ACL Anthology. 2024. p. 219-288

Not everything is online grooming: False risk finding in large language model assessments of human conversations

Abstract

Conference

Research Groups and Themes

Access to Document

Handle.net

Other files and links

Fingerprint

Cite this