This is a Drill

: Developing Contextual Methods for Responsible AI with Industry Practitioners

Abstract

As artificial intelligence (AI) applications are increasingly being disseminated across a variety of industries, researchers, journalists and civil groups have questioned the ethics and values that these AI systems have been developed with or embody. Conversations about AI ethics have been sparked by a series of scandals involving AI from racially discriminatory facial recognition software to colonial practices in the data work birthing conferences, legislation and research agendas dedicated to algorithmic impacts on society.

A lot of this work so far has concentrated on producing generalised regulation and frameworks, which lack the contextual detail that is key to ethical decision making. Technical practitioners, who often lack training in ethical discussion, need tools to support them in applying the abstract frameworks to the context of the industries, user groups and organisations they are operating in.

This thesis explores using narrative methods such as design fiction and role play to create spaces for discussing ethics and values of data and AI systems in their application contexts. During the course of my PhD, I carried out the following investigations using a research through design approach:

- I investigated a data science team's responsible AI practices using design fiction memos
- I developed the method called Data Ethics Emergency Drill (DEED), which expands on the concept of ethical role play by crafting contextual scenarios with data science teams
- I evaluated the method with three industry data teams in five iterations
- I created and evaluated a toolbox to help disseminate the method and make it accessible to other teams

The main contributions of this thesis are twofold: firstly, the DEED method, as an example of how to design and carry out ethical discussions of data and AI in context and secondly an analysis of responsible AI practice in industry teams through the lens of speculative methods.

Date of Award	8 Apr 2025
Original language	English
Awarding Institution	University of Bristol
Sponsors	LV= General Insurance
Supervisor	Kenton O'Hara (Supervisor), Edwin D. Simpson (Supervisor) & Paul Marshall (Supervisor)