Ethical, political and epistemic implications of machine learning (mis)information classification: Insights from an interdisciplinary collaboration between social and data scientists

Machine learning (ML) classification models are becoming increasingly popular for tackling the sheer volume and speed of online misinformation. A key consideration when building these models is the legitimacy, authoritativeness and objectivity of the sources of ‘truth’ employed for model training and testing. This has political, ethical and epistemic implications which are rarely addressed in technical papers. Despite (and due to) their reported high performance, ML-driven moderation systems have the potential to shape online public debate and create downstream negative impacts, highlighting the need for their responsible development. This article presents findings from a responsible innovation (RI) inflected collaboration between science and technology studies scholars and data scientists developing ML (mis)information classification models. Following an interactive co-ethnographic process, we identify a series of algorithmic contingencies –key moments during ML model development which could lead to different future outcomes, uncertainty and harmful effects. We conclude by offering practical recommendations that emerged from the RI collaboration relating to how to assess and mitigate the potential failures of ML tools for combating online misinformation.