Explainability fact sheets: A framework for systematic assessment of explainable approaches

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

Abstract

Explanations in Machine Learning come in many forms, but a consensus regarding their desired properties is yet to emerge. In this paper we introduce a taxonomy and a set of descriptors that can be used to characterise and systematically assess explainable systems along five key dimensions: functional, operational, usability, safety and validation. In order to design a comprehensive and representative taxonomy and associated descriptors we surveyed the eXplainable Artificial Intelligence literature, extracting the criteria and desiderata that other authors have proposed or implicitly used in their research. The survey includes papers introducing new explainability algorithms to see what criteria are used to guide their development and how these algorithms are evaluated, as well as papers proposing such criteria from both computer science and social science perspectives. This novel framework allows to systematically compare and contrast explainability approaches, not just to better understand their capabilities but also to identify discrepancies between their theoretical qualities and properties of their implementations. We developed an operationalisation of the framework in the form of Explainability Fact Sheets, which enable researchers and practitioners alike to quickly grasp capabilities and limitations of a particular explainable method. When used as a Work Sheet, our taxonomy can guide the development of new explainability approaches by aiding in their critical evaluation along the five proposed dimensions.

Original languageEnglish
Title of host publicationFAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
PublisherAssociation for Computing Machinery (ACM)
Pages56-67
Number of pages12
ISBN (Electronic)9781450369367
DOIs
Publication statusPublished - 27 Jan 2020
Event3rd ACM Conference on Fairness, Accountability, and Transparency, FAT* 2020 - Barcelona, Spain
Duration: 27 Jan 202030 Jan 2020

Publication series

NameFAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

Conference

Conference3rd ACM Conference on Fairness, Accountability, and Transparency, FAT* 2020
CountrySpain
CityBarcelona
Period27/01/2030/01/20

Structured keywords

  • Digital Health

Keywords

  • AI
  • Desiderata
  • Explainability
  • Fact Sheet
  • Interpretability
  • ML
  • Taxonomy
  • Transparency
  • Work Sheet

Fingerprint Dive into the research topics of 'Explainability fact sheets: A framework for systematic assessment of explainable approaches'. Together they form a unique fingerprint.

  • Projects

    SPHERE2

    Craddock, I. J.

    1/10/1830/09/21

    Project: Research, Parent

    Cite this

    Sokol, K., & Flach, P. (2020). Explainability fact sheets: A framework for systematic assessment of explainable approaches. In FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 56-67). (FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency). Association for Computing Machinery (ACM). https://doi.org/10.1145/3351095.3372870