Contrastive Policy Explanation for Lay Users

: A Study on AI Planning and Reinforcement Learning

Abstract

Recent studies have highlighted the need for enhancing the explainability of autonomous agents in decision-making problems. In the field of explainable AI, lay users are important but often under-represented stakeholders. The policy of an agent, whether computed by AI planning or learnt through reinforcement learning, is crucial for users to comprehend its decisions and behaviour. In this thesis, we explore contrastive explanations for AI planning and reinforcement learning policies, focussing on improving comprehension and trust for lay users.
We begin this thesis by introducing the policy explanation problem and the importance of contrastive explanations. We provide a technical background on AI planning and reinforcement learning, highlighting similarities in their formulation within the Markov Decision Process. We then develop the contrastive and policy explanation approaches targeted at AI planning and reinforcement learning. The core contributions are two-fold: First, we conduct a user study on contrastive explanations for multi-effector temporal planning in a smart home application. A novel custom domain-dependent planner is developed to handle non-stationary costs, while contrastive explanations are provided to lay users. Results demonstrate that explanations enhance user satisfaction and perceived system helpfulness. Second, we propose a framework for generating contrastive visual explanations of reinforcement learning policies by comparing behaviour under actual and hypothetical reward functions. This approach provides global policy-level explanations through trajectory visualisation and local explanations of action-value functions. Additionally, we contribute a simulator based on the olfactory search problem as an extension for discussing policy learning and interpretability in partially observable environments.
This thesis aims to address explainable AI through contrastive explanations from a user-centric perspective. Key innovations include formalising contrastive questions as planning model restrictions within the smart home domain and visualising policy contrastively through critical states. Future directions encompass developing hypothesis-driven decision support, addressing uncertainty in explanations, and adapting to broader classes of learning systems.

Date of Award	10 Dec 2024
Original language	English
Awarding Institution	University of Bristol
Supervisor	Weiru Liu (Supervisor) & Kevin McAreavey (Supervisor)