Towards Cooperative MARL in Industrial Domains

Student thesis: Doctoral ThesisDoctor of Philosophy (PhD)

Abstract

This thesis investigates the application of Deep Multi-Agent Reinforcement Learning (DMARL) to problems within telecommunications and logistics. These sectors are exemplary of a common class of industrial systems that are comprised of large numbers of interconnected and interdependent assets. Traditionally, optimisation of these systems is achieved through the utilisation of heuristic and/or human expertise. However, due to their inherent complexity and dimensionality, these approaches are often significantly sub-optimal. Deep Reinforcement Learning (DRL) has proved successful in several real-world problems, but the aforementioned characteristic of these domains precludes its direct application. Alternatively, we can instantiate each asset as an agent and apply DMARL methodologies. This addresses the dimensionality but requires cooperative behaviours to be induced. Herein, we detail our efforts deriving novel cooperative DMARL solutions to representative problems in our target industrial domains. Our telecommunications work considers network maintenance planning, which requires a finite amount of maintenance resources to be assigned among network equipment. The logistics work explores the order-picking problem, in which human and robot workers must collaborate to collect and deliver items distributed around a commercial warehouse. In both cases, we develop and empirically validate novel DMARL algorithms within simulated environments and demonstrate improvements over relevant industrial heuristics. Furthermore, inspired by these domains, we anticipate language playing a key part in future industrial systems as a means to enable cooperation among diverse sets of agents. As such, we conduct investigations into the fundamental challenges of automatically establishing a common language from scratch for agents to effectively communicate. Collectively, this work serves as a first step towards exploiting the potential of cooperative DMARL for industrial applications and provides paths towards their realisation within real-world settings.
Date of Award20 Jun 2023
Original languageEnglish
Awarding Institution
  • University of Bristol
SupervisorRobert J Piechocki (Supervisor) & Raul Santos-Rodriguez (Supervisor)

Cite this

'