Overcoming Relative Overgeneralisation for Cooperative Multi-Agent Reinforcement Learning

15^th June 2023, 13:00 Meeting Room 101 Ashton Building 1st Floor
Bei Peng
CS, University of LIverpool

Abstract

Many real-world problems involve multiple agents acting and interacting in a shared environment to achieve some common goal, which can be naturally modelled as cooperative multi-agent systems. Multi-agent reinforcement learning (MARL) can be used to learn optimal decision making in many of these complex uncertain and dynamic multi-agent systems. In this talk, I first introduce the MARL paradigm and overview some of the key challenges in developing MARL algorithms that can efficiently learn decentralised policies for a group of agents. I then focus on discussing one of our recent works that addresses the open problem of relative overgeneralisation, which can prevent agents from solving cooperative tasks requiring significant coordination. In this work, we propose a new MARL approach called Universal Value Exploration (UneVEn) to better overcome relative overgeneralisation. To learn a target task exhibiting relative overgeneralisation, UneVEn learns a set of related tasks simultaneously and uses the policies of already solved related tasks to improve the joint exploration process of all agents in the target task. Our empirical results show that UneVEn can solve challenging cooperative tasks where other state-of-the-art MARL methods fail.