PhD defence of Paul Barde – Coordination in Cooperative Multi-Agent Learning

Abstract

Exploring the process by which autonomous agents coordinate represents a pivotal advancement toward emulating populations, which encompasses diverse applications in robotics, game theory, economics, and social sciences. Specifically, the mechanisms that allow cooperating learners to converge toward coherent team strategies remain poorly understood, and yet, understudied. The myriad of challenges that hinder concurrent learning -- environment non-stationarity, intricate credit assignments, exponential complexity, etc. -- has fragmented the focus of the literature such that the underlying mechanisms of successful multi-agent coordination remain to be pinpointed. This work puts forward and explores three key elements of coordination, namely, shared incentives, interactions, and the use of internal models. We scrutinize this take by inspecting how these components can enable coordination in several multi-agent learning paradigms that cover the well-established Multi-Agent Reinforcement Learning framework, its offline interaction-less variation, and the Architect-Builder Problem, a novel reward-less interactive learning setting. Our research derives fresh insights into fostering coordination and while the implications of multi-agent learning extend across various fields, we are particularly interested in planning solutions to societal and ecological challenges by simulating how populations would react to changes in their environment.