Abstract
Modern Reinforcement Learning (RL) algorithms are able to outperform humansin a wide variety of tasks. Multi-agent reinforcement learning (MARL) settingspresent additional challenges, and successful cooperation in mixed-motivegroups of agents depends on a delicate balancing act between individual andgroup objectives. Social conventions and norms, often inspired by humaninstitutions, are used as tools for striking this balance. In this paper, we examine a fundamental, well-studied social convention thatunderlies cooperation in both animal and human societies: dominancehierarchies. We adapt the ethological theory of dominance hierarchies to artificialagents, borrowing the established terminology and definitions with as fewamendments as possible. We demonstrate that populations of RL agents, operatingwithout explicit programming or intrinsic rewards, can invent, learn, enforce,and transmit a dominance hierarchy to new populations. The dominancehierarchies that emerge have a similar structure to those studied in chickens,mice, fish, and other species.