Abstract
This review addresses the problem of learning abstract representations of themeasurement data in the context of Deep Reinforcement Learning (DRL). While thedata are often ambiguous, high-dimensional, and complex to interpret, manydynamical systems can be effectively described by a low-dimensional set ofstate variables. Discovering these state variables from the data is a crucialaspect for (i) improving the data efficiency, robustness, and generalization ofDRL methods, (ii) tackling the curse of dimensionality, and (iii) bringinginterpretability and insights into black-box DRL. This review provides acomprehensive and complete overview of unsupervised representation learning inDRL by describing the main Deep Learning tools used for learningrepresentations of the world, providing a systematic view of the method andprinciples, summarizing applications, benchmarks and evaluation strategies, anddiscussing open challenges and future directions.