Abstract
Reinforcement Learning (RL), bolstered by the expressive capabilities of DeepNeural Networks (DNNs) for function approximation, has demonstratedconsiderable success in numerous applications. However, its practicality inaddressing various real-world scenarios, characterized by diverse andunpredictable dynamics, noisy signals, and large state and action spaces,remains limited. This limitation stems from poor data efficiency, limitedgeneralization capabilities, a lack of safety guarantees, and the absence ofinterpretability, among other factors. To overcome these challenges and improveperformance across these crucial metrics, one promising avenue is toincorporate additional structural information about the problem into the RLlearning process. Various sub-fields of RL have proposed methods forincorporating such inductive biases. We amalgamate these diverse methodologiesunder a unified framework, shedding light on the role of structure in thelearning problem, and classify these methods into distinct patterns ofincorporating structure. By leveraging this comprehensive framework, we providevaluable insights into the challenges of structured RL and lay the groundworkfor a design pattern perspective on RL research. This novel perspective pavesthe way for future advancements and aids in developing more effective andefficient RL algorithms that can potentially handle real-world scenariosbetter.