Abstract
Offline reinforcement learning has emerged as a promising technology byenhancing its practicality through the use of pre-collected large datasets.Despite its practical benefits, most algorithm development research in offlinereinforcement learning still relies on game tasks with synthetic datasets. Toaddress such limitations, this paper provides autonomous driving datasets andbenchmarks for offline reinforcement learning research. We provide 19 datasets,including real-world human driver's datasets, and seven popular offlinereinforcement learning algorithms in three realistic driving scenarios. We alsoprovide a unified decision-making process model that can operate effectivelyacross different scenarios, serving as a reference framework in algorithmdesign. Our research lays the groundwork for further collaborations in thecommunity to explore practical aspects of existing reinforcement learningmethods. Dataset and codes can be found in https://sites.google.com/view/ad4rl.