Abstract
Reinforcement Learning is a promising tool for learning complex policies evenin fast-moving and object-interactive domains where human teleoperation orhard-coded policies might fail. To effectively reflect this challengingcategory of tasks, we introduce a dynamic, interactive RL testbed based onrobot air hockey. By augmenting air hockey with a large family of tasks rangingfrom easy tasks like reaching, to challenging ones like pushing a block byhitting it with a puck, as well as goal-based and human-interactive tasks, ourtestbed allows a varied assessment of RL capabilities. The robot air hockeytestbed also supports sim-to-real transfer with three domains: two simulatorsof increasing fidelity and a real robot system. Using a dataset ofdemonstration data gathered through two teleoperation systems: a virtualizedcontrol environment, and human shadowing, we assess the testbed with behaviorcloning, offline RL, and RL from scratch.