Abstract
Deep reinforcement learning (DRL) is playing an increasingly important rolein real-world applications. However, obtaining an optimally performing DRLagent for complex tasks, especially with sparse rewards, remains a significantchallenge. The training of a DRL agent can be often trapped in a bottleneckwithout further progress. In this paper, we propose RICE, an innovativerefining scheme for reinforcement learning that incorporates explanationmethods to break through the training bottlenecks. The high-level idea of RICEis to construct a new initial state distribution that combines both the defaultinitial states and critical states identified through explanation methods,thereby encouraging the agent to explore from the mixed initial states. Throughcareful design, we can theoretically guarantee that our refining scheme has atighter sub-optimality bound. We evaluate RICE in various popular RLenvironments and real-world applications. The results demonstrate that RICEsignificantly outperforms existing refining schemes in enhancing agentperformance.