Abstract
Recent work has shown that deep neural networks are capable of approximatingboth value functions and policies in reinforcement learning domains featuringcontinuous state and action spaces. However, to the best of our knowledge noprevious work has succeeded at using deep neural networks in structured(parameterized) continuous action spaces. To fill this gap, this paper focuseson learning within the domain of simulated RoboCup soccer, which features asmall set of discrete action types, each of which is parameterized withcontinuous variables. The best learned agent can score goals more reliably thanthe 2012 RoboCup champion agent. As such, this paper represents a successfulextension of deep reinforcement learning to the class of parameterized actionspace MDPs.