Abstract
Portfolio optimization tasks describe sequential decision problems in whichthe investor's wealth is distributed across a set of assets. Allocationconstraints are used to enforce minimal or maximal investments into particularsubsets of assets to control for objectives such as limiting the portfolio'sexposure to a certain sector due to environmental concerns. Although methodsfor constrained Reinforcement Learning (CRL) can optimize policies whileconsidering allocation constraints, it can be observed that these generalmethods yield suboptimal results. In this paper, we propose a novel approach tohandle allocation constraints based on a decomposition of the constraint actionspace into a set of unconstrained allocation problems. In particular, weexamine this approach for the case of two constraints. For example, an investormay wish to invest at least a certain percentage of the portfolio into greentechnologies while limiting the investment in the fossil energy sector. We showthat the action space of the task is equivalent to the decomposed action space,and introduce a new reinforcement learning (RL) approach CAOSD, which is builton top of the decomposition. The experimental evaluation on real-worldNasdaq-100 data demonstrates that our approach consistently outperformsstate-of-the-art CRL benchmarks for portfolio optimization.