FD-RLPO: Feature Domain-based Reinforcement Learning Framework for Portfolio Optimization

Portfolio optimization is a critical issue in finance. In the past decade, reinforcement learning has advanced con-siderably in this area owing to its outstanding ability to solve complex sequential decision-making problems. Nevertheless, ex-isting methods still have great room for improvement in fully considering the dynamic fluctuations of intra-domain and inter-domain features in financial markets. To address this issue, we propose FD-RLPO, a Feature Domain-based Reinforcement Learning Framework for Portfolio Optimization, which inte-grates Prediction and Relation Modules while leveraging rein-forcement learning agents for adaptive strategy adjustment. In particular, the Prediction Module utilizes an inversion mechanism within a Transformer architecture to effectively extract sequential fluctuations in intra-domain features. Meanwhile, the Relation Module employs a self-supervised approach to capture inter-domain relational distribution patterns, mitigating the impact of data noise on reconstructed asset relationships. Ultimately, the Decision Module constructs a comprehensive representation of the market feature domain by synthesizing the outputs of the preceding modules and leveraging the Actor-Critic Network to achieve portfolio optimization. The simultaneous perception of both intra-domain and inter-domain features enables FD-RLPO to dynamically adjust to the volatility of financial markets and increase returns. Experiments on the CSI-300 and NASDAQ-lOO datasets demonstrate that FD-RLPO outperforms previous state-of-the-art methods in key metrics, including the Annualized Rate of Return (ARR) and Annualized Sharpe Ratio (ASR). Ablation studies further validate the effectiveness of each component.