Collaborative Multi-Agent Reinforcement Learning Model for Portfolio Management

The portfolio management problem refers to dynamically making investment decisions and allocating funds across multiple assets with a fixed principal, in order to maximize investment returns while effectively controlling risk. In recent years, deep learning and reinforcement learning have been introduced to better predict price trends and manage risks due to their powerful learning and generalization capabilities. However, how to solve the problem of cooperation and information sharing between assets to obtain good investment decisions and fund allocation at the same time, remains a challenging issue. In this paper, we propose a cooperative multi-agent reinforcement learning model for portfolio management, where each agent is responsible for making investment decisions for an asset, and funds are allocated based on the decisions of all agents. By designing a reward function that considers both individual agent's profit and global profit, we aim to optimize investment strategies and fund allocation, pursuing the maximum overall return on the system. The experimental results on futures indicate that the proposed model demonstrates superior performance. The study also analyzes the impact of the ratio between individual agent's profit and other agents' profits in the reward function, and proposes a dynamic adjustment strategy to optimize model performance. Additionally, the impact of the discreteness of the action space is studied in the paper.