DSAC: Distributional Soft Actor-Critic for Risk-Sensitive Reinforcement Learning
We present Distributional Soft Actor-Critic (DSAC), a distributional reinforcement learning (RL) algorithm that combines the strengths of distributional information of accumulated rewards and entropy-driven exploration from Soft Actor-Critic (SAC) algorithm. DSAC models the randomness in both action and rewards, surpassing baseline performances on various continuous control tasks. Unlike standard approaches that solely maximize expected rewards, we propose a unified framework for risk-sensitive learning, one that optimizes the risk-related objective while balancing entropy to encourage exploration. Extensive experiments demonstrate DSAC’s effectiveness in enhancing agent performances for both risk-neutral and risk-sensitive control tasks.
Authors
- Judy Chuyi Chen (ORCID: https://orcid.org/0009-0003-5875-1878)
- Qianchuan Zhao (ORCID: https://orcid.org/0000-0002-7952-5621)
- Xiaoteng Ma (ORCID: https://orcid.org/0000-0002-7250-6268)
- Zhengyuan Zhou (ORCID: https://orcid.org/0000-0002-0005-9411)
- Li Xia (ORCID: https://orcid.org/0000-0002-3193-7336)
- Jun Yang
Publication Details
- Journal
- Journal of Artificial Intelligence Research
- Published
- 2025-06-25
- DOI
- https://doi.org/10.1613/jair.1.17526
- Citations
- 15
- Primary Topic
- Reinforcement Learning in Robotics
- Type
- article
- Field-Weighted Citation Impact
- 19.85
Funders
- National Science Foundation
- National Natural Science Foundation of China
- Sun Yat-sen University
- Guangdong Province Key Laboratory of Computational Science
- National Key Research and Development Program of China
- Basic and Applied Basic Research Foundation of Guangdong Province
- Division of Electrical, Communications and Cyber Systems