Adaptive Confidence-Weighted LLM Infusion for Financial Reinforcement Learning

We propose an enhanced version of FinRL-DeepSeek for the "FinRL Contest 2025 Task 1", a framework that integrates Large Language Model (LLM) signals into Reinforcement Learning (RL) for stock trading. The original approach employed fixed LLM-based perturbations (e. g., on actions or trajectory returns), overlooking each signal’s uncertainty and thereby risking the amplification of noise.To address these limitations, we introduce Adaptive Confidence-Weighted LLM Infusion for two types of infusions. First, trading-actions scores, which aligns each LLM-derived factor around a neutral baseline based on the LLM’s self-reported confidence. High-confidence signals preserve their strength, while low-confidence signals are dampened. Second, controlling the exploration-exploitation trade-off by entropy regularisation.This dynamic adaptation is implemented in both PPO and CVaR-PPO, enabling more robust, context-aware policy updates. Empirical backtests on real market data demonstrate that our method improves stability and risk-adjusted returns, underscoring the value of explicitly modeling confidence in LLM-augmented RL trading strategies.<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1, 2</sup>