World-model-inspired latent action priors for home energy optimization

The integration of renewable energy sources into residential microgrids introduces operational challenges associated with intermittency, electricity-price volatility, and the progressive degradation of battery storage systems. Reinforcement learning has emerged as a promising approach for battery scheduling, yet many existing methods remain vulnerable to actions that fall outside the support of the training data and often generalize poorly across operating regimes. This work proposes an offline reinforcement learning framework that combines a conditional variational autoencoder with latent action priors in order to constrain policy optimization to a structured latent action space learned from expert demonstrations generated by entropy-regularized and deterministic actor-critic controllers. The proposed methodology consists of three stages: first, learning a latent representation of admissible battery actions from expert trajectories through a conditional variational autoencoder; second, extracting an offline policy within the learned latent manifold through implicit value-based policy improvement; and third, deploying an out-of-distribution detection mechanism that gradually blends the learned controller with a conservative fallback policy under distributional uncertainty. Experiments on the GEFCom2014 benchmark, the StoreNet House 4 scenario, and the CityLearn environment show that the proposed method achieves the highest daily net benefit among value-based and offline controllers, with improvements of up to 61.9% over deep Q-learning and 7.9% over standard model predictive control on GEFCom2014. In addition, the method converges substantially faster than competing baselines, requiring up to nine times less training time in CityLearn. At the same time, the results show that the approach does not uniformly dominate all references: in the policy-based comparison on CityLearn, deep deterministic policy gradients achieve a lower monetary cost, and in standard continuous-control benchmarks, proximal policy optimization and twin delayed deterministic policy gradients outperform the proposed method on specific metrics in selected environments. Ablation studies further show that expert-dataset diversity and rare-event coverage are essential for policy quality, while posterior-collapse diagnostics confirm that the learned latent representation remains informative throughout training. Overall, the proposed framework constitutes a competitive and computationally efficient alternative for offline energy management, particularly in settings where safe exploration, fast convergence, and degradation-aware control are more critical than unconstrained asymptotic performance.

Authors

Institutions

Publication Details

Journal
Computers & Electrical Engineering
Published
2026-11-15
DOI
https://doi.org/10.1016/j.compeleceng.2026.111652
Primary Topic
Smart Grid Energy Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

World-model-inspired latent action priors for home energy optimization

Fernando De la Prieta, Albano Carrera, Guillermo Hernández, Sebastián López Flórez
Computers & Electrical Engineering
Smart Grid Energy Management
article

World-model-inspired latent action priors for home energy optimization

Fernando De la Prieta, Albano Carrera, Guillermo Hernández, Sebastián López Flórez
article en

Abstract

The integration of renewable energy sources into residential microgrids introduces operational challenges associated with intermittency, electricity-price volatility, and the progressive degradation of battery storage systems. Reinforcement learning has emerged as a promising approach for battery scheduling, yet many existing methods remain vulnerable to actions that fall outside the support of the training data and often generalize poorly across operating regimes. This work proposes an offline reinforcement learning framework that combines a conditional variational autoencoder with latent action priors in order to constrain policy optimization to a structured latent action space learned from expert demonstrations generated by entropy-regularized and deterministic actor-critic controllers. The proposed methodology consists of three stages: first, learning a latent representation of admissible battery actions from expert trajectories through a conditional variational autoencoder; second, extracting an offline policy within the learned latent manifold through implicit value-based policy improvement; and third, deploying an out-of-distribution detection mechanism that gradually blends the learned controller with a conservative fallback policy under distributional uncertainty. Experiments on the GEFCom2014 benchmark, the StoreNet House 4 scenario, and the CityLearn environment show that the proposed method achieves the highest daily net benefit among value-based and offline controllers, with improvements of up to 61.9% over deep Q-learning and 7.9% over standard model predictive control on GEFCom2014. In addition, the method converges substantially faster than competing baselines, requiring up to nine times less training time in CityLearn. At the same time, the results show that the approach does not uniformly dominate all references: in the policy-based comparison on CityLearn, deep deterministic policy gradients achieve a lower monetary cost, and in standard continuous-control benchmarks, proximal policy optimization and twin delayed deterministic policy gradients outperform the proposed method on specific metrics in selected environments. Ablation studies further show that expert-dataset diversity and rare-event coverage are essential for policy quality, while posterior-collapse diagnostics confirm that the learned latent representation remains informative throughout training. Overall, the proposed framework constitutes a competitive and computationally efficient alternative for offline energy management, particularly in settings where safe exploration, fast convergence, and degradation-aware control are more critical than unconstrained asymptotic performance.

Computers & Electrical EngineeringVol. 141
Universidad de Salamanca (ES), Technological University of Pereira (CO)
Smart Grid Energy Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon