Human-in-the-Loop Reinforcement Learning for Energy-Efficient Smart Home Recommendations

Abstract Smart homes generate a continuous stream of sensor, occupancy and appliance data that is well suited to data-driven energy optimisation, yet the majority of deployed home energy management systems still rely on static rule-based control and manual scheduling. Such systems adapt poorly to evolving occupant routines, seasonal variation and conflicting preferences among household members, which results either in avoidable energy waste or in comfort losses. This paper presents a recommender system whose policy is learned entirely from occupant feedback: a Proximal Policy Optimization (PPO) agent is trained online from accept and reject signals, with no pre-collected preference corpus, and is paired with a deterministic rule engine that supplies a behavioural prior during the cold-start period. The system is embedded in a complete Internet of Things (IoT) architecture comprising a layered application server, a relational data model and an Android client. The agent observes a nine-dimensional state vector combining temporal, environmental, device-state and rule-baseline features, and emits a device action together with a confidence score and a natural-language justification. Recommendations are validated against a priority-ordered safety rule set and reconciled across occupants through a combined weighted-average and majority-voting strategy before being delivered to the user. The nine PPO hyperparameters were tuned by Bayesian optimisation using a Tree-structured Parzen Estimator over a lightweight surrogate environment. The system was evaluated on a synthetic but behaviourally grounded one-year dataset generated at a thirty-minute resolution for a three-occupant, six-room dwelling, under three user-feedback regimes and against a rule-based baseline and a Deep Q-Network (DQN) variant. Under full acceptance the system reduced annual consumption by 14.2% on average, reaching 27.8% in the most favourable month, while under a realistic mixed-feedback regime it delivered 6.4% average annual savings with an agent confidence of approximately 74%. PPO outperformed the DQN variant in every month of the simulated year, with the gap widening in periods dominated by continuous set-point control. The results indicate that policy-gradient reinforcement learning driven by human feedback and anchored by a rule-based prior is an effective basis for adaptive, comfort-preserving energy recommendations in residential settings.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-12
DOI
https://doi.org/10.21203/rs.3.rs-11168153/v1
Primary Topic
Smart Grid Energy Management
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Human-in-the-Loop Reinforcement Learning for Energy-Efficient Smart Home Recommendations

Youcef Zafoune, Youcef Guenfaf, Douniazed Daouadji, Amdjed Baha-eddine Chaabane et al.
Research Square
Smart Grid Energy Management
preprint

Human-in-the-Loop Reinforcement Learning for Energy-Efficient Smart Home Recommendations

Youcef Zafoune, Youcef Guenfaf, Douniazed Daouadji, Amdjed Baha-eddine Chaabane, Abdnour Benmeridja, Dounia Fatma Zohra Rehab, Imad Yacine Karoum
preprint en

Abstract

Abstract Smart homes generate a continuous stream of sensor, occupancy and appliance data that is well suited to data-driven energy optimisation, yet the majority of deployed home energy management systems still rely on static rule-based control and manual scheduling. Such systems adapt poorly to evolving occupant routines, seasonal variation and conflicting preferences among household members, which results either in avoidable energy waste or in comfort losses. This paper presents a recommender system whose policy is learned entirely from occupant feedback: a Proximal Policy Optimization (PPO) agent is trained online from accept and reject signals, with no pre-collected preference corpus, and is paired with a deterministic rule engine that supplies a behavioural prior during the cold-start period. The system is embedded in a complete Internet of Things (IoT) architecture comprising a layered application server, a relational data model and an Android client. The agent observes a nine-dimensional state vector combining temporal, environmental, device-state and rule-baseline features, and emits a device action together with a confidence score and a natural-language justification. Recommendations are validated against a priority-ordered safety rule set and reconciled across occupants through a combined weighted-average and majority-voting strategy before being delivered to the user. The nine PPO hyperparameters were tuned by Bayesian optimisation using a Tree-structured Parzen Estimator over a lightweight surrogate environment. The system was evaluated on a synthetic but behaviourally grounded one-year dataset generated at a thirty-minute resolution for a three-occupant, six-room dwelling, under three user-feedback regimes and against a rule-based baseline and a Deep Q-Network (DQN) variant. Under full acceptance the system reduced annual consumption by 14.2% on average, reaching 27.8% in the most favourable month, while under a realistic mixed-feedback regime it delivered 6.4% average annual savings with an agent confidence of approximately 74%. PPO outperformed the DQN variant in every month of the simulated year, with the gap widening in periods dominated by continuous set-point control. The results indicate that policy-gradient reinforcement learning driven by human feedback and anchored by a rule-based prior is an effective basis for adaptive, comfort-preserving energy recommendations in residential settings.

Research Square
University of Sciences and Technology Houari Boumediene (DZ)
Affordable and clean energy
Smart Grid Energy Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon