PhD Forum Abstract: Hybrid MCTS for Heat Pump Control

Heat pump control under dynamic environmental conditions and time-varying electricity pricing poses significant challenges for energy efficiency and cost optimisation. Conventional methods such as PID controllers cannot adapt effectively to such non-stationary conditions, while model-free reinforcement learning approaches require extensive training and struggle with long-horizon constraints. This work investigates planning-based approaches, focusing on Monte Carlo Tree Search (MCTS) and its learning-augmented successor MuZero, as promising alternatives for adaptive and efficient heat pump control. A simplified immersion hot water heater environment is developed as a proof-of-concept testbed, capturing the core challenge of reaching a target temperature at a specified time while minimising energy use. Experimental evaluation shows that MCTS achieved target tracking with a 7% reduction in energy consumption compared to a bang-bang baseline, without requiring any training. In contrast, PPO, used as a baseline learning method, achieved higher efficiency (up to 19% savings) but only after extensive training and hyperparameter tuning. This work establishes planning-based control as a viable research direction for scalable, adaptive, and cost-effective energy management in built environments.