Historical optimization decision-assisted adversarial inverse reinforcement learning for smart home energy management

Smart home energy management (SHEM) plays a vital role in reducing energy costs and enhancing the utilization of renewable energy. Among the various SHEM approaches, deep reinforcement learning (DRL)-based methods have attracted significant attention due to their ability to support real-time, online energy management without requiring forecasts of uncertain parameters such as dynamic electricity prices, outdoor temperature, renewable generation, and load demand. However, a major limitation of existing DRL-based SHEM methods is their reliance on manually designed reward functions, which adds complexity in terms of selecting appropriate reward structures and tuning parameters. To address this challenge, this paper proposes an energy management method for smart homes based on adversarial inverse reinforcement learning (AIRL). In the proposed method, historical optimal decisions are first obtained by solving a deterministic optimization problem using historical data and an optimization solver. These optimal decisions serve as expert demonstrations in the AIRL module, which is designed to train a discriminator that distinguishes expert demonstrations from the state-action transitions generated by a DRL agent policy. Simultaneously, the agent policy is updated to generate transitions that confuse the discriminator, thereby learning an effective control policy without the need for explicit reward engineering. The proposed method eliminates the need for predictive models of uncertain parameters and manually crafted reward functions. Simulation results using real-world data traces validate the effectiveness, data efficiency, and robustness of the proposed method.