On the global feature importance for interpretable and trustworthy heat demand forecasting

The paper introduces the explainable artificial intelligence methodology to assess the global feature importance of the machine learning models used for heat demand forecasting in intelligent control of district heating systems, with motivation facilitate their interpretability and trustworthiness, hence addressing the challenges related to adherence to communal standards, customer satisfaction and liability risks. Methodology involves generation of global feature importance insights by using four different approaches, namely intrinsic (ante-hoc) interpretability of gradient boosting method and selected post-hoc methods, namely partial dependence, accumulated local effects and SHAP and qualitative analysis of those insights in context of expected behavior of district heating systems and comparative analysis. None of the selected methods assume feature permutation or perturbations which can introduce bias due to introduction of random unrealistic values of data instances. The accumulated local effects and SHAP have been found as most reliable methods for determining the feature importance, taking into account feature interactions and non-linearities. The accumulated local effects plots with transmitted energy across the range of ambient temperatures closely resemble the shape of the control curve, which is the evidence of accurate model, as well as suitability of explanation method. By providing the insights which align with the domain expertise, the discussion confirms the value of using explainable artificial intelligence stack as mandatory layer in assessing the performance of machine learning models, especially in high risk artificial intelligence systems, such as those whose use is anticipated in the district heating systems.