Modelling local electricity consumption by incorporating data of social media using natural language processing

Understanding and estimating power demand and energy consumption for customer billing, grid planning, incorporating flexibility in the grid and enabling ecological transition need the most accurate forecasts at local level (e.g. at the level of town, city, regions etc.). Most models used by ENEDIS (the French DSO) takes weather information and calendar data (i.e. weekends, public holidays and season) as inputs. However, significant errors are observed on certain days, which can be attributed to irregular human activities. This research contributes to build a better understanding of these events. We have been able to develop a process to optimise the correlation between the energy consumption and events related variables created using NLP. These variables are created using unstructured textual data obtained from social media using NLP. Not only this method could be used to enhance forecasting accuracy by “neutralizing” the special days and stopping it to interfere with other variable estimation, but it could also help us monitor the overall impact of some events on electricity consumption. The results show that certain events have significant impact on local energy consumption and incorporating these event variables in baseline model helps in reducing the prediction error up to 7%.