Context-aware data driven sensor data analysis: With application to H2S concentration prediction in urban drainage networks

• Data-driven methodology was successfully applied to real-world datasets from urban drainage networks in Switzerland, demonstrating its applicability to analyse big data from sensors. • Context-aware machine learning integrates system variables with contextual variables for enhanced prediction accuracy. • The Deep Neural Network (DNN) achieved superior performance, with R² values ranging from 0.906 to 0.927, outperforming other models like Random Forest Regressor and Gradient Boosting. • Using only system variables or incorporating contextual information leads to different data-driven model selection. This paper presents a context-aware data-driven approach for the analysis of big data from sensors. Different from conventional methods, this approach incorporates exogenous variables or contextual information that influences the dynamic behaviour of the monitored system. In the context of water distribution systems, for example, key system variables including water demand variations and pressure are significantly affected by factors like time of day, the day of the week, unusual events, seasonal variations and weather conditions. This contextual information creates dynamic relationships between water demand and pressure, which are critical for understanding system behaviour. Specifically, the context-aware method will use present and past observed values from sensors (which are normally time-series data recording the system’s dynamic behaviour), in addition to also including contextual information regarding the spatial context (e.g., the correlation between the values of different sensors) and temporal context (e.g., correlation between observed values and days of the week and time of the day). The method is applied to the prediction of Hydrogen Sulphide (H 2 S) concentration in a real-world urban drainage network, based on the analysis of big real-time data sets from different sensors. Although the datasets are variables with non-uniform time intervals, uncertainties, and faulty data, the context-aware method identifies the correlations among different datasets to predict the concentration of H 2 S with high accuracy (R 2 > 0.92; RMSE = 0.029). The method is also proven robust for a Deep Neural Networks approach.