A multimodal approach combining wearable sensors and point of view videos to evaluate stress in urban cycling

Sustainable and active urban mobility are critical for addressing the pressing challenges of climate change, urban congestion, and public health in cities. External conditions caused by the spatiotemporal nature of urban environments pose stressful factors hindering the adoption of active modes of transportation, and are mainly evaluated through qualitative feedback, or labour-intensive workflows processing multi-modal information. To investigate stress caused by static and dynamic elements of the natural and built environment, experienced while cycling in urban areas, we propose a multi-modal, scalable methodology that combines wearable sensors and cyclist’s POV videos. By spatially aggregating EDA based stress responses, captured by wearable sensors, we are able to identify spatial clusters of stress, which are used as label class to alleviate issues concerning spatial autocorrelation. Combining this with CV models to automatically process synchronously recorded cyclist’s point of view videos, captured by low-cost camera devices mounted on the bikes (chest of the cyclist or handlebar of the bicycle), we derive spatiotemporal elements of the natural and built environment and link them to measured stress. We show that the confidence threshold for semantic segmentation can be used to handle obstructed areas, caused by improper camera positioning, and that CV-based urban features are sufficient to predict stressful urban areas. Our model demonstrates strong performance in classifying spatial clusters of stress, achieving an accuracy of 0.745 and a recall of 0.615 when distinguishing between hot spots (areas with concentrated stress measurements) and cold spots (regions with clustered non-stress measurements).