Mobility Scooter Riding Behavior Stability Analysis Based on Multimodal Contrastive Learning

Mobility scooters are popular among people with limited mobility, providing convenient transportation in their communities and thus enhancing their life quality. However, due to various reasons, a high number of mobility scooter accidents have been reported. In this paper, we tackle its safety issues by performing riding behavior stability analysis using both video and motion sensor data. We design a transformer-based cross-modal encoder to generate the embeddings representing the riding stability and develop a binary classifier based on the embeddings to classify riding behaviors as stable or unstable. The advantage of this work is that the multimodal contrastive learning approach enables the neural network to understand the correlations across two modalities for the same events, and to distinguish the unstable riding behaviors from stable, so that incontext stability analysis becomes possible. We have conducted extensive experiments based on real-world mobility scooter riding data labeled by medical practitioners. The results have shown a high level of classification accuracy of our system, across different settings, in ablation study as well as comparative study.