Vertical federated learning for transport mode detection using multi-modality data

• A semi-supervised vertical federated learning framework for transport mode detection is introduced. • The framework integrates multi-modality features without sharing raw IMU, GPS, or cellular data. • The proposed representation alignment and knowledge distillation enhance weak-modality learning. • The proposed framework supports reliable single-modality inference under missing or sparse data conditions. • Experiments on the SHL dataset show substantial gains over state-of-the-art baselines. Transport mode detection (TMD) is vital for intelligent transportation systems and urban computing. However, most existing approaches rely on data from a single modality, such as GPS trajectories or inertial measurement units. This limits their effectiveness in dynamic real-world scenarios. While some studies have improved transport mode detection by combining multiple modalities, heterogeneities in sampling frequency, spatial precision, signal coverage, and occasional missing-modality conditions, together with the requirement for strict temporal alignment, constrain the fused data to the resolution of the weakest modality and lead to substantial information loss. Furthermore, centralizing data from all modalities is often impractical, as the different data types are held by separate parties that may lack labeled data and may also be unwilling to share their raw data due to privacy or commercial concerns. To address these issues, we propose a semi-supervised vertical federated learning (VFL) framework for TMD that uses IMU, GPS, and mobile phone network data. In this framework, each party independently trains an attention-based autoencoder to encode local features. The hidden features are then sent to a central server for classification, and the classification loss is propagated back for local model updating. Given the high-frequency nature of IMU data, we have designed a dynamic sub-segment sampling strategy to adapt to transport mode detection tasks with different temporal resolutions. In addition, the framework adopts distillation and representation alignment to mitigate the impact of missing or weak modalities, and it supports both single-modality and multi-modality inference. We compared the proposed model with several state-of-the-art models, exploring the effectiveness of the different components of the VFL framework. The results demonstrate that our model consistently outperforms multiple baselines, and the proposed VFL framework effectively enhances local inference performance, which can be extended to various model architectures.