FedHyperM: modality-sensitive federated hypergraph learning for heterogeneous multimodal edge intelligence

Abstract Federated learning enables edge devices to collaboratively train models in edge–cloud environments without sharing raw sensor data. In practical edge deployments, however, devices often have access to different subsets of modalities due to hardware heterogeneity, varying sensing conditions, or temporary sensor failures. Such modality heterogeneity makes conventional federated aggregation unreliable, as parameters learned from different modality evidence may not be directly comparable. Meanwhile, this setting also raises privacy concerns, since image, depth, and signal data may reveal sensitive user or environmental information. To address these challenges, we propose FedHyperM, a privacy-preserving federated multimodal learning framework for edge devices. FedHyperM performs hypergraph-based multimodal prediction by treating modality-specific features as nodes and adaptively constructing hyperedges to capture high-order correlations among multimodal features. On the cloud server, FedHyperM further introduces modality-sensitive aggregation (MSA), which uses the minimum edit distance (MinED) between available modality names to weight uploaded hypergraph learning parameters during aggregation. Experiments on the UR-FALL dataset under different modality existence rate (MER) settings demonstrate that FedHyperM consistently outperforms representative federated multimodal baselines, including FedAvg, FedCor, FedProx, AutoFed, and FedInMM. Compared with the five baselines, FedHyperM improves average accuracy by 3.64% to 22.33% and average F1-score by 2.91% to 26.82%.