Tuner: A New Approach For 3D Semantic Segmentation Using Federated Architecture

3D computer vision (CV) applications have been gaining great popularity in various real-world applications, such as autonomous driving, environmental surveillance, and medical diagnostics. A significant obstacle in advancing 3D CV applications is the scarcity of large-scale, high-quality training datasets. Gathered 3D datasets have the issue of data heterogeneity since comprehensive capturing of 3D representations requires specialized tools and systematic approaches like scanning the object or scene from multiple viewpoints. This frequently results in datasets that are not only limited in size but also exhibit significant variation in attributes like resolutions, lighting conditions, and the distribution of labels. To address these challenges, this paper introduces Tuner, a novel strategy for training 3D CV models using datasets from various sources. Tuner leverages a federated learning (FL) framework, allowing the incorporation of extensive data samples. It achieves this through a unique and streamlined model structure linked to each participant’s model, designed to effectively address the issue of data heterogeneity. Our experiments show that Tuner can outperform other FL algorithms on various 3D segmentation tasks.