FedCRAC: Improving Federated Classification Performance on Long-Tailed Data via Classifier Representation Adjustment and Calibration

Federated learning has been a popular distributed training paradigm that enables to train a shared model with data privacy protection. However, non-Independent Identically Distribution and long-tailed data distribution characteristics across mobile devices results in evident performance degradation, especially for classification tasks. Although plenty of research studies devote to alleviating classification performance degradation caused by highly-skewed data distribution, they still cannot improve the distinguishability of model representation on hard-to-learn tail classes, and face obvious divergence of local classifiers in FL setting. To this end, we propose Federated Classifier Representation Adjustment and Calibration to improve the representation distinguishability of tail classes and achieve inter-client representation alignment with acceptable resource consumption on attaching operations. We first design a Class Similarity-Aware Margin matrix to enlarge class representation discrepancy and improve local classifier discriminability on tail classes during client-side local training process. To mitigate the divergence of local classifiers across clients, we further propose the Self Distillation Classifier Calibration to achieve the aggregated global classifier calibration with the assistance of generated pseudo representation samples via self-distillation manner. We conduct various experiments under wide-range long-tailed and heterogeneous data settings. Experimental results show that FedCRAC outperforms state-of-the-art methods in terms of accuracy and resource consumption.