EDFFF: Ensemble Distillation Framework based on Fourier Features

Domain generalization enables the model to extrapolate its learning to an unfamiliar test distribution, showcasing its ability to adapt and apply knowledge across diverse domains. Deep convolutional neural networks have significantly improved the generalization ability of models due to their enhanced fitting capabilities and deep feature extraction abilities. Current domain generalization models are still susceptible to irrelevant factors and do not effectively address the correlation shift caused by these factors or the diversity shift caused by style differences. To tackle these issues, we propose an Ensemble Distillation Framework based on Fourier Features (EDFFF). This framework utilizes features learned by teacher models through distillation learning to train student models. Specifically, phase reconstruction is employed to retain main body information while amplitude mixing is used to mitigate style differences between domains during the data manipulation. Furthermore, ensemble teacher models learn from images augmented by different augmentation strategies, emphasizing the heterogeneity of features learned by the student model from these ensemble teacher models. Extensive experiments demonstrate that EDFFF achieves superior accuracy on both correlation shift datasets and diversity shift datasets.