Efficient image dehazing with synergic expert modulation

Abstract Recent developments in context modulation mechanisms have achieved significant improvements in performance, as well as better trade-offs between model accuracy and efficiency. These mechanisms operate on input through context modeling and then leverage these contexts to modulate projected input features. However, existing methods have limitations. First, they cannot adaptively learn the extracted hierarchical context or ignore the complementarity of the cross-scale context. Second, these methods do not adequately address the unique frequency characteristics of hazy images. In response, we propose a synergic expert modulation (SEM) mechanism to explicitly model context information. Specifically, the SEM consists primarily of two mixture of spatial experts (MSE) modules that handle features of different scales and one mixture of frequency experts (MFE) module that operates within the frequency domain. The MSE learns hierarchical features of various granularities in an adaptive manner, guided by multiple gating experts and a routing network. The MFE specializes in mining frequency contexts guided by multiple frequency experts and a routing network. At the micro level, each frequency expert operates in two stages: spectral filtering and spectral learning. The former performs mask filtering to enhance the weights of low-frequency components, and the latter performs Fourier amplitude and phase decoupled learning, thus promoting the removal of haze information and global context learning. Finally, the obtained contexts are integrated to modulate the projected feature, thereby significantly enhancing cross-domain feature synergies. The proposed network, referred to as the synergic expert modulation network, is constructed by inserting SEM-based building blocks into the U-Net architecture to increase efficiency. Extensive experiments demonstrate that our network achieves state-of-the-art performance on multiple datasets for the image dehazing task while incurring lower computational costs.