MFCNet: Multimodal Feature Fusion Network for RGB-T Vehicle Density Estimation

The basic task of vehicle density estimation is to use image information to estimate the distribution and quantity of vehicles within it. However, many previous methods only use the optical information in red-green–blue (RGB) images, which makes it difficult to effectively identify potential vehicles under poor light, strong reflections, and bad weather, resulting in unsatisfactory density estimation performance. To address these problems, we consider introducing thermal images to provide a richer source of information for the vehicle density estimation task, and propose a multimodal feature fusion network (MFCNet) for accurate RGB-Thermal (RGB-T) vehicle density estimation. First, multimodal features are cross-integrated through the attention-guided multiscale feature fusion coordination module (MFFC) to compensate for the limitations of single modal features. Following this, the edge feature calibration module (EFC) is utilized to correct the spatial misalignment regions between modalities. Subsequently, the adaptive deep fusion module (ADFM) is applied to further refine the features on the global scale and improve the intermodality correlation. Finally, the features of different stages are fused step by step to obtain the final fused feature, which is fed into a simple regression header to generate a pixel-level vehicle density map. Experimental results show that the GAME2 and root mean square error of the proposed method are reduced to 5.21 and 3.54 on the DroneVehicle dataset, respectively. Compared with existing vehicle density estimation methods, MFCNet achieves competitive accuracy and can be applied to the vehicle density estimation task in unconstrained scenarios. Our codes will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/QLingX/MFCNet</uri>.