MDI-YOLO a lightweight transformer-CNN-based multidimensional feature fusion model for small object detection

To address the problem of insufficient detection accuracy in drone aerial and remote sensing images due to factors such as small target size, background interference, and occlusion, we propose a lightweight small object detection model, MDI-YOLO, based on multi-dimensional feature fusion of Transformer and CNN. In the MDI-YOLO model, we utilize a channel grouping strategy that combines the advantages of Transformer and CNN, proposing the C2f-Multi-Head Self-Attention-Convolutional Gated Linear Unit-Convolutional Neural Network (C2f-MCC) structure to improve the ability of the You Only Look Once v8 (YOLOv8) backbone network in extracting global features. Additionally, we propose Directional Fusion Attention (DFA), an attention mechanism that focuses on spatial and channel features across different dimensions, enhancing the model's feature representation ability. Finally, we design the Inner-Shape-Intersection over Union (Inner-Shape-IoU) loss function, which thoroughly evaluates the bounding boxes by considering their shape, scale, and position, thereby improving the model's precision in locating objects. The findings from the experiments reveal that the proposed detection model improves [email protected] by 4% and [email protected]:0.95 by 2.5% on the VisDrone2019 dataset with nearly the number of parameters remaining unchanged. On the DOTAv1.0 dataset, [email protected] is increased by 3.3%, and [email protected]:0.95 by 2.8%. The improved detection model not only enhances recognition accuracy but also maintains lightweight characteristics, making it suitable for drone aerial and remote sensing image detection, and strengthening the network's robustness and generalization ability.