Performance and Stability Trade-offs of Attention Mechanisms in YOLOv8n-based Indoor Crowd Detection

This study investigates the effects of attention mechanisms—CBAM, SE, ECA, MHSA, NAM, and a baseline— on YOLOv8n for indoor crowd detection. Models were trained over 0-300 epochs, with key metrics (mAP@50, recall, precision, mAP@50-95) evaluated across phases. Results reveal: initial rapid improvement (0-50 epochs) across all models, with the baseline advancing fastest yet unstable; intermediate slowed growth (50-150 epochs), CBAM and SE stable, ECA and NAM with minimal fluctuations; final plateauing (150-300 epochs), baseline maintaining peak accuracy with volatility, MHSA declining slightly, suggesting overfitting. Conclusions: baseline excels in accuracy but requires stability enhancement; CBAM and SE balance precision and robustness optimally; ECA and NAM provide superior stability for critical applications; MHSA needs hyperparameter tuning. This work identifies key performance-stability trade-offs, offering novel insights for deploying attention-based detectors in surveillance.