Cross-Modal Vehicle Detection, Tracking, and Counting from Hyperspectral to RGB Imagery Using Spatial–Spectral Fusion and Knowledge Distillation

Accurate vehicle detection, tracking, and counting are central to intelligent transportation systems. Most existing approaches rely on RGB imagery and often suffer from performance degradation when vehicles are partially blocked, lighting conditions change, backgrounds are complex, or traffic is heavy. Hyperspectral imaging provides additional spectral information that can help distinguish vehicles more effectively, although its use in end-to-end traffic analysis and its transferability to conventional RGB imagery remain relatively unexplored. In this study, we propose a unified vehicle-analysis framework combining Edge-Guided Dimensionality Reduction (EGDR), a Spatial Context Enhancement Module (SCEM), and teacher-guided pseudo-label knowledge distillation. EGDR preserves spectral, edge, and multi-scale spatial information in a compact three-channel representation, while SCEM enhances contextual feature extraction. An EGDR+SCEM YOLOv11m detector is used as a teacher, from which a lightweight YOLOv11s student is trained through knowledge distillation. The student detector is then combined with tracking and trajectory-based counting to form a complete vehicle-analysis pipeline. On the HSI-Drive dataset, the distilled student improved mAP50 from 0.740 ± 0.020 to 0.780 ± 0.016 and mAP50:95 from 0.508 ± 0.018 to 0.532 ± 0.006 across six training seeds while retaining the same 9.41M-parameter, approximately 21-GFLOP student architecture. We further evaluated direct cross-modal generalization by testing the hyperspectral-trained model on RGB UA-DETRAC video sequences without RGB-domain fine-tuning or adaptation. Distillation improved RGB mAP50:95 from 0.5377 to 0.5809 and increased trajectory-based counting accuracy from 84.72% to 87.50%, while reducing average counting error from 15.28% to 12.50%. Thus, supervision derived from hyperspectral-trained vehicle representations can improve lightweight detector generalization to RGB traffic imagery and support detection, tracking, and counting across sensing modalities.

Authors

Institutions

Publication Details

Journal
Sensors
Published
2026-11-19
DOI
https://doi.org/10.3390/s26227304
Primary Topic
Advanced Neural Network Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Cross-Modal Vehicle Detection, Tracking, and Counting from Hyperspectral to RGB Imagery Using Spatial–Spectral Fusion and Knowledge Distillation

Himanshu Upadhyay, Viktoria Meskova, Malek Adjouadi, Jayesh Soni et al.
Sensors
Advanced Neural Network Applications
article

Cross-Modal Vehicle Detection, Tracking, and Counting from Hyperspectral to RGB Imagery Using Spatial–Spectral Fusion and Knowledge Distillation

Himanshu Upadhyay, Viktoria Meskova, Malek Adjouadi, Jayesh Soni, Naphtali David Rishe
article en

Abstract

Accurate vehicle detection, tracking, and counting are central to intelligent transportation systems. Most existing approaches rely on RGB imagery and often suffer from performance degradation when vehicles are partially blocked, lighting conditions change, backgrounds are complex, or traffic is heavy. Hyperspectral imaging provides additional spectral information that can help distinguish vehicles more effectively, although its use in end-to-end traffic analysis and its transferability to conventional RGB imagery remain relatively unexplored. In this study, we propose a unified vehicle-analysis framework combining Edge-Guided Dimensionality Reduction (EGDR), a Spatial Context Enhancement Module (SCEM), and teacher-guided pseudo-label knowledge distillation. EGDR preserves spectral, edge, and multi-scale spatial information in a compact three-channel representation, while SCEM enhances contextual feature extraction. An EGDR+SCEM YOLOv11m detector is used as a teacher, from which a lightweight YOLOv11s student is trained through knowledge distillation. The student detector is then combined with tracking and trajectory-based counting to form a complete vehicle-analysis pipeline. On the HSI-Drive dataset, the distilled student improved mAP50 from 0.740 ± 0.020 to 0.780 ± 0.016 and mAP50:95 from 0.508 ± 0.018 to 0.532 ± 0.006 across six training seeds while retaining the same 9.41M-parameter, approximately 21-GFLOP student architecture. We further evaluated direct cross-modal generalization by testing the hyperspectral-trained model on RGB UA-DETRAC video sequences without RGB-domain fine-tuning or adaptation. Distillation improved RGB mAP50:95 from 0.5377 to 0.5809 and increased trajectory-based counting accuracy from 84.72% to 87.50%, while reducing average counting error from 15.28% to 12.50%. Thus, supervision derived from hyperspectral-trained vehicle representations can improve lightweight detector generalization to RGB traffic imagery and support detection, tracking, and counting across sensing modalities.

SensorsVol. 26(22)
Florida International University (US)
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon