AviaTAD-LGH: A Multi-Task Spatio-Temporal Action Detector with Lightweight Gradient Harmonization for Real-Time Avian Behavior Monitoring

Fine-grained spatio-temporal action detection in continuous, unconstrained field videos remains a formidable challenge due to severe background clutter, high inter-class similarity, and the scarcity of domain-specific benchmarks. To address these limitations, we first introduce a large-scale Wintering-Crane Benchmark, providing dense, individual-level bounding box annotations for six complex behaviors across diverse habitat scenes. Leveraging this data, we propose AviaTAD-LGH, a real-time multi-task framework that incorporates auxiliary motion supervision into a dual-pathway 3D backbone to enhance feature discriminability. A critical bottleneck in such multi-task settings is the negative transfer caused by conflicting optimization objectives. To resolve this, we present Lightweight Gradient Harmonization (LGH), a plug-and-play optimization strategy that dynamically modulates task weights based on the cosine similarity of gradient directions. This mechanism effectively aligns optimization trajectories without introducing inference latency. Extensive experiments demonstrate that AviaTAD-LGH achieves a state-of-the-art mAP of 68.60%, surpassing strong public baselines by 7.44% and improving upon the single-task baseline by 2.80%, with significant gains observed on ambiguous dynamic classes. The proposed pipeline enables efficient, scalable ecological monitoring suitable for edge deployment.

Authors

Institutions

Publication Details

Journal
Sensors
Published
2026-03-27
DOI
https://doi.org/10.3390/s26072088
Primary Topic
Human Pose and Action Recognition
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

AviaTAD-LGH: A Multi-Task Spatio-Temporal Action Detector with Lightweight Gradient Harmonization for Real-Time Avian Behavior Monitoring

Haifang Jian, Markus P. Eichhorn, Wanting Peng, Ramiro D. Crego et al.
Sensors
Human Pose and Action Recognition
article

AviaTAD-LGH: A Multi-Task Spatio-Temporal Action Detector with Lightweight Gradient Harmonization for Real-Time Avian Behavior Monitoring

Haifang Jian, Markus P. Eichhorn, Wanting Peng, Ramiro D. Crego, Ning Xin, Jun Du, Z. Q. Xie, Mengdi Fu, Hongchang Wang, Wenhui Yang
article en

Abstract

Fine-grained spatio-temporal action detection in continuous, unconstrained field videos remains a formidable challenge due to severe background clutter, high inter-class similarity, and the scarcity of domain-specific benchmarks. To address these limitations, we first introduce a large-scale Wintering-Crane Benchmark, providing dense, individual-level bounding box annotations for six complex behaviors across diverse habitat scenes. Leveraging this data, we propose AviaTAD-LGH, a real-time multi-task framework that incorporates auxiliary motion supervision into a dual-pathway 3D backbone to enhance feature discriminability. A critical bottleneck in such multi-task settings is the negative transfer caused by conflicting optimization objectives. To resolve this, we present Lightweight Gradient Harmonization (LGH), a plug-and-play optimization strategy that dynamically modulates task weights based on the cosine similarity of gradient directions. This mechanism effectively aligns optimization trajectories without introducing inference latency. Extensive experiments demonstrate that AviaTAD-LGH achieves a state-of-the-art mAP of 68.60%, surpassing strong public baselines by 7.44% and improving upon the single-task baseline by 2.80%, with significant gains observed on ambiguous dynamic classes. The proposed pipeline enables efficient, scalable ecological monitoring suitable for edge deployment.

SensorsVol. 26(7)
Chinese Academy of Sciences (CN), University College Cork (IE), Shandong Normal University (CN), Shanghai Tongji Urban Planning and Design Institute (CN), Institute of Semiconductors (CN), Chinese Research Academy of Environmental Sciences (CN)
Reduced inequalities
Openalex Percentile: Top 4%
Human Pose and Action Recognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.