Learning Observers’ Gaze Dynamics: An Efficient and Mobile Sport Scenery Recognition Pipeline

This study addresses the challenge of semantically sorting complex scenes in a mobile environment by processing multimodal visual inputs to create detailed landscape representations. Central to the approach is a streamlined multi-layer hierarchical model that mimics human attention dynamics, using the BING objectness metric to quickly identify significant areas by recognizing objects across different scales and contexts. To enhance feature extraction, time-sensitive and manifold-guided selectors are employed to prioritize high-quality visual features, while a low-rank active learning (LAL) algorithm simulates human-like focus on key visual zones, specifically in sports scenes. The model generates a Gaze Shift Path (GSP), which directs the collection of composite CNN features, ultimately classifying the scenes into distinct landscape types using a support vector machine (SVM). Experimental results on seven scene image sets have shown that our method outperforms the others by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2\% \sim 5\%$ </tex-math></inline-formula>. Additionally, our calculated deep GSP features can greatly facilitate image clustering. Last but not least, our visualized GSPs are over 90% consistent with real-world human gaze behaviors, which explains the competitiveness of our method.