Hierarchical Hybrid Deep Riemannian Network with Joint Metric Learning for Visual Classification

Modeling visual data on non-Euclidean spaces, such as Grassmann and Symmetric Positive Definite (SPD) Riemannian manifolds, has shown superior advances in visual classification. To extract complementary features, recent approaches fuse multiple manifolds using deep learning architectures, achieving superior performance compared to single-manifold networks. However, they usually fuse manifold networks in a naive joint manner, which may not extract the fine-grained intrinsic features. Alternatively, metric learning is often used to enhance the discrimination of the loss function in Riemannian manifold networks. However, the random sampling scheme of batch in Riemannian manifold networks may not ensure sufficient intra-class samples for positive pairs, which may result in instability for training. In this paper, we propose to fuse two branches of hybrid manifold networks, with each fusing the Grassmann and SPD manifold networks, in a hierarchical decoupling manner. Two branches of the hybrid manifold network are individually optimized by decoupling strategies to alleviate the problem of gradient conflict between branches. Subsequently, the outputs of two branches are hierarchically concatenated to capture the fine-grained intrinsic information. Furthermore, we remove the intra-class scatter constraint in the common metric learning regularization term. Then, we exploit a dynamic positive pair construction strategy to extract sufficient intra-class samples from two branches to circumvent the instability of metric learning caused by insufficient positive pairs. Extensive experimental results on multiple visual classification tasks validate the effectiveness of the proposed method.

Authors

Institutions

Publication Details

Journal
ACM Transactions on Multimedia Computing Communications and Applications
Published
2026-11-19
DOI
https://doi.org/10.1145/3862853
Primary Topic
Image Retrieval and Classification Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Hierarchical Hybrid Deep Riemannian Network with Joint Metric Learning for Visual Classification

Guofeng Yan, Jiao Du, Hengliang Tan, Shuo Yang et al.
ACM Transactions on Multimedia Computing Communications and Applications
Image Retrieval and Classification Techniques
article

Hierarchical Hybrid Deep Riemannian Network with Joint Metric Learning for Visual Classification

Guofeng Yan, Jiao Du, Hengliang Tan, Shuo Yang, Yujie Wu
article en

Abstract

Modeling visual data on non-Euclidean spaces, such as Grassmann and Symmetric Positive Definite (SPD) Riemannian manifolds, has shown superior advances in visual classification. To extract complementary features, recent approaches fuse multiple manifolds using deep learning architectures, achieving superior performance compared to single-manifold networks. However, they usually fuse manifold networks in a naive joint manner, which may not extract the fine-grained intrinsic features. Alternatively, metric learning is often used to enhance the discrimination of the loss function in Riemannian manifold networks. However, the random sampling scheme of batch in Riemannian manifold networks may not ensure sufficient intra-class samples for positive pairs, which may result in instability for training. In this paper, we propose to fuse two branches of hybrid manifold networks, with each fusing the Grassmann and SPD manifold networks, in a hierarchical decoupling manner. Two branches of the hybrid manifold network are individually optimized by decoupling strategies to alleviate the problem of gradient conflict between branches. Subsequently, the outputs of two branches are hierarchically concatenated to capture the fine-grained intrinsic information. Furthermore, we remove the intra-class scatter constraint in the common metric learning regularization term. Then, we exploit a dynamic positive pair construction strategy to extract sufficient intra-class samples from two branches to circumvent the instability of metric learning caused by insufficient positive pairs. Extensive experimental results on multiple visual classification tasks validate the effectiveness of the proposed method.

ACM Transactions on Multimedia Computing Communications and Applications
Guangzhou University (CN)
Image Retrieval and Classification Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon