Deep Uncertainty-aware Probabilistic Hashing for Cross-modal Retrieval

Due to its outstanding computational efficiency and low storage requirements, hashing technology has become a research hotspot in large-scale multimedia retrieval. In cross-modal hashing, the key lies in mapping samples from different data sources into a common discrete space. However, most existing methods assume that the input data is of high quality and completeness. When samples are incomplete or degraded (e.g., blurry images or incomplete text), the absence or ambiguity of semantic information inevitably compromises retrieval accuracy. Traditional deterministic embedding methods typically map multi-modal samples to a single point in the embedding space without considering uncertainty. As a result, inherent noise or feature ambiguity in the inputs may lead to distorted or shifted binary representations. To resolve this problem, this article proposes a novel Deep Uncertainty-aware Probabilistic Hashing (DUaPH) method that models the uncertainty of multi-modal samples. By capturing the underlying distribution of heterogeneous data, DUaPH effectively mitigates feature ambiguity and inherent noise, enhancing the robustness of cross-modal retrieval. Specifically, each heterogeneous sample is mapped to a multivariate Gaussian distribution, where the mean represents the most probable semantic features, and the variance reflects the sample uncertainty. A semantic feature matching mechanism is introduced to dynamically adjust the importance of feature dimensions, prioritizing those with higher certainty. Then, a semantic feature fusion mechanism is developed to integrate the semantic features from multi-modal sample pairs, producing a new distribution with reduced uncertainty and improved semantic alignment. Extensive experiments on four benchmark datasets demonstrate that DUaPH significantly improves robustness and retrieval performance under conditions of semantic ambiguity and data uncertainty. The source code is available at https://github.com/QinLab-WFU/DUaPH .