Zero-Shot Recognition for Healthcare Social Networks via Tensor-Based Vision-Semantic Manifold Alignment

Healthcare social networks (HSNs) are pivotal in spreading healthcare knowledge, providing support to both potential patients and medical professionals, and enhancing healthcare services. However, identifying unseen data in HSN poses a significant challenge due to their intrinsic heterogeneity, dynamic characteristics, and the scarcity of labeled data. Employing semantic knowledge transfer for class-agnostic zero-shot recognition stands out as a promising and innovative solution to this problem, but the visual-semantic gap and domain shift problems considerably hinder advancements in zero-shot recognition capabilities. Previous zero-shot models often impose constraints between vision and semantics in the loss part without explicitly injecting intermodality guidance into the feature refinement process. This article yields a novel zero-shot recognition framework for HSN, named the dual tensor prototype graph network, devoted to improving the performance of recognizing unseen objects in HSN leveraging semantic knowledge. We have developed an iterative and interactive updating strategy for dual tensor prototype graphs, explicitly leveraging the distribution information from one modality to guide the prototype graph updates of another modality. We constrain the update process of the dual prototype graphs by several tailored loss functions and episodic training, alleviating the inconsistency between semantic and visual manifolds. Extensive comparative experiments conducted on two medical imaging datasets and five zero-shot benchmarks affirm the stronger generalization ability of our proposed method compared with other advanced approaches, showing the potential of addressing zero-shot problems in HSN.