Deep Differential Lifelong Cross-modal Hashing for Stream Medical Data Retrieval

With the explosive growth of stream medical multi-modal data, it is significant to develop an efficient cross-modal retrieval algorithm to achieve effective medical data search. Within it, deep cross-modal hashing which maps cross-modal data into low-dimensional Hamming space where similarity in high-dimension space is preserved has made much progress. However, most of deep cross-modal hashing algorithms are usually facing disability of adapting to dynamic stream medical data, non-differentiable optimization, and unaligned semantic across modalities. To address these, we, in this article, propose a novel deep differential lifelong cross-modal hashing method for large-scale stream medical data retrieval. Specifically, we first design lifelong learning module to keep the learned hash code of base data unchanged and directly learn hash code of incremental data with new categories to achieve continuous retrieval of stream medical data, which effectively mitigates catastrophic forgetting, as well as significantly reduces training time and computation resource. Then, we introduce differential cross-modal hashing module to generate discriminative binary hash codes, which yields continuous and differentiable optimization and improves accuracy. Besides, we design semantic alignment module which embeds intra-modal and inter-modal losses to maintain the semantic similarity and dis-similarity among stream medical data across modalities. Extensive experiments on benchmark medical datasets show that our proposed method can retrieve dynamic stream medical cross-modal data effectively and obtain higher retrieval performance comparing with recent state-of-the-art approaches.