Prospective Layout-Guided Multi-Modal Online Hashing

In real-world scenarios, the data usually appears in a streaming fashion. To achieve remarkable retrieval performance in such scenarios, online multi-modal hashing has drawn great research attention due to its high retrieval speed and low storage cost. However, existing online multi-modal hashing methods still fail to achieve satisfactory retrieval performance in the scenarios where the new streaming datapoints all belong to the new classes. Therefore, to further improve the retrieval performance in these scenarios, we propose a novel Prospective Layout-Guided Multi-modal Online Hashing, termed PLG-MOH. Specifically, PLG-MOH first establishes the layout of the Hamming space by generating a series of hashing centers to split the space. Each hashing center will be gradually assigned to a new appearing class, and these assigned centers correspond one-to-one with the classes. Moreover, we propose a novel prospective layout-guided loss, which leverages all the hashing centers, including those not yet assigned to the classes, to supervise the training of hashing model. As the unassigned hashing centers will be designated to the new classes emerging in the future, it signifies that during each round of training, PLG-MOH has already considered the forthcoming data from new classes in the future rounds. Consequently, PLG-MOH can effectively adapt its hashing functions to address the new arriving samples and learn semantic similarity-preserved hash codes for them, meanwhile it can effectively retain the information learned from the old data. Extensive experiments on two public datasets demonstrate that the proposed PLG-MOH achieves better retrieval performance than state-of-the-art baselines on online scenarios.