RE-GZSL: Relation Extrapolation for Generalized Zero-Shot Learning

Unlike Conventional Zero-Shot Learning (CZSL) which only focuses on the recognition of unseen classes by using a classifier trained on seen classes and semantic embeddings, Generalized Zero-Shot Learning (GZSL) requires a classifier trained on seen classes to recognize objects from both seen and unseen classes. To tackle this problem, feature generative-based models have been proposed to synthesize visual features for unseen classes conditioned on their semantic descriptors. However, they treat these semantic descriptors as independent individuals without exploring their structural relations among categories. We propose a novel approach, dubbed Relation Extrapolation based feature generation for GZSL (RE-GZSL), which generates features of unseen classes by borrowing some features that are extrapolated from seen classes based on semantic relations. In RE-GZSL, a visual-semantic relations alignment loss and an instance-prototype contrastive loss are presented to align visual relations with semantic relations. To maintain the information of the visual features before and after the alignment, a discrimination preservation loss is further introduced. Besides, a feature mixing module is built to synthesize features for unseen classes, which are more realistic and tightly related to seen classes. Experimental results demonstrate that RE-GZSL outperforms competitors on four benchmark datasets. Comprehensive ablation studies and analyses are provided to dissect what factors led to this success. Code is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Barcaaaa/RE-GZSL</uri>.