Zero-shot learning based on the fusion of global and local representations

Abstract Zero-shot learning (ZSL) endeavors to extend knowledge to novel classes by capitalizing on the semantic overlap across different categories. Contemporary ZSL approaches concentrate on isolating image-specific local features pertinent to attributes and using these to align class semantic vectors. However, existing methods often overlook the integration of global and local representation, a synthesis that could significantly enhance zero-shot recognition accuracy. This paper introduces an innovative ZSL technique, termed ZSL based on the fusion of global representation and local representation (ZGLR). Our approach incorporates a Transformer encoder constructed upon attribute prototypes to extract attribute-level features, which are then mapped to local representation to align with class semantic vectors. Concurrently, we introduce a discriminative semantic-visual mapping network that embeds class semantic vectors into the visual domain, thereby aligning global representation. During training, both global and local representation are optimized in tandem, while in the testing phase, the outcomes from both representational levels are consolidated to bolster classification precision. Practical results from three benchmark ZSL datasets demonstrate the superiority of our put-forward ZGLR solution.