Overlooked Factors in Continual Zero-Shot Learning: Inflexible Semantic Prototypes, Simplistic Loss Functions, and SGD Noise

Zero-Shot Learning (ZSL) enables the recognition of unseen classes by transferring semantic knowledge from seen classes, typically through shared attributes. Continual Zero-Shot Learning (CZSL) advances the concept by enabling the model to continuously recognize new classes while retaining the ability to recognize previously learned ones. A common approach in CZSL is generative replay, where pseudo-data is synthesized to retain recognition of old classes while adapting to new ones. Despite this, existing methods rely on manually defined semantic prototypes, which may not align well with diverse visual representations. In this paper, we propose a method that refines semantic prototypes in the continual setting, treating them similarly to visual feature generation for better alignment with category semantics. We also introduce a contrastive loss on generated embeddings, which outperforms simplistic loss functions used in prior work. Furthermore, we investigate the impact of SGD (Stochastic Gradient Descent) noise, a factor often overlooked in CZSL, highlighting its significant role in model convergence and generalization. Our contributions improve CZSL performance and enhance the understanding of SGD noise in this context.