Prompt-stable knowledge distillation of vision-language models for efficient waste classification in material recovery facilities

• Domain specific knowledge distillation framework using Vision-Language Models for reliable waste sorting. • Hybrid prompt learning enhances generalization and reduces prompt sensitivity. • Efficient and scalable distilled student models match teacher performance with better generalization and real-time deployability. Waste classification in material recovery facilities is difficult to scale due to limited labelled data and the high variability in waste appearance. Vision-language models address these challenges through strong zero-shot performance when supported by well-designed prompt engineering, enabling high accuracy without task-specific fine-tuning. However, their large size, architectural complexity, slow inference speed, and sensitivity to the wording of text descriptions limit their suitability for real-time sorting environments. This study introduces a knowledge distillation framework that transfers the semantic capability of large vision-language models into compact student models. A key innovation is a hybrid prompt learning strategy in which the teacher model uses detailed, optimized textual descriptions while the student model learns from simple, generic prompts during distillation. This design improves the student model’s stability across varied textual descriptions and reduces the need for dynamic prompting to accommodate differences in waste appearance. Experiments on a multi-class waste dataset show that the best distilled student model surpasses the strongest teacher by over 13 % in classification accuracy, improves prompt stability by 79.5 %, and achieves 84.2 % faster inference with 80.0 % lower architectural complexity. The resulting model addresses key practical limitations of vision-language models for waste classification, delivering a complete, practical solution for real-time sorting in material recovery facilities.