Enhanced Waste Sorting Technology by Integrating Hyperspectral and RGB Imaging

Identifying the material composition of objects is crucial for many recycling sector applications. Traditionally, object classification relies either on hyperspectral imaging (HSI), which analyses the chemometric properties of objects to infer material types, or on RGB imaging, which captures an object’s visual appearance and compares it to a reference sample. While both approaches have their strengths, each also suffers from limitations, particularly in challenging scenarios such as robotic municipal waste sorting, where objects are often heavily deformed or contaminated with various forms of dirt, complicating material recognition. This work presents a novel method for material-based object classification that jointly exploits HSI and RGB imaging. The proposed approach aims to mitigate the weaknesses of each technique while amplifying their respective advantages. It involves the real-time alignment of HSI and RGB data streams to ensure reliable result correlation, alongside a machine learning framework that learns to exploit the strengths and compensate for the weaknesses of each modality across different material types. Experimental validation on a municipal waste sorting facility demonstrates that the combined HSI–RGB approach significantly outperforms the individual methods, achieving robust and accurate classification even in highly challenging conditions.