BEE: Opportunistic Heat-based Bio-sensing for Produce Quality Monitoring

Fruits and vegetables have a short shelf life, resulting in significant amounts of unsold produce in retail stores, which contributes to food waste and financial losses. To address this issue, retailers require innovative technologies that enhance stock estimation and influence customer behavior. Monitoring systems that track expiration dates and assess the quality of organic products show promise, but they must be easy to deploy, maintain, and scale for widespread adoption. We introduce BEE, an innovative solution for produce quality estimation that leverages collaborative heat-based bio-sensing from customer interactions. BEE addresses a critical gap in current solutions: lack of user-friendly approaches that are easy to deploy and manage. BEE operates by capturing superficial heat residuals left by customers interacting with fresh produce using a thermal camera, which are then used to construct thermal dissipation profiles that enable accurate quality estimation. Constructing these profiles from sparse and variable thermal residuals in different environments, however, is highly challenging. To overcome this challenge, the key innovations in BEE are the use of opportunistically collected thermal residuals for profiling produce, integration of novel AI image enhancement techniques for improving the quality of thermal residual, AI-based calibration to overcome variations in interactions and consumer characteristics, and a novel computer vision pipeline to facilitate effective estimation. We validate the practicality of BEE through rigorous experiments, demonstrating its ability to capture decay patterns from human heat residuals. Our results indicate that BEE can achieve over 95% accuracy in estimating produce quality while operating robustly across varying environmental conditions and outperforming existing state-of-the-art solutions. Additionally, user studies reveal that BEE is intuitive and user-friendly, achieving the 90th percentile in an user satisfaction survey.