Video based deep learning deciphers honeybee waggle dances in natural conditions

Abstract Urbanization and industrial agriculture are a threat to wild and managed honey-bees, crucial pollinators of the natural- and agro-ecosystems components of the landscapes. Understanding bee colonies’ foraging behaviors within these landscapes is essential for managing human-bee conflicts and sustaining their vital pollination services. Objectives To understand how bees use their surroundings, researchers often decode bee waggle dances, a behavior that communicates navigational information about desirable foraging sites to their nest mates. This process is carried out manually, which is time-consuming, prone to human error and requires specialized skills. We aim at developing an automatic pipeline to detect and translate waggle dances in natural conditions. Methods We introduce a novel deep learning-based pipeline that automatically detects and measures waggle runs, the core movement of the waggle dance, under natural recording conditions for the first time. With this information we can estimate the spatial and temporal dynamics of bee foraging behavior. Results Comparison of our pipeline with analysis made by human experts revealed that our procedure is able to detect 100% of waggle runs on the testing dataset, with a run duration Root Mean Squared Error (RMSE) of less than a second, and a run angle RMSE of 0.21 radians. It is also generalizable to other recording conditions and bee species. Conclusion Our approach enables precise measurement of direction and duration, enabling the spatial and temporal analysis of bee foraging behavior on an unprecedented scale compared to traditional manual methods, contributing to preserving biodiversity and ecosystem services.