Data-driven listening intention detection for hearables

Commercially available hearing aids are capable to automatically adapt the built-in “sound enhancing” technology according to the “acoustic scene.” Individualization of this technology can be done with assisted or manual intervention. However, without continuous self-adjustment the mapping of “acoustic scene” and “sound enhancement” setting stays fixed, i.e., the hearing device does not adapt to an acute switch of the user’s “listening intention.” This ability, to selectively attend and switch attention between various acoustic sources, is a landmark of the healthy hearing system and seriously degraded even with a mild hearing deficit. In this work, we developed a system based on “listening intention detection” which can integrate data from multiple sensors to classify complex acoustic environments, infer the acoustic source attended by the listener, and adjust to individual user preferences. This is achieved through a comprehensive analysis of the user’s surroundings and complemented with implicit feedback signals from the user himself, without introducing invasive sensors. The algorithm can be adjusted to meet specific technical constraints for edge devices.