Identifying noise-emitting road users by combining multimodal sensor data

We present a system for identifying noise offenders in road traffic. The system is designed to automatically detect excessively loud road users, whose noise pollution annoys residents and harms their health. The multimodal system consists of acoustic and visual sensors, and a mobile user interface. The acoustic sensors include a calibrated sound pressure level meter (SLM) and an acoustic camera. The SLM monitors the acoustic scene and notifies in case of loud sound events. The acoustic camera assigns a direction to the sound event, and visualizes the measured sound field on its camera image. The visual component synchronously evaluates images from an RGB and a thermal camera. It monitors the observed road section and generates synchronized RGB and thermal measurement images at up to 10 Hz. The vehicle causing the noise is visually identified and tracked within a defined time window to determine the optimal position for license plate recognition. The thermal images are analyzed to identify potentially overheated vehicle components. The mobile user interface is installed on a tablet computer. In the case of a relevant acoustic incident, the user, e.g. a police officer, receives a notification and can quickly review the current data to take further action.