A model for improving the performance of large language models on soundscape description tasks

Generative AI is increasingly used in acoustic scene analysis and environmental soundscape description.Audio Language Models (ALMs) are key in this field but have two main drawbacks: they don't provide human-like survey responses and require significant computational resources, making real-time monitoring impractical.These issues stem from differences in how ALMs and humans assess and store sensory experiences.We propose a computationally efficient model with a three-layer architecture.The top layer features an advanced ALM for recognizing, grouping, and remembering audio events.The intermediate layer includes a local salient change detector based on sound feature embedding, which identifies meaningful changes in the auditory environment.When a change is detected, the ALM interprets and remembers the sound.The bottom layer has a memory cell mimicking echoic memory, retaining and reconstructing recent audio inputs.This reconstructed memory is passed to the ALM for recognition when triggered by the salient detector.Tested on the Urban Soundscapes of the World dataset, our model balances computational efficiency with real-time recognition accuracy, paving the way for scalable, intelligent acoustic monitoring solutions.