A multichannel audio tagging and localization system for home surveillance

Audio Tagging (AT) is a critical technique in smart home applications, enabling continuous monitoring of specific sound events for subsequent surveillance systems. Although Deep Neural Networks (DNNs) can provide promising tagging results, the performance degrades significantly in adverse acoustic environments. In addition, if the AT system can provide both the tagging and the location results, it can greatly enhance the system’s ability to capture sound events. To this end, a multichannel audio tagging system based on Convolutional Neural Network (CNN) is proposed to simultaneously tag and localize the sound event. Interchannel Phase Differences (IPDs) between each pair of microphones are used as the spatial feature for the input to the model. Instead of predicting the angle or zone index, the proposed system outputs a unit vector pointing to the sound event for localization. A novel loss function is introduced that computes the square of the cosine similarity between the ground truth and the estimated unit vector, allowing for more accurate localization performance. Experimental results show that the proposed system outperforms the single-channel baseline under strong interference, making it well suited for real-world applications. In addition, the model trained with the proposed loss function can greatly improve the localization accuracy.