HiGT-Fuse: Hierarchical Graph-Transformer Fusion Framework for Self-Supervised Howling Detection and Filtering

Audio signal processing has always been plagued with the problem of acoustic howling, especially when it comes to real-time and open microphone applications. In this paper, we present HiGT-Fuse, a novel Hierarchical Graph-Transformer Fusion based howling detection and suppression with self-supervised learning to achieve robust and real-time howling detection and suppression. We develop graphs-multi-resolution ones-on time-frequency spectrograms and these are reflections of fine and coarse-grained spectral dynamics. In order to solve the limitations of the fixed graph structures, we propose a meta-learned edge function that is able to dynamically modify the graph structure. Individual graphs are encoded via a deep Graph Neural Network (GNN) which is enhanced with a Transformer encoder to enable long range temporal dependencies. The learning strategy is a contrastive curriculum, which combines incremental augmentation and dual-loss training, and enables the model to learn the abnormal patterns without the need of labeled data. Learned embeddings are used to calculate anomaly scores and a dynamically configurable thresholding module, which uses a Gated Recurrent Unit (GRU), is used to identify howling events. We design a two-stream GRU controller which, in real time, parameterizes a digital notch filter which is used to reject the detected anomalies. Learned embeddings are used to compute the anomaly scores and a real-time thresholding module based on GRU is used to localize events. A two-stream GRU controller is then used to adapt a digital notch filter to suppress the detected howling. The average F1-score, Perceptual Evaluation of Speech Quality (PESQ) improvement, Signal-to-Distortion Ratio (SDR) and Real-Time Factor (RTF) of HiGT-Fuse are 0.91, 3.03, 3.03 dB and 0.37x respectively, which means that the system is approximately 2.7 times faster than real time; also, it outperforms the traditional baselines, including AFC, and recent deep learning models, including Temporal-Spectral Transformer (TST) and Bootstrap Your Own Latent (BY The system is highly generalizable to both real-world and synthetic data sets including LibriSpeech, GWA, VCTK, REVERB and VoiceBank-DEMAND, in both Signal-to-Noise Ratio (SNR) and reverberation settings. HiGT-Fuse is a novel standard in autonomous and low-latency audio stabilization and can be widely applied to anomaly detection in the spectral domain.