Patch-AST: A Patch-Based Audio Spectrogram Transformer for Multiclass Environmental Noise Classification in Noisy Speech Under Variable SNR Conditions

Abstract—Reliable identification of the acoustic environment surrounding a speech signal supports speech enhancement, hearing-aid noise suppression, adaptive speech recognition and context-aware voice assistants. This paper presents Patch-AST, a compact Audio Spectrogram Transformer that classifies four background-noise environments (Airport, Babble, Restaurant and Station) mixed with clean speech at six signal-to-noise ratios (SNRs) from −5 to 10dB. Log-Mel spectrograms are split into frame-level tokens, linearly embedded, given learned positional encodings, and processed by a six-layer multi-head self-attention encoder with 20,099,844 parameters. The model is trained from scratch on a corpus balanced across classes and SNR levels of 120,000 synthetically mixed utterances (84,000 training, 18,000 validation and 18,000 test) using a class-weighted focal loss, AdamW, a one-cycle learning-rate schedule and early stopping. A short analysis of the mixing model shows that the noise contributes 76% of the mixture energy at −5dB but only 9% at 10dB, which explains why accuracy decreases as SNR increases. On the test set the model reaches 94.59% accuracy (95% Wilson interval 94.25–94.91%), a macro-F1 of 0.95, per-class Brier scores between 0.0129 and 0.0201, and an area under the receiver operating characteristic curve (ROC-AUC) of 1.00 for every class. Accuracy exceeds 98% at −5dB and falls to about 84% at 10dB, and two-dimensional embedding projections show largely separable classes. Index Terms—Audio spectrogram transformer, Mel spectrogram, noise classification, self-attention, signal-to-noise ratio, speech enhancement