Danger Detection Through Audio Captioning and Classification Fusion

Sound is inherently ambiguous, which makes certainty critical in danger recognition, where false alarms and missed detection both carry real consequences. The definition of danger adopted here extends beyond ISO 7731:2003, which covers only intentionally designed alarm signals, to naturally occurring dangerous events such as glass breaking, gunshots, and human distress sounds, following the acoustic surveillance literature. Existing systems largely treat danger as multiclass classification without semantic reasoning about context, or rely on one-shot audio-to-word predictions that do not describe the scene. We instead treat danger as an outcome of the full acoustic scene, requiring semantically rich descriptions. The proposed framework trains a CNN for binary danger/no-danger classification on ESC-50, UrbanSound8K, and AudioSet, and in parallel extracts captions with the pretrained CoNeTTE model, scores them through a custom danger lexicon with SBERT similarity, and fine-tunes a DistilBERT classifier on the resulting labeled captions. The two classifiers are combined through a rule-based uncertainty fusion strategy, in which an inexpensive first-stage decision resolves clear-cut cases and the semantic classifier is reserved for ambiguous ones, reflecting the low-cost deployment demands of practical systems. On UrbanSound8K, fusion achieves a danger class recall of 0.88 (F1 0.66), improving over the standalone CNN (recall 0.70) and matching the semantic classifier alone (0.86) while invoking it for only 45% of clips, at about half of its measured computational cost, and preserving a human-readable intermediate description of each decision. We further evaluate the rule on two external, unseen datasets (XD-Violence and DESED) from different acoustic domains to assess generalization.