Privacy Failure in Split Audio-Visual Separation for Ecological Acoustic Monitoring under Tight Uplink Budgets

Passive acoustic monitoring stations record year-round soundscapes in forests, wetlands and coastal waters, and the recordings carry not only birds, frogs and cetaceans but also the voices of rangers, researchers and visitors. Because such stations have neither mains power nor fibre, recent deployments split the model: a light front end runs at the station and only an intermediate representation is uplinked to a cloud separator. This paper shows that the representation is not privacy-neutral. On a split audio-visual separator, a black-box inversion trained on a few hundred uplinked vectors reconstructs speech that is intelligible enough to be verified as the same speaker, and the leakage grows with separation quality because both depend on the same timbre cues. We study this failure in the ecological setting and propose an interface that separates what separation needs from what privacy must hide. An identity-decoupled split interface keeps a content stream and replaces the facial feature map with four activity tokens, while a mutual-information penalty removes speaker statistics from the uplinked vector. A budget-conditioned allocator spends the scarce uplink on the token groups with the largest marginal separation gain per bit, gated by an estimate of visual reliability under rain, dusk and infrared night. A conformal risk gate turns an auditable bound on speaker-verification success into an upload decision with an explicit degradation ladder. Across AVSpeech, VGGSound and four BEANS subsets, the interface recovers 10.4 dB SI-SNRi at 16 kbps on the mixed station set, 0.8 dB above a static-weight audio-visual baseline, while raising the equal error rate of reconstructed speech from 10.9% to 29.4% and cutting end-to-end latency by 6%.

Authors

Institutions

Publication Details

Published
2026-11-13
DOI
https://doi.org/10.22541/essoar.15012185/v1
Primary Topic
Speech and Audio Processing
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Privacy Failure in Split Audio-Visual Separation for Ecological Acoustic Monitoring under Tight Uplink Budgets

Siyi Rong, Zihan Gai, Yuze Qiao, Hanwen Cen et al.
1 links
Speech and Audio Processing
preprint

Privacy Failure in Split Audio-Visual Separation for Ecological Acoustic Monitoring under Tight Uplink Budgets

Siyi Rong, Zihan Gai, Yuze Qiao, Hanwen Cen, Jingpei Luan
preprint en

Abstract

Passive acoustic monitoring stations record year-round soundscapes in forests, wetlands and coastal waters, and the recordings carry not only birds, frogs and cetaceans but also the voices of rangers, researchers and visitors. Because such stations have neither mains power nor fibre, recent deployments split the model: a light front end runs at the station and only an intermediate representation is uplinked to a cloud separator. This paper shows that the representation is not privacy-neutral. On a split audio-visual separator, a black-box inversion trained on a few hundred uplinked vectors reconstructs speech that is intelligible enough to be verified as the same speaker, and the leakage grows with separation quality because both depend on the same timbre cues. We study this failure in the ecological setting and propose an interface that separates what separation needs from what privacy must hide. An identity-decoupled split interface keeps a content stream and replaces the facial feature map with four activity tokens, while a mutual-information penalty removes speaker statistics from the uplinked vector. A budget-conditioned allocator spends the scarce uplink on the token groups with the largest marginal separation gain per bit, gated by an estimate of visual reliability under rain, dusk and infrared night. A conformal risk gate turns an auditable bound on speaker-verification success into an upload decision with an explicit degradation ladder. Across AVSpeech, VGGSound and four BEANS subsets, the interface recovers 10.4 dB SI-SNRi at 16 kbps on the mixed station set, 0.8 dB above a static-weight audio-visual baseline, while raising the equal error rate of reconstructed speech from 10.9% to 29.4% and cutting end-to-end latency by 6%.

Kunming University of Science and Technology (CN)
Speech and Audio Processing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon