A2SSC: An Agent-based Adaptive Semantic Speech Communication System

Abstract In wireless environments with weak coverage or rapidly changing channels, reliable speech transmission is challenging due to fluctuating channel quality and limited bandwidth. Semantic communication offers a promising solution by focusing on the transmission of essential information rather than raw signals. Although existing semantic speech communication systems have made progress, they still lack decision mechanisms that are aware of both content and channel, limiting their robustness and flexibility. In this paper, we propose an agent-based adaptive semantic speech communication system (A2SSC). The system encodes textual content and speaker characteristics separately, and adaptively allocates transmission resources based on the importance of the speech content and the current channel conditions. At the same time, we incorporate an agent-based architecture to ensure system flexibility and scalability, and introduce an online learning mechanism that allows the system to continuously adapt to changing channel conditions after deployment. Experimental results show that A2SSC achieves comparable word error rate (WER) and speaker similarity (SS) to state-of-the-art baselines, while reducing the maximum num-ber of transmitted symbols by 62.5%. Moreover, the system maintains robust performance across varying channel conditions, demonstrating its effectiveness for reliable and efficient speech transmission in challenging wireless environments.