Semantic Feature Scheduling and Rate Control in Multi-Modal Distributed Network

Traditional scheduling algorithms make decisions based solely on the channel conditions, without considering the transmitted semantic content. However, in multi-modal semantic communication systems, there is redundancy and variance in the importance of semantic features across modalities for task performance. To address this issue, we propose a novel feature scheduling and error probability control scheme that balances channel diversity and semantic diversity in semantic communication systems. Specifically, we first introduce a semantic feature importance metric to measure each feature’s contribution to the inference performance of semantic task. Using this metric, we formulate and solve an optimization problem to reduce overall latency while guaranteeing semantic task performance. Our detailed analysis examines feature selection strategies and transmission rate optimization to illustrate scheduling decisions based on both channel fading and semantic content. Consequently, we develop both optimal and low-complexity feature transmission scheduling schemes based on the optimization solution. Extensive experiments over multi-modal semantic communication systems validate that the semantic feature importance metric can reveal the importance of features from different modalities and accordingly affect their transmission rate. Additionally, the proposed schemes significantly reduce the system latency compared to the traditional methods.