SeER: A Unified Transmission Performance Metric for Wireless Semantic Communication

Semantic communication (SemCom) with intelligent transceivers focusing on meaning delivery has shown great potential for robust data transmission and spectrum saving. Currently, the metrics used in SemCom are primarily derived from those adopted in the machine learning (ML) field. As a result, existing design approaches are not capable of differentiating whether performance loss is caused by poor ML models or unreliable data transmission. Meanwhile, we realize that the classic transmission metric, i.e., Bit Error Rate (BER), seems not measure the reliability of semantic transmission correctly. Thus, the lack of a metric to measure semantic transmission is a fundamental problem. To address this issue, the goal of this article is to provide a unified metric, defined as the Semantic Error Ratio (SeER), specifically to measure the performance of semantic transmission, shielding the impacts from the ML model adopted by semantic extraction and recovery. On this basis, two possible expressions to describe SeER are proposed from the perspective of semantic similarity and semantic significance. Simulation results demonstrate that the SeER metric can objectively portray the reliability of semantic transmission, even implying a more consistently negative relation with the task performance, compared to the classic BER in traditional syntactic communication.