Towards Task-Scalable Semantic Communications: A Preprocessing Enhanced Image Transmission Framework

Recently, more and more images are transmitted to the receiver for the machine analysis tasks instead of purely for the human-vision task. However, most current machine-type semantic communication systems are optimized towards a specific task or a given set of tasks, without considering the scalability to new tasks. In this work, we propose a preprocessing enhanced image semantic communication system to address this challenge. Instead of optimizing directly towards a specific task, our system first optimizes a neural codec for preserving the holistic image information under the trade-off between bandwidth and distortion. Then, we reveal that the performance drop on the primary task can be ascribed to a lack of generalization of the task head to images with overall artifacts caused by compression and noisy channels. Accordingly, by finetuning a preprocessing module based on the pretrained neural codec under the trade-off between task accuracy and distortion, most loss on the primary task accuracy can be recovered with a trivial degradation on distortion, in this way, the task-scalable ability can also be preserved well due to the high quality on the holistic image. Experiments verify the effectiveness of our method in driving the primary task and the scalability to new tasks.