Compression of Self-Supervised Representations for Machine Vision

Coding for machines is an emerging research topic aiming at compressing visual signals for machine analysis. Most existing research on it consider supervised feature compression, i.e., compressing features for a particular task and dataset. This paper explores a more flexible scenario in which the machine vision task (e.g., the classes to predict) is unknown during training and encoding (i.e., only decided during decoding). To achieve this goal, we compress self-supervised learning (SSL) features, which are not tied to a particular dataset but can be used for various tasks without re-training. Empirical studies are provided to analyze and derive an SSL feature compression system. Despite its simplicity, we show that linear transform coding achieves comparable or better rate-accuracy performance for SSL features compared to more advanced techniques. Although SSL feature compression performs slightly worse than its supervised counterpart, it generalizes well for out-of-distribution datasets. We highlight the use cases of various feature compression schemes and provide insights into developing future work. The code is made publicly available<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>https://gitlab.com/viper-purdue/dino-compress.