Steganography in Large Language Models

The development of deep learning has provided new momentum for steganography. However, existing model steganography methods are generally applicable to CNNs models, which suffer from low embedding capacity and poor robustness. To this end, we propose a stego scheme designed for large language models based on the Transformer architecture. Using the powerful feature representation ability and multilayer self-attention mechanism of Transformer, a large amount of secret data can be embedded without significantly affecting the performance of the model. In our scheme, the sender uses matrix multiplication to encode the coverage parameters of a specific Transformer block to embed secret data during the training process of the large language models. Ordinary users can use the stego model for text classification, text generation, and other routine tasks, while receivers can use a secret key to extract the secret data from the stego model, allowing for covert communication of secret data. Experimental results affirm the efficacy of our scheme in terms of embedding capacity, undetectability, and robustness.