WL-WEM Combining Low-Cost Watermark Enhancement Modules for In-Generation Watermarking

In general, modifying the latent diffusion model (LDM) decoder to achieve in-generation watermarking cannot introduce tremendous computational burden, which easily leads to non-convergence. This necessarily increases the difficulty of embedding the watermark into the LDM decoder due to the need to strike a balance among imperceptibility, robustness and computational cost. We realize the difficulty and design two lightweight watermarking modules, namely a low-cost watermark redundancy enhancement module (WREM) and a latent-guided watermark enhancement module (LWEM), aiming at reducing the modifications to the LDM decoder as much as possible while maintaining the generation quality and enhancing the robustness. Specifically, WREM, specially designed for shallow layers, utilizes a small number of repetition operations to strengthen the robustness of the watermark, and adopts a low-cost sub-pixel convolution layer to achieve dimension consistency between the watermark residual and the input latent, greatly reducing the computational cost while enhancing the integration of watermark features and the latent feature. LWEM, tailored for deep layers, innovatively exploits a simple bilinear interpolation to strengthen the robustness of the watermark, and fuses watermark features and the latent feature using a cheap convolution layer so as to generate the watermark residual with relatively low impact on the input latent. Combining WREM and LWEM, we construct a lightweight encoder-noiselayer-decoder in-generation watermarking method dubbed WL-WEM pursuing a satisfactory balance among three metrics including computational cost, generation quality and robustness. Experimental results also demonstrate that the proposed WL-WEM outperforms several related works in balancing three metrics.