Controllable Face Synthesis via Bi-Level Routing Attention Zigzag Mamba Diffusion

With the popularity of mamba, the combination of MAMBA and diffusion model has become a new research hotspot. The existing working Zigma introduces mamba module under the framework of stochastic interpolation diffusion model, which achieves good results, but has high computational complexity and memory consumption. In this work we propose a modified zigma diffusion model for controllable face synthesis by introducing the Bi-Level Routing Attention mechanism, which enables more flexible computation allocation and content awareness. In zigma's Mamba block, we inserted a biformer block in front of the cross attention block, and reduced the number of blocks accordingly to avoid the model being too large. We trained the model on the mulitcelebA 256*256 face dataset, compared it with the previous methods in terms of authenticity and text consistency, and conducted a user study to obtain good experimental results, verifying the effectiveness of this work on face image synthesis tasks.