Enhancing image consistency through zone-restricted self-attention for advanced generation and editing

In the current field of image generation and editing based on diffusion models, ensuring the consistency of the same foreground object/character across various backgrounds remains a challenge. Here, we introduce the Zone- Restricted Self-Attention (ZRSA) mechanism, which integrates semantic segmentation to guide consistent generation and complex non-rigid editing while preserving texture and identity. Our method outperforms existing techniques, improving text-image consistency by 1% and foreground character consistency by 13%, without the need for fine-tuning. This work makes a significant impact on high-quality image synthesis and editing by providing a user-friendly solution that requires no tuning.