Dynamic gating and attention masking for multimodal conditional diffusion image translation in autonomous driving scenarios

Virtual-to-real image translation is important for autonomous driving scenario generation, but existing conditional diffusion models still struggle to maintain semantic contour and spatial depth consistency when multimodal conditions exhibit varying reliability. In this study, we propose an adaptive multimodal conditional diffusion framework that dynamically regulates multimodal feature reliability through dynamic gating and attention masking.Semantic segmentation maps and depth maps are used as dual conditions. The dynamic gating module adaptively rebalances multimodal features by estimating spatial semantic importance and channel-level modal contribution, while the attention masking module suppresses interference from non-critical regions and enhances key traffic elements such as vehicles, road boundaries, and traffic signs. The optimized conditional features are injected into the U-Net denoising network to guide noise prediction with aligned semantic and depth constraints at the pixel level. Experiments on CARLA-to-Cityscapes translation show that the proposed method achieves an FID of 42.80 and an LPIPS-diversity value of 0.412. Downstream evaluations further obtain an mIoU of 82.2% and a Depth MAE of 0.12 m, indicating improved semantic structure preservation and spatial depth consistency.These results indicate that adaptive multimodal conditioning improves image realism, diversity, and semantic-depth consistency in autonomous driving image translation.