From RGB to polarization: synergizing physics-based priors with diffusion models for polarization generation

Polarization observations encode surface geometry and material responses that are difficult to capture directly from RGB images, providing valuable complementary cues for visual tasks such as semantic segmentation, depth and normal estimation. However, collecting dense polarization requires specialized sensors, which limits its broader application in complex-scene perception. Existing approaches either rely heavily on fixed analytical models, which struggle to describe non-ideal polarization responses and are often restricted to single-object scenes, or adopt learning-based paradigms without explicit physical constraints, making it difficult to balance physical fidelity and generalization. To address these limitations, we propose a novel physics-guided two-stage framework that decomposes polarization generation into a Physics-Based Analytical Initialization Module (PAIM) and a Diffusion Residual Refinement Module (DRRM). In the PAIM, the framework estimates surface normals, pixel-wise refractive indices, and reflection mixture weights from an RGB image. It then fuses diffuse and specular components in normalized Stokes space to construct a physically interpretable initial polarization estimate. The DRRM employs a one-step diffusion model for residual refinement. This design preserves the interpretable physical structure while compensating for high-frequency details and non-ideal deviations. We further introduce discrepancy-aware loss and structural-consistency loss to emphasize challenging regions and jointly constrain polarization magnitude and orientation, thereby preserving polarization physics. Experiments on HAMMER and OutdoorPol show that our method achieves the lowest errors, together with strong overall reconstruction performance. Ablation studies further validate the effectiveness of each component. In addition, zero-shot generalization experiments and downstream task evaluations demonstrate strong cross-domain generalization and practical value for downstream perception.