UDDSFormer: An Efficient Hardware-Oriented Transformer Framework for Unified Demosaicing, Denoising, and Super-Resolution

Reconstructing high-quality, high-resolution RGB images from raw sensor data is a critical task in the image signal processing (ISP) pipeline. However, cascading these processing stages often leads to error accumulation and imposes excessive computational and memory overheads, thus limiting efficient deployment on resource-constrained embedded edge devices. To address these challenges, we propose UDDSFormer—a hardware-oriented Transformer framework designed specifically for the unified RAW-to-RGB reconstruction task. First, the framework introduces an adaptive deformable sampling mechanism that aggregates features aligned with image structures by predicting offsets relative to a regular grid. Subsequently, a geometry decomposition module models horizontal and vertical dependencies through two directional branches and sums their separately normalized axial value responses for feature aggregation. Furthermore, on the Kodak24 test set, UDDSFormer achieves an 8.63× reduction in parameter count while improving PSNR by about 4.22 dB compared to the reference lightweight model, benefiting from unified modeling with deformable sampling. Finally, quantization-aware training experiments show that configurations with 8-bit or 6-bit weights and 8-bit activations achieve validation PSNR values approximately 0.2 dB below the FP32 baseline, whereas reducing activation precision to 4 bits leads to a substantial performance decrease. Overall, these results demonstrate that the unified framework achieves a favorable balance among reconstruction quality, computational efficiency, and hardware deployment potential.