A Simple Extension of Bayer Demosaicing for Lightweight Non-Bayer Demosaicing

Recent advancements in image sensor technology have introduced non-Bayer color filter arrays (CFAs), such as Quad (2 × 2), Nona (3 × 3), and Q×Q (4 × 4) patterns, which improve performance under low-light conditions compared to the traditional Bayer CFA. Although numerous deep learning models exist for Bayer demosaicing, directly applying them to non-Bayer CFAs often yields suboptimal results, despite identical input-output dimensions and differences only in pixel-level color arrangements. In this work, we identify <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pixel discontinuity</i>, a structural mismatch introduced by modern non-Bayer CFAs, as the key factor behind this performance gap. Then, based on our investigation, we propose a simple yet effective solution to mitigate it: inserting a pixel unshuffle layer at the input of existing Bayer demosaicing networks. This key operation spatially restructures the input, enabling seamless adaptation to non-Bayer patterns <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">without</i> altering the original network architecture. Our method significantly improves demosaicing performance with almost no increase in both network size and computational cost. Moreover, it generalizes across various CFA patterns, including Quad, Nona, and Q×Q, facilitating the reuse of Bayer-trained models for a wide range of sensor designs. Experimental results on synthetic and real non-Bayer data demonstrate the practical effectiveness of our approach, removing the need for custom model design for other non-Bayer CFA types.