Bypassing ISP: Transfer Learning-Based Object Detection on Raw Bayer Images with Faster R-CNN

Object detection models traditionally rely on RGB images, which are processed through computationally intensive image signal processing (ISP) pipelines to enhance visual quality. However, such pipelines can reduce the fidelity of raw image details, potentially affecting detection accuracy. This paper presents a novel transfer learning-based framework to enhance object detection performance directly on raw Bayer images. Our approach leverages a pre-trained Faster R-CNN model trained on RGB images, followed by selective fine-tuning of the last five layers of the ResNet-50 backbone while keeping the earlier layers frozen to adapt the model to the unique characteristics of Bayer images. Experimental results on the KITTI dataset and its generated Bayer dataset demonstrate that our method significantly outperforms traditional Faster R-CNN on Bayer images and achieves performance comparable to RGB-based models. This framework provides an effective solution for low-power edge computing by bypassing ISP and directly processing raw Bayer images without increasing model complexity.