Evaluation of impact of image augmentation techniques on two tasks: Window detection and window states detection

• Nine distinct image augmentation techniques—Brightness, Contrast, Perspective, Rotate, Scale, Shear, Translate, Blurring, and a combined application termed "A Sum of Techniques"—were applied to two tasks: window detection and window state detection. • These techniques were demonstrated using three deep learning-based object detection methods: Faster R-CNN, YOLO, and R-FCN, with five widely-used architectures: ResNet-50, ResNet-101, MobileNetV3, EfficientNetV2, and InceptionV3. • The results highlight that the most effective augmentation techniques vary not only across different architectures but also between different performance metrics. • It was found that integrating multiple augmentation techniques is not always more effective than using individual methods alone, indicating that augmentation strategies must be specifically tailored to each architecture. The detection of windows and their states from building façade images is crucial for applications such as digital twins and building performance simulations. Deep learning using object detection algorithms can achieve this with reliable accuracy, but its success depends on diverse and accurate datasets. While image augmentation can enhance dataset diversity, its impact on detecting windows and their states from building façade images remains understudied. This research introduces nine augmentation techniques: brightness, hue, saturation, contrast, scale, translation, cutout, weather, and combinations thereof. These techniques are evaluated using Faster R-CNN, YOLOv8, and R-FCN across various architectures, including ResNet-101, ResNet-152, MobileNetV3, EfficientNetV2, and InceptionV3, with AP50 and IoU as performance metrics. Results show that the most effective augmentation techniques vary by architecture and performance metric, highlighting the need for tailored augmentation strategies. Combining multiple techniques does not always outperform using individual methods, suggesting that augmentation strategies should be specifically adapted to each architecture. These findings indicate that the proposed augmentation techniques can improve window and window state detection and may also extend to related tasks such as window type detection and building age classification.