Using synthetic images to train neural networks for tramline detection

In recent years, the ability to generate synthetic image data and the quality of such data have significantly improved. This advancement facilitates the training of neural networks, which ­require large amounts of training data. The ability to train neural networks more quickly and easily provides significant benefits for precision farming applications, particularly in auto-steering situations where Global Navigation Satellite System (GNSS) cannot be utilized. In these non-GNSS scenarios, a neural network combined with a single camera can act as an effective alternative to GNSS. This is especially useful in situations where GNSS signals are unavailable, e. g., because the initial seeding was performed without GNSS-based equipment, resulting in missing positional data, or due to signal obstruction caused by environmental factors, like overhead agrivoltaics systems. This paper explores the process of generating synthetic images using a game engine and a diffusion model. These synthetic images will be utilized to train a neural network for tramline detection. The neural network, when trained with real images, achieved a mean Intersection over Union (mIoU) of 81.7 %. Incorporating synthetic images into the training process increased the mIoU to 83.3 %, resulting in ­improved tramline detection.