A methodology for image reproduction in virtual environments applied to an agricultural assistance function

• Methodology for realistic image data generation in VR environments • Process of parameter identification for real camera characteristics and device positioning • Process of image reconstruction with camera model in agricultural VR environment • Evaluation of the methodology by comparing the real image and virtually generated image With the digitalization of mobile work machines, camera-based monitoring of machine conditions is becoming increasingly important. In the present application of an agricultural cultivator, the wear condition of its tines is monitored using a camera system and intelligent image evaluation. Extensive test drives are needed to develop such systems. This work proposes a methodology to reproduce real image data in high-resolution virtual environments to enable realistic virtual testing processes. The methodology consists of subprocesses for parameter identification and image reconstruction. In the context of parameter identification, accurate camera modelling is essential, requiring parameterization based on real camera calibration data, including image distortion. The subsequent image reconstruction process covers the fine-tuning of the virtual environment and its camera model, as well as the recording of the virtually generated images. After describing the methodology, the processes are evaluated using the example of the tool monitoring system for the above-mentioned cultivator. A comparison of the virtually generated images with the images from the real camera shows a good match of the images. Visible deviations are due to usage-related deformations of the real system. However, the deviations are small and the image quality high enough that the image recognition of the monitoring system can identify the cultivator tines and their elements. The evaluation shows that the potential for improving the methodology lies mainly in the automation of its process steps. It also highlights the importance of the robustness of tool monitoring systems with regard to geometric changes to the overall system.