Calibration of the omnidirectional vision system for robotics sorting system

Sorting objects is a relevant field in the robotics direction and includes a complex approach in order to meet the objectives. The important part of this system is a vision system. Vision system should provide a correct distance to obstacles to make the robot operate with them properly. In this paper is considered an omnidirectional vision system consisting of the fisheye camera and structured light. Towards other vision systems, an omnidirectional one has its own distinctive advantages as it can provide more information of the environment on the image snapshot. The advantage of the use of the structured light is its simple detection the image. However, before moving to the distance measurements the vision system must be calibrated. This paper proposed an improved extrinsic calibration of the vision system. The calibrated vision system is evaluated experimentally by using simulated and real data to demonstrate that the proposed calibration method is more robust and provides a higher measurement accuracy in comparison to other methods. Obstacle distances were obtained with a measurement error did not exceed 0.5% for synthetic data, did not exceed 1.6% for real data and computation time for the calibration process decreased by 12%. This paper also proposed a simulator which was useful during experiments part. By simulator it was possible to generate datasets of different vision system configurations to test the proposed calibration method and compare it with other methods before moving to real experiments. Built using Unity, the simulator integrates vision system, industrial robot, elements of the environment, and allows one to generate synthetic photo-realistic datasets with automatic ground truth annotations. The proposed simulator and supporting materials are available online: https://github.com/kholodilinivan/RoboticsSystemCalibration .