Performance Evaluation and Improvement for RGB‐D Cameras on High‐Throughput Phenotyping Robots

ABSTRACT RGB‐D cameras are widely used in indoor robots. However, their ranging capability for agricultural robots under natural lighting still needs to be evaluated. Especially in the field of robotics‐based high‐throughput crop phenotyping, the measurement accuracy of phenotypic parameters is deeply related to the ranging performances of RGB‐D cameras. In this paper, we propose a depth‐ranging evaluation framework and an online ranging compensation strategy for RGB‐D cameras on phenotyping robots. The goal is to acquire high‐quality depth‐ranging performances for plant phenotyping tasks. First, we evaluate ranging performances of RealSense D435i and Kinect V2 under typical phenotyping scenes with different lighting conditions, verify their feasibility on different maize organ observations in different growth periods, and give the optimal observation ranging areas. Second, we employ image brightness to reflect the lighting situations, and propose a novel ranging compensation strategy to decrease the lighting influences in real‐time. The results of sufficient field experiments show that RealSense D435i has better ranging performances than Kinect V2 for crop phenotyping, especially for open‐field, in‐row, and close‐range observations. The optimal ranging area of RealSense D435i is within a region of [0.16–1.2] m. However, Kinect V2 is not suitable for field phenotyping robots due to significant interference from natural sunlight, limited measurement range, and instability in depth measurements under outdoor conditions. In addition, we also verify that our online depth error compensation strategy can effectively reduce the influences of lighting intensity and target distance on the depth ranging of RGB‐D cameras. Although we test and verify our ranging evaluation framework and ranging error compensation strategy with two old‐fashion cameras, the framework and strategy are generic and applicable to other new RGB‐D cameras.