Evaluating performance and generalizability of Learning from Demonstration for the harvesting of apples & pears

Automating agricultural tasks is challenging because of the large amount of variation in the environment and variation in tasks. Learning from Demonstration (LfD) is a promising method that allows robots to adjust to new environments and tasks easily through an easy training procedure performed by the end-user. The novelty of this work is that it is the first time LfD was tested in the orchard on harvesting fruits. We performed apple and pear harvesting with the LfD-method trained on five, ten, twenty and forty demonstrations to evaluate the effect of increasing the number of demonstrations. In order to focus on the performance of the LfD algorithm, we defined the grasp pose of each execution by positioning the end-effector in the correct pose. Based on the performance in the orchard, we did an additional generalization experiment to gain insights into the effect of fruit pose on the performance of both harvesting tasks. Apple harvesting was performed best by training with 20 demonstrations and resulted in 90% of the fruits grasped and 65% picked. Pear harvesting was performed best by training with 40 demonstrations and resulted in 65% of the fruits grasped and 50% picked. In the generalization experiment, it was found that additional demonstrations allowed for better performance in a broader area. Through the combination of the presented analyses, the training process can be improved by analysing the trained model before testing in the orchard, reducing the number of fruits damaged in testing. • Variation makes automating agricultural task challenging. • Learning from Demonstration can enable robots to deal with variation. • First evaluation of Learning from Demonstration on apple and pear harvesting. • 65% of apples and 50% of pears were picked, similar to the state of art robots. • To learn both tasks, at most three hours of demonstrations were required.