Comparing Human and Automatic Judging for Precision Aerobatics

There is growing interest in the use of flight data recorders as a training aid for aerobatic pilots and judges in order to make reliable judging more widely available, make more effective use of flight hours and reduce ambiguity in the process. This paper describes an assimilation of data from Acrowrx, which produces a flight logger, plotting tool and data sharing infrastructure for full-scale aerobatics, and Flight Coach, which focuses on open source flight data processing for model-scale and full-scale aerobatics. The most recent development from Flight Coach is an automatic judging system that calculates scores for aerobatic flights following a process mirroring that used by human judges. In this work the Flight Coach automatic judging system is applied to flight data shared in the Acrowrx system and the results are compared to scores submitted by expert human judges. Cases are presented where the system is working well, where the human judge was in error and where the judging process can be improved. Comparisons such as these are used in the development of the judging process and will be a major part of promoting pilots' and competition organisers' confidence in the system.