From COACH to COACH+: Automating output checking with human-in-the-loop

This paper presents COACH+ (COmputer-Assisted Output CHecking with Human-in-the-loop), an extension of COACH by (Slokom et al. COACH: computer Assisted output CHecking with human-in-the-loop. U N Econ Comm Eur Conf Eur Stat) that integrates machine learning models with human expertise to automate output-checking processes. Building upon the foundation of COACH, COACH+ improves the collaborative capabilities between automated algorithms and human chequers. The primary objective remains to facilitate the assessment of outputs generated by researchers to determine their suitability for public release. We first provide an overview of our initial iteration of COACH, which revolutionised output checking by combining machine learning models with human expertise. Then, we present the advancements introduced in COACH+, which are divided into two main aspects. In the backend, we have integrated a secondary machine learning algorithm, a convolutional neural network, designed to analyze images such as plots and determine their safety for release. In the front end, we have minimised reliance on human chequers. Unlike the previous COACH version, where human intervention was required to input certain values before making predictions, COACH+ now allows human chequers to upload Excel files and figures directly.