Semi-supervised Learning to Defer Algorithm for Lung Disease Diagnosis

Recent research highlights the advantages of leveraging complementary strengths of both human expert and model in decision-making processes. Learning to Defer(L2D) is proposed to build a system consisting of both and improve the performance of expert or model alone. For each instance, L2D algorithms search for the optimal decision-maker between model and human expert to improve the human-ai system accuracy. However, most previous work is based on the assumption that human predictions are available for every instance, which is unrealistic in practical scenarios due to the high expense of manual annotation. To address this, we consider L2D problem where human predictions are available only for a subset of image. We propose a multistep framework for this scenario and integrate a consistency regularization loss to learn expert capability from limited human predictions. The consistency regularization loss is designed to encourage the expert to learn intrinsic information and make similar predictions for similar instances. Empirical validation on real-world data from airspace opacity diagnosis shows that the proposed framework not only outperforms several competitive baselines but also enhances the performance of various L2D algorithms under constraints of minimal human predictions.