Instance-wise Joint Feature and Expert Decision Acquisition for Classification

In many real–world applications, such as medical diagnosis, financial decision–making, and AI–driven content moderation, obtaining features and expert inputs is both time– consuming and expensive. To address these challenges, we present a supervised learning framework for sequential feature and expert decision acquisition. The objective of this framework, which operates in two phases, is to assign labels to each instance while reducing the cost of acquiring features and requesting experts’ decisions. The process begins with an initial belief about the label of a data instance, and at each step, there is a choice to either stop or continue acquiring features. Once feature acquisition terminates, the framework either assigns a label, or switches to expert decision acquisition, both based on the acquired features. The framework’s performance is evaluated using thirteen publicly available datasets, showing improvements in accuracy while acquiring fewer features and expert decisions on average than baselines. Its generalizability across expert types is demonstrated through experiments using standard supervised learning models, widely–used ensemble methods, and neural networks as experts, highlighting its broad applicability across domains.