Evaluation and Learning with Multiple Inaccurate True Targets

In many real-world machine learning (ML) scenarios, obtaining accurate true targets (ATTs) for evaluation and learning is difficult, expensive, or fundamentally infeasible. This article proposes a unified scientific paradigm—evaluation and learning with multiple inaccurate true targets (MIATTs)—that addresses this challenge by integrating the fundamental principles of two recently proposed frameworks: Logical Assessment Formula (LAF) and Undefinable True Target Learning (UTTL). Both LAF and UTTL operate under a relaxed but shared assumption that the true target for a given ML task is not assumed to exist as a well-defined object in the real world, which motivates us to define MIATTs as a collection of weak yet partially informative targets, each capturing a different aspect of the underlying true target. Building on this foundation, we present a comprehensive theoretical framework, which encompasses MIATTs generation, model construction, metric formulation, and model optimization, for formalizing the evaluation and learning of predictive models with MIATTs. The article offers a principled and practical alternative for ML scenarios marked by true target ambiguity, providing a viable path where conventional ATT-based methods prove inadequate or inapplicable.