DataEval: enhancing end-to-end AI development through comprehensive data analysis

In high-stakes environments, such as defense applications where systems must be responsible, equitable, traceable, reliable, and governable, mitigating risks from AI systems is critical for adoption. AI models can encounter conditions leading to model fault scenarios, resulting in system or performance failures. Many fault conditions stem from data issues, such as insufficient data, poor data quality, bias, and dataset shifts. Establishing trustworthy and reliable AI systems requires the right resources to mitigate the data problems prior to deployment. DataEval, a Python-based library developed under the Joint AI Test Infrastructure Capability (JATIC) program, provides metrics that enable AI/ML model developers and Test and Evaluation (T&E) engineers the ability to verify that performance requirements can be met by the data used for developing models, and the ability to evaluate and monitor data for sources of possible fault conditions in order to mitigate potential risks. DataEval facilitates this through three key functionalities. First, model-agnostic metrics establish whether datasets meet performance requirements for operational AI models or not by assessing dataset sufficiency for training and testing (i.e., determining whether the dataset contains enough diverse, representative samples to meet performance requirements). Second, model-specific metrics predict the impact of additional data on model performance and support decision-making in model selection. Third, post-deployment monitoring metrics enable engineers to detect when an operational environment deviates from training conditions, evaluate the impact on model performance, and determine when retraining is necessary. DataEval identifies potential failure modes early by providing metrics throughout the machine learning lifecycle that identify data issues (such as dataset anomalies and drift) at each stage and supports the development of robust, trustworthy AI systems to enable safe deployment of AI systems in high-stakes environments, where understanding and mitigating operational risks is paramount. The official implementation can be found at: https://github.com/aria-ml/dataeval.