An Ontology-Based Framework for Automated Dataset Validation

With the rapid development of artificial intelligence, the importance of data quality has become increasingly prominent. The existing data quality assessment methods, such as accuracy, completeness, consistency, effectiveness, timeliness, uniqueness, intelligibility assessment and data quality indicators, lack user perspective and may not accurately reflect the applicability and practicability of the data in practical application, resulting in the evaluation results not matching with the specific needs and expectations of users. In this paper, we propose OmniSense, a framework of data quality assessment based on domain model and knowledge inference. Using domain model as a bridge, the link between the original data set and the user's needs is established, and the automatic data quality assessment from the user's perspective is realized. This approach enables the developer to estimate the support of the dataset for the implementation of the function before starting training, thereby reducing the cost of retraining due to the inapplicability of the dataset.