D3MV: Detecting Deficient Data in Intelligent Software Systems via Model Verification

The integration of data-driven, intelligent components, particularly those based on Artificial Neural Network (ANN), is pivotal for enabling software systems to adapt to dynamic environments. However, the performance of these hybrid systems is critically dependent on the quality of their training data. Deficiencies in this data can propagate through the ANN, leading to violations of key system properties that are difficult to trace back to their root cause. To address this challenge, this paper introduces D3MV, a novel verification-based methodology for tracing system-level property violations back to specific deficient training data. Our approach involves constructing a unified system model (Adaptive Petri Net) that integrates traditional components with ANNs, extracting interpretable fuzzy rules from the trained network to bridge the semantic gap, and employing model checking for formal verification. When a property is violated, a novel inverse mapping technique leverages the implicated fuzzy rules to pinpoint the responsible data samples in the training set. Experimental validation using a manufacturing case study demonstrates that the method D3MV successfully identified deficient data causing property violations. After replacing 200 problematic data samples and updating the model, the revised system met all specified performance metrics. This preliminary result suggests that by improving data quality, our approach helps ensure system reliability.