Enhancing Electric Power System Reliability through Advanced Data Quality Assurance with PowerDataQ Framework

In the case of big data implementations in electric power industry, there is increasing need for establishing good frameworks for the quality and soundness of the big data that feed key processes and initiatives. As a new and unique system, PowerDataQ has been tailored exclusively for evaluation of the quality of the numeric values associated with the electric power. It uses advanced machine learning algorithms and the most complex techniques used in data mining to address issues to do with accuracy, consistency, completeness and timeliness of data. PowerDataQ has been structured to have a multilayered architecture with an embedded real-time data validation and anomaly detection and data cleaning modules to deal with issues that are characteristic of big and heterogeneous data coming from many sources. There is emphasis on domain-specific ontologies for improving semantic comprehension of quality metrics, which in turn are made more context specific. Heavily supported by real-world experiments, the good performance of PowerDataQ is shown in terms of catching data quality problems and enhancing the accuracy and robustness of following power system studies.