Adaptive Framework for Comprehensive Quality Assessment in Unstructured Big Data

The current growth and evolution have led to the emergence of structures such as unstructured big data in different fields creating a need and demand for effective quality assessment models specifically designed for big data. This work presents a new Big Data Quality Assessment Model (BDQAM) that is tailored for use with unstructured data since they are highly diverse, a have no prior planned structure, and semantics are complex. The BDQAM, therefore, uses cutting edge machine learning algorithms and natural language processing tools to perform a real time assessment and validation on big data with the view of determining the quality of the data accumulated. The quality assessment is based on key quality dimensions and the assessment of accuracy, completeness, consistency and timeliness; the model is designed to learn from experience and improve the results. The effectiveness of the proposed model is affirmed by testing on a variety of large-scale unstructured datasets across different domains together with substantial enhancements in precision and recall measurements in contrast with the conventional quality assessment methodologies. The study results enhance practice in big data analysis by offering a solution that can be applied and is effective in checking the validity of Big Data inside decision-making processes.