Innovative Data Modeling Concepts for Big Data Analytics: Probabilistic Cardinality and Replicability Notations

The evolving practice of big data analytics encompasses the aggregation of data from multiple sources, with the imperative of delivering metrics and reports that maintain a high standard of reliability and consistency. As stakeholders may interpretat the data and associated metrics differently throughout the process, this often have to make assumptions, which can lead to inconsistencies in metrics aggregation. Our work addresses the limitation of traditional data modeling methods, which often fail to capture the nuances of the relationships among various data sources. We propose two conceptual data modeling concepts: probabilistic cardinality and metric replicability along with definitions, notation and illustrative examples, as well as the general big data analytics framework that is used for discussing the role and implementation of the concepts. Application of proposed concepts is illustrated through two applied case studies highlighting variety of ways in which they reduce risk of inconsistent aggregation and reporting of metrics.