DataSentinel X: A Framework for Data Trust Assessment, Anomaly Detection, and Security Monitoring in Data Engineering Systems
Abstract Data reliability is an important concern for organizations that depend on information systems for daily operations and decision making. Problems such as missing records, delayed updates, unexpected schema changes, and suspicious access activities can reduce confidence in datasets. This research presents DataSentinel X, a framework designed to combine data quality assessment, security monitoring, anomaly detection, and incident investigation in a single system. The framework evaluates dataset trust using five dimensions, integrity, freshness, schema stability, access security, and lineage confidence. These dimensions are combined through a weighted scoring method to produce an overall trust score. The system was developed using Python, FastAPI, SQLite, React, and TypeScript. Experimental evaluation was conducted using controlled synthetic events. In a small detection experiment involving 14 labeled events, the rule based detection engine achieved 75% precision and 75% recall. Additional experiments examined trust score sensitivity, duplicate event processing, and scoring method comparisons. The findings provide preliminary evidence of consistent behavior under selected synthetic conditions. However, further testing with larger datasets and real operational environments is required. Research type: Independent research manuscript, preprint. Research area: Data Engineering, Data Quality, Cybersecurity, Data Observability. Source code: https://github.com/Faizan-AiEngineer/datasentinel-x Research limitations: The evaluation uses synthetic data and deterministic detection rules. The manuscript has not undergone peer review.
Authors
- Faizan Akhtar
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-15
- DOI
- https://doi.org/10.5281/zenodo.24246076
- Primary Topic
- Data Quality and Management
- Type
- preprint