Proposed Metadata Management Model for Data Lakehouse
The exponential growth in the amount and diversity of data organizations generate has presented significant challenges in data storage, processing, and management. Data Lakehouses, a hybrid solution that combines the capabilities of Data Warehouses and Data Lakes, have been developed to address these challenges. However, metadata management in these environments is crucial to prevent data from becoming inaccessible and unwieldy. We propose a metadata management model for a Data Lakehouse based on a functional architecture that integrates functional metadata features for a Data Lake with business and maintenance metadata for a Data Warehouse. We include the results of an initial implementation and discuss its impact on data accessibility and governance.
Authors
- Ania Cravero (ORCID: https://orcid.org/0000-0003-0883-7254)
- Pablo Rain
Institutions
- Universidad de La Frontera (CL)
Publication Details
- Published
- 2024-10-28
- DOI
- https://doi.org/10.1109/sccc63879.2024.10767658
- Primary Topic
- Data Quality and Management
- Type
- article
- Field-Weighted Citation Impact
- 0.00