Data Science Masters Thesis :Building Bridges in Research: Integrating Provenance and Data Management in Virtual Research Environments.

Machine learning workflows in modern scientific research are often fragmented acrossmultiple tools and environments, resulting in scattered or incomplete metadata.This fragmentation makes it impracticable to determine how models were trained,which datasets were utilized, and what decisions impacted experimental results.As a result, transparency, reproducibility, and auditability deteriorate, limiting thelong-term reliability and reusable nature of machine learning research.This thesis examines into how semantic provenance techniques might be includedinto Virtual Research Environments (VREs) in order to address these problems. Bydefining key metadata needs across the machine learning lifecycle, we create a formalmapping to community standards like as FAIR, FAIR4ML, PROV-O, Croissant, andMLSea. Based on this basis, we suggest a strategy for integrating diverse metadatasources into a semantically rich representation.This approach is implemented within a modular framework that combines Jupyter-Hub, GitHub, DBRepo, and Invenio, with MLflow support for semi-automatedmetadata gathering. Key aspects including dataset provenance, training setups, run-time environment, and justification information are exported in machine-readableforms (JSON-LD and RDF/XML). A Streamlit-based dashboard makes it easier tovisualize, compare experiment runs, and do SPARQL-based provenance queries.

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2025-01-01
DOI
https://doi.org/10.5281/zenodo.16890558
Primary Topic
Theoretical and Computational Physics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Data Science Masters Thesis :Building Bridges in Research: Integrating Provenance and Data Management in Virtual Research Environments.

Zenodo (CERN European Organization for Nuclear Research)
Theoretical and Computational Physics
article

Data Science Masters Thesis :Building Bridges in Research: Integrating Provenance and Data Management in Virtual Research Environments.

article en

Abstract

Machine learning workflows in modern scientific research are often fragmented acrossmultiple tools and environments, resulting in scattered or incomplete metadata.This fragmentation makes it impracticable to determine how models were trained,which datasets were utilized, and what decisions impacted experimental results.As a result, transparency, reproducibility, and auditability deteriorate, limiting thelong-term reliability and reusable nature of machine learning research.This thesis examines into how semantic provenance techniques might be includedinto Virtual Research Environments (VREs) in order to address these problems. Bydefining key metadata needs across the machine learning lifecycle, we create a formalmapping to community standards like as FAIR, FAIR4ML, PROV-O, Croissant, andMLSea. Based on this basis, we suggest a strategy for integrating diverse metadatasources into a semantically rich representation.This approach is implemented within a modular framework that combines Jupyter-Hub, GitHub, DBRepo, and Invenio, with MLflow support for semi-automatedmetadata gathering. Key aspects including dataset provenance, training setups, run-time environment, and justification information are exported in machine-readableforms (JSON-LD and RDF/XML). A Streamlit-based dashboard makes it easier tovisualize, compare experiment runs, and do SPARQL-based provenance queries.

Zenodo (CERN European Organization for Nuclear Research)
Industry, innovation and infrastructure
Openalex Percentile: Top 59%
Theoretical and Computational Physics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.