Democratizing Scientific Machine Learning through Expert-Encoded Workflow Templates

The increasing complexity of AI/ML software stacks and diverse accelerator hardware on High-Performance Computing (HPC) systems creates a significant productivity barrier for domain scientists. Configuring optimized, reproducible AI workflows often requires specialized systems expertise that researchers may not possess. To tackle this, we introduce Drona, a workflow engine that encapsulates expert knowledge into reusable, domain-specific templates. Drona guides researchers through complex setups via a simple web interface, dynamically generating optimized and transparent workflows. We demonstrate Drona's power and flexibility through an in-depth analysis of a Hugging Face template that automates the model inference and fine-tuning lifecycle, from proactive resource validation and distributed training configuration to seamless reproducibility. Drona lowers the barrier to entry for advanced AI/ML on HPC, improves scientific reproducibility, and empowers a wider community of researchers to leverage accelerated computing.

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2025-10-26
DOI
https://doi.org/10.5281/zenodo.17450204
Primary Topic
Scientific Computing and Data Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Democratizing Scientific Machine Learning through Expert-Encoded Workflow Templates

Zenodo (CERN European Organization for Nuclear Research)
Scientific Computing and Data Management
article

Democratizing Scientific Machine Learning through Expert-Encoded Workflow Templates

article en

Abstract

The increasing complexity of AI/ML software stacks and diverse accelerator hardware on High-Performance Computing (HPC) systems creates a significant productivity barrier for domain scientists. Configuring optimized, reproducible AI workflows often requires specialized systems expertise that researchers may not possess. To tackle this, we introduce Drona, a workflow engine that encapsulates expert knowledge into reusable, domain-specific templates. Drona guides researchers through complex setups via a simple web interface, dynamically generating optimized and transparent workflows. We demonstrate Drona's power and flexibility through an in-depth analysis of a Hugging Face template that automates the model inference and fine-tuning lifecycle, from proactive resource validation and distributed training configuration to seamless reproducibility. Drona lowers the barrier to entry for advanced AI/ML on HPC, improves scientific reproducibility, and empowers a wider community of researchers to leverage accelerated computing.

Zenodo (CERN European Organization for Nuclear Research)
Mitchell Institute (US), Texas A&M University (US)
Decent work and economic growth
Openalex Percentile: Top 26%
Scientific Computing and Data Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Democratizing Scientific Machine Learning through Expert-Encoded Workflow Templates · Zenodo (CERN European Organization for Nuclear Research) (2025) | TGRS Research Map | TGRS