Democratizing Scientific Machine Learning through Expert-Encoded Workflow Templates
The increasing complexity of AI/ML software stacks and diverse accelerator hardware on High-Performance Computing (HPC) systems creates a significant productivity barrier for domain scientists. Configuring optimized, reproducible AI workflows often requires specialized systems expertise that researchers may not possess. To tackle this, we introduce Drona, a workflow engine that encapsulates expert knowledge into reusable, domain-specific templates. Drona guides researchers through complex setups via a simple web interface, dynamically generating optimized and transparent workflows. We demonstrate Drona's power and flexibility through an in-depth analysis of a Hugging Face template that automates the model inference and fine-tuning lifecycle, from proactive resource validation and distributed training configuration to seamless reproducibility. Drona lowers the barrier to entry for advanced AI/ML on HPC, improves scientific reproducibility, and empowers a wider community of researchers to leverage accelerated computing.
Institutions
- Mitchell Institute (US)
- Texas A&M University (US)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2025-10-26
- DOI
- https://doi.org/10.5281/zenodo.17450204
- Primary Topic
- Scientific Computing and Data Management
- Type
- article
- Field-Weighted Citation Impact
- 0.00