mlrl-testbed: A command line utility for tabular machine learning experiments

The Python package mlrl-testbed provides a command line utility designed to support researchers in conducting reproducible machine learning experiments.It offers a straightforward, easily configurable, and extensible workflow that supports the full experimental lifecycle:• Loading a dataset.• Splitting it into training and test sets.• Training one or more models.• Evaluating the models' predictive performance.• Saving experimental results to output files.By default, mlrl-testbed executes a single experiment using a given dataset and parameter setting.However, it can also be operated in the following modes:• Batch mode: Allows running multiple independent experiments with varying datasets and parameter settings.Installing the optional package mlrl-testbed-slurm enables experiments to be run via the Slurm Workload Manager 1 .• Read mode: Allows inspecting the results of previous experiments and saving them to new output files.When viewing results obtained in batch mode, results are automatically aggregated across different experiments.• Run mode: Allows re-running previously conducted experiments with the option to partly override their configuration.Experiments for which results are already available can be skipped.Originally developed to support work on the BOOMER algorithm (Rapp et al., 2021;Rapp, 2021), mlrl-testbed has since evolved into a standalone utility for empirical machine learning studies.

Authors

Publication Details

Journal
The Journal of Open Source Software
Published
2026-11-13
DOI
https://doi.org/10.21105/joss.10609
Primary Topic
Machine Learning and Data Classification
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

mlrl-testbed: A command line utility for tabular machine learning experiments

Michael T. Rapp
1 links
The Journal of Open Source Software
Machine Learning and Data Classification
article

mlrl-testbed: A command line utility for tabular machine learning experiments

Michael T. Rapp
article en

Abstract

The Python package mlrl-testbed provides a command line utility designed to support researchers in conducting reproducible machine learning experiments.It offers a straightforward, easily configurable, and extensible workflow that supports the full experimental lifecycle:• Loading a dataset.• Splitting it into training and test sets.• Training one or more models.• Evaluating the models' predictive performance.• Saving experimental results to output files.By default, mlrl-testbed executes a single experiment using a given dataset and parameter setting.However, it can also be operated in the following modes:• Batch mode: Allows running multiple independent experiments with varying datasets and parameter settings.Installing the optional package mlrl-testbed-slurm enables experiments to be run via the Slurm Workload Manager 1 .• Read mode: Allows inspecting the results of previous experiments and saving them to new output files.When viewing results obtained in batch mode, results are automatically aggregated across different experiments.• Run mode: Allows re-running previously conducted experiments with the option to partly override their configuration.Experiments for which results are already available can be skipped.Originally developed to support work on the BOOMER algorithm (Rapp et al., 2021;Rapp, 2021), mlrl-testbed has since evolved into a standalone utility for empirical machine learning studies.

The Journal of Open Source SoftwareVol. 11(127)
Machine Learning and Data Classification
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon