Optimizing Data Analytics Workflows Through User-Driven Experimentation: Progress and Updates

This paper provides a progress update to the doctoral symposium paper accepted at CAIN 2024 [1], which introduced the Experimentation Engine for optimizing data analytics workflows through user-driven experimentation. Over the past year, the framework has been significantly refined to better address diverse user needs by adapting the experimentation process for different user types, including domain experts and data scientists. The updated Experimentation Engine emphasizes continuous user involvement throughout the experimentation process, facilitating seamless interaction and iterative refinement. A demonstration video of the initial prototype, showcasing its capabilities, is available at https://doi.org/10.5281/zenodo.14288226. These enhancements aim to further streamline the experimentation process, providing accurate, meaningful, and trustworthy results while dynamically adapting to varying user requirements and system constraints. The key updates and lessons learned during this process are discussed in this paper.