ALF: An Automated Low-code Flexible Framework for Data Workflows

The demand for data science innovation is growing across all sectors. There is no shortage of new big-data applications and tools for deploying data science projects. Data scientists typically create data workflows manually or by writing scripts to automate parts of the process. However, basic scripting is insufficient for large-scale projects and performance optimizations. This poses great technical challenges for data scientists. In this paper, we present the Automated Low-code Flexible (ALF) Framework, designed to remove technical barriers and streamline data workflows. ALF also provides a holistic view of workflow construction and end-to-end workflow management. Its design is supported by a case study, demonstrating ALF’s ability to integrate a diverse array of applications, alongside performance enhancement tools and workflow inspection capabilities.