AstroFlow: A customizable workflow management system for astronomical data production and case study of EP-WXT

In the era of time-domain astronomy, automated and efficient data processing is a crucial factor in ensuring astronomical scientific output. With the international collaboration trend in large-scale sky survey projects, data processing algorithms developed through multi-institutional collaboration face challenges in system integration, scheduling capability, extensibility, and traceability. In this work, we employ containerized execution modules to resolve dependency issues in multi-institutional software integration, implement complex workflow orchestration through message queue control modules, including cyclic dependencies, dynamic parallel relationships, and resource/priority-based scheduling. The system also non-intrusively implements the International Virtual Observatory Alliance (IVOA) provenance data model through aspect injection mechanisms, ensuring traceability and reproducibility of data products. In Einstein Probe (EP) data processing, the system has processed over 20,000 general observations, 170,000 FXT observations, and 21,000 VHF and Beidou alerts, supported over 300 agile algorithm updates, and detected more than 17,000 sources and nearly 200 transient events. The system demonstrates excellent generality and portability, smoothly migrating from the single-module LEIA to the 12-module EP, and supporting data fusion for new payload FXT and multiple data transmission links. This workflow management system provides a flexible, extensible, and standardized data processing framework for future large-scale astronomical observation projects, effectively addressing key issues such as multi-institutional collaboration, complex workflow orchestration, and scientific data reproducibility.