AI-POWERED ETL WORKFLOW ORCHESTRATION WITH SELFADJUSTING DATA TRANSFORMATIONS
The increasing complexity of e-commerce data environments provides a requirement for effective Extract,Transform, and Load (ETL) processes for enhancing data-based decision-making. This work introduces anAI-optimized ETL workflow management framework with adaptively self-tuning data transformations forenhancing efficiency, scalability, and data quality. Using machine learning and dynamic algorithms, thesuggested framework optimizes extraction, transformation, and loading functions adaptively. It enables realtime data processing, which offers seamless integration with changing e-commerce requirements. Researchemphasizes the importance of AI to provide automated data transformation logic, minimize human touch,and improve ETL efficiency. The intelligent features are intelligent schema discovery, auto anomalyhandling, and dynamic process adjustment, which guarantee enhanced data consistency and reliability.Performance analysis confirms that the AI-based ETL system maximizes processing time greatly withoutcompromising data integrity. The results indicate that AI-powered orchestration of ETL pipelines canimprove operational efficiency, facilitate large-scale analytics, and help companies gain actionable insightsbetter. Future work can investigate adding federated learning to further enhance distributed data processingand strengthen security in multi-source data systems.
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2025-03-23
- DOI
- https://doi.org/10.5281/zenodo.15071366
- Primary Topic
- Big Data and Digital Economy
- Type
- article
- Field-Weighted Citation Impact
- 0.00