Big Data Analytics in Industry 4.0: Automating Production Metrics With an Edge-Based Architecture

This paper presents the development and evaluation of a Big Data Analytics (BDA) platform designed for Industry 4.0 applications, focusing on the integration, processing, and visualization of data at the edge. The platform integrates tools such as Apache Spark, Airflow, MinIO, Airbyte, and Streamlit, to facilitate automated data collection and report generation. Deployed in a real manufacturing environment in São Paulo, Brazil, the solution aims to minimize data latency by avoiding cloud transmission and leveraging edge computing. Automated daily reports have been generated since May 2024, improving decision-making and reducing human error associated with manual data integration. This work provides a practical contribution to industrial data integration, highlighting the benefits of edge-based data processing, automation, and visualization to optimize production efficiency and decision-making processes.