The Proposal and Validation of a Distributed Real-Time Data Management Framework Based on Edge Computing with OPC Unified Architecture and Kafka

With the advent of Industry 4.0, the manufacturing industry is facing unprecedented data challenges. Sensors, PLCs, and various types of automation equipment in smart factories continue to generate massive amounts of heterogeneous data, but existing systems generally have bottlenecks in data collection standardization, real-time processing capabilities, and system scalability, which make it difficult to meet the needs of efficient collaboration and dynamic decision making. This study proposes a multi-level industrial data processing framework based on edge computing that aims to improve the response speed and processing ability of manufacturing sites to data and to realize real-time decision making and lean management of intelligent manufacturing. At the edge layer, the OPC UA (OPC Unified Architecture) protocol is used to realize the standardized collection of heterogeneous equipment data, and a lightweight edge-computing algorithm is designed to complete the analysis and processing of data so as to realize a visualization of the manufacturing process and the inventory in a production workshop. In the storage layer, Apache Kafka is used to implement efficient data stream processing and improve the throughput and scalability of the system. The test results show that compared with the traditional workshop, the framework has excellent performance in improving the system throughput capacity and real-time response speed, can effectively support production process judgment and status analysis on the edge side, and can realize the real-time monitoring and management of the entire manufacturing workshop. This research provides a practical solution for the industrial data management system, not only helping enterprises improve the transparency level of manufacturing sites and the efficiency of resource scheduling but also providing a practical basis for further research on industrial data processing under the “edge-cloud collaboration” architecture in the academic community.