A General-Purpose Middleware System for Edge-side Data Processing

Edge computing, including fog computing, processes data measured by their sensing systems. Such data often contain noise, losses, and non-canonical representations, necessitating preliminary processing to enhance data quality and convert it into certain formats. While data processing tasks have been centralized in data centers so far, edge computing, including fog computing, decentralizes these tasks to edge nodes. This paper presents a data processing framework for edge computing. The contribution of the framework is to adapt existing popular data processing approaches, originally designed for use in data centers consisting of high-performance servers connected through wide-band networks, for use in edge computing. The framework hides network configurations and node failures, enabling developers to focus solely on data processing tasks without worrying about underlying infrastructural complexities. This paper details the framework’s architecture, discusses its integration with edge and fog computing environments, and evaluates its performance, demonstrating its superior efficiency and effectiveness in handling edge-based data processing challenges.