Quality of Experience Based Dynamic Path Serverless Data Pipelines in Edge/Fog Computing

ABSTRACT Recently, the usage of Internet of Things (IoT) devices has been increasing drastically and thus producing huge amounts of data. To handle this big data, researchers have proposed hybrid model SDP (Serverless data pipeline) approaches, supporting intermediate data processing across the fog topology. However, these predetermined path SDPs never consider the task expectation requirements and current node capabilities, and data flows through fixed paths. But in shared network environments like fog and cloud, we cannot expect that the resources will always be reserved for the pipeline. This predetermined SDP's behavior ultimately produces a low Quality of Experience (QoE) for the user regarding pipeline performance. This paper proposes a QoE‐based dynamic path SDP, which can dynamically reroute its data path on the nodes and offer better QoE. For the node selection, we used the hierarchical fuzzy‐based placement strategy and demonstrated the approach with an SDP image processing application. The designed QoE‐based SDP outperforms the Predetermined SDP in handling real‐time data without any pipeline interrupts. The experiment results showed that the proposed method is almost free from the bottleneck effect with a loss of 2% packet drops. In Predetermined SDP, we found 74% of packet loss when the load on the SDP is significant.