Utility Aware Collaborative Edge Inference for Fog Computing Networks: A Neural Bandits Approach

Resource-constrained artificial intelligence of Things (AIoT) devices face significant challenges in performing complex deep neural network (DNN) inference. Collaborative edge inference within fog computing networks (FCNs) offers a promising solution by enabling these devices to offload parts of their computations to nearby idle devices. In this work, we propose the group neural upper confidence bounds (G-NeuralUCB) approach for optimizing offloading decisions and DNN partitioning strategies, addressing the challenges posed by the dynamic nature of network conditions and system parameters. Our approach models intricate utility functions that integrate latency and energy consumption, while dynamically balancing exploration and exploitation through upper confidence bounds. Extensive experiments within an FCN show that G-NeuralUCB significantly outperforms baseline methods in cumulative regret and utility, emphasizing its adaptability to changing network conditions. The results validate G-NeuralUCB as an effective solution for real-time optimization in collaborative edge inference within FCNs.