Stochastic Optimization Method for Multi-exit Deep Neural Networks for Edge Intelligence Applications

As a novel intelligent computing paradigm,edge intelligence can effectively enhance the response speed of intelligent inference tasks on embedded edge devices.Age of information(AoI),an important metric for measuring data freshness,is of great significance to the computing resource overhead and real-time response of edge intelligence applications.This work studies the resource allocation optimization problem for multi-exit deep neural networks(DNNs) that takes into account the uncertainty of AoI caused by exit probabilities and introduces a probabilistic constraint on system AoI.The stochastic optimization theory is incorporated to make decision on the most appropriate exit configuration for the purpose of minimizing the resource overhead of multi-exit DNNs.A cuckoo search-based metaheuristic algorithm is proposed to solve the stochastic optimization problem with the probabilistic AoI constraint.The metaheuristic predicts the statistical distribution of system AoI based on the exit probabilities,calculates the resource consumption according to a specified AoI threshold and uses it as the fitness value of the corresponding cuckoo individual,and iteratively updates the cuckoo population to explore the exit configuration solution leading to the lowest computing resource overhead.Experimental results on various DNN models show that compared with deterministic optimization methods,the stochastic optimization approach can produce better exit configuration solutions,significantly reducing resource overhead while satisfying the probabilistic AoI constraint.