RTDeepEnsemble: Real-Time DNN Ensemble Method for Machine Perception Systems

Deep Neural Networks (DNNs) are crucial for enhancing environmental perception in embedded intelligent systems, particularly in applications such as drones and autonomous vehicles. These mission-critical applications necessitate both computational accuracy and real-time performance. A significant challenge arises from the unpredictable execution times of DNNs and the variable periods of perception tasks. To address this, we present RTDeepEnsemble, an innovative framework designed for the real-time execution of DNN ensembles. RTDeepEnsemble dynamically manages the DNN ensemble as an imprecise computation model, judiciously selecting the optimal DNN for execution at runtime. This framework effectively arranges the DNN execution sequence, estimating utility through a refined Boosting algorithm. Additionally, RTDeepEnsemble prioritizes mandatory subtasks based on the Earliest Deadline First scheduling and employs a dynamic programming method to schedule optional subtasks. Comprehensive evaluations on the Nvidia Jetson Xavier NX platform, within a real-time object detection context, demonstrate RTDeepEnsemble's superiority in both accuracy and deadline miss rate. Notably, while maintaining real-time performance, we observe an accuracy increase of approximately 10 %, underscoring RTDeepEnsemble's potential as a solution for real-time machine perception.