Construction of a Reliability Measurement Scheme for Deep Learning Vision Algorithms based on Failure Modes

With the application of vision algorithms in multitask scenarios such as image classification, object detection and object tracking, the traditional software reliability measurement scheme is difficult to meet the needs of new technologies. Therefore, the research related to the reliability measurement of deep learning vision algorithms has become an urgent need. In this paper, we collect and analyse failure cases in five vision application scenarios, such as image classification, object detection, object tracking, and semantic segmentation, to carry out failure analysis and fault tree analysis. Using the defective causes of the failure cases, ten reliability measurements of deep learning vision algorithms are established, covering four stages: dataset, algorithm training, algorithm inference and algorithm deployment. Through experiments, the influence of each index in the reliability measurement scheme was analysed. The experimental results show that all the measurements in the reliability measurement scheme constructed in this paper have a certain degree of influence, and the three measurements of data balance defect, interference data, and data accuracy defect have the highest influence on the vision task.