DNN Testing, Regression Testing and Software Reliability Prediction

In this issue, we are pleased to present three papers where the first two focus on testing and the third one focuses on software reliability prediction. The first paper, “Validity Matters: Uncertainty-Guided Testing of Deep Neural Networks” by Zhouxian Jiang, Honghui Li, Rui Wang, Xuetao Tian, Ci Liang, Fei Yan, Junwen Zhang and Zhen Liu, presents the uncertainty-guided deep learning testing approach (UCTest) to address the safety and security challenges of deep neural networks (DNNs). Their large-scale empirical study on the effectiveness of 11 predictive uncertainty metrics for identifying invalid test inputs shows that predictive entropy with dropout-enable, ensemble-based metrics (with or without dropout-enable) provides the top detection performance. The authors design UCTest to generate valid and authentic test inputs by jointly optimizing the loss function and uncertainty. Their experimental results show that UCTest outperforms baseline approaches in generating valid and diverse test inputs while imposing less computational overhead (Recommended by Long Wang). The second paper, “Generation of Regression Tests From Logs With Clustering Guided by Usage Patterns” by Frédéric Tamagnan, Alexandre Vernotte, Fabrice Bouquet and Bruno Legeard, presents a new approach to improve regression testing by automating the creation of test suites using user traces fed into clustering pipelines used to select appropriate test suites during regression testing. The authors propose a new metric based on pattern mining to quantify the statistical coverage of prevalent user paths, to help determine the optimal number of clusters within a clustering pipeline. The authors also propose two criteria to systematically evaluate and rank clustering pipelines. The experimental results show the potential effectiveness of the proposed automated approach compared with manually crafted test suites (Recommended by Dan Hao). The third paper, “Software Reliability Prediction by Adaptive Gated Recurrent Unit-Based Encoder-Decoder Model with Ensemble Empirical Mode Decomposition” by Minghao Yang, Shunkun Yang and Chong Bian, points out the problem of prediction lag in software reliability prediction and presents a new model to effectively eliminate the prediction lag and conduct accurate prediction. The model first reduces the autocorrelation and nonstationarity of fault data by using first-order difference and EEMD to clearly characterize the changing trend of the data. The model then combines the frequency-specific ED-GRU networks to adaptively learn the nonlinear fluctuation trend of fault data under different frequency scales and obtain accurate prediction final results after aggregation. The experimental results show that the autocorrelation and nonstationarity of fault data are important reasons for the prediction lags of nonparametric models, and the proposed model can effectively improve the prediction accuracy by eliminating these lags with strong stability and applicability (Recommended by Min Xie). We hope that these papers will inspire further research in related directions. The authors have nothing to report.