Using Metamorphic Relations to Improve Accuracy and Robustness of Deep Neural Networks

When applying metamorphic testing to a deep neural network (DNN), the DNN could have an "acceptable" level of accuracy but it performs poorly against some metamorphic relations (MRs). Such a DNN is considered not robust against these MRs. Improving both accuracy and robustness of a DNN is non-trivial because improving one aspect may adversely affect the other aspect. To alleviate this trade-off problem, we proposed a regularization-based method, in which an optimization function is designed to balance a DNN's accuracy and robustness. Then, we designed a reinforcement-learning-based algorithm to optimize this function. We tested our training method with two datasets (SVHN and CIFAR10), and each dataset with two DNN models. When comparing ours with the other six benchmark methods, we found the DNNs trained with our method have a better balance between accuracy and robustness.