<i>SPLITZ</i>: Certifiable Robustness via Split Lipschitz Randomized Smoothing

Certifiable robustness gives the guarantee that small perturbations around an input to a classifier will not change the prediction. There are two approaches to provide certifiable robustness to adversarial examples– a) explicitly training classifiers with small Lipschitz constants, and b) Randomized smoothing, which adds random noise to the input to create a smooth classifier. We propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SPLITZ</i>, a practical and novel approach which leverages the synergistic benefits of both the above ideas into a single framework. Our main idea is to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">split</i> a classifier into two halves, constrain the Lipschitz constant of the first half, and smooth the second half via randomization. Motivation for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SPLITZ</i> comes from the observation that many standard deep networks exhibit heterogeneity in Lipschitz constants across layers. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SPLITZ</i> can exploit this heterogeneity while inheriting the scalability of randomized smoothing. We present a principled approach to train <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SPLITZ</i> and provide theoretical analysis to derive certified robustness guarantees during inference. We present a comprehensive comparison of robustness-accuracy trade-offs and show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SPLITZ</i> consistently improves on existing state-of-the-art approaches in the MNIST, CIFAR-10 and ImageNet datasets. For instance, with ℓ<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> norm perturbation budget of ϵ = 1, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SPLITZ</i> achieves 43.2% top-1 test accuracy on CIFAR-10 dataset compared to state-of-art top-1 test accuracy 39.8%.