Metamers in deep learning for synthetic aperture sonar image classification and their significance in understanding deep networks

Metamers in deep learning are images that appear different but elicit the same feature vector response at the penultimate layer. These images can be easily discovered using optimization methods built into existing deep learning training frameworks. Typically, metamers look nothing like the training data, despite having feature vectors that match exactly with existing training samples. This phenomenon emerges as a function of network layer depth, where initially dissimilar representations converge to a representation of an existing training image. Conversely, simple transforms like image despeckling may have the opposite effect: initial representations similar to a training sample diverge as a function of layer depth until they diverge so much that they become misclassified. In this discussion, we explore this phenomenon, provide practical examples, and aim to stimulate conversation about this intriguing behavior of deep networks.