Neural Networks with Malignant Neurons: Robust Models for Smart Manufacturing

In smart manufacturing, robust and reliable neural networks are critical for ensuring seamless operations, particularly in adversarial or noisy environments. However, existing regularization techniques, such as Dropout, were primarily designed for improving generalization in clean settings, and they often fall short in adversarial scenarios. This reveals a key gap in current research: the lack of regularization strategies explicitly designed to improve adversarial robustness. In this paper, we propose Adversarial Regularization (AR), a novel technique that enhances network robustness by designating certain neurons as “malignant” during training. These malignant neurons are trained not to minimize the loss but to maximize it, introducing probabilistic adversarial effects that actively challenge the network. Unlike Dropout, which deactivates neurons to prevent co-adaptation, AR simulates adversarial contributions to improve resilience against perturbations. We position AR within the broader landscape of regularization approaches, providing theoretical insights and justifications for its scaling factors. Additionally, we introduce a hybrid approach that combines Dropout and AR for enhanced flexibility. Experimental evaluations on public datasets for classification tasks show that AR achieves competitive performance on clean data and outperforms Dropout under adversarial noise, either alone or as part of a hybrid technique. Comparative analyses of accuracy and loss dynamics further demonstrate AR’s robustness and generalization capabilities. By directly addressing the gap in adversarial regularization, AR emerges as a promising technique for developing resilient neural networks tailored to the demands of smart manufacturing applications.