Investigation of Feature Distribution and Network Weight Updates in the Machine Unlearning Process

Machine unlearning refers to the process of expunging previously learned information and data from a machine learning model to achieve the objective of privacy protection. In this research, we explore two prevalent methods for unlearning, namely, label reassignment and model manipulation. Using the CIFAR-100 classification problem with ResNet-50 architecture as an example, we examine the efficacy of these two mechanisms and their variants in the corresponding unlearning task. We further investigate the changes in feature distribution and the extent of weight updates across various network layers throughout the unlearning process. Experimental results indicate that the degree of network variation is proportional to the number of removed classes. When employing the label reassignment method, the variation is concentrated in the final stage and fully connected layers. On the other hand, using the weight resetting strategy affects more network layers, with the impact gradually decreasing from the later layers to the middle and earlier layers. Overall, when the categories to be forgotten are less than 10%, no significant impact on feature extraction is observed.