Non-Gradient Quaternion Training Matrix Modifications for Color-Image Distillation

This paper develops a new multi-stage image distillation method that combines two well-known techniques. In the first stage, our method creates a matrix from all training images. In the next stage, it adapts a modified principal component analysis (M-PCA) approach to transform the training matrix. In the third stage, Singular Value Decomposition (SVD) further refines the training-image matrix through low-rank reconstruction and controlled row selection. In the fourth stage, rotation of small 2×2 matrix blocks on the entire left singular matrix is conducted. The upper m (user-selected number) rows of the reconstructed matrix are selected and transformed back to images, which we call distilled images. This dataset is significantly smaller yet retains the critical information needed for accurate classification. We validated the novelty and the advantages of the new method by applying the Baseline, ResNet50V2, and ConvNetD4 CNNs and the public image databases Digit-MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100, BloodMNIST, and Tiny ImageNet. Experimental results show that by distilling 50 images per class from CIFAR-10, CIFAR-100, and Tiny ImageNet, the proposed method achieves superior test accuracies of 75.59%, 56.27%, and 31.75%, respectively, when evaluated on ResNet and ConvNetD4.