A spherical manifold learning technique to enhance machine learning and explainable artificial intelligence systems

Machine learning (ML) techniques generally bias towards the training set; hence, they lose their ability to generalize ML models, efficiently. In other words, they experience overfitting problems; thus, they show poor performance on unseen data. To address overfitting problems, regularization approaches have been proposed in ML literature. A common approach is to add penalty terms to avoid the ML models closely fitting the training data. However, this approach is not suitable for developing artificial intelligence (AI) systems that require both quality and explainability since the penalty terms are sensitive to randomness and uncertainty. Hence, this paper proposes a manifold learning technique to address the overfitting problem and enhance the predictive performance and the explainability of the models. The novel idea is to first magnify the overfitting problem and then integrate manifold-based regularization to enhance the quality, the diversity, and the geometrical explainability of the features. Therefore, the proposed manifold learning technique (i) vectorizes the data points, generates feature vectors, and constructs a finite feature space to train ML models, (ii) applies principal component analysis (PCA) to transform feature vectors and construct an eigenvector space that is formed by the principal components (PCs)—this mathematical space supplies information to the models to closely fit the training data, and (iii) projects the eigenvectors into a spherical manifold (i.e., sum-of-squares manifold) formed by a chosen set of principal components and then train the model on this dimensionality-reduced manifold—this manifold can generalize the models to address the overfitting problem. Simulations using the subsets of a wine dataset and a widely used birds’ images dataset, along with support vector machine and random forest classifiers show that the addition of a spherical manifold to a PCA-based feature space can significantly improve the prediction accuracy and the explainability (e.g., SHAP) of the classifiers.