Parameter Estimation for Support Vector Machines Using Markov Chain Monte Carlo Simulation

One of the widely used methods in machine learning is the support vector machine (SVM). The SVM is a data classification algorithm based on labeled training data; it is widely used in fields, such as image classification and machine learning. In the SVM, parameter estimation is frequently performed based on mathematical programming methods, such as Sequential Minimal Optimization (SMO), Studies have proposed various algorithms, including interior-point methods. However, although the variables used in an SVM model are generally interrelated, traditional SVMs based on optimization problems cannot consider the relationships between variables in the model, such as the variance-covariance relationship observed in regression analysis. To address this, we attempted to estimate the parameters of an SVM by considering the relationships between the variables through Markov Chain Monte Carlo (MCMC) simulations. Furthermore, through verification experiments using real data, we demonstrated the possibility of estimating the variance and covariance of parameters from sampled data and improving the estimation accuracy through data over-sampling.