A Mathematical Introduction to SVMs with Self-Concordant Kernel

A derivation of so-called “soft-margin support vector machines with kernel” is presented along with elementary proofs that do not rely on concepts from functional analysis such as Mercer’s theorem or reproducing kernel Hilbert spaces which are frequently cited in this context. The analysis leads to new continuity properties of the kernel functions, in particular a self-concordance condition for the kernel. Practical aspects concerning the im- plementation and the choice of the kernel are addressed and illustrated with some numerical examples. The derivations are intended for a general audience, requiring basic knowledge of calculus and linear algebra, while some more advanced results from optimization theory are being introduced in a self-contained form.