10.3 Support Vector Machines

Enlarging the feature space of the support vector classifier accommodates nonlinar relationships. Support vector machines do this in a specific way, using kernals. The kernal is a generalization of the inner product with form \(K(x_i, x_i^{'})\). So the linear kernal is simply

\[K(x_i, x_i^{'}) = \langle x, x_i \rangle\]

and the solution is

\[f(x) = \beta_0 + \sum_{i = 1}^n {\alpha_i K(x_i, x_i^{'})}\]

\(K\) can take onother form instead, such as polynomial

\[K(x, x') = (\gamma \langle x, x' \rangle + c_0)^d\]

or radial

\[K(x, x') = \exp\{-\gamma ||x - x'||^2\}.\]