Kernel Principal Component Analysis Action Set

The Exact Method

This section describes the exact KPCA method. For more information, see Schölkopf, Smola, and Müller (1998).

Suppose that the mean of the data in a reproducing kernel Hilbert space (RKHS) is

where x Subscript i is a q times 1 data point in the original space and phi left-parenthesis dot right-parenthesis is the mapping function that maps the original data to some high-dimensional RKHS. Then, assuming that the data vector x Subscript i is already centered, the covariance matrix is

The eigensystem of normal upper C is

Schölkopf, Smola, and Müller (1998) show that the eigenvectors can be expressed as a linear combination of features in RKHS:

Therefore, finding the eigenvectors is equivalent to finding the coefficients alpha Subscript i.

By substituting the preceding equation into the equation immediately before it, for the j th eigenvector, you get

Then, plugging in the kernel and rearranging the equation, you get

where the n times n matrix K is called the kernel matrix, whose k l th element is the value of the kernel function k left-parenthesis x Subscript k Baseline comma x Subscript l Baseline right-parenthesis equals phi left-parenthesis x Subscript k Baseline right-parenthesis Superscript upper T Baseline phi left-parenthesis x Subscript l Baseline right-parenthesis, and bold-italic alpha Subscript j is an n times 1 vector, whose l th element is the coefficient alpha Subscript j l.

Next, the factor K is removed from both sides, and finally the eigensystem can be written in terms of kernel matrix K as

Then, a normalization condition is applied such that the eigenvector v Subscript j in the RKHS satisfies

By left-multiplying upper K bold-italic alpha Subscript j Baseline equals n lamda Subscript j Baseline bold-italic alpha Subscript j by bold-italic alpha Subscript j Superscript upper T and using the normalization condition, you get

The projection of a new or old data point x onto the j th principal component is

Last updated: November 04, 2020