Kernel Principal Component Analysis Action Set
The Exact Method
This section describes the exact KPCA method. For more information, see Schölkopf, Smola, and Müller (1998).
Suppose that the mean of the data in a reproducing kernel Hilbert space (RKHS) is
where is a data point in the original space and is the mapping function that maps the original data to some high-dimensional RKHS. Then, assuming that the data vector is already centered, the covariance matrix is
Schölkopf, Smola, and Müller (1998) show that the eigenvectors can be expressed as a linear combination of features in RKHS:
Therefore, finding the eigenvectors is equivalent to finding the coefficients .
By substituting the preceding equation into the equation immediately before it, for the eigenvector, you get
Then, plugging in the kernel and rearranging the equation, you get
where the matrix K is called the kernel matrix, whose element is the value of the kernel function , and is an vector, whose element is the coefficient .
Next, the factor K is removed from both sides, and finally the eigensystem can be written in terms of kernel matrix K as
Then, a normalization condition is applied such that the eigenvector in the RKHS satisfies
By left-multiplying by and using the normalization condition, you get
The projection of a new or old data point x onto the principal component is