The PCA Procedure

The PCA procedure implements several methods of calculating principal components: eigenvalue decomposition (EIG), the nonlinear iterative partial least squares (NIPALS) method, the iterative method based on the Gram-Schmidt orthogonalization (ITERGS) of Andrecut (2009), and the iterative method based on random projection (RANDOM) of Halko, Martinsson, and Tropp (2011). The EIG, NIPALS, and ITERGS methods provide exact principal component solutions, whereas the RANDOM method gives approximate principal components.

Each method has computational strengths; Table 15.6 provides guidelines for selecting an efficient (fast) method. The guidelines include the concept of a "small to moderate" and "moderate to large" number of variables. This concept depends on the number of data set observations, the number of principal components required, the PCA method parameter settings, and the grid computing environment. As a simple illustration of the efficiency relationship between the EIG and RANDOM methods, consider a scenario in which one principal component is extracted from a data set that consists of 1,000,000 observations that are distributed across a grid of 16 servers. Figure 15.5 depicts execution time for RANDOM divided by the execution time of EIG (labeled "Execution Time Ratio") as a function of the number of data set variables. Where execution time ratio is greater than 1, EIG is more efficient than RANDOM. For this scenario, "small to moderate" means approximately 700 variables or less and "moderate to large" means anything more than 700 variables.

Figure 15.5: Illustration of Efficiency Crossover Point for EIG and RANDOM Methods