The PCA Procedure

Example 18.2 Extracting Principal Components with NIPALS

This example demonstrates the NIPALS method in PROC PCA, which extracts principal components successively. The data that this example uses are from the Getting Started section; they provide crime rates per 100,000 people in seven categories for each of the 50 US states in 1977. The following DATA step generates the data:

data mycas.Crime;
   title 'Crime Rates per 100,000 Population by State';
   input State $1-15 Murder Rape Robbery Assault
         Burglary Larceny Auto_Theft;
   datalines;
Alabama        14.2 25.2  96.8 278.3 1135.5 1881.9 280.7
Alaska         10.8 51.6  96.8 284.0 1331.7 3369.8 753.3
Arizona         9.5 34.2 138.2 312.3 2346.1 4467.4 439.5
Arkansas        8.8 27.6  83.2 203.4  972.6 1862.1 183.4
California     11.5 49.4 287.0 358.0 2139.4 3499.8 663.5

   ... more lines ...   

Wisconsin       2.8 12.9  52.2  63.7  846.9 2614.2 220.7
Wyoming          .  21.9  39.7 173.9  811.6 2772.2 282.0
;

The following statements use PROC PCA to extract principal components by using the NIPALS method, and produce default plots:

ods graphics on;

proc pca data=mycas.Crime method=nipals plots;
run;

Output 18.2.1 displays the PROC PCA output. The "Model Information" table shows that the NIPALS method is used to extract principal components. The "Explained Variation of Variables" table lists the fraction of variation that is accounted for in each variable by each of the seven principal components. All the variation in each variable is accounted for by seven principal components because there are only seven variables. The eigenvalues indicate that two or three components provide a good summary of the data: two components account for 76% of the total variance, and three components account for 87%. Subsequent components account for less than 5% each.

Note that in the Getting Started section, the principal components are extracted from the same data by using the eigenvalue decomposition method; the "Eigenvalues" table that is generated there matches the one generated by the NIPALS method. Also, the eigenvectors in the "Eigenvectors" table match the loading factors in the "Loadings" table.

Output 18.2.1: Results of Principal Component Analysis Using NIPALS

Crime Rates per 100,000 Population by State

The PCA Procedure

Model Information
Data SourceCRIME
Component Extraction MethodNIPALS

Number of Variables7
Number of Principal Components7

Number of Observations Read50
Number of Observations Used48

Centering and Scaling Information
VariableSubtracted offDivided by
Murder7.516673.93059
Rape26.0750010.81304
Robbery127.5562588.49374
Assault214.58750100.64360
Burglary1316.37917423.31261
Larceny2696.88542714.75023
Auto_Theft383.97917194.37033

Explained Variation of Variables
VariablePrin1Prin2Prin3Prin4Prin5Prin6Prin7
Murder0.371170.855390.877900.895620.975550.991431.00000
Rape0.762420.799170.840590.841990.850650.990411.00000
Robbery0.637830.640640.821640.929420.997880.999921.00000
Assault0.635170.791270.793410.917810.988220.995131.00000
Burglary0.789130.844140.881830.882070.885440.948001.00000
Larceny0.513730.721780.937180.954790.954920.955301.00000
Auto_Theft0.336380.657460.904810.961970.996230.997061.00000

Eigenvalues
 EigenvalueDifferenceProportionCumulative
14.0458242.7817950.57800.5780
21.2640300.5165290.18060.7586
30.7475000.4211750.10680.8653
40.3263250.0611190.04660.9120
50.2652070.0368430.03790.9498
60.2283640.1056130.03260.9825
70.122750 0.01751.0000

Loadings
VariablePrin1Prin2Prin3Prin4Prin5Prin6Prin7
Murder0.30289-0.618930.17353-0.233080.54896-0.26371-0.26428
Rape0.43410-0.17053-0.235390.065400.180750.782320.27946
Robbery0.397050.047130.49208-0.57470-0.508080.094520.02497
Assault0.39622-0.35142-0.053430.61744-0.51525-0.17395-0.19921
Burglary0.441640.20861-0.22454-0.027500.11273-0.523400.65085
Larceny0.356340.40570-0.53681-0.232310.02172-0.04085-0.60346
Auto_Theft0.288340.504000.575240.418530.359390.06024-0.15487


PROC PCA produces the scree plot as shown in Output 18.2.2 by default when ODS Graphics is enabled and no plot request is specified in the PLOTS option. You can obtain more plots by specifying the plot requests in the PLOTS option in the PROC PCA statement.

The scree plot on the left shows that the eigenvalue of the first component is approximately 4 and the eigenvalue of the second component is largely decreased to under 1.5. The variance-explained plot on the right shows that the first two principal components account for nearly 80% of the total variance.

Output 18.2.2: Scree Plot from the PCA Procedure

 Scree Plot from the PCA Procedure


In addition to the scree plot, PROC PCA also produces the component pattern plot and the component pattern profile plot. The following statements request these plots:

proc pca data=mycas.Crime method=nipals
         plots=(pattern(ncomp=3) patternprofile);
run;

The pairwise component pattern plots are shown in Output 18.2.3 through Output 18.2.5. The pattern plots show the following:

  • All variables positively and evenly correlate with the first principal component (Output 18.2.3 and Output 18.2.4).

  • The variable Auto_Theft correlates highly with the second component, and the variable Murder correlates highly but negatively with the second component (Output 18.2.3).

  • All the variables have low correlations (either positively or negatively) with the third component (Output 18.2.4).

  • The variable Auto_Theft correlates highly with the second component, but it has low correlation with the third component; the variable Murder correlates highly but negatively with the second component, but it has very low correlation with the third component (Output 18.2.5).

Output 18.2.3: Pattern Plot of Component 2 by Component 1

 Pattern Plot of Component 2 by Component 1


Output 18.2.4: Pattern Plot of Component 3 by Component 1

 Pattern Plot of Component 3 by Component 1


Output 18.2.5: Pattern Plot of Component 3 by Component 2

 Pattern Plot of Component 3 by Component 2


Output 18.2.6 shows a component pattern profile. As shown in the pattern plots, the nearly horizontal profile of the first component indicates that the first component is mostly correlated evenly across all variables.

Output 18.2.6: Component Pattern Profile Plot from the PCA Procedure

 Component Pattern Profile Plot from the PCA Procedure


Last updated: September 13, 2022