The DISCRIM Procedure

Example 36.3 Normal-Theory Discriminant Analysis of Iris Data

(View the complete code for this example.)

In this example, PROC DISCRIM uses normal-theory methods to classify the iris data used in Example 36.1. The POOL=TEST option tests the homogeneity of the within-group covariance matrices (Output 36.3.3). Since the resulting test statistic is significant at the 0.10 level, the within-group covariance matrices are used to derive the quadratic discriminant criterion. The WCOV and PCOV options display the within-group covariance matrices and the pooled covariance matrix (Output 36.3.2). The DISTANCE option displays squared distances between classes (Output 36.3.4). The ANOVA and MANOVA options test the hypothesis that the class means are equal, by using univariate statistics and multivariate statistics; all statistics are significant at the 0.0001 level (Output 36.3.5). The LISTERR option lists the misclassified observations under resubstitution (Output 36.3.6). The CROSSLISTERR option lists the observations that are misclassified under cross validation and displays cross validation error-rate estimates (Output 36.3.7). The resubstitution error count estimate, 0.02, is not larger than the cross validation error count estimate, 0.0267, as would be expected because the resubstitution estimate is optimistically biased. The OUTSTAT= option generates a TYPE=MIXED (because POOL=TEST) output data set containing various statistics such as means, covariances, and coefficients of the discriminant function (Output 36.3.8).

The following statements produce Output 36.3.1 through Output 36.3.8:

title 'Discriminant Analysis of Fisher (1936) Iris Data';
title2 'Using Quadratic Discriminant Function';

proc discrim data=sashelp.iris outstat=irisstat
   wcov pcov method=normal pool=test
   distance anova manova listerr crosslisterr;
   class Species;
   var SepalLength SepalWidth PetalLength PetalWidth;
run;

proc print data=irisstat;
   title2 'Output Discriminant Statistics';
run;

Output 36.3.1: Quadratic Discriminant Analysis of Iris Data

Discriminant Analysis of Fisher (1936) Iris Data
Using Quadratic Discriminant Function

The DISCRIM Procedure

Total Sample Size150DF Total149
Variables4DF Within Classes147
Classes3DF Between Classes2

Number of Observations Read150
Number of Observations Used150

Class Level Information
SpeciesVariable
Name
FrequencyWeightProportionPrior
Probability
SetosaSetosa5050.00000.3333330.333333
VersicolorVersicolor5050.00000.3333330.333333
VirginicaVirginica5050.00000.3333330.333333


Output 36.3.2: Covariance Matrices

Discriminant Analysis of Fisher (1936) Iris Data
Using Quadratic Discriminant Function

The DISCRIM Procedure
Within-Class Covariance Matrices

Species = Setosa, DF = 49
VariableLabelSepalLengthSepalWidthPetalLengthPetalWidth
SepalLengthSepal Length (mm)12.424897969.921632651.635510201.03306122
SepalWidthSepal Width (mm)9.9216326514.368979591.169795920.92979592
PetalLengthPetal Length (mm)1.635510201.169795923.015918370.60693878
PetalWidthPetal Width (mm)1.033061220.929795920.606938781.11061224


 

Species = Versicolor, DF = 49
VariableLabelSepalLengthSepalWidthPetalLengthPetalWidth
SepalLengthSepal Length (mm)26.643265318.5183673518.289795925.57795918
SepalWidthSepal Width (mm)8.518367359.846938788.265306124.12040816
PetalLengthPetal Length (mm)18.289795928.2653061222.081632657.31020408
PetalWidthPetal Width (mm)5.577959184.120408167.310204083.91061224


 

Species = Virginica, DF = 49
VariableLabelSepalLengthSepalWidthPetalLengthPetalWidth
SepalLengthSepal Length (mm)40.434285719.3763265330.328979594.90938776
SepalWidthSepal Width (mm)9.3763265310.400408167.137959184.76285714
PetalLengthPetal Length (mm)30.328979597.1379591830.458775514.88244898
PetalWidthPetal Width (mm)4.909387764.762857144.882448987.54326531


 

Discriminant Analysis of Fisher (1936) Iris Data
Using Quadratic Discriminant Function

The DISCRIM Procedure

Pooled Within-Class Covariance Matrix, DF = 147
VariableLabelSepalLengthSepalWidthPetalLengthPetalWidth
SepalLengthSepal Length (mm)26.500816339.2721088416.751428573.84013605
SepalWidthSepal Width (mm)9.2721088411.538775515.524353743.27102041
PetalLengthPetal Length (mm)16.751428575.5243537418.518775514.26653061
PetalWidthPetal Width (mm)3.840136053.271020414.266530614.18816327


Within Covariance Matrix Information
SpeciesCovariance
Matrix Rank
Natural Log of the
Determinant of the
Covariance Matrix
Setosa45.35332
Versicolor47.54636
Virginica49.49362
Pooled48.46214

Output 36.3.3: Homogeneity Test

Discriminant Analysis of Fisher (1936) Iris Data
Using Quadratic Discriminant Function

The DISCRIM Procedure
Test of Homogeneity of Within Covariance Matrices

Chi-SquareDFPr > ChiSq
140.94305020<.0001

Since the Chi-Square value is significant at the 0.1 level, the within covariance matrices will be used in the discriminant function.
Reference: Morrison, D.F. (1976) Multivariate Statistical Methods p252.



Output 36.3.4: Squared Distances

Discriminant Analysis of Fisher (1936) Iris Data
Using Quadratic Discriminant Function

The DISCRIM Procedure

Squared Distance to Species
From SpeciesSetosa VersicolorVirginica
Setosa0103.19382168.76759
Versicolor323.06203013.83875
Virginica706.0849417.866700

Generalized Squared Distance to Species
From SpeciesSetosa VersicolorVirginica
Setosa5.35332110.74017178.26121
Versicolor328.415357.5463623.33238
Virginica711.4382625.413069.49362


Output 36.3.5: Tests of Equal Class Means

Discriminant Analysis of Fisher (1936) Iris Data
Using Quadratic Discriminant Function

The DISCRIM Procedure

Univariate Test Statistics
F Statistics, Num DF=2, Den DF=147
VariableLabelTotal
Standard
Deviation
Pooled
Standard
Deviation
Between
Standard
Deviation
R-SquareR-Square
/ (1-RSq)
F ValuePr > F
SepalLengthSepal Length (mm)8.28075.14797.95060.61871.6226119.26<.0001
SepalWidthSepal Width (mm)4.35873.39693.36820.40080.668849.16<.0001
PetalLengthPetal Length (mm)17.65304.303320.90700.941416.05661180.16<.0001
PetalWidthPetal Width (mm)7.62242.04658.96730.928913.0613960.01<.0001

Average R-Square
Unweighted0.7224358
Weighted by Variance0.8689444

Multivariate Statistics and F Approximations
S=2 M=0.5 N=71
StatisticValueF ValueNum DFDen DFPr > F
Wilks' Lambda0.02343863199.158288<.0001
Pillai's Trace1.1918988353.478290<.0001
Hotelling-Lawley Trace32.47732024582.208203.4<.0001
Roy's Greatest Root32.191929201166.964145<.0001
NOTE: F Statistic for Roy's Greatest Root is an upper bound.
NOTE: F Statistic for Wilks' Lambda is exact.


Output 36.3.6: Misclassified Observations: Resubstitution

Discriminant Analysis of Fisher (1936) Iris Data
Using Quadratic Discriminant Function

The DISCRIM Procedure
Classification Results for Calibration Data: SASHELP.IRIS
Resubstitution Results using Quadratic Discriminant Function

Posterior Probability of Membership in Species
ObsFrom SpeciesClassified into
Species
Setosa VersicolorVirginica
53VersicolorVirginica*0.00000.33590.6641
55VersicolorVirginica*0.00000.15430.8457
103VirginicaVersicolor*0.00000.60500.3950

* Misclassified observation


Discriminant Analysis of Fisher (1936) Iris Data
Using Quadratic Discriminant Function

The DISCRIM Procedure
Classification Summary for Calibration Data: SASHELP.IRIS
Resubstitution Summary using Quadratic Discriminant Function

Number of Observations and Percent Classified
into Species
From SpeciesSetosa VersicolorVirginica Total
Setosa
50
100.00
0
0.00
0
0.00
50
100.00
Versicolor
0
0.00
48
96.00
2
4.00
50
100.00
Virginica
0
0.00
1
2.00
49
98.00
50
100.00
Total
50
33.33
49
32.67
51
34.00
150
100.00
Priors
0.33333
 
0.33333
 
0.33333
 
 
 

Error Count Estimates for Species
 Setosa VersicolorVirginica Total
Rate0.00000.04000.02000.0200
Priors0.33330.33330.3333 


Output 36.3.7: Misclassified Observations: Cross Validation

Discriminant Analysis of Fisher (1936) Iris Data
Using Quadratic Discriminant Function

The DISCRIM Procedure
Classification Results for Calibration Data: SASHELP.IRIS
Cross-validation Results using Quadratic Discriminant Function

Posterior Probability of Membership in Species
ObsFrom SpeciesClassified into
Species
Setosa VersicolorVirginica
52VersicolorVirginica*0.00000.31340.6866
53VersicolorVirginica*0.00000.16160.8384
55VersicolorVirginica*0.00000.07130.9287
103VirginicaVersicolor*0.00000.66320.3368

* Misclassified observation


Discriminant Analysis of Fisher (1936) Iris Data
Using Quadratic Discriminant Function

The DISCRIM Procedure
Classification Summary for Calibration Data: SASHELP.IRIS
Cross-validation Summary using Quadratic Discriminant Function

Number of Observations and Percent Classified
into Species
From SpeciesSetosa VersicolorVirginica Total
Setosa
50
100.00
0
0.00
0
0.00
50
100.00
Versicolor
0
0.00
47
94.00
3
6.00
50
100.00
Virginica
0
0.00
1
2.00
49
98.00
50
100.00
Total
50
33.33
48
32.00
52
34.67
150
100.00
Priors
0.33333
 
0.33333
 
0.33333
 
 
 

Error Count Estimates for Species
 Setosa VersicolorVirginica Total
Rate0.00000.06000.02000.0267
Priors0.33330.33330.3333 


Output 36.3.8: Output Statistics from Iris Data

Discriminant Analysis of Fisher (1936) Iris Data
Output Discriminant Statistics

ObsSpecies_TYPE__NAME_SepalLengthSepalWidthPetalLengthPetalWidth
1 N 150.00150.00150.00150.00
2SetosaN 50.0050.0050.0050.00
3VersicolorN 50.0050.0050.0050.00
4VirginicaN 50.0050.0050.0050.00
5 MEAN 58.4330.5737.5811.99
6SetosaMEAN 50.0634.2814.622.46
7VersicolorMEAN 59.3627.7042.6013.26
8VirginicaMEAN 65.8829.7455.5220.26
9SetosaPRIOR 0.330.330.330.33
10VersicolorPRIOR 0.330.330.330.33
11VirginicaPRIOR 0.330.330.330.33
12SetosaCSSCPSepalLength608.82486.1680.1450.62
13SetosaCSSCPSepalWidth486.16704.0857.3245.56
14SetosaCSSCPPetalLength80.1457.32147.7829.74
15SetosaCSSCPPetalWidth50.6245.5629.7454.42
16VersicolorCSSCPSepalLength1305.52417.40896.20273.32
17VersicolorCSSCPSepalWidth417.40482.50405.00201.90
18VersicolorCSSCPPetalLength896.20405.001082.00358.20
19VersicolorCSSCPPetalWidth273.32201.90358.20191.62
20VirginicaCSSCPSepalLength1981.28459.441486.12240.56
21VirginicaCSSCPSepalWidth459.44509.62349.76233.38
22VirginicaCSSCPPetalLength1486.12349.761492.48239.24
23VirginicaCSSCPPetalWidth240.56233.38239.24369.62
24 PSSCPSepalLength3895.621363.002462.46564.50
25 PSSCPSepalWidth1363.001696.20812.08480.84
26 PSSCPPetalLength2462.46812.082722.26627.18
27 PSSCPPetalWidth564.50480.84627.18615.66
28 BSSCPSepalLength6321.21-1995.2716524.847127.93
29 BSSCPSepalWidth-1995.271134.49-5723.96-2293.27
30 BSSCPPetalLength16524.84-5723.9643710.2818677.40
31 BSSCPPetalWidth7127.93-2293.2718677.408041.33
32 CSSCPSepalLength10216.83-632.2718987.307692.43
33 CSSCPSepalWidth-632.272830.69-4911.88-1812.43
34 CSSCPPetalLength18987.30-4911.8846432.5419304.58
35 CSSCPPetalWidth7692.43-1812.4319304.588656.99
36 RSQUARED 0.620.400.940.93
37SetosaCOVSepalLength12.429.921.641.03
38SetosaCOVSepalWidth9.9214.371.170.93
39SetosaCOVPetalLength1.641.173.020.61
40SetosaCOVPetalWidth1.030.930.611.11
41VersicolorCOVSepalLength26.648.5218.295.58
42VersicolorCOVSepalWidth8.529.858.274.12
43VersicolorCOVPetalLength18.298.2722.087.31
44VersicolorCOVPetalWidth5.584.127.313.91
45VirginicaCOVSepalLength40.439.3830.334.91
46VirginicaCOVSepalWidth9.3810.407.144.76
47VirginicaCOVPetalLength30.337.1430.464.88
48VirginicaCOVPetalWidth4.914.764.887.54
49 PCOVSepalLength26.509.2716.753.84
50 PCOVSepalWidth9.2711.545.523.27
51 PCOVPetalLength16.755.5218.524.27
52 PCOVPetalWidth3.843.274.274.19
53 BCOVSepalLength63.21-19.95165.2571.28
54 BCOVSepalWidth-19.9511.34-57.24-22.93
55 BCOVPetalLength165.25-57.24437.10186.77
56 BCOVPetalWidth71.28-22.93186.7780.41
57 COVSepalLength68.57-4.24127.4351.63
58 COVSepalWidth-4.2419.00-32.97-12.16
59 COVPetalLength127.43-32.97311.63129.56
60 COVPetalWidth51.63-12.16129.5658.10
61SetosaSTD 3.523.791.741.05
62VersicolorSTD 5.163.144.701.98
63VirginicaSTD 6.363.225.522.75
64 PSTD 5.153.404.302.05
65 BSTD 7.953.3720.918.97
66 STD 8.284.3617.657.62
67SetosaCORRSepalLength1.000.740.270.28
68SetosaCORRSepalWidth0.741.000.180.23
69SetosaCORRPetalLength0.270.181.000.33
70SetosaCORRPetalWidth0.280.230.331.00
71VersicolorCORRSepalLength1.000.530.750.55
72VersicolorCORRSepalWidth0.531.000.560.66
73VersicolorCORRPetalLength0.750.561.000.79
74VersicolorCORRPetalWidth0.550.660.791.00
75VirginicaCORRSepalLength1.000.460.860.28
76VirginicaCORRSepalWidth0.461.000.400.54
77VirginicaCORRPetalLength0.860.401.000.32
78VirginicaCORRPetalWidth0.280.540.321.00
79 PCORRSepalLength1.000.530.760.36
80 PCORRSepalWidth0.531.000.380.47
81 PCORRPetalLength0.760.381.000.48
82 PCORRPetalWidth0.360.470.481.00
83 BCORRSepalLength1.00-0.750.991.00
84 BCORRSepalWidth-0.751.00-0.81-0.76
85 BCORRPetalLength0.99-0.811.001.00
86 BCORRPetalWidth1.00-0.761.001.00
87 CORRSepalLength1.00-0.120.870.82
88 CORRSepalWidth-0.121.00-0.43-0.37
89 CORRPetalLength0.87-0.431.000.96
90 CORRPetalWidth0.82-0.370.961.00
91SetosaSTDMEAN -1.010.85-1.30-1.25
92VersicolorSTDMEAN 0.11-0.660.280.17
93VirginicaSTDMEAN 0.90-0.191.021.08
94SetosaPSTDMEAN -1.631.09-5.34-4.66
95VersicolorPSTDMEAN 0.18-0.851.170.62
96VirginicaPSTDMEAN 1.45-0.254.174.04
97 LNDETERM 8.468.468.468.46
98SetosaLNDETERM 5.355.355.355.35
99VersicolorLNDETERM 7.557.557.557.55
100VirginicaLNDETERM 9.499.499.499.49
101SetosaQUADSepalLength-0.090.060.020.02
102SetosaQUADSepalWidth0.06-0.08-0.010.01
103SetosaQUADPetalLength0.02-0.01-0.190.09
104SetosaQUADPetalWidth0.020.010.09-0.53
105SetosaQUAD_LINEAR_4.46-0.763.36-3.13
106SetosaQUAD_CONST_-121.83-121.83-121.83-121.83
107VersicolorQUADSepalLength-0.050.020.04-0.03
108VersicolorQUADSepalWidth0.02-0.10-0.010.10
109VersicolorQUADPetalLength0.04-0.01-0.100.13
110VersicolorQUADPetalWidth-0.030.100.13-0.44
111VersicolorQUAD_LINEAR_1.801.600.33-1.47
112VersicolorQUAD_CONST_-76.55-76.55-76.55-76.55
113VirginicaQUADSepalLength-0.050.020.05-0.01
114VirginicaQUADSepalWidth0.02-0.08-0.010.04
115VirginicaQUADPetalLength0.05-0.01-0.070.01
116VirginicaQUADPetalWidth-0.010.040.01-0.10
117VirginicaQUAD_LINEAR_0.741.320.620.97
118VirginicaQUAD_CONST_-75.82-75.82-75.82-75.82