The DISCRIM Procedure

Example 36.2 Bivariate Density Estimates and Posterior Probabilities

(View the complete code for this example.)

In this example, four more discriminant analyses of iris data are run with two quantitative variables: petal width and petal length. The following statements produce Output 36.2.1 through Output 36.2.5:

title 'Discriminant Analysis of Fisher (1936) Iris Data';
proc template;
   define statgraph scatter;
      begingraph;
         entrytitle 'Fisher (1936) Iris Data';
         layout overlayequated / equatetype=fit;
            scatterplot x=petallength y=petalwidth /
                        group=species name='iris';
            layout gridded / autoalign=(topleft);
               discretelegend 'iris' / border=false opaque=false;
            endlayout;
         endlayout;
      endgraph;
   end;
run;

proc sgrender data=sashelp.iris template=scatter;
run;

The scatter plot in Output 36.2.1 shows the joint sample distribution.

Output 36.2.1: Joint Sample Distribution of Petal Width and Petal Length in Three Species

Joint Sample Distribution of Petal Width and Petal Length in Three Species


Another data set is created for plotting, containing a grid of points suitable for contour plots. The following statements create the data set:

data plotdata;
   do PetalLength = -2 to 72 by 0.5;
      do PetalWidth= - 5 to 32 by 0.5;
         output;
      end;
   end;
run;

Three macros are defined as follows to make contour plots of density estimates, posterior probabilities, and classification results:

%let close = thresholdmin=0 thresholdmax=0 offsetmin=0 offsetmax=0;
%let close = xaxisopts=(&close) yaxisopts=(&close);

proc template;
   define statgraph contour;
      begingraph;
         layout overlayequated / equatetype=equate &close;
            contourplotparm x=petallength y=petalwidth z=z /
                            contourtype=fill nhint=30;
            scatterplot x=pl y=pw / group=species name='iris'
                        includemissinggroup=false primary=true;
            layout gridded / autoalign=(topleft);
               discretelegend 'iris' / border=false opaque=false;
            endlayout;
         endlayout;
      endgraph;
   end;
run;

%macro contden;
   data contour(keep=PetalWidth PetalLength species z pl pw);
      merge plotd(in=d) sashelp.iris(keep=PetalWidth PetalLength species
                                     rename=(PetalWidth=pw PetalLength=pl));
      if d then z = max(setosa,versicolor,virginica);
   run;

   title3 'Plot of Estimated Densities';

   proc sgrender data=contour template=contour;
   run;
%mend;

%macro contprob;
   data posterior(keep=PetalWidth PetalLength species z pl pw into);
      merge plotp(in=d) sashelp.iris(keep=PetalWidth PetalLength species
                                     rename=(PetalWidth=pw PetalLength=pl));
      if d then z = max(setosa,versicolor,virginica);
      into = 1 * (_into_ =: 'Set') + 2 * (_into_ =: 'Ver') +
             3 * (_into_ =: 'Vir');
   run;

   title3 'Plot of Posterior Probabilities ';

   proc sgrender data=posterior template=contour;
   run;
%mend;
%macro contclass;
   title3 'Plot of Classification Results';

   proc sgrender data=posterior(drop=z rename=(into=z)) template=contour;
   run;
%mend;

A normal-theory analysis (METHOD=NORMAL) assuming equal covariance matrices (POOL=YES) illustrates the linearity of the classification boundaries. These statements produce Output 36.2.2:

title2 'Using Normal Density Estimates with Equal Variance';

proc discrim data=sashelp.iris method=normal pool=yes
   testdata=plotdata testout=plotp testoutd=plotd
   short noclassify crosslisterr;
   class Species;
   var Petal:;
run;

%contden
%contprob
%contclass

Output 36.2.2: Normal Density Estimates with Equal Variance

Discriminant Analysis of Fisher (1936) Iris Data
Using Normal Density Estimates with Equal Variance

The DISCRIM Procedure

Total Sample Size150DF Total149
Variables2DF Within Classes147
Classes3DF Between Classes2

Number of Observations Read150
Number of Observations Used150

Class Level Information
SpeciesVariable
Name
FrequencyWeightProportionPrior
Probability
SetosaSetosa5050.00000.3333330.333333
VersicolorVersicolor5050.00000.3333330.333333
VirginicaVirginica5050.00000.3333330.333333

Discriminant Analysis of Fisher (1936) Iris Data
Using Normal Density Estimates with Equal Variance

The DISCRIM Procedure
Classification Results for Calibration Data: SASHELP.IRIS
Cross-validation Results using Linear Discriminant Function

Posterior Probability of Membership in Species
ObsFrom SpeciesClassified into
Species
Setosa VersicolorVirginica
53VersicolorVirginica*0.00000.21300.7870
100VersicolorVirginica*0.00000.31180.6882
103VirginicaVersicolor*0.00000.84530.1547
113VirginicaVersicolor*0.00000.83220.1678
124VirginicaVersicolor*0.00000.80570.1943
136VirginicaVersicolor*0.00000.89030.1097

* Misclassified observation


Discriminant Analysis of Fisher (1936) Iris Data
Using Normal Density Estimates with Equal Variance

The DISCRIM Procedure
Classification Summary for Calibration Data: SASHELP.IRIS
Cross-validation Summary using Linear Discriminant Function

Number of Observations and Percent Classified
into Species
From SpeciesSetosa VersicolorVirginica Total
Setosa
50
100.00
0
0.00
0
0.00
50
100.00
Versicolor
0
0.00
48
96.00
2
4.00
50
100.00
Virginica
0
0.00
4
8.00
46
92.00
50
100.00
Total
50
33.33
52
34.67
48
32.00
150
100.00
Priors
0.33333
 
0.33333
 
0.33333
 
 
 

Error Count Estimates for Species
 Setosa VersicolorVirginica Total
Rate0.00000.04000.08000.0400
Priors0.33330.33330.3333 

Discriminant Analysis of Fisher (1936) Iris Data
Using Normal Density Estimates with Equal Variance

The DISCRIM Procedure
Classification Summary for Test Data: WORK.PLOTDATA
Classification Summary using Linear Discriminant Function

Observation Profile for Test Data
Number of Observations Read11175
Number of Observations Used11175

Number of Observations and Percent Classified
into Species
 Setosa VersicolorVirginica Total
Total
3670
32.84
4243
37.97
3262
29.19
11175
100.00
Priors
0.33333
 
0.33333
 
0.33333
 
 
 


disx2fc
External File:images/disx2fc1.png
External File:images/disx2fc2.png

A normal-theory analysis assuming unequal covariance matrices (POOL=NO) illustrates quadratic classification boundaries. These statements produce Output 36.2.3:

title2 'Using Normal Density Estimates with Unequal Variance';

proc discrim data=sashelp.iris method=normal pool=no
   testdata=plotdata testout=plotp testoutd=plotd
   short noclassify crosslisterr;
   class Species;
   var Petal:;
run;

%contden
%contprob
%contclass

Output 36.2.3: Normal Density Estimates with Unequal Variance

Discriminant Analysis of Fisher (1936) Iris Data
Using Normal Density Estimates with Unequal Variance

The DISCRIM Procedure

Total Sample Size150DF Total149
Variables2DF Within Classes147
Classes3DF Between Classes2

Number of Observations Read150
Number of Observations Used150

Class Level Information
SpeciesVariable
Name
FrequencyWeightProportionPrior
Probability
SetosaSetosa5050.00000.3333330.333333
VersicolorVersicolor5050.00000.3333330.333333
VirginicaVirginica5050.00000.3333330.333333

Discriminant Analysis of Fisher (1936) Iris Data
Using Normal Density Estimates with Unequal Variance

The DISCRIM Procedure
Classification Results for Calibration Data: SASHELP.IRIS
Cross-validation Results using Quadratic Discriminant Function

Posterior Probability of Membership in Species
ObsFrom SpeciesClassified into
Species
Setosa VersicolorVirginica
53VersicolorVirginica*0.00000.09030.9097
100VersicolorVirginica*0.00000.46750.5325
103VirginicaVersicolor*0.00000.72880.2712
113VirginicaVersicolor*0.00000.51960.4804
136VirginicaVersicolor*0.00000.83350.1665

* Misclassified observation


Discriminant Analysis of Fisher (1936) Iris Data
Using Normal Density Estimates with Unequal Variance

The DISCRIM Procedure
Classification Summary for Calibration Data: SASHELP.IRIS
Cross-validation Summary using Quadratic Discriminant Function

Number of Observations and Percent Classified
into Species
From SpeciesSetosa VersicolorVirginica Total
Setosa
50
100.00
0
0.00
0
0.00
50
100.00
Versicolor
0
0.00
48
96.00
2
4.00
50
100.00
Virginica
0
0.00
3
6.00
47
94.00
50
100.00
Total
50
33.33
51
34.00
49
32.67
150
100.00
Priors
0.33333
 
0.33333
 
0.33333
 
 
 

Error Count Estimates for Species
 Setosa VersicolorVirginica Total
Rate0.00000.04000.06000.0333
Priors0.33330.33330.3333 

Discriminant Analysis of Fisher (1936) Iris Data
Using Normal Density Estimates with Unequal Variance

The DISCRIM Procedure
Classification Summary for Test Data: WORK.PLOTDATA
Classification Summary using Quadratic Discriminant Function

Observation Profile for Test Data
Number of Observations Read11175
Number of Observations Used11175

Number of Observations and Percent Classified
into Species
 Setosa VersicolorVirginica Total
Total
1382
12.37
1345
12.04
8448
75.60
11175
100.00
Priors
0.33333
 
0.33333
 
0.33333
 
 
 


disx2kc
External File:images/disx2kc1.png
External File:images/disx2kc2.png

A nonparametric analysis (METHOD=NPAR) follows, using normal kernels (KERNEL=NORMAL) and equal bandwidths (POOL=YES) in each class. The value of the radius parameter r that, assuming normality, minimizes an approximate mean integrated square error is 0.50 (see the section Nonparametric Methods). These statements produce Output 36.2.4:

title2 'Using Kernel Density Estimates with Equal Bandwidth';

proc discrim data=sashelp.iris method=npar kernel=normal
   r=.5 pool=yes testoutd=plotd testdata=plotdata testout=plotp
   short noclassify crosslisterr;
   class Species;
   var Petal:;
run;

%contden
%contprob
%contclass

Output 36.2.4: Kernel Density Estimates with Equal Bandwidth

Discriminant Analysis of Fisher (1936) Iris Data
Using Kernel Density Estimates with Equal Bandwidth

The DISCRIM Procedure

Total Sample Size150DF Total149
Variables2DF Within Classes147
Classes3DF Between Classes2

Number of Observations Read150
Number of Observations Used150

Class Level Information
SpeciesVariable
Name
FrequencyWeightProportionPrior
Probability
SetosaSetosa5050.00000.3333330.333333
VersicolorVersicolor5050.00000.3333330.333333
VirginicaVirginica5050.00000.3333330.333333

Discriminant Analysis of Fisher (1936) Iris Data
Using Kernel Density Estimates with Equal Bandwidth

The DISCRIM Procedure
Classification Results for Calibration Data: SASHELP.IRIS
Cross-validation Results using Normal Kernel Density

Posterior Probability of Membership in Species
ObsFrom SpeciesClassified into
Species
Setosa VersicolorVirginica
53VersicolorVirginica*0.00000.08000.9200
100VersicolorVirginica*0.00000.41230.5877
103VirginicaVersicolor*0.00000.74740.2526
113VirginicaVersicolor*0.00000.58630.4137
136VirginicaVersicolor*0.00000.83580.1642

* Misclassified observation


Discriminant Analysis of Fisher (1936) Iris Data
Using Kernel Density Estimates with Equal Bandwidth

The DISCRIM Procedure
Classification Summary for Calibration Data: SASHELP.IRIS
Cross-validation Summary using Normal Kernel Density

Number of Observations and Percent Classified
into Species
From SpeciesSetosa VersicolorVirginica Total
Setosa
50
100.00
0
0.00
0
0.00
50
100.00
Versicolor
0
0.00
48
96.00
2
4.00
50
100.00
Virginica
0
0.00
3
6.00
47
94.00
50
100.00
Total
50
33.33
51
34.00
49
32.67
150
100.00
Priors
0.33333
 
0.33333
 
0.33333
 
 
 

Error Count Estimates for Species
 Setosa VersicolorVirginica Total
Rate0.00000.04000.06000.0333
Priors0.33330.33330.3333 

Discriminant Analysis of Fisher (1936) Iris Data
Using Kernel Density Estimates with Equal Bandwidth

The DISCRIM Procedure
Classification Summary for Test Data: WORK.PLOTDATA
Classification Summary using Normal Kernel Density

Observation Profile for Test Data
Number of Observations Read11175
Number of Observations Used11175

Number of Observations and Percent Classified
into Species
 Setosa VersicolorVirginica Total
Total
3195
28.59
2492
22.30
5488
49.11
11175
100.00
Priors
0.33333
 
0.33333
 
0.33333
 
 
 


disx2qc
External File:images/disx2qc1.png
External File:images/disx2qc2.png

Another nonparametric analysis is run with unequal bandwidths (POOL=NO). These statements produce Output 36.2.5:

title2 'Using Kernel Density Estimates with Unequal Bandwidth';

proc discrim data=sashelp.iris method=npar kernel=normal
   r=.5 pool=no testoutd=plotd testdata=plotdata testout=plotp
   short noclassify crosslisterr;
   class Species;
   var Petal:;
run;

%contden
%contprob
%contclass

Output 36.2.5: Kernel Density Estimates with Unequal Bandwidth

Discriminant Analysis of Fisher (1936) Iris Data
Using Kernel Density Estimates with Unequal Bandwidth

The DISCRIM Procedure

Total Sample Size150DF Total149
Variables2DF Within Classes147
Classes3DF Between Classes2

Number of Observations Read150
Number of Observations Used150

Class Level Information
SpeciesVariable
Name
FrequencyWeightProportionPrior
Probability
SetosaSetosa5050.00000.3333330.333333
VersicolorVersicolor5050.00000.3333330.333333
VirginicaVirginica5050.00000.3333330.333333

Discriminant Analysis of Fisher (1936) Iris Data
Using Kernel Density Estimates with Unequal Bandwidth

The DISCRIM Procedure
Classification Results for Calibration Data: SASHELP.IRIS
Cross-validation Results using Normal Kernel Density

Posterior Probability of Membership in Species
ObsFrom SpeciesClassified into
Species
Setosa VersicolorVirginica
53VersicolorVirginica*0.00000.05160.9484
100VersicolorVirginica*0.00000.37730.6227
103VirginicaVersicolor*0.00000.78260.2174
136VirginicaVersicolor*0.00000.88020.1198

* Misclassified observation


Discriminant Analysis of Fisher (1936) Iris Data
Using Kernel Density Estimates with Unequal Bandwidth

The DISCRIM Procedure
Classification Summary for Calibration Data: SASHELP.IRIS
Cross-validation Summary using Normal Kernel Density

Number of Observations and Percent Classified
into Species
From SpeciesSetosa VersicolorVirginica Total
Setosa
50
100.00
0
0.00
0
0.00
50
100.00
Versicolor
0
0.00
48
96.00
2
4.00
50
100.00
Virginica
0
0.00
2
4.00
48
96.00
50
100.00
Total
50
33.33
50
33.33
50
33.33
150
100.00
Priors
0.33333
 
0.33333
 
0.33333
 
 
 

Error Count Estimates for Species
 Setosa VersicolorVirginica Total
Rate0.00000.04000.04000.0267
Priors0.33330.33330.3333 

Discriminant Analysis of Fisher (1936) Iris Data
Using Kernel Density Estimates with Unequal Bandwidth

The DISCRIM Procedure
Classification Summary for Test Data: WORK.PLOTDATA
Classification Summary using Normal Kernel Density

Observation Profile for Test Data
Number of Observations Read11175
Number of Observations Used11175

Number of Observations and Percent Classified
into Species
 Setosa VersicolorVirginica Total
Total
1370
12.26
1505
13.47
8300
74.27
11175
100.00
Priors
0.33333
 
0.33333
 
0.33333
 
 
 


disx2vc
External File:images/disx2vc1.png
External File:images/disx2vc2.png