The HPFOREST Procedure

This example uses diabetes data to illustrate PROC HPFOREST. Diabetes is a major American disease. The American Diabetes Association estimates that over 8% of Americans have diabetes, and diabetes costs Americans over $175 billion a year. The National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) has been studying diabetes and obesity in the Akimel O’otham (formerly known as Pima Indians) in Arizona for over 30 years. Smith et al. (1988) prepared some of the NIDDK data for forecasting the onset of diabetes and then donated the data for community use. Since then, the data have been applied to dozens of experimental algorithms for predicting the onset of diabetes.

The Pima Indians Diabetes data set is in the UCI Machine Learning Repository (Lichman 2013) and is available at http://archive.ics.uci.edu/ml/datasets/Pima+Indians+Diabetes.^[3]The following SAS statements create a SAS data set from a URL:

   data diabetes;
      %let url=//archive.ics.uci.edu/ml/machine-learning-databases;
      infile "http:&url/pima-indians-diabetes/pima-indians-diabetes.data"
            device=url delimiter=',';
      input NumPregnancies
            PlasmaGlucose
            DiastolicBloodPr
            TricepsSkinfold
            HrSerumInsulin
            BodyMassIndex
            DiabetesPedigreeFn
            Age
            Diabetes $
      ;
   run;

The variable diabetes has values 0 and 1, 1 indicating the presence of diabetes. The other variables are raw measures on interval scales, except for DiabetesPedigreeFn, which is an interval variable created by Smith et al. (1988) to capture the family history of diabetes. PROC HPFOREST uses an interval scale for numeric variables and a nominal scale for categorical variables unless the scale is specified. The following statements run PROC HPFOREST and save the model in a binary file:

proc hpforest data=diabetes ;
   input NumPregnancies
         PlasmaGlucose
         DiastolicBloodPr
         TricepsSkinfold
         HrSerumInsulin
         BodyMassIndex
         DiabetesPedigreeFn
         Age ;
   target Diabetes;
   ods output FitStatistics=fitstats;
   save file="model";
run;

Figure 7.1 shows that the program ran locally and that four threads were used. The default number of threads is the number of processors in the computer. The listing also shows the values of the training parameters and the number of observations (768). No parameters are specified in the PROC HPFOREST statement; therefore, all the values are default. The maximum number of decision trees to create is 50. The VARS_TO_TRY= option value is 3, indicating that three of the eight input variables are randomly selected to be considered for a splitting rule.

Figure 7.2 shows the baseline fit statistics. PROC HPFOREST first computes baseline statistics without using a model. The listing shows a baseline misclassification rate of 0.349, because that is the proportion of observations for which the value of diabetes is 1.

Figure 7.3 shows the first 10 and last 10 observations of the fit statistics. When PROC HPFOREST runs in single-machine mode, it computes fit statistics for a sequence of forests that have an increasing number of trees. As the number of trees increases, the fit statistics usually improve (decrease) at first and then level off and fluctuate in a small range. Forest models provide an alternative estimate of average square error and misclassification rate, called the out-of-bag (OOB) estimate. The OOB estimate is a convenient substitute for an estimate that is based on test data and is a less biased estimate of how the model will perform on future data. For more information, see the section Bagging the Data. The listing shows that the OOB error estimate is worse (larger) than the estimate that evaluates all observations on all trees. This is usual. The OOB misclassification rate for the model fluctuates between 0.230 and 0.238, which is in a range much less than the baseline rate of rate of 0.349. Therefore, you can conclude that the model is good.

Estimates of variable importance appear after the fit statistics. The Number of Rules column in Figure 7.4 shows the number of splitting rules that use a variable. The section Measuring Variable Importance explains the measures of importance. Each measure is computed twice: once on training data and once on out-of-bag data. As with fit statistics, the out-of-bag estimates are less biased. The rows are sorted by the OOB Gini measure, which is a more stringent measure than the OOB margin measure. The OOB Gini column is negative for seven variables, and the OOB margin column is negative for two variables. The splitting rules that involve these two variables are, on average, spurious. The main conclusion from fitting the forest model to these data is that plasmaGlucose is the most important predictor of future onset of diabetes.