The TREESPLIT Procedure
Splitting Criteria
The goal of recursive partitioning, as described in the section Building a Decision Tree, is to subdivide the predictor space in such a way that the values of the response variable for the observations within a terminal node are similar as possible. The TREESPLIT procedure provides two types of criteria for splitting a parent node: criteria that maximize a decrease in node impurity, as defined by an impurity function, and criteria that are defined by a statistical test. You can specify the criterion in the GROW statement. If you do not specify a criterion in the GROW statement, the entropy criterion is used for classification trees and the RSS criterion is used for regression trees.
Criteria Based on Impurity
The entropy, Gini index, and RSS criteria decrease impurity. The impurity of a parent node is defined as , a nonnegative number that is equal to zero for a pure node (a node in which all the observations have the same value of the response variable). Nodes in which the observations have very different values of the response variable have a large impurity.
The TREESPLIT procedure selects the best split variable and the best split value to produce the highest reduction in impurity,
where denotes the bth child node, p(|) is the proportion of observations in that are assigned to , and B is the number of branches after splitting .
Impurity Reduction Criteria for Classification Trees
The following impurity reduction criteria available for classification trees are based on different impurity functions i():
Entropy criterion: The entropy impurity of node is defined as
where is the proportion of observations that have the jth response value.
Gini index criterion: This criterion defines i() as the Gini index that corresponds to the ASE of a class response and is given by
For more information, see Hastie, Tibshirani, and Friedman (2009).
Impurity Reduction Criterion for Regression Trees
Only one impurity reduction criterion, the RSS criterion, is available for regression trees. This criterion, also called the ANOVA criterion, defines i() as the residual sum of squares,
where is the number of observations in , is the response value of observation i, and is the average response of the observations in .
Criteria Based on Statistical Test
The chi-square, F test, and CHAID criteria are defined by statistical tests. These criteria calculate the worth of a split by testing for a significant difference in the response variable across the branches that are defined by a split. The worth is defined as , where p is the p-value of the test. You can adjust the p-values for these criteria by specifying the BONFERRONI option in the GROW statement.
Statistical Criterion for Classification Trees
In the chi-square criterion for categorical response variables, the worth is based on the p-value for the Pearson chi-square test, which compares the frequencies of the levels of the response across the child nodes.
Statistical Criterion for Regression Trees
In the F-test criterion for continuous response variables, the worth is based on the F test for the null hypothesis that the means of the response values are identical across the child nodes. The test statistic is
where
Statistical Criterion for Both Classification and Regression Trees
Available for both categorical and continuous response variables, CHAID, first described by Kass (1980), is an approach that regards every possible split as representing a test. CHAID tests the hypothesis of no association between the values of the response (target) variable and the branches of a node. The Bonferroni adjusted probability is defined as m, where is the significance level of a test and m is the number of independent tests.