The TREESPLIT Procedure
Variable Importance
A training data table can contain a large number of predictor variables, some of which are useful for predicting the response variable and others of which are not. You can use the TREESPLIT procedure to select the most useful predictor variables on the basis of variable importance. (See Example 21.3: Assessing Variable Importance.) Variable importance is an indication of which predictor variables are the most useful for predicting the response variable.
The most important variables might not be the ones near the top of the tree. The TREESPLIT procedure implements several methods for computing variable importance, which are described in the following subsections. By default, PROC TREESPLIT calculates the variable importance by three methods (count-based, change in the residual sum of square errors, and relative importance), and outputs the results from these variable importance methods in one table. You can request that the TREESPLIT procedure also calculate the variable importance by random branch assignment (RBA) by specifying the RBAIMP option.
Count-Based Importance Method
PROC TREESPLIT calculates two count-based variable importance methods:
Count-based variable importance simply counts the number of times in the tree that a particular variable is used in a split.
If the NSURROGATES= option is used, then surrogate-count-based variable importance also counts the number of times that a variable is used in a surrogate splitting rule.
Residual Sum of Squares (RSS) Importance Method
This method measures variable importance based on the change of RSS when a split is found at a node. The change for variable v is
where
If the change in RSS is negative (which is possible when you measure the validation set), then the change is set to 0.
The residual sum of squares for regression trees is defined as
where
The residual sum of squares for classification trees is defined as
where
If surrogate rules are in effect, they are also credited with a portion of the change in RSS. The credit is proportional to the agreement between the primary and surrogate splitting rules at the node. The agreement at node d (denoted by ) is defined as
where is the number of nonmissing observations and is the number of observations that were assigned to i by both the primary and surrogate rules.
The change in RSS from the surrogate rules for variable v is defined as
The RSS-based importance for a single variable, v, is then defined as
where D is the total number of nodes in which v was used as the splitting variable.
Relative Variable Importance Method
The relative variable importance metric is a number between 0 and 1, which is calculated in two steps:
PROC TREESPLIT finds the maximum RSS-based variable importance.
For each variable, PROC TREESPLIT calculates the relative variable importance as the RSS-based importance of this variable divided by the maximum RSS-based importance among all the variables.
The RSS and relative importance are calculated from the validation data. If no validation data exist, they are calculated instead from the training data.
Variable Interaction Importance
In some cases, interactions of variables are of more interest than a single variable. When you specify the VII= option in the PROC TREESPLIT statement, the procedure computes variable interaction importance as follows.
The two-way interaction importance for variables v and w is
where
d and e denote nodes, with node d being the parent of node e
v and w are variables, with v being the splitting variable for node d and w being the splitting variable for node e
this summation is across all parent-child node combinations with respect to the variables v and w
The three-way interaction importance for variables v, w, and x is
where
d, e, and f denote nodes, with node d being the parent of node e, which is the parent of node f
v, w, and x are variables, with v being the splitting variable for node d, w being the splitting variable for node e, and x being the splitting variable for node f
this summation is across all parent-child-grandchild node combinations with respect to the variables v, w, and x
When comparing variable importance values, you should only compare a two-way interaction with another two-way interaction, or a three-way interaction with another three-way interaction. Comparing a two-way interaction to a three-way interaction is not meaningful, because the values for the importance are not on the same scale.
In addition to calculating the variable interaction importance, you can also generate SAS DATA step code to create interaction variables and save the code to a file by using the VIICODE statement. When you run the generated SAS DATA step code on the original data or on new data, interaction variables are created.
Created interaction variables are of the form , , or , where the values used in the calculations come from the variables whose interactions are determined to be important. The values used are as follows:
For continuous variables, the raw value of the variable is used.
For categorical variables, the proportion of observations in the training set that is assigned to a specific branch is used.
Random Branch Assignment Importance Method
The random branch assignment (RBA) method computes the importance of an input variable v by comparing how well the data fit the predictions before and after they are modified. Neville and Tan (2014) motivate and introduce the RBA method of variable importance. To modify the predictions, the TREESPLIT procedure replaces all splitting rules that use variable v by a rule that randomly assigns an observation to a branch. The probability of assigning an observation to a branch is proportional to the number of observations that are assigned to the branch in the current data. The current data are the training data when RBA is computed during training. Otherwise, the current data are those being scored on an existing model.
The RBA importance can be expressed mathematically as
where is the modified prediction for observation i and is the standard prediction.
For an continuous response, PROC TREESPLIT computes the RBA importance of squared error loss. For a categorical response, PROC TREESPLIT uses the misclassification rate as the loss function.
Neville and Tan (2014) motivate and introduce the RBA method of variable importance.