The FOREST Procedure

Residual Sum of Squares Importance Method

The residual sum of squares (RSS) for regression trees is defined as

where

i is an observation on leaf
is the predicted value of the response variable of observation i
is the actual value of the response variable on leaf

The residual sum of squares (RSS) for classification trees is defined as

where

is the actual response level
is the number of observations on leaf that have response level
is the posterior probability for the response level on leaf
is the posterior probability for the actual response level on leaf

For a single tree in the forest, the RSS-based metric measures variable importance based on the change in RSS when a split is found at a node. The change is

where

d denotes the node
i denotes the index of a child that this node includes
is the RSS if the node is treated as a leaf
is the RSS of the node after it has been split

If the change in RSS is negative (which is possible when you use the validation set), then the change is set to 0.

The RSS-based importance for a single tree is then defined as

where D is the total number of nodes.

The RSS variable importance for the forest is the average of the RSS variable importance across all trees in the forest.