The HPFOREST Procedure

Breiman’s method is also called a permutation-based or randomization method. Breiman’s method calculates importance as

where is the prediction for observation i and is the prediction for observation i after randomizing the values of input v. In Breiman’s writings, the sum uses only out-of-bag observations, and randomizing is done by permuting the out-of-bag values of v. Originally Breiman (2001) uses misclassification as the loss function. Breiman and Cutler (2003) retract that, saying misclassification loss is too volatile with many variables. Instead they recommend the margin for a nominal target: the probability of the true class minus the maximum probability among the other classes. Breiman (2001) bases the loss on the entire forest, not a single tree. Today authors generally compute the importance for each tree and then average these (Berk 2008; Grömping 2009).