The FOREST Procedure

PROC FOREST Statement

PROC FOREST <options>;

The PROC FOREST statement invokes the procedure. Table 2 summarizes the options in the PROC FOREST statement.

Table 2: PROC FOREST Statement Options

Option	Description
Basic Options
BINMETHOD=	Specifies how to bin interval inputs prior to training
DATA=	Specifies the name of the input table
INBAGFRACTION=	Specifies the fraction of the training data to use for growing each tree
INMODEL=	Specifies a saved forest model to use to score a new table
ISOLATION	Generates an isolation forest for anomaly detection instead of a forest for target prediction
LOH=	Specifies the number of variables to preselect using the Loh method
NOPRINT	Suppresses ODS output
NTHREADS=	Specifies the number of threads to use in the computation
NTREES=	Specifies the number of trees to grow in the forest model
NUMBIN=	Specifies the number of bins for continuous variables
OUTMODEL=	Specifies the data table to score the forest model
PRINTTARGET	Create tables that contain information about columns in the output
RBAIMP	Creates a variable importance table by using random branch assignment
SEED=	Specifies the random number seed to use for model building
VARS_TO_TRY=	Specifies the number of variables to examine at each node split
VII=	Calculates the importance of the specified types of variable interactions
VOTE=	Specifies the method for calculating the predicted probabilities for a nominal target
Splitting Options
ASSIGNMISSING=	Specifies how to handle missing values in a predictor variable
MAXBRANCH=	Specifies the maximum number of splits per node
MAXDEPTH=	Specifies the maximum tree depth
MINLEAFSIZE=	Specifies the minimum number of observations per leaf
MINUSEINSEARCH=	Specifies the minimum number of observations to use with the USEINSEARCH policy for handling missing values
NOMSEARCH=	Specifies the method to use to find a split on a nominal variable

You can specify the following options:

ASSIGNMISSING=NONE |MACSMALL |USEINSEARCH

specifies how PROC FOREST creates a default splitting rule that is used to handle missing values and unknown levels. An unknown level is a level of a categorical predictor that does not exist in the training data but is encountered during scoring.

This option controls how missing values are used in model training, and controls the creation of the default splitting rule.

The primary splitting rule for a node is created during model training. During model scoring, observations are assigned to a node in a tree based upon the primary splitting rule if the rule’s variable is not missing. If the variable is missing for the observation, then the default splitting rule is used.

The default splitting rule enables all data to be scored, even if the primary rule cannot be used on a particular observation.

You can specify one of the following values to determine the default splitting rule:

NONE: excludes observations that have any missing variables from training the forest model. In the scoring phase, this default rule assigns observations that have missing values of an interval predictor variable to the branch with the smallest predictor values, and assigns observations that have unknown and missing nominal levels to the branch with the most training observations.
MACSMALL: treats a missing value as a separate, legitimate value in the search for a split for the primary splitting rule. Missing values in interval inputs are treated as less than any other number. In the scoring phase, this default rule assigns missing interval inputs to the leftmost branch of the split, and unknown nominal levels to the largest branch in the split.
USEINSEARCH: treats a missing value as a separate, legitimate value in the search for a split for the primary splitting rule. Missing values in interval inputs are treated as a special level that is used during the split process. In the scoring phase, this default rule assigns missing interval inputs to the branch determined during forest growing, and unknown nominal levels to the largest branch in the split.

By default, ASSIGNMISSING=USEINSEARCH.

BINMETHOD=BUCKET |QUANTILE

specifies how to bin interval input variables prior to growing the forest model. The number of bins that are created is determined by the NUMBIN= option.

You can specify one of the following values:

BUCKET: bins interval input variables into fixed-width bins. The width of each bin for a particular variable is calculated by subtracting the smallest value among all observations from the largest value among all observations, and then dividing that result by the number of bins.
QUANTILE: bins interval input variables into bins according to their quantile. The width of the bins for a particular variable is not fixed, but the number of observations in each bin is approximately equal.

By default, BINMETHOD=QUANTILE.

DATA=CAS-libref.data-table

names the input data table for PROC FOREST to use. The default is the most recently created data table. CAS-libref.data-table is a two-level name, where

CAS-libref: refers to a collection of information that is defined in the LIBNAME statement and includes the caslib, which includes a path to the data, and a session identifier, which defaults to the active session but which can be explicitly defined in the LIBNAME statement. For more information about CAS-libref, see the section Using CAS Sessions and CAS Engine Librefs.
data-table: specifies the name of the input data table.

INBAGFRACTION=number

specifies the fraction of the random bootstrap sample of the training data to be used for growing each tree in the forest, where number is a value between 0 and 1. Observations can be duplicated in the sample. The number of distinct observations in the sample will be less than the number of observations available for training even when number is 1.

By default, INBAGFRACTION=0.6. This value can be tuned with the AUTOTUNE statement.

INMODEL=<CAS-libref.>data-table

specifies the data table that you have previously saved as a forest model by using the OUTMODEL= option in a previous run of PROC FOREST. CAS-libref.data-table is a two-level name, where CAS-libref refers to the caslib and session identifier, and data-table specifies the name of the input data table. For more information about this two-level name, see the DATA= option and the section Using CAS Sessions and CAS Engine Librefs.

When you use the INMODEL= option, the OUTPUT statement is required and any other options in the PROC FOREST statement, except for NOPRINT and VOTE=, are ignored.

The data table used in the INMODEL= option must include the attributes that are associated with the table produced by the OUTMODEL= option, or the FOREST procedure will error.

ISOLATION <(SAMPLEN=number)>

creates an isolation forest for anomaly detection instead of creating a forest for target prediction. For more information about isolation forests, see section Isolation Forests.

You can specify the following option:

SAMPLEN=number

specifies the number of observations, sampled without replacement, to use in each tree of the isolation forest.

By default, SAMPLEN=100.

When the ISOLATION option is specified, the GROW and TARGET statements are ignored and the BINMETHOD, INBAGFRACTION, RBAIMP, VARS_TO_TRY, and VII options in the PROC FOREST statement are ignored. Fit statistics are not computed.

LOH=L

specifies a number of variables (L) that are preselected to consider for candidate splits for each node. The variables are selected using the Loh method.

If L is less than the value of the VARS_TO_TRY= option (m), then the variables are selected from among the m variables. If L is greater than or equal to m, or if no L is specified, then the Loh method is not used.

MAXBRANCH=b

specifies the maximum number of children per node in the tree. PROC FOREST tries to create this number of children unless it is impossible (for example, if a split variable does not have enough levels).

By default, MAXBRANCH=2.

MAXDEPTH=number

specifies the maximum depth of the tree to be grown. The number of levels in a tree is equal to the depth plus one.

By default, MAXDEPTH=20.

MINLEAFSIZE=number

specifies the minimum number of observations that each child of a split must contain in the training data table in order for the split to be considered. The count of observations includes those that are excluded from training as a consequence of specifying a value less than 1 in the INBAGFRACTION= option. By default, MINLEAFSIZE=5. This value can be tuned with the AUTOTUNE statement.

MINUSEINSEARCH=number

specifies a threshold for using missing values in the split search when ASSIGNMISSING=USEINSEARCH. If the number of observations in which the splitting variable has missing values is greater than or equal to number, then PROC FOREST uses the USEINSEARCH policy to handle missing values for that variable.

By default, MINUSERINSEARCH=1.

NOMSEARCH(suboption …)

specifies search methods for splitting on a nominal variable. For more information, see the section Splitting Nominal Values. If you do not specify this option, the default values of the suboptions determine the search method. You can specify one or more of the following suboptions:

MAXCATEGORIES=number

specifies the maximum number of categories to use in a splitting rule. The input is eligible for splitting nodes in which no more than number categories appear. The value of number must be greater than 1.

By default, MAXCATEGORIES=128.

SHRINKAGE=number

specifies how much weight to give the average gradient when you combine it with the average gradient within a category. You use this option only when sorting the categories. The value of number must be nonnegative.

By default, SHRINKAGE=10.

SORT=number

specifies the minimum cardinality in the node of the nominal variable for using the sort method. The sort method sorts the categories and then examines binary splits that preserve the ordering. The value of number must be a nonnegative integer. If number is greater than the value of the MAXCATEGORIES=number option, then the variable is not used in the node.

By default, SORT=10.

NOPRINT

suppresses ODS output.

NTHREADS=number-of-threads

specifies the number of threads to use in the computation. The default value is the number of CPUs available on the machine.

NTREES=number

specifies the number of trees to grow in the forest model.

By default, NTREES=100. This value can be tuned with the AUTOTUNE statement.

NUMBIN=number

specifies the number of bins in which to bin the interval input variables. PROC FOREST bins continuous predictors to a fixed bin size. This option controls the number of bins and thereby also the size of the bins.

By default, NUMBIN=50. This value can be tuned with the AUTOTUNE statement.

OUTMODEL=<CAS-libref.>data-table

specifies the data table to which to save the forest model. CAS-libref.data-table is a two-level name, where CAS-libref refers to the caslib and session identifier, and data-table specifies the name of the output data table. For more information about this two-level name, see the DATA= option and the section Using CAS Sessions and CAS Engine Librefs.

The data table that results from this option contains information about each node and each tree in the forest model, including the splitting variables, the child nodes, the number of observations at each node, and the predicted response at each node. This table also has attributes attached that contain information about the input variables, the target variable, and the forest model.

PRINTTARGET

outputs tables that indicate generated columns in the OUT= table from the OUTPUT statement. For a continuous response, PROC FOREST generates an output table named PredName, which indicates the name of the predicted value column. For a categorical response, PROC FOREST generates an output table named PredIntoName, which indicates the name of the predicted value column, and also an output table named PredProbName, which indicates the names of the predicted probability columns.

RBAIMP

creates a variable importance table by using random branch assignment (RBA). This table is created in addition to the normal variable importance table that is calculated using the residual sum of squares (RSS) error. For more information about RBA and RSS variable importance, see the section Measuring Variable Importance.

SEED=number

specifies the initial seed for random number generation for model building. The value of number must be an integer. If you do not specify a seed or you specify a value less than or equal to 0, the seed is generated from reading the time of day from the computer’s clock.

VARS_TO_TRY=m M=m

specifies the number of input variables to consider splitting on in a node, where m ranges from 1 to the number of input variables.

By default, m is the square root of the number of input variables. This value can be tuned with the AUTOTUNE statement.

VII=2 |3 INTERACTIONIMP=2 |3

calculates the variable interaction importance, which is described in the section Variable Interaction Importance. You can specify the following values:

2: calculates the importance of all two-way variable interactions.
3: calculates the importance of all three-way and all two-way variable interactions.

If you do not specify this option, then the variable interaction importance is not calculated.

VOTE=MAJORITY |PROBABILITY

specifies how to calculate the predicted probability of the target levels for a nominal target. The predicted level is the level that has the highest predicted probability. This option affects the scoring and fit statistics of the forest model. You can specify the following values:

MAJORITY: specifies that the predicted probability of each target level is equal to the number of trees in the forest that predicted that level as the target, divided by the total number of trees in the forest.
PROBABILITY: specifies that the predicted probability of each target level is equal to the probability of that level averaged over each tree in the forest.

When using the INMODEL= option, you can specify how to calculate the predicted probability of the target levels, even if the model was trained with a different value for the VOTE= option.

By default, VOTE=PROBABILITY.

Last updated: November 11, 2020