GRADBOOST Procedure
PROC GRADBOOST Statement
PROC GRADBOOST <options>;
The PROC GRADBOOST statement invokes the procedure. Table 2 summarizes the options in this statement.
Table 2: PROC GRADBOOST Statement Options
| Option | Description |
|---|---|
| Basic Options | |
| ADDTREES | Adds trees to an already generated boosting model, which is specified in the INMODEL= option |
| BINMETHOD= | Specifies how to bin interval inputs prior to training |
| DISTRIBUTION= | Specifies the distribution of the objective function |
| EARLYSTOP= | Specifies how to perform early stopping during model training |
| INMODEL= | Specifies a saved gradient boosting model to use for scoring a new table |
| LASSO= | Specifies the L1 norm regularization parameter |
| LEARNINGRATE= | Specifies the learning rate for each tree |
| NOPRINT | Suppresses ODS output |
| NTHREADS= | Specifies the number of threads to use in the computation |
| NTREES= | Specifies the number of trees to grow in the boosting model |
| NUMBIN= | Specifies the number of bins for continuous variables |
| OFFSET= | Specifies an offset variable to use with DISTRIBUTION=POISSON or TWEEDIE |
| OUTMODEL= | Specifies the data table to which the gradient boosting model is to be saved |
| PRINTTARGET | Creates tables that contain information about columns in the output |
| RBAIMP | Creates a variable importance table by using random branch assignment |
| RIDGE= | Specifies the L2 norm regularization parameter |
| SAMPLINGRATE= | Specifies the fraction of the training data to use for growing each tree |
| SEED= | Specifies the random number seed to use for model building |
| VARS_TO_TRY= | Specifies the number of variables to examine at each node split |
| VII= | Calculates the importance of the specified types of variable interactions |
| Splitting Options | |
| ASSIGNMISSING= | Specifies how to handle missing values in a predictor variable |
| MAXBRANCH= | Specifies the maximum number of splits per node |
| MAXDEPTH= | Specifies the maximum tree depth |
| MINLEAFSIZE= | Specifies the minimum number of observations per leaf |
| MINUSEINSEARCH= | Specifies the minimum number of observations to use with the USEINSEARCH policy for handling missing values |
| NOMSEARCH= | Specifies the method to use to find a split on a nominal variable |
You also specify the following options:
- ADDTREES
adds trees to an already generated gradient boosting model, which is specified in the INMODEL= option. The resulting model will consist of all the trees in the original model, plus a number of new trees, where the number of new trees is specified in the NTREES= option. When you update a model, you can use the original training data or you can use new data. If you use new data, then the new data must have all the variables that the original training data have.
- ASSIGNMISSING=NONE | MACSMALL | USEINSEARCH
-
specifies how to handle missing values during training and creates a splitting rule to handle missing values and unknown levels during scoring. An unknown level is a level of a categorical predictor variable that does not exist in the training data but is encountered during scoring.
During model training, PROC GRADBOOST searches for the best splitting rule for each node, as described in the section Training a Decision Tree. During model scoring, observations are assigned to a node in a tree based on the best splitting rule if that rule’s variable is not missing. If the variable is missing for the observation, then the default splitting that is specified by this option is used. The default splitting rule enables all data to be scored, even if the best splitting rule cannot be used on a particular observation.
You can specify one of the following values:
- NONE
during training, excludes observations that have any missing variables from the model. In the scoring phase, this default rule assigns observations that have missing values of an interval predictor variable to the branch with the smallest predictor values, and assigns observations that have unknown and missing nominal levels to the branch with the most training observations.
- MACSMALL
during training, treats a missing value as a separate, legitimate value in the search for a split for the primary splitting rule. Missing values in interval predictor variables are treated as less than any other number. In the scoring phase, this default rule assigns observations that have missing values of interval predictor variables to the leftmost branch of the split, and assigns observations that have unknown nominal levels to the largest branch in the split.
- USEINSEARCH
during training, treats a missing value as a separate, legitimate value in the search for a split for the primary splitting rule. Missing values in interval predictor variables are treated as a special level that is used during the split process. In the scoring phase, this default rule assigns observations that have missing values of interval predictor variables to the branch determined during the model growing, and assigns observations that have unknown nominal levels to the largest branch in the split.
By default, ASSIGNMISSING=USEINSEARCH.
- BINMETHOD=BUCKET | QUANTILE
-
specifies how to bin interval input variables prior to growing the gradient boosting model. The number of bins that are created is determined by the NUMBIN= option.
You can specify one of the following values:
- BUCKET
bins interval input variables into fixed-width bins. The width of each bin for a particular variable is calculated by subtracting the smallest value among all observations from the largest value among all observations, and then dividing that result by the number of bins.
- QUANTILE
bins interval input variables into bins according to their quantile. The width of the bins for a particular variable is not fixed, but the number of observations in each bin is approximately equal.
By default, BINMETHOD=QUANTILE.
- DATA=libref.data-table
-
names the input data table for PROC GRADBOOST to use. The default is the most recently created data table. libref.data-table is a two-level name, where
- libref
refers to a collection of information that is defined in the LIBNAME statement and includes the
library, which includes a path to the data. For more information about libref, see the section Using SAS Viya Workbench.- data-table
specifies the name of the input data table.
- DISTRIBUTION=BINARY | GAUSSIAN | MULTINOMIAL | POISSON | TWEEDIE(POWER=p)
specifies the distribution of the objective function. The default distribution is GAUSSIAN, BINARY, or MULTINOMIAL for an interval, binary, or nominal target, respectively. The POISSON distribution is appropriate for count data. The TWEEDIE distribution is useful for modeling total losses in insurance. The default value of POWER=p is 1.5, and
p 
. - EARLYSTOP(suboption …)
-
specifies options for stopping the gradient boosting model training early. For more information, see the section Early Stopping. Two approaches to early stopping are available: stagnation and threshold. You can specify one or more of the following suboptions:
- METRIC=MCR | LOGLOSS
-
specifies whether to use the misclassification rate or the log-loss prediction error in early stopping for a binary or nominal target.
By default, METRIC=MCR. Average square error is always used for an interval target.
- MINIMUM=NO | YES
-
specifies whether the stagnation approach should count iterations starting from the iteration that has the smallest error.
By default, MINIMUM=NO.
- STAGNATION=number
-
specifies the number of iterations in the gradient boosting model to consider for early stopping, where number must be a nonnegative integer. If STAGNATION=0, then the stagnation approach is not used. If you want to use autotuning by specifying the AUTOTUNE statement, but you do not want autotuning to use early stopping, then specify STAGNATION=0.
By default, STAGNATION=0. When you specify the AUTOTUNE statement, STAGNATION=4 by default.
-
THRESHOLD=
-
specifies the threshold value that stops training when the error equals or exceeds it.
By default, THRESHOLD=0 and the threshold approach to early stopping is not used.
- THRESHOLDITER=T
specifies the minimum number of training iterations to run before the threshold approach is invoked.
- TOLERANCE=number
-
specifies the number to be used as the tolerance for the stagnation approach to early stopping, where number must be nonnegative.
By default, TOLERANCE=0.
- INMODEL=libref.data-table
-
specifies the data table that you previously saved as a gradient boosting model by using the OUTMODEL= option in a previous run of PROC GRADBOOST. libref.data-table is a two-level name, where libref refers to the library, and data-table specifies the name of the input data table. For more information about this two-level name, see the DATA= option and the section Using SAS Viya Workbench.
You can specify the INMODEL= option for the following purposes:
To score new data. In this case, both the DATA= option and the OUTPUT statement are required, and any other options, except for NOPRINT, are ignored.
To add trees to an existing gradient boosting model. In this case, the GRADBOOST procedure uses the originally saved model and adds new trees to it by using the same or new data. If new data are used, then they must contain all the variables of the original data. When you use the INMODEL= option to add trees to an existing gradient boosting model, you must also specify the ADDTREE option. In this case, all other training options are also honored.
The data table used in the INMODEL= option must include the attributes that are associated with the table produced by the OUTMODEL= option, or PROC GRADBOOST issues an error.
-
LASSO=number
L1=number -
specifies the L1 norm regularization parameter, where number must be nonnegative.
By default, LASSO=0. This value can be tuned with the AUTOTUNE statement.
- LEARNINGRATE=number
-
specifies the learning rate for the gradient boosting algorithm, where number must be between 0 and 1, inclusive.
By default, LEARNINGRATE=0.1. This value can be tuned with the AUTOTUNE statement.
- MAXBRANCH=b
-
specifies the maximum number of children per node in the tree. PROC GRADBOOST tries to create this number of children unless it is impossible (for example, if a split variable does not have enough levels).
By default, MAXBRANCH=2.
- MAXDEPTH=number
-
specifies the maximum depth of the tree to be grown. The number of levels in a tree is equal to the depth plus 1.
By default, MAXDEPTH=4.
This value can be tuned with the AUTOTUNE statement. In the ODS output that corresponds to tables that are generated using the AUTOTUNE statement, the term "MAXLEVEL" is used. The value of MAXLEVEL is equal to the value of the MAXDEPTH= option plus 1.
-
MINLEAFSIZE=number
LEAFSIZE=number -
specifies the minimum number of observations that each child of a split must contain in the training data table in order for the split to be considered. The count of observations includes those that are excluded from training as a consequence of specifying a value less than one in the SAMPLINGRATE= option.
By default, MINLEAFSIZE=5. This value can be tuned with the AUTOTUNE statement.
- MINUSEINSEARCH=number
-
specifies a threshold for using missing values in the split search when ASSIGNMISSING=USEINSEARCH. If the number of observations in which the splitting variable has missing values is greater than or equal to number, then PROC GRADBOOST uses the USEINSEARCH policy to handle missing values for that variable.
By default, MINUSEINSEARCH=1.
- NOMSEARCH(suboption …)
-
specifies search methods for splitting on a nominal variable. For more information, see the section Splitting Nominal Values. If you do not specify this option, the default values of the suboptions determine the search method. You can specify one or more of the following suboptions:
- NOPRINT
- NTHREADS=number
The gradient boosting algorithm automatically determines the default value on the basis of the properties of the training data, model, and hardware that you use. If you specify this option, the specified number of threads is used if it is less than or equal to the maximum number of threads that Viya allows. If the number of threads is larger than the maximum number of threads that Viya allows, the maximum number allowed is used instead.
- NTREES=number
-
specifies the number of trees to grow in the gradient boosting model.
By default, NTREES=100. This value can be tuned with the AUTOTUNE statement.
-
NUMBIN=number
NBINS=number -
specifies the number of bins to use for binning the interval input variables. PROC GRADBOOST bins continuous predictors to a fixed bin size. This option controls the number of bins and thereby also the size of the bins.
By default, NUMBIN=50.
- OFFSET=variable
specifies an offset variable to be used with DISTRIBUTION=POISSON or TWEEDIE.
- OUTMODEL=<libref.>data-table
-
specifies the data table to which to save the gradient boosting model. libref.data-table is a two-level name, where libref refers to the library, and data-table specifies the name of the output data table. For more information about this two-level name, see the DATA= option and the section Using SAS Viya Workbench.
The data table that results from this option contains information about each node and each tree in the gradient boosting model, including the splitting variables, the child nodes, the number of observations at each node, and the predicted response at each node. This table also has attributes attached that contain information about the input variables, the target variable, and the gradient boosting model.
- PRINTTARGET
outputs tables that indicate generated columns in the OUT= table from the OUTPUT statement. For a continuous response, PROC GRADBOOST generates an output table named PredName, which indicates the name of the predicted value column. For a categorical response, PROC GRADBOOST generates an output table named PredIntoName, which indicates the name of the predicted value column, and also an output table named PredProbName, which indicates the names of the predicted probability columns.
- RBAIMP
creates a variable importance table by using random branch assignment (RBA). This table is created in addition to the normal variable importance table that is calculated using the residual sum of squares (RSS) error. For more information about RBA and RSS variable importance, see the section Measuring Variable Importance.
-
RIDGE=number
L2=number -
specifies the L2 norm regularization parameter on prediction. The value of number must be nonnegative.
By default, RIDGE=1. This value can be tuned with the AUTOTUNE statement.
- SAMPLINGRATE=number
-
specifies the fraction of the training data to be used for growing each tree in the boosting model.
By default, SAMPLINGRATE=0.5. This value can be tuned with the AUTOTUNE statement.
- SEED=number
specifies the initial seed for random number generation for model building. The value of number must be an integer. If you do not specify a seed or you specify a value less than or equal to 0, the seed is generated by reading the time of day from the computer’s clock.
-
VARS_TO_TRY=m
M=m -
specifies the number of input variables to consider splitting on in a node, where m ranges from 1 to the number of input variables.
By default, m is the number of input variables. This value can be tuned with the AUTOTUNE statement.
-
VII=2 | 3
INTERACTIONIMP=2 | 3 -
calculates the variable interaction importance, which is described in the section Interaction Importance. You can specify the following values:
- 2
calculates the importance of all two-way variable interactions.
- 3
calculates the importance of all three-way and all two-way variable interactions.
If you do not specify this option, then the variable interaction importance is not calculated.