The TREESPLIT Procedure

AUTOTUNE Statement

AUTOTUNE <options>;

The AUTOTUNE statement searches for the best combination of values of the MAXDEPTH= and NUMBIN= options in the PROC TREESPLIT statement and the criterion in the GROW statement. If you specify the AUTOTUNE statement, then you cannot also specify k-fold cross validation with cost-complexity pruning.

Table 21.2 summarizes the options you can specify in the AUTOTUNE statement.

Table 21.2: AUTOTUNE Statement Options

Option	Description
EVALHISTORY=	Specifies how to report the evaluation history of the tuner
FRACTION=	Specifies the fraction of observations to use for validation
KFOLD=	Specifies the number of folds for k-fold cross validation
MAXBAYES=	Specifies the maximum number of points in the kriging model
MAXEVALS=	Specifies the maximum number of evaluations
MAXITER=	Specifies the maximum number of iterations when SEARCHMETHOD=GA or SEARCHMETHOD=BAYESIAN
MAXTIME=	Specifies the maximum time for all iterations
MAXTRAINTIME=	Specifies the maximum time for a model train
NPARALLEL=	Specifies the number of parallel sessions
NSUBSESSIONWORKERS=	Specifies the number of workers in parallel sessions
OBJECTIVE=	Specifies the objective function
POPSIZE=	Specifies the population size when SEARCHMETHOD=GA or SEARCHMETHOD=BAYESIAN
SAMPLESIZE=	Specifies the sample size when SEARCHMETHOD=LHS or SEARCHMETHOD=RANDOM
SEARCHMETHOD=	Specifies the search method that the optimizer uses
TARGETEVENT=	Specifies the target event for ROC-based calculations
TRAINFRACTION=	Specifies the fraction of observations to use for training
TUNINGPARAMETERS=	Specifies the custom tuning parameters
USEPARAMETERS=	Specifies how to handle the TUNINGPARAMETERS= option

EVALHISTORY=ALL |LOG |NONE |TABLE

specifies how to report the evaluation history of the tuner.

You can specify one of the following values:

ALL: reports each evaluation in the log and creates the EvaluationHistory ODS table.
LOG: prints the following information to the log for each evaluation: evaluation number, objective value, best objective value up to that point, evaluation time, and elapsed time since the beginning of the tuning process.
NONE: suppresses reporting of evaluations in the log and does not create the EvaluationHistory ODS table.
TABLE: creates the EvaluationHistory ODS table, which contains all evaluated points. The table contains columns for the evaluation number, all tuning parameters, and the objective function value.

By default, EVALHISTORY=TABLE

FRACTION=number

specifies the fraction of all data to be used for validation, where number must be between 0.01 and 0.99, inclusive. If you specify this option, the tuner uses a single-partition validation for finding the objective value (validation error estimate). Using this option might not be advisable for small or unbalanced data tables, where the random assignment of the validation subset might not provide a good estimate of error. For large, balanced data tables, a single-partition validation is usually sufficient for estimating error; a single partition is more efficient than cross validation in terms of the total execution time.

If a PARTITION statement is specified, the validation partition defined in that statement is used, and this option is ignored. You cannot specify this option in combination with the KFOLD= option.

If the TRAINFRACTION= option is used, then by default the value of the FRACTION= option is one minus the value of the TRAINFRACTION= option. Otherwise, by default, FRACTION=0.3.

KFOLD=number

specifies the number of folds (partitions) in the cross validation process, where number must be between 2 and 20, inclusive. If you specify this option, the tuner uses cross validation to find the objective value. In cross validation, each model evaluation requires number of training executions (on number–1 data folds) and number of scoring executions (on one hold-out fold). Thus, the evaluation time is increased by approximately a factor of number. For small to medium data tables or for unbalanced data tables, cross validation provides on average a better representation of error across the entire data table (a better generalization error).

If you do not specify either this option or the FRACTION= option, then the default of FRACTION=0.3 is used. If a PARTITION statement is specified, the validation partition defined in that statement is used, and this option is ignored. You cannot specify this option in combination with the FRACTION= option.

MAXBAYES=number

specifies the maximum number of points in the Kriging model. This option is only honored when SEARCHMETHOD=BAYESIAN, and has a minimum value of 10.

By default, MAXBAYES=100.

MAXEVALS=number

specifies the maximum number of configuration evaluations allowed for the tuner, where number must be an integer greater than or equal to 3. When the number of evaluations is reached, the tuner terminates the search and returns the results. To produce a single objective function value (validation error estimate), each configuration evaluation requires either a single model training and scoring execution on a validation partition, or a number of training and scoring executions equal to the value of the KFOLD= option for cross validation. The MAXEVALS= option might lead to termination before the value of the MAXITER= option or the MAXTIME= option is reached.

By default, MAXEVALS=50.

MAXITER=number

specifies the maximum number of iterations of the optimization tuner, where number must be greater than or equal to 1. Each iteration normally involves a number of objective evaluations up to the value of the POPSIZE= option. The MAXITER= option is used only when SEARCHMETHOD=GA or SEARCHMETHOD=BAYESIAN; it is ignored when SEARCHMETHOD=RANDOM or SEARCHMETHOD=LHS. The MAXITER= option might lead to termination before the value of the MAXEVALS= option or the MAXTIME= option is reached.

By default, MAXITER=5.

MAXTIME=number

specifies the maximum time (in seconds) allowed for the tuner, where number must be greater than or equal to 1. When this value is reached, the tuner terminates the search and returns results. The actual run time for optimization might be longer because it includes the remaining time needed to finish the current evaluation. For long-running model training (large data tables), the actual run time might significantly exceed number. The MAXTIME= option might lead to termination before the value of the MAXEVALS= option or the MAXITER= option is reached.

By default, MAXTIME=36000.

MAXTRAINTIME=number

specifies the maximum time allowed for a single model train. The model train is terminated if it exceeds this time, and the objective value is set to missing.

By default, there is no maximum time allowed for a single train.

NPARALLEL=number

specifies the number of evaluations to be performed in parallel, where number must be greater than or equal to 0.

By default, NPARALLEL=0, which indicates that the value of number is determined as follows:

If SEARCHMETHOD=GA or SEARCHMETHOD=BAYESIAN, then the number of parallel evaluations is equal to the value of the POPSIZE= option minus 1.
If SEARCHMETHOD=LHS or SEARCHMETHOD=RANDOM, then the number of parallel evaluations is equal to the value of the SAMPLESIZE= option, with a maximum value of 32.

NSUBSESSIONWORKERS=number

specifies the number of workers to use in parallel subsessions. When alternative configurations are evaluated in parallel, a number of subsessions is created by the tuner, with each using multiple workers. By default, the number of workers that are used in a parallel subsession is determined as described in the section Determining the Number of Parallel Evaluations.

OBJECTIVE=function

specifies which measure of model performance the tuner uses as the objective function.

You can specify one of the following values for function. Some values can be specified only when the target variable is of a particular type, as shown in parentheses.

ASE: uses average squared error as the objective function.
AUC: uses area under the curve as the objective function (nominal type only).
F05: uses the F0.5 coefficient as the objective function (nominal type only).
F1: uses the F1 coefficient as the objective function (nominal type only).
GAMMA: uses the gamma coefficient as the objective function (nominal type only).
GINI: uses the Gini coefficient as the objective function (nominal type only).
KS: uses the Kolmogorov-Smirnov coefficient as the objective function (nominal type only).
MAE: uses the mean absolute error as the objective function (interval type only).
MCE: uses the misclassification rate as the objective function (nominal type only).
MCLL: uses the multiclass log loss as the objective function (nominal type only).
MISC: uses the misclassification error percentage as the objective function (nominal type only).
MSE: uses the mean squared error as the objective function (interval type only).
MSLE: uses the mean squared logarithmic error as the objective function (interval type only).
RASE: uses the root average squared error as the objective function.
RMAE: uses the root mean absolute error as the objective function (interval type only).
RMSLE: uses the root mean squared logarithmic error as the objective function (interval type only).
TAU: uses the tau coefficient as the objective function (nominal type only).

By default, OBJECTIVE=MISC for nominal targets, and OBJECTIVE=MSE for interval targets.

POPSIZE=number

specifies the maximum population size, where population is the number of configuration evaluations in one iteration (population). The number must be greater than or equal to 1. In some cases, the tuner algorithm might generate a number of new configurations that is smaller than number. The POPSIZE= option is used only when SEARCHMETHOD=GA or SEARCHMETHOD=BAYESIAN; it is ignored when SEARCHMETHOD=RANDOM or SEARCHMETHOD=LHS.

By default, POPSIZE=10.

SAMPLESIZE=number

specifies the total number of evaluations, where number must be greater than or equal to 1. This option is only used when SEARCHMETHOD=RANDOM or SEARCHMETHOD=LHS; it is ignored when SEARCHMETHOD=GA or SEARCHMETHOD=BAYESIAN.

By default, SAMPLESIZE=50.

SEARCHMETHOD=BAYESIAN |GA |GRID |LHS |RANDOM

specifies the search method to use for tuning. You can specify the following values:

BAYESIAN: builds a kriging surrogate model to approximate the objective value and uses this surrogate model to generate new alternative configurations at each iteration. The kriging model is continuously updated during the search process.
GA: uses an initial Latin hypercube sample that seeds a genetic algorithm to generate a new population of alternative configurations at each iteration.
GRID: uses all combinations of selected values of the hyperparameters.
LHS: uses a Latin hypercube to generate a single sample of configurations that is uniform in each tuning parameter but random in combinations.
RANDOM: generates a single sample of purely random configurations.

By default, SEARCHMETHOD=GA.

TARGETEVENT=string

specifies the target event to use for calculating the selected objective function. This option is ignored when the value of the OBJECTIVE= option is not AUC, F1, F05, GINI, GAMMA, TAU, or KS.

If you do not specify the TARGETEVENT= option, the tuner selects one of the target levels and uses it for calculating the specified objective function.

TRAINFRACTION=number

specifies the fraction of all data to be used for training, where number must be between 0.01 and 0.99, inclusive. If you specify this option, the tuner uses a single-partition validation to find the objective value (validation error estimate). Using this option might not be advisable for small or unbalanced data tables, where the random assignment of the validation subset might not provide a good estimate of error. For large, balanced data tables, a single-partition validation is usually sufficient for estimating error; a single partition is more efficient than cross validation in terms of the total execution time.

If a PARTITION statement is specified, the validation partition defined in that statement is used, and this option is ignored. You cannot specify this option in combination with the KFOLD= option.

If the FRACTION= option is used, then by default the value of the TRAINFRACTION= option is one minus the value of the FRACTION= option. Otherwise, by default, TRAINFRACTION=0.7.

TUNINGPARAMETERS=(suboption |…|<suboption>) TUNEPARMS=(suboption |…|<suboption>)

specifies which parameters to tune and which ranges to tune over. If USEPARAMETERS=STANDARD, this option is ignored.

You can specify one or more of the following suboptions:

CRITERION (VALUES=value-list INIT=value EXCLUDE)

specifies information about the splitting criteria to use for tuning the decision tree. For more information about the criteria, see the GROW statement.

You can specify the following additional suboptions:

VALUES=value-list

specifies a list of splitting criteria to consider during tuning, where value-list is a space separated list that can include one or more of the following values: CHAID, CHISQUARE, FTEST, GAIN, GINI, and VARIANCE.

INIT=value

specifies the initial splitting criterion for the tuner to use. You can specify the following values: CHAID, CHISQUARE, FTEST, GAIN, GINI, IGR, RSS, or VARIANCE.

By default, INIT=IGR for nominal targets and INIT=VARIANCE for interval targets.

EXCLUDE

excludes the splitting criterion from the tuning process.

MAXDEPTH (LB=number UB=number VALUES=value-list INIT=number EXCLUDE)

specifies information about the maximum depth to grow the decision tree to use for tuning the decision tree. For more information, see the MAXDEPTH= option.

You can specify the following additional suboptions:

LB=number

specifies the minimum depth to consider during tuning. If you specify this suboption, you cannot specify the VALUES= suboption.

By default, LB=1.

UB=number

specifies the maximum depth to consider during tuning. If you specify this suboption, you cannot specify the VALUES= suboption.

By default, UB=19.

VALUES=value-list

specifies a list of depth values to consider during tuning, where value-list is a space-separated list of positive integers. If you specify this suboption, you cannot specify either the LB= or UB= suboption.

INIT=number

specifies the initial depth for the tuner to use.

By default, INIT=10.

EXCLUDE

excludes depth from the tuning process.

NUMBIN (LB=number UB=number VALUES=value-list INIT=number EXCLUDE)

specifies information about the number of bins in which to bin the interval inputs while tuning the decision tree. For more information, see the NUMBIN= option.

You can specify the following additional suboptions:

LB=number

specifies the minimum number of bins to consider during tuning. If you specify this suboption, you cannot specify the VALUES= suboption.

By default, LB=20.

UB=number

specifies the maximum number of bins to consider during tuning. If you specify this suboption, you cannot specify the VALUES= suboption.

By default, UB=200.

VALUES=value-list

specifies a list of numbers of bins to consider during tuning, where value-list is a space-separated list of positive integers. If you specify this suboption, you cannot specify either the LB= or UB= suboption.

INIT=number

specifies the initial number of bins for the tuner to use.

By default, INIT=20.

EXCLUDE

excludes the number of bins from the tuning process.

USEPARAMETERS=tuning-parameter-option

specifies which set of parameters to tune.

You can specify the following tuning-parameter-options:

STANDARD: tunes using the default bounds and initial values for all parameters.
CUSTOM: tunes only the parameters that are specified in the TUNINGPARAMETERS= option.
COMBINED: tunes the parameters that are specified in the TUNINGPARAMETERS= option and uses default bounds and initial values to tune all other parameters.

Last updated: December 21, 2018