The NNET Procedure

AUTOTUNE Statement

AUTOTUNE <options>;

The AUTOTUNE statement activates the tuning optimization algorithm, which searches for the best hidden layers and regularization parameters based on the problem and specified options. If ALGORITHM=SGD or ALGORITHM=ADAM, the algorithm also searches for the best values of the learning rate and annealing rate. When you specify the AUTOTUNE statement, PROC NNET might ignore any specified HIDDEN statement depending on the values of USEPARAMETERS= and TUNINGPARAMETERS= options. You cannot specify both the AUTOTUNE statement and the CROSSVALIDATION statement in the same procedure run.

Table 1 summarizes the options you can specify in the AUTOTUNE statement. For more information about all options except the TUNINGPARAMETERS= option, see the section AUTOTUNE Statement in Chapter 2, Shared Concepts. The TUNINGPARAMETERS= option is described in detail following Table 1.

Table 1: AUTOTUNE Statement Options

Option	Description
APPENDLOOKUP	Specifies that the table specified in the HISTORYTABLE= option contain the rows from the table specified in the LOOKUPTABLE= option
EVALHISTORY=	Specifies how to report the evaluation history of the tuner
FRACTION=	Specifies the fraction of observations to use for validation
HISTORYTABLE=	Specifies the CAS table that contains the evaluation history
KFOLD=	Specifies the number of folds for k-fold cross validation
LIVEUPDATE	Specifies that the table specified in the HISTORYTABLE= option be updated at every evaluation
LOOKUPTABLE=	Specifies the CAS table to use for evaluation lookup
MAXBAYES=	Specifies the maximum number of points in the kriging model
MAXEVALS=	Specifies the maximum number of evaluations
MAXITER=	Specifies the maximum number of iterations when SEARCHMETHOD=GA or SEARCHMETHOD=BAYESIAN
MAXTIME=	Specifies the maximum time for all iterations
MAXTRAINTIME=	Specifies the maximum time for a model training
NCONVITER=	Specifies the number of convergence iterations
NOGRIDSHUFFLE	Requests that the grid points not be shuffled
NOLOCALSEARCH	Disables local search optimization
NPARALLEL=	Specifies the number of parallel sessions
NSUBSESSIONWORKERS=	Specifies the number of workers in parallel sessions
OBJECTIVE=	Specifies the objective function
POPSIZE=	Specifies the population size when SEARCHMETHOD=GA or SEARCHMETHOD=BAYESIAN
SAMPLESIZE=	Specifies the sample size when SEARCHMETHOD=LHS or SEARCHMETHOD=RANDOM
SEARCHMETHOD=	Specifies the search method that the optimizer uses
SECONDOBJECTIVE=	Specifies the second objective to use for tuning
SELECTINITPOINT	Specifies that the tuner select the best evaluation from the lookup table
TARGETEVENT=	Specifies the target event for ROC-based calculations
TRAINFRACTION=	Specifies the fraction of observations to use for training
TUNINGPARAMETERS=	Specifies the custom tuning parameters
USEPARAMETERS=	Specifies how to handle the TUNINGPARAMETERS= option

TUNINGPARAMETERS=(suboption |…|<suboption>) TUNEPARMS=(suboption |…|<suboption>)

specifies which parameters to tune and which ranges to tune over. If USEPARAMETERS=STANDARD, this option is ignored.

You can specify one or more of the following suboptions:

ANNEALINGRATE (LB=number UB=number VALUES=value-list INIT=number EXCLUDE)

specifies the range of the annealing rates to use in the tuning process. This option is valid only when ALGORITHM=SGD or ADAM in the OPTIMIZATION statement.

You can specify the following additional suboptions:

LB=number

specifies the minimum annealing rate to use in the tuning process, where number is a nonnegative double. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, LB=1E–13.

UB=number

specifies the maximum annealing rate to use in the tuning process, where number is a nonnegative double. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, UB=1E–2.

VALUES=value-list

specifies a list of annealing rates to use in the tuning process, where value-list is a space-separated list of nonnegative doubles. If you specify this suboption, you cannot specify either the LB= or UB= suboption.

INIT=number

specifies the initial annealing rate in the tuning process.

By default, INIT=1E–06.

EXCLUDE

excludes the annealing rate from the tuning process. If you specify this suboption, any specified LB=, UB=, VALUES=, and INIT= suboptions are ignored.

LEARNINGRATE (LB=number UB=number VALUES=value-list INIT=number EXCLUDE)

specifies the range of the learning rates to use in the tuning process. This option is valid only when ALGORITHM=SGD or ADAM in the OPTIMIZATION statement.

You can specify the following additional suboptions:

LB=number

specifies the minimum learning rate to use for tuning, where number is a nonnegative double. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, LB=1E–6.

UB=number

specifies the maximum learning rate to use for tuning, where number is a nonnegative double. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, UB=1E–1.

VALUES=value-list

specifies a list of learning rates to use in the tuning process, where value-list is a space-separated list of nonnegative doubles. If you specify this suboption, you cannot specify either the LB= or UB= suboption.

INIT=number

specifies the initial learning rate to use for tuning.

By default, INIT=1E–03.

EXCLUDE

excludes the learning rate from the tuning process. If you specify this suboption, any specified LB=, UB=, VALUES=, and INIT= suboptions are ignored.

NHIDDEN (LB=number UB=number VALUES=value-list INIT=number EXCLUDE)

specifies the tuning range of the number of hidden layers in the network.

You can specify the following additional suboptions:

LB=number

specifies the minimum number of hidden layers, where number is an integer between 0
and 5. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, LB=0.

UB=number

specifies the maximum number of hidden layers, where number is an integer between 0
and 5. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, UB=2.

VALUES=value-list

specifies a list of numbers of hidden layers to be searched in the tuning process, where value-list is a space-separated list of nonnegative integers. If you specify this suboption, you cannot specify either the LB= or UB= suboption.

INIT=number

specifies the initial number of hidden layers for the tuning process.

By default, INIT=0.

EXCLUDE

excludes the number of hidden layers from the tuning process. If you specify this suboption, any specified LB= and UB= suboptions are ignored and you cannot specify any NUNITSi suboption.

NUNITSi (LB=number UB=number VALUES=value-list INIT=number EXCLUDE)

specifies tuning information for neurons in the ith hidden layer, where i is any integer 1–5, inclusive. By default, up to two hidden layers are tried during tuning. An NUNITSi suboption takes effect only when the value of the UB= suboption in the NHIDDEN suboptions is greater than or equal to i.

You can specify the following additional suboptions:

LB=number

specifies the minimum number of neurons in the ith hidden layer, where number must be a nonnegative integer. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, LB=1.

UB=number

specifies the maximum number neurons in the ith hidden layer, where number must be a nonnegative integer. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, UB=, where n is the number of model inputs.

VALUES=value-list

specifies a list of candidate numbers of neurons in the ith hidden layer to be searched in the tuning process, where value-list is a space-separated list of nonnegative integers. If you specify this suboption, you cannot specify either the LB= or UB= suboption.

INIT=number

specifies the initial number of neurons in the ith hidden layer.

By default, INIT=1.

EXCLUDE

excludes the number of neurons in the ith hidden layer from the tuning process. If you specify this suboption, any specified LB=, UB=, VALUES=, and INIT= suboptions are ignored. If you specify this suboption, you cannot specify the NHIDDEN suboption or any other NUNITSi option.

REGL1 (LB=number UB=number VALUES=value-list INIT=number EXCLUDE)

specifies the range of L1 regularization values in the tuning process.

You can specify the following additional suboptions:

LB=number

specifies the minimum L1 regularization value in the tuning process, where number is a nonnegative double. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, LB=0.

UB=number

specifies the maximum L1 regularization value in the tuning process, where number is a nonnegative double. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, UB=10.

VALUES=value-list

specifies a list of L1 regularization values to be searched in the tuning process, where value-list is a space-separated list of nonnegative doubles. If you specify this suboption, you cannot specify either the LB= or UB= suboption.

INIT=number

specifies the initial L1 regularization value in the tuning process.

By default, INIT=0.

EXCLUDE

excludes L1 regularization from the tuning process. If you specify this suboption, any specified LB=, UB=, VALUES=, and INIT= suboptions are ignored.

REGL2 (LB=number UB=number VALUES=value-list INIT=number EXCLUDE)

specifies the range of L2 regularization values in the tuning process.

You can specify the following additional suboptions:

LB=number

specifies the minimum L2 regularization value in the tuning process, where number is a nonnegative double. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, LB=0.

UB=number

specifies the maximum L2 regularization value in the tuning process, where number is a nonnegative double. If you specify this suboption, you cannot also specify the VALUES= suboption.

By default, UB=10.

VALUES=value-list

specifies a list of L2 regularization values to be searched in the tuning process, where value-list is a space-separated list of nonnegative doubles. If you specify this suboption, you cannot specify either the LB= or UB= suboption.

INIT=number

specifies the initial L2 regularization value in the tuning process.

By default, INIT=0.

EXCLUDE

excludes L2 regularization from the tuning process. If you specify this suboption, any specified LB=, UB=, VALUES=, and INIT= suboptions are ignored.

Last updated: November 11, 2020