NNET Procedure

This example illustrates how to use the AUTOTUNE statement to search for the best set of hyperparameters within the domains that you specify. The data set (iris) is the same data set that is used in the section Getting Started: NNET Procedure. The AUTOTUNE statement searches for the best network for iris within two hidden layers (each of which has specified ranges), and it also searches for the best L1 and L2 regularization values based on the specified ranges. Only one controller node is used in the example.

You can load the sashelp.iris data set into your CAS session by naming your CAS engine libref in the first statement of the following DATA step. These statements assume that your CAS engine libref is named mylib, but you can substitute any appropriately defined CAS engine libref.

data mylib.iris;
   set sashelp.iris;
run;

The following statements run PROC NNET and output the results to ODS tables. The AUTOTUNE statement activates the tuning optimization algorithm, which applies the specified ranges in the local searching process.

proc nnet data=mylib.iris;
   input SepalLength SepalWidth PetalLength PetalWidth;
   target Species / level=nominal;
   train outmodel=mylib.nnetModel2 seed=1517878693;
   autotune useparameters=custom objective=MCE searchmethod=GA
            tuningparameters=(nhidden(LB=1 UB=2 INIT=1)
                              nunits1(LB=1 UB=10 INIT=1)
                              nunits2(LB=1 UB=15 INIT=2)
                              regl1(LB=1e-04 UB=1e-02 INIT=1e-03)
                              regl2(LB=1e-04 UB=1e-02 INIT=1e-03)
                              );
   optimization algorithm=LBFGS maxiter=100;
run;

Output 25.2.1 shows the setup values that are used by the tuner.

Output 25.2.2 shows the results that are reported by the NNET procedure. The first row displays results from the default settings, the second row displays the best results that are found by the tuner, and the third row displays the second-best results that are found.

Output 25.2.3 shows the best values of the tuning parameters from the tuning process.

Output 25.2.4 shows the tuner summary.

Output 25.2.5 shows the run time for each task during the searching process. It is evident that the tuner spent the vast majority of its time on training; this behavior is similar to most tuner runs. Therefore, it is important to understand that tuning might take a very long time by nature if the training time is long. Typically, networks that have more neurons or larger training samples take more time; also, if the value of the MAXITER= option is very large and the nonlinear objective function converges slowly, the run time could be very long. In general, tuner performance should not be a concern, because you typically use tuning only once in a while.

Output 25.2.6 shows the relative importance of each hyperparameter.

Output 25.2.7 displays a scatter plot of all configurations that the tuner tried. The objective values are shown on the Y axis, and the evaluation numbers are shown on the X axis.

The plot in Output 25.2.8 displays how the best found objective value and the elapsed time changed with each iteration of the tuner.