Autotune Action Set

Tuning the Hyperparameters of a Gradient Boosting Tree Model

This section contains PROC CAS code.

Note: Input data must be accessible in your CAS session, either as a CAS table or as a transient-scope table. A CAS table has a two-level name: the first level is your CAS engine libref, and the second level is the table name. You refer to this table in the CAS procedure by specifying only the second level. For more information about two-level names, see Chapter 2, Shared Concepts (SAS Viya: Machine Learning Procedures). A transient-scope table is called directly from the action and exists in memory for the duration of the action. For more information about accessing data, see SAS Viya: System Programming Guide. For more information about PROC CAS and programming in CASL, see SAS Cloud Analytic Services: CASL Programmer’s Guide and SAS Cloud Analytic Services: CASL Reference.

This example illustrates the use of the tuneGradientBoostTree action to automatically tune the hyperparameters of a gradient boosting tree model. The following DATA step loads the iris data set from the Sashelp library into a data table named mycas.iris. These statements assume that the CAS engine libref is named mycas, but you can substitute any appropriately defined CAS engine libref.

  data mycas.iris;
     set sashelp.iris;
  run;

The following PROC CAS code uses the tuneGradientBoostTree action to automatically tune the hyperparameters of a gradient boosting tree model that is trained on the iris data table. Note that the syntax of the trainOptions parameter here is the same as the syntax of the gbtreeTrain action. For more information about the gbtreeTrain action, see "Decision Tree Action Set," SAS Visual Analytics: Programming Guide.

  proc cas noqueue;
     autotune.tuneGradientBoostTree /
        trainOptions = {
           table     = {name="iris"},
           inputs    = {"SepalLength",
                        "SepalWidth",
                        "PetalLength",
                        "PetalWidth"
                       },
           target     = "Species",
           nominal    = {"Species"},
           casout     = {name="gradboost_iris_model",
                         replace=true
                        }
        }
       tunerOptions={seed=12345}

       /* Tuning Parameters

       You do not need to specify any tuning parameters for the default
       tuning process. If you want to make adjustments to the default
       tuning process, uncomment the following block of code and change
       any of the tuning parameters' attributes.

       tuningParameters={
          {name="nTree",           lb=20,   ub=150,  initValue=100,  exclude=false},
          {name="m",               lb=1,    ub=4,    initValue=4,    exclude=false},
          {name="learningRate",    lb=0.01, ub=1.0,  initValue=0.1,  exclude=false},
          {name="subSampleRate",   lb=0.1,  ub=1.0,  initValue=0.5,  exclude=false},
          {name="lasso",           lb=0.0,  ub=10.0, initValue=0.0,  exclude=false},
          {name="ridge",           lb=0.0,  ub=10.0, initValue=1.0,  exclude=false},
          {name="nBins",           lb=20,   ub=100,  initValue=50,   exclude=false},
          {name="maxLevel",        lb=2,    ub=7,    initValue=5,    exclude=false},
          {name="leafSize",        valueList={1, 5, 10, 20, 40, 80, 160, 320},
                                                     initValue=5,    exclude=true }
       }

       */

     ;
     ods output TunerResults      = TuneResults(keep=MisclassErr);
     ods output EvaluationHistory = EvalHistory;
     ods output IterationHistory  = IterHistory;
     run;
  quit;

The following optional code includes two DATA steps and two SGPLOT procedure steps that generate the following two plots from the tuneGradientBoostTree action results:

  • history of all evaluations (the objective value versus the evaluation number, grouped by iteration)

  • iteration history (the best objective value and total tuning time versus the iteration number)

 data _null_;
    set TuneResults;
    if _n_ = 1 then call symput('default', MisclassErr);
    else if _n_ = 2 then call symput('best', MisclassErr);
 run;

 data EvalHistory;
    set EvalHistory;
    default = &default.;
    best = &best.;
 run;

 proc sgplot data=EvalHistory;
    title "Tune GradBoost with GA on Iris data";
    scatter x=Evaluation y=MisclassErr / group=Iteration;
    series x=Evaluation y=default /
       curvelabel="default" lineattrs=(pattern=shortdash);
    series x=Evaluation y=best /
       curvelabel="best" lineattrs=(color=green pattern=shortdash);
 run;
 proc sgplot data=IterHistory;
    title "Tune GradBoost with GA on Iris data";
    series x=Iteration y=Best_Obj;
    series x=Iteration y=Time_Sec / y2axis;
 run;

Output 7.8.1 shows the "Tuner Information" output table, which displays the values of parameters that the tuner used.

Output 7.8.1: Tuner Information Output

Results from decisionTree.gbtreeTrain

Tuner Information
Model TypeGradient Boosting Tree
Tuner Objective FunctionMisclassification Error Percentage
Search MethodGA
Population Size10
Maximum Iterations5
Maximum Tuning Time in Seconds36000
Validation TypeSingle Partition
Validation Partition Fraction0.30
Log Level2
Seed12345
Number of Parallel Evaluations23
Number of Workers per Subsession1


Output 7.8.2 shows the "Tuner Results" output table, which displays the default configuration (Evaluation 0) and up to 10 of the best configurations that the tuner found. For each configuration, the hyperparameter values and objective value are provided.

Output 7.8.2: Tuner Results Output

Tuner Results
Default and Best Configurations
EvaluationMaximum
Tree Levels
Number
of Bins
Number of
Variables
to Try
Learning
Rate
Sampling
Rate
LassoRidgeMisclassification
Error
Percentage
Evaluation
Time
in Seconds
055040.1000000.50000001.0000006.671.10
279130.8900000.9000002.2222221.1111114.4414.13
576410.2300000.60000004.4444444.4414.13
858220.4500000.4000007.7777785.5555564.4414.13
965630.1200001.0000006.6666672.2222224.4414.88
1156030.5050000.5500005.0000005.0000004.4414.13
1356010.5050000.5500005.0000005.0000004.4414.88
1456031.0000000.5500005.0000005.0000004.4414.88
1556030.0100000.5500005.0000005.0000004.4414.88
1656030.5050001.0000005.0000005.0000004.4414.88
1956030.5050000.55000005.0000004.4414.88


Output 7.8.3 shows the "Tuner Iteration History" output table, which displays the results of each iteration. For each iteration, the number of evaluations, the best objective value so far, and the tuning time in seconds are provided.

Output 7.8.3: Tuner Iteration History Output

Tuner Iteration History
IterationEvaluationsBest
Objective
Elapsed
Time
in Seconds
016.671.10
1254.4417.58
2464.4419.20
3664.4420.86
4854.4422.43
51044.4423.93


Output 7.8.4 shows the "Tuner Evaluation History" output table, which displays all model configurations that the tuner evaluated. For each configuration, the hyperparameter values and objective value are provided.

Output 7.8.4: Tuner Evaluation History Output

Tuner History
All Evaluated Configurations
EvaluationIterationMaximum
Tree Levels
Number
of Bins
Number of
Variables
to Try
Learning
Rate
Sampling
Rate
LassoRidgeMisclassification
Error
Percentage
Evaluation
Time
in Seconds
0055040.1000000.50000001.0000006.671.10
1124740.0100000.3000008.8888893.3333336.6714.13
2179130.8900000.9000002.2222221.1111114.4414.13
3122040.7800000.2000004.44444406.6714.13
41410040.5600000.5000003.33333310.0000006.6714.13
5176410.2300000.60000004.4444444.4414.13
6137310.3400000.80000010.0000006.6666676.6714.13
7152920.6700000.1000005.5555568.88888966.6714.13
8158220.4500000.4000007.7777785.5555564.4414.13
9165630.1200001.0000006.6666672.2222224.4414.88
10155040.1000000.50000001.0000006.676.15
11156030.5050000.5500005.0000005.0000004.4414.13
12156040.5050000.5500005.0000005.0000006.6714.88
13156010.5050000.5500005.0000005.0000004.4414.88
14156031.0000000.5500005.0000005.0000004.4414.88
15156030.0100000.5500005.0000005.0000004.4414.88
16156030.5050001.0000005.0000005.0000004.4414.88
17156030.5050000.1000005.0000005.00000066.6714.88
18156030.5050000.55000010.0000005.0000006.6714.88
19156030.5050000.55000005.0000004.4414.88
20156030.5050000.5500005.00000010.0000004.4414.88
21156030.5050000.5500005.00000004.4414.88
221510030.5050000.5500005.0000005.0000004.4414.88
23152030.5050000.5500005.0000005.0000006.6714.88
24176030.5050000.5500005.0000005.0000004.449.23
25126030.5050000.5500005.0000005.0000006.670.75
262610010.3434570.4645718.8539667.3492036.670.92
27234030.6734570.2645715.5206331.7936474.441.60
28259640.7106680.6826282.8260325.9415934.441.28
292310030.9285191.0000001.4440611.5001924.441.62
30244540.8185190.4451213.6662830.38908115.560.99
31276020.1793200.6460731.4259404.1884844.441.55
32257820.3993200.4460735.9860555.2995954.441.52
33269430.9686010.9142911.1106970.3171644.441.41
34247610.4186010.8142918.8884745.8727206.671.39
35276030.5050000.55000010.0000005.0000006.671.04
36276031.0000000.5500005.0000005.0000004.441.58
37276030.0100000.5500005.0000005.0000004.441.62
38276030.5050001.0000005.0000005.0000004.441.60
39276030.5050000.1000005.0000005.00000066.671.48
40272030.5050000.5500005.0000005.0000006.671.16
41276030.5050000.55000005.0000004.441.55
42276030.5050000.5500005.00000010.0000004.441.61
43276030.5050000.5500005.000000013.331.54
442710030.5050000.5500005.0000005.0000004.441.59
45276040.5050000.5500005.0000005.0000006.671.46
46276020.5050000.5500005.0000005.0000004.441.52
47348720.5099390.3455108.2789545.8582808.890.92
48366920.2899390.5455103.7188404.7471694.441.54
49369230.9057310.8625831.6491130.8674054.441.52
50355310.0733060.91174610.0000000.6744504.441.04
51365730.1326250.9246725.5512572.6407114.441.64
52365530.1058520.7165607.9165971.7532624.441.65
53375920.1651720.7304862.6758703.7195244.441.26
54379321.0000001.0000000.00126304.441.65
55369530.8774920.8324621.7165992.3038624.441.54
56366030.5050000.5500005.0000005.0000004.441.52
57376030.5050000.5500002.5000005.0000004.441.64
58376030.5050000.3250005.0000005.0000004.441.66
59376030.5050000.5500007.5000005.0000004.441.60
60376030.2575000.5500005.0000005.0000004.441.61
61376030.5050000.7750005.0000005.0000004.441.16
62378030.5050000.5500005.0000005.0000004.441.56
63374030.5050000.5500005.0000005.0000006.671.48
64376030.5050000.5500005.0000007.5000004.441.63
65376030.5050000.5500005.0000002.5000004.441.56
66376030.7525000.5500005.0000005.0000004.441.57
674610031.0000001.0000000.64687704.441.54
68468020.5881280.6990482.7166042.8684484.441.45
69427220.4914640.8358251.6305012.2657504.440.98
70468930.9293940.8878221.59256404.441.50
71469230.9011550.8577011.6600501.1002144.441.55
72455210.0632240.9316385.6587280.3402654.441.55
73475820.1550900.7503783.4796643.3853394.441.47
74466920.2779730.5393483.7590614.8225644.441.18
75469230.8937650.8564211.6893340.9428014.441.41
76476030.6287500.5500005.0000005.0000004.441.14
77476030.5050000.5500003.7500005.0000004.441.57
78476030.5050000.5500005.0000006.2500004.441.48
79476030.5050000.6625005.0000005.0000006.671.39
80476030.5050000.4375005.0000005.0000004.441.53
81476030.5050000.5500006.2500005.0000004.441.51
82476030.5050000.5500005.0000003.7500004.440.84
83477030.5050000.5500005.0000005.0000004.441.19
84475030.5050000.5500005.0000005.0000006.671.47
85476030.3812500.5500005.0000005.0000004.441.49
865610031.0000000.9706680.94357404.441.46
87567720.4998540.6535953.0133003.4246154.440.79
88555310.0767120.9478785.8981770.7873484.441.48
89579931.0000000.8631171.7084620.5126846.671.47
90537920.6222600.8425211.6496291.8356324.441.42
91555310.1123600.8155831.6031583.3442694.441.50
92527320.5220510.8374581.6327072.1787334.441.07
93523720.0100001.0000007.22915004.441.39
94556510.3079490.8231984.2870261.5189764.441.50
95576030.5668750.5500005.0000005.0000004.440.93
96576030.4431250.5500005.0000005.0000004.441.49
97576030.5050000.5500005.0000004.3750004.441.43
98576030.5050000.4937505.0000005.0000004.441.43
99576030.5050000.5500005.6250005.0000004.441.30
100576030.5050000.5500004.3750005.0000004.441.11
101576030.5050000.5500005.0000005.6250004.441.48
102576530.5050000.5500005.0000005.0000004.441.46
103575530.5050000.5500005.0000005.0000006.671.49
104576030.5050000.6062505.0000005.0000004.441.47


Output 7.8.5 shows the "Best Configuration" output table, which displays the values of hyperparameters and the objective value for the best configuration that the tuner found.

Output 7.8.5: Best Configuration Output

Best Configuration
Evaluation2
Number of Variables to Try3
Learning Rate0.89
Sampling Rate0.9
Lasso2.22222222
Ridge1.11111111
Number of Bins91
Maximum Tree Levels7
Misclassification Error Percentage4.44


Output 7.8.6 shows the "Tuner Summary" output table, which displays statistics about the tuning process.

Output 7.8.6: Tuner Summary Output

Tuner Summary
Initial Configuration Objective Value6.6667
Best Configuration Objective Value4.4444
Worst Configuration Objective Value66.6667
Initial Configuration Evaluation Time in Seconds1.0983
Best Configuration Evaluation Time in Seconds0.6439
Number of Improved Configurations1
Number of Evaluated Configurations104
Total Tuning Time in Seconds24.5008
Parallel Tuning Speedup5.7540


Output 7.8.7 shows the "Tuner Task Timing" output table, which displays timing information about the different tasks that were performed during the tuning process.

Output 7.8.7: Tuner Task Timing Output

Tuner Task Timing
TaskSecondsPercent
Model Training79.9156.68
Model Scoring45.5932.34
Total Objective Evaluations125.5189.03
Tuner15.4710.97
Total CPU Time140.98100.00


Output 7.8.8 shows the "Output CAS Tables" output table, which displays a list of CAS tables that were created during the tuning process.

Output 7.8.8: Output CAS Tables

Output CAS Tables
CAS LibraryNameNumber
of Rows
Number
of Columns
YOUR_CAS_LIBgradboost_iris_model3220


Output 7.8.9 shows the "Hyperparameter Importance" output table, which displays the relative importance of each hyperparameter during the tuning process, normalized by the effect of the most influential hyperparameter (importance set to 1) and listed in order of decreasing effect.

Output 7.8.9: Hyperparameter Importance Output

Hyperparameter Importance
HyperparameterRelative
Importance
SUBSAMPLERATE1.0000
LEARNINGRATE0.1588
NBINS0.1240
RIDGE0.1166
MAXLEVEL0.0757
LASSO0.0652
M0.0134


The first PROC SGPLOT step creates a scatter plot of the objective value (misclassification error in this case) evaluation history by iteration, which is shown in Output 7.8.10. The second PROC SGPLOT step creates a line plot of the best objective value by iteration and a line plot of the total tuning time by iteration, which are shown in Output 7.8.11.

Output 7.8.10: Tuner Evaluation History Plot

Tuner Evaluation History Plot


Output 7.8.11: Tuner Iteration History Plot

Tuner Iteration History Plot


Tuning the Hyperparameters of a Gradient Boosting Tree Model

This section contains Lua code for the analysis in the CASL version of this example, which contains details about the results.

Note: In order to run this code, the data that are described in the CASL version need to be accessible to the CAS server. One way to do this is to convert the iris data to the comma-separated-value (CSV) file iris.csv and then use the following code to load the CSV file into CAS:

s:loadtable{casLib="casuser", path="iris.csv"}

For more information about coding in Lua, see Getting Started with SAS Viya for Lua and SAS Viya: System Programming Guide.

The following Lua code uses the tuneGradientBoostTree action to automatically tune the hyperparameters of a gradient boosting tree model that is trained on the iris data table. The data and the parameters are the same as in the PROC CAS version of this example. Note that the syntax of the trainOptions parameter here is the same as the syntax of the gbtreeTrain action. For more information about the gbtreeTrain action, see "Decision Tree Action Set," SAS Visual Analytics: Programming Guide.

   result = s:autotune_tuneGradientBoostTree{
        trainOptions = {
           table     = {name="iris"},
           inputs    = {"SepalLength",
                        "SepalWidth",
                        "PetalLength",
                        "PetalWidth"
                       },
           target    = "Species",
           nominal   = {"Species"},
           casout    = {name = "gradboost_iris_model",
                        replace=true
                       }
       },
       tunerOptions={seed=12345}

       --[[ Tuning Parameters

       You do not need to specify any tuning parameters for the default
       tuning process. If you want to make adjustments to the default
       tuning process, uncomment the following block of code and change
       any of the tuning parameters' attributes.

       , tuningParameters={
          {name="nTree",         lb=20,   ub=150,  initValue=100,  exclude=false},
          {name="m",             lb=1,    ub=4,    initValue=4,    exclude=false},
          {name="learningRate",  lb=0.01, ub=1.0,  initValue=0.1,  exclude=false},
          {name="subSampleRate", lb=0.1,  ub=1.0,  initValue=0.5,  exclude=false},
          {name="lasso",         lb=0.0,  ub=10.0, initValue=0.0,  exclude=false},
          {name="ridge",         lb=0.0,  ub=10.0, initValue=1.0,  exclude=false},
          {name="nBins",         lb=20,   ub=100,  initValue=50,   exclude=false},
          {name="maxLevel",      lb=2,    ub=7,    initValue=5,    exclude=false},
          {name="leafSize",      valueList={1, 5, 10, 20, 40, 80, 160, 320},
                                                   initValue=5,    exclude=true }
       }

       --]]

   }

The following commands display the tables that are produced by this action call:

print(result.TunerInfo)
print(result.TunerResults)
print(result.IterationHistory)
print(result.EvaluationHistory)
print(result.BestConfiguration)
print(result.TunerSummary)
print(result.TunerTiming)
print(result.TunerCasOutputTables)
print(result.HyperparameterImportance)

For details about the results of this analysis, see the PROC CAS version of this example.

Tuning the Hyperparameters of a Gradient Boosting Tree Model

This section contains Python code for the analysis in the CASL version of this example, which contains details about the results.

Note: In order to run this code, the data that are described in the CASL version need to be accessible to the CAS server. One way to do this is to convert the iris data to the comma-separated-value (CSV) file iris.csv and then use the following code to load the CSV file into CAS:

s.upload_file('iris.csv')

For more information about coding in Python, see Getting Started with SAS Viya for Python and SAS Viya: System Programming Guide.

The following Python code uses the tuneGradientBoostTree action to automatically tune the hyperparameters of a gradient boosting tree model that is trained on the iris data table. The data and the parameters are the same as in the PROC CAS version of this example. Note that the syntax of the trainOptions parameter here is the same as the syntax of the gbtreeTrain action. For more information about the gbtreeTrain action, see "Decision Tree Action Set," SAS Visual Analytics: Programming Guide.

   result = s.autotune.tuneGradientBoostTree(
        trainOptions = {
           "table"   : {"name":"iris"},
           "inputs"  : {"SepalLength",
                        "SepalWidth",
                        "PetalLength",
                        "PetalWidth"
                       },
           "target"  : "Species",
           "nominal" : {"Species"},
           "casout"  : {"name":"gradboost_iris_model"}
        },
        tunerOptions={"seed":12345}

   # Tuning Parameters

   # You do not need to specify any tuning parameters for the default
   # tuning process. If you want to make adjustments to the default
   # tuning process, uncomment the following block of code and change
   # any of the tuning parameters' attributes.

   # , tuningParameters=[
   #    {"name":"nTree",
   #         "lb":20,   "ub":150,  "initValue":100,  "exclude":"false"},
   #    {"name":"m",
   #         "lb":1,    "ub":4,    "initValue":4,    "exclude":"false"},
   #    {"name":"learningRate",
   #         "lb":0.01, "ub":1.0,  "initValue":0.1,  "exclude":"false"},
   #    {"name":"subSampleRate",
   #         "lb":0.1,  "ub":1.0,  "initValue":0.5,  "exclude":"false"},
   #    {"name":"lasso",
   #         "lb":0.0,  "ub":10.0, "initValue":0.0,  "exclude":"false"},
   #    {"name":"ridge",
   #         "lb":0.0,  "ub":10.0, "initValue":1.0,  "exclude":"false"},
   #    {"name":"nBins",
   #         "lb":20,   "ub":100,  "initValue":50,   "exclude":"false"},
   #    {"name":"maxLevel",
   #         "lb":2,    "ub":7,    "initValue":5,    "exclude":"false"},
   #    {"name":"leafSize",
   #        "valueList":{1, 5, 10, 20, 40, 80, 160, 320},
   #                                   "initValue":5, "exclude":"true"}
   #  ]

   )

The following commands display the tables that are produced by this action call:

print(result.TunerInfo)
print(result.TunerResults)
print(result.IterationHistory)
print(result.IterationHistory)
print(result.EvaluationHistory)
print(result.BestConfiguration)
print(result.TunerSummary)
print(result.TunerTiming)
print(result.TunerCasOutputTables)
print(result.HyperparameterImportance)

For details about the results of this analysis, see the PROC CAS version of this example.

Tuning the Hyperparameters of a Gradient Boosting Tree Model

This example is not available for the R programming language.

Last updated: August 04, 2026