The REGSELECT Procedure

This example continues the investigation of the baseball data set that were introduced in the section Getting Started: REGSELECT Procedure. That example uses the default stepwise selection method, which is based on the SBC criterion to select a model. This example explores model selection by using LASSO and elastic net selection methods with validation.

You can use the following DATA step to load the Sashelp.Baseball data set into your CAS session:

     data mycas.baseball;
        set sashelp.baseball;
     run;

These statements assume that your CAS engine libref is named mycas, as in the section Using CAS Sessions and CAS Engine Librefs, but you can substitute any appropriately defined CAS engine libref.

The following statements perform LASSO selection with validation for the baseball data:

  ods graphics on;
  proc regselect data=mycas.baseball;
    partition roleVar = league(train='National' validate = 'American');
    class division;
    model logSalary = division nAtBat nHits nHome nRuns nRBI nBB
                      yrMajor crAtBat crHits crHome crRuns crRbi
                      crBB nOuts nAssts nError;
    selection  method = lasso(choose=validate) plots=all;
  run;

The PARTITION statement assigns observations to training and validation roles on the basis of the values of the input variable league. The CHOOSE=VALIDATE option in the SELECTION statement selects the model that yields the smallest average square error (ASE) value for the validation data.

Output 22.2.1 shows that 124 observations are used for model training, 139 observations are used for model validation, and the number of effects is 19.

The "Selection Summary" table in Output 22.2.2 shows the effect that was added or dropped at each step of the selection process together with the SBC and validation ASE for the model at each step. You see that the local minimum of SBC occurs at step 11 and that three more steps are completed before the procedure stops (because STOPHORIZON=3). The model that has lowest SBC is not used as the selected model because CHOOSE=VALIDATE. Instead, the model at step 14 (where the validation ASE achieves a local minimum value) is selected. Details of this selected model are shown in Output 22.2.3.

The coefficient panel in Output 22.2.4 enables you to visualize the selection process. In this plot, standardized coefficients of all the effects that are selected at some step of the LASSO method are plotted as a function of the step number. The panel’s lower plot shows how the criterion that is used to choose the selected model changes as effects enter or leave the model.

The criterion panel in Output 22.2.5 shows how the various fit criteria change as the LASSO selection method proceeds. This plot shows that if you had used the SBC statistic as the CHOOSE= criterion, then the selected model would be the model at step 11. However, the AIC and validation ASE continue to decline for all steps of the selection process.

The "Parameter Estimates" table in Output 22.2.3 shows that the coefficients of the parameters are in different scales. In this case, you can use the adaptive LASSO, in which adaptive weights are used for penalizing different coefficients in the L1 penalty. For more information about this selection method, see the section Adaptive LASSO Selection in Chapter 2, Shared Concepts.

The following statements perform adaptive LASSO selection with validation for the baseball data:

 proc regselect data=mycas.baseball;
   partition roleVar = league(train='National' validate = 'American');
   class division;
   model logSalary = division nAtBat nHits nHome nRuns nRBI nBB
                     yrMajor crAtBat crHits crHome crRuns crRbi
                     crBB nOuts nAssts nError;
   selection  method = lasso(adaptive choose=validate);
 run;

The results of adaptive LASSO selection are shown in Output 22.2.6. You can see that the selected model for adaptive LASSO selection has fewer parameters and a lower validation ASE than LASSO selection has.

If you are interested only in the catchers’ salaries, you can use the following statements to perform LASSO selection for the baseball_C data:

  proc regselect data = mycas.baseball(where=(Position='C'));
    partition roleVar = league(train='National' validate = 'American');
    class division;
    model logSalary = division nAtBat nHits nHome nRuns nRBI nBB
                      yrMajor crAtBat crHits crHome crRuns crRbi
                      crBB nOuts nAssts nError;
    selection  method = lasso(choose=validate);
  run;

Output 22.2.7 shows that 15 observations were used for training and the number of effects is 19. This implies that you cannot obtain a good result by using LASSO selection because the number of effects is greater than the number of observations. In this case, you can use elastic net selection instead of LASSO selection. For more information about this selection method, see the section Elastic Net Selection in Chapter 2, Shared Concepts.

The following statements perform elastic net selection for the baseball_C data:

   proc regselect data=mycas.baseball(where=(Position='C'));
     partition roleVar = league(train='National' validate = 'American');
     class division;
     model logSalary = division nAtBat nHits nHome nRuns nRBI nBB
                       yrMajor crAtBat crHits crHome crRuns crRbi
                       crBB nOuts nAssts nError;
     selection  method = elasticnet(choose=validate);
     ods output ElasticNetSummary=ElasticNetSummary;
   run;

You can specify the L2= suboption of the METHOD option in the SELECTION statement if you have a good estimate of the ridge regression parameter. If not, you can leave the L2= suboption unspecified; in this case, PROC REGSELECT estimates this regression parameter according to the criterion that is specified in the CHOOSE= option, by trying a series of candidate values for the ridge regression parameter. The optimal regression parameter is set to the value that achieves the minimum validation ASE.

The "Elastic Net Selection Summary" table in Output 22.2.10 shows that the optimal L2 value is 0.14677993 and the minimal validation ASE is 0.17967. Output 22.2.9 shows the details of the model that is selected by elastic net method with optimal L2 value. If you compare the ASE values for the model in Output 22.2.9 and the model in Output 22.2.8, you see that the elastic net method selects a model that has a lower validation ASE than the LASSO method does in this case.

Similar to the previous analysis, which used the adaptive LASSO selection, you can also perform adaptive elastic net selection on the data when parameter estimates has different scales. The following statements perform adaptive elastic net selection for the baseball_C data:

proc regselect data=mycas.baseball(where=(Position='C'));
     partition roleVar = league(train='National' validate = 'American');
     class division;
     model logSalary = division nAtBat nHits nHome nRuns nRBI nBB
                       yrMajor crAtBat crHits crHome crRuns crRbi
                       crBB nOuts nAssts nError;
     selection  method = elasticnet(adaptive choose=validate);
run;
ods graphics off;

The "Adaptive Elastic Net Selection Summary" table in Output 22.2.12 shows that the optimal L2 value is 0.0215 and the minimal validation ASE is 0.1661. Output 22.2.11 shows that adaptive elastic net selection can produce better results in the sense of a lower validation ASE and a sparser model. For more information about this method, see the section Adaptive Elastic Net Selection in Chapter 2, Shared Concepts.