REGSELECT Procedure
SELECTION Statement
SELECTION <METHOD=method <(method-options)>><options>;
The SELECTION statement performs model selection by examining whether effects should be added to or removed from the model according to rules that are defined by model selection methods. The statement is fully documented in the section SELECTION Statement in Chapter 2, Shared Concepts.
The REGSELECT procedure supports the following values of the METHOD= option in the SELECTION statement:
- BACKWARD
specifies the backward elimination method, which starts with all effects in the model and deletes effects.
- BESTSUBSET
specifies the best-subset selection method, which uses the branch-and-bound algorithm (Furnival and Wilson 1974) to efficiently search candidate models without examining all possible subsets.
- ELASTICNET
specifies the elastic net method, which is an extension of LASSO that estimates parameters by using a version of ordinary least squares in which both the sum of the absolute regression coefficients and the sum of the squared regression coefficients are constrained.
- FORWARD
specifies the forward selection method, which starts with no effects in the model and adds effects.
- FORWARDSWAP
specifies forward-swap selection, which is an extension of the forward selection method. Before any addition step, PROC REGSELECT makes all pairwise swaps of one effect in the model and one effect out of the current model that improve the selection criterion. When the selection criterion is R-square, this method is the same as the MAXR method in the REG procedure in SAS/STAT software.
- LAR
specifies the least angle regression method. Like forward selection, this method starts with no effects in the model and adds effects. The parameter estimates at any step are "shrunk" when compared to the corresponding least squares estimates. If the model contains classification variables, then these classification variables are split. For more information about split classification variables, see the section Classification Variables and the SPLIT Option.
- LASSO
specifies the LASSO method, which adds and deletes parameters based on a version of ordinary least squares in which the sum of the absolute regression coefficients is constrained. If the model contains classification variables, then these classification variables are split. For more information about split classification variables, see the section Classification Variables and the SPLIT Option.
- MCP
specifies the MCP method (Zhang 2010), which uses the minimax concave penalty as the penalty function in penalized least squares estimation.
- NONE
specifies no model selection.
- SCAD
specifies the SCAD method (Fan and Li 2001), which uses smoothly clipped absolute deviation as the penalty function in penalized least squares estimation.
- STEPWISE
specifies the stepwise regression method, which is similar to the forward selection method except that effects already in the model do not necessarily stay there. ??citation??
By default, METHOD=STEPWISE.
The SELECT=, CHOOSE=, and STOP= method-options default to the Schwarz Bayesian information criterion (SBC) except in best-subset selection. For the best-subset selection method, the default SELECT= criterion is ADJRSQ. These defaults differ from their corresponding defaults in the HPLOGISTIC procedure in SAS/STAT software.
The DETAILS=ALL and DETAILS=STEPS options produce the "ANOVA," "Fit Statistics," and "Parameter Estimates" tables, which provide information about the model that is selected at each step of the selection process.
If you specify the PLOTS=CRITERIA or PLOTS=ALL option, then a plot of the fit criteria by the selection step is created. The default fit criteria that are shown in this plot are the ADJRSQ, AIC, AICC, ASE, and SBC statistics. If you also specify a PARTITION statement, then the ASE statistics for all roles that are defined in the PARTITION statement are included. If you use the CP or PRESS statistic as the SELECT=, STOP=, or CHOOSE= criterion in the SELECTION statement, then these statistics are also added to the criterion panel.
If you specify the PLOTS=FITBYROLE or PLOTS=ALL option and a PARTITION statement, then a plot of the ASE by the selection step for each role is created.
For best-subset selection, only a plot of the fit criteria is created if you specify the PLOTS=CRITERIA or PLOTS=ALL option. The fit criteria that are shown in the plot are the ADJRSQ, AIC, BIC, CP, RSQUARE, and SBC statistics.
For SCAD and MCP selection, no plot is produced in this release.