Shared Concepts
Folded Concave Penalized Selection
This section applies to the REGSELECT procedure.
Recall that adaptive LASSO selection consists of a two-stage approach that involves an initial estimator to reduce bias for large regression coefficients. An alternative single-stage approach is to use a penalty that tapers off as 
becomes larger in absolute value.
More specifically, the folded concave penalized (FCP) selection coefficients 
are the solution to the regularized least squares
where 
is a folded concave penalty.
Figure 6: Penalty Functions: LASSO, SCAD, and MCP
Figure 6 shows the penalty functions in the LASSO, SCAD, and MCP methods. When the absolute value of 
becomes large, the corresponding folded concave penalty value increases much more slowly than the LASSO penalty value.
If you have a good estimate of 
or 
, you can specify the value in either the ALPHA= or LAMBDA= option. If you omit the values for and , then by default the procedure searches for a pair of 
that is optimal according to the current value of the CHOOSE= criterion (by default, CHOOSE=SBC).
A variety of nonconvex penalties have been proposed. Two of the earliest and most influential nonconvex penalties are the smoothly clipped absolute deviation (SCAD) and the minimax concave penalty (MCP).
SCAD
In SCAD selection (Fan and Li 2001), the takes the following form:
The SCAD estimator 
solves the following minimization problem:
The quadratic program (QP) reformulation of the preceding SCAD problem is given by
where 
and 
.
Liu, Yao, and Li (2016) show that the preceding QP (1) is equivalent to the following mixed integer linear program (MILP):
where 
is a properly large constant.
MCP
In MCP selection (Zhang 2010), the takes the following form:
The MCP estimator 
solves the following minimization problem:
The QP reformulation of the preceding MCP problem is given by
Liu, Yao, and Li (2016) show that the preceding QP (3) is equivalent to the following MILP:
where is a properly large constant.
SOLVER
When you specify SOLVER=MILP, the coefficients are obtained by solving the minimization problem (2) or (3). Furthermore, you can also specify the BIGM= option for the 
value and the INTTOL= option for the tolerance of integer variables 
. By default, 
and INTTOL=1E–7.
When you specify SOLVER=NLP, the coefficients are obtained by solving the minimization problem (1) or (3).
LAMBDAGRID
When you search for the optimal model, the default values of 
and 
are, respectively,
where 
is the standard deviation of 
, m is the length of , and 
is the natural logarithm.
Suppose that the number of searching steps is 
. You can define the value by specifying the MAXITERLAMBDA= option. When you specify LAMBDAGRID=LINSPACE, the values of series are generated as follows:
where 
.
When you specify LAMBDAGRID=LOGSPACE, the values of series are generated as follows:
where 
.