GAMSELECT Procedure

(View the complete code for this example.)

This example continues the analysis of building an additive model for the data table that is formed in Example 13.2.

The following program uses the shrinkage method to perform the selection:

proc gamselect data=mylib.one seed=123 plots=all;
   model y = &splineTerms;
   selection method=shrinkage;
run;

Output 13.3.1 shows that the selected model converged and the fittings for all the regularization parameter candidates succeeded.

Output 13.3.2 shows the number of tries and successes of regularization parameter fittings. It also shows how many iterations the selected model required and how many effects were selected.

Output 13.3.3 displays the upper bounds for the regularization parameters that PROC GAMSELECT computed from the input data table and the selected values after the grid search.

Output 13.3.4 shows that six effects are selected: the intercept, the splines of four variables (v1–v4) that construct the true model, and one additional spline of variable v15.

Output 13.3.5 displays the fit statistics for the model; in this case, there is only one—the ASE.

Output 13.3.6 displays the values of sparsity penalty, smoothness penalty, and generalized ridge penalty for the selected spline terms. Compared to the first four splines, the last spline of variable v15 has somewhat smaller penalty values.

The “Smoothing Component Panel” displays the fitted curves of the five selected spline terms. The vertical scale of Spline(v15) indicates that the contribution from this effect is small compared to the contribution from the other spline effects.

Meier, Van de Geer, and Bühlmann (2009) propose an adaptive operation that forms sparsity penalties and smoothness penalties in a data-adaptive way, which is similar to the idea of the adaptive LASSO (Zou 2006). In the context of shrinkage selection, this adaptive option corresponds to setting the WEIGHT1 and WEIGHT2 options in individual spline terms, on the basis of previously computed sparsity penalty and smoothness penalty values. Using the notation in the section Shrinkage, you compute the adaptive weights as

where and are computed by using the selected regularization parameters (Output 13.3.3) divided by the reported sparsity penalty and smoothness penalty values, respectively, shown in Output 13.3.6.

The following program uses the computed values of and from the previous analysis and performs one additional step of additive model selection in an adaptive fashion:

proc gamselect data=mylib.one seed=123 plots=all;
   model y=spline(v1 / weight1=0.716  weight2=0.075)
           spline(v2 / weight1=0.693  weight2=0.206)
           spline(v3 / weight1=1.046  weight2=17.62)
           spline(v4 / weight1=0.488  weight2=0.090)
           spline(v15/ weight1=343.5  weight2=34.68);
   selection method=shrinkage;
run;

Output 13.3.8 shows that the intercept and splines of the four variables that construct the true response were selected in the adaptive model.

Output 13.3.9 displays the fit statistics for the adaptive model; in this case, there is only one—the ASE.

Output 13.3.10 shows the fitted curves from the adaptive model.