GENSELECT Procedure
Example 14.1 Model Selection
(View the complete code for this example.)
The following statements examine the same data set that is used in the section Getting Started: GENSELECT Procedure, but they request model selection via the forward selection technique. Model effects are added in the order of their significance until no more effects significantly improve the current model. The DETAILS=ALL option in the SELECTION statement requests that all tables that are related to model selection be produced.
The data set getStarted is shown in the section Getting Started: GENSELECT Procedure. It contains 100 observations on a count response variable (Y), a continuous variable (Total) to be used in Example 14.2, and five categorical variables (C1–C5), each of which has four numerical levels.
A log-linked Poisson regression model is specified by using classification effects for variables C1–C5. The following statements request model selection by the forward selection method:
ods graphics on;
proc genselect data=mylib.getStarted;
class C1-C5;
model Y = C1-C5 / Distribution=Poisson;
selection method=forward details=all plots=all;
run;
ods graphics off;
The model selection tables are shown in Output 14.1.1 through Output 14.1.3. Selection graphics produced by the PLOTS= option are displayed in Output 14.1.6 and Output 14.1.7.
The "Selection Information" table in Output 14.1.1 summarizes the settings for the model selection. The effect that produces the smallest approximate SBC is added to the model. The forward selection stops three steps after the smallest SBC is obtained, or when all effects have been added to the model.
Output 14.1.1: Selection Information
| Selection Information | |
|---|---|
| Selection Method | Forward |
| Select Criterion | SBC |
| Stop Criterion | SBC |
| Effect Hierarchy Enforced | None |
| Stop Horizon | 3 |
The "Selection Summary" table in Output 14.1.2 shows the effects that were added to the model and the approximate SBC. Step 0 refers to the null model that contains only an intercept. In the next step, effect C2 resulted in the smallest approximate SBC among the candidate effects. In step 2, the smallest SBC when an effect is added to a model that contains the intercept and C2 was achieved by adding C5 to the model. Similarly, in steps 3, 4, and 5, effects were added in order of the minimum SBC among remaining candidates. Finally, the model in step 2 was chosen as the best because it had the smallest SBC among all the candidate models.
Output 14.1.2: Selection Summary Information
| Selection Summary | |||
|---|---|---|---|
| Step | Effect Entered | Number Effects In | SBC |
| 0 | Intercept | 1 | 354.7980 |
| 1 | C2 | 2 | 342.8795 |
| 2 | C5 | 3 | 333.8182* |
| 3 | C1 | 4 | 341.7993 |
| 4 | C4 | 5 | 351.8375 |
| 5 | C3 | 6 | 363.8674 |
| * Optimal Value Of Criterion | |||
| Selection stopped at a local minimum of the STOP criterion. |
| The model at step 2 is selected. |
| Selected Effects: | Intercept C2 C5 |
|---|
For each step of the selection process, the DETAILS=ALL option displays the candidate effects for entering the model along with their SELECT= criterion. Output 14.1.3 displays this table for the first step; the other steps are not shown here.
Output 14.1.3: Step1 Entry Candidates
| Entry Candidates | ||
|---|---|---|
| Rank | Effect | SBC |
| 1 | C2 | 342.8795 |
| 2 | C5 | 347.5376 |
| 3 | C1 | 355.7067 |
| 4 | C4 | 361.7379 |
| 5 | C3 | 365.7339 |
The DETAILS=ALL option also displays the dimensions, fit statistics, and parameter estimates at each step of the selection process; these details are not shown here.
Output 14.1.4 displays information about the selected model. Notice that the –2 log-likelihood value in the "Fit Statistics" table is larger than the value for the full model in Figure 6. This is expected because the selected model contains only a subset of the parameters. Because the selected model is more parsimonious than the full model, the information criteria AIC, AICC, and BIC are smaller than in the full model, indicating a better fit.
Output 14.1.4: Fit Statistics
| Fit Statistics | |
|---|---|
| -2 Log Likelihood | 303.58036 |
| AIC (smaller is better) | 317.58036 |
| AICC (smaller is better) | 318.79775 |
| SBC (smaller is better) | 335.81655 |
The parameter estimates of the selected model are shown in Output 14.1.5. Notice that the effects are listed in the "Parameter Estimates" table in the order in which they were specified in the MODEL statement and not the order in which they were added to the model.
Output 14.1.5: Parameter Estimates
| Parameter Estimates | |||||
|---|---|---|---|---|---|
| Parameter | DF | Estimate | Standard Error | Chi-Square | Pr > ChiSq |
| Intercept | 1 | 0.528024 | 0.215231 | 6.0186 | 0.0142 |
| C2 0 | 1 | 0.962636 | 0.236921 | 16.5088 | <.0001 |
| C2 1 | 1 | 0.845132 | 0.233634 | 13.0851 | 0.0003 |
| C2 2 | 1 | 0.213247 | 0.248896 | 0.7341 | 0.3916 |
| C2 3 | 0 | 0 | . | . | . |
| C5 0 | 1 | -0.763491 | 0.207082 | 13.5933 | 0.0002 |
| C5 1 | 1 | -0.785915 | 0.201271 | 15.2471 | <.0001 |
| C5 2 | 1 | -0.541212 | 0.208661 | 6.7275 | 0.0095 |
| C5 3 | 0 | 0 | . | . | . |
The coefficient panel in Output 14.1.6 enables you to visualize the selection process. In this plot, standardized coefficients of all the effects that are selected at some step of the stepwise method are plotted as a function of the step number. This enables you to assess the relative importance of the effects that are selected at any step of the selection process and to know when effects entered the model. The lower plot in the panel shows how the criterion that is used to choose the selected model changes as effects enter or leave the model.
Output 14.1.6: Coefficient Progression

The criterion panel in Output 14.1.7 provides a graphical view of the progression of the fit criteria as the selection process evolves.
Output 14.1.7: Criterion Panel
