The QTRSELECT Procedure
Displayed Output
The following sections describe the output produced by PROC QTRSELECT. The output is organized into various tables, which are discussed in their order of appearance.
Selection Information
When you specify the SELECTION statement, the QTRSELECT procedure produces by default a series of tables that display information about the model selection. The "Selection Information" table informs you about the model selection method; select, stop, and choose criteria; and other parameters that govern the selection. You can suppress this table by specifying DETAILS=NONE in the SELECTION statement.
Number of Observations
The "Number of Observations" table displays the number of observations read from the input data table and the number of observations used in the analysis. If you use a PARTITION statement, the table also displays the number of observations used for each data role.
Class Level Information
The "Class Level Information" table lists the levels of every variable that you specify in the CLASS statement. You should check this information to make sure that the data are correct. You can adjust the order of the CLASS variable levels by specifying the ORDER= option in the CLASS statement. You can suppress the "Class Level Information" table completely or partially by specifying the NOCLPRINT= option in the PROC QTRSELECT statement.
If the classification variables use a nonsingular parameterization, the "Class Level Information" table also displays the reference value for each variable.
Dimensions
The "Dimensions" table displays the number of effects and the number of parameters from which the selected model is chosen. If you use split classification variables, then this table also displays the number of effects after splitting is taken into account.
Entry and Removal Candidates
When you specify the DETAILS=ALL or DETAILS=STEPS option in the SELECTION statement, the QTRSELECT procedure produces "Entry Candidates" and "Removal Candidates" tables that display the effect names and values of the criterion used to select entering or departing effects at each step of the selection process. The effects are displayed in sorted order from best to worst of the selection criterion.
Selection Summary
When you specify the SELECTION statement, the QTRSELECT procedure produces the "Selection Summary" table, which displays information about the sequence of steps of the selection process. For each step, the effect that was entered or dropped is displayed along with the statistics used to select the effect, stop the selection, and choose the selected model. For all criteria that you can use for model selection, the steps at which the optimal values of these criteria occur are also indicated.
You can suppress the display of the "Selection Summary" table by specifying DETAILS=NONE in the SELECTION statement.
Stop Reason
The "Stop Reason" table displays the reason why the selection stopped. To facilitate programmatic use of this table, an integer code is assigned to each reason and is included if you use an ODS OUTPUT statement or a DISPLAYOUT statement to output this table. The reasons and their associated codes follow:
Code | Stop Reason |
|---|---|
1 | All eligible effects are in the model. |
2 | All eligible effects have been removed. |
3 | Specified maximum number of steps done. |
4 | The model contains the specified maximum number of effects. |
5 | The model contains the specified minimum number of effects (for backward selection). |
6 | The stopping criterion is at a local optimum. |
7 | No suitable add or drop candidate could be found. |
8 | Adding or dropping any effect does not improve the selection criterion. |
9 | No candidate meets the appropriate SLE or SLS significance level. |
10 | Stepwise selection is cycling. |
11 | The model is an exact fit. |
12 | Dropping an effect would result in an empty model. |
You can suppress the display of the "Stop Reason" table by specifying DETAILS=NONE in the SELECTION statement.
Selection Reason
When you specify the SELECTION statement, the QTRSELECT procedure produces a simple table that explains why the final model was selected.
You can suppress the display of the "Selection Reason" table by specifying DETAILS=NONE in the SELECTION statement.
Selected Effects
When you specify the SELECTION statement, the QTRSELECT procedure produces a simple table that lists which effects were selected for the final model.
Fit Statistics
The "Fit Statistics" table displays fit statistics for the selected model. The statistics include the following:
Objective Function, total sum of check losses. Objective Function is denoted as in Table 18.9.
R1, a measure between 0 and 1 that indicates the portion of the (corrected) total check losses attributed to the fit rather than left to residuals. It is calculated as . It is the quantile regression counterpart of the linear regression R square.
Adj R1, the adjusted R1, a version of R1 that has been adjusted for degrees of freedom. It is calculated as
where if the intercept is forced in and otherwise , n is the number of observations used to fit the model, and p is the number of parameters in the model.
the fit criteria AIC, AICC, and SBC if they are used in the selection process. For the formulas to evaluate these criteria, see Table 18.9.
the average check loss (ACL) on the training, validation, and test data
You can request the "Fit Statistics" tables for the model at each step of the selection process by specifying the DETAILS= option in the SELECTION statement.
Parameter Estimates
The "Parameter Estimates" table displays the parameters in the selected model and their estimates. The information displayed for each parameter in the selected model includes the following:
the parameter label that includes the effect name and level information for effects that contain classification variables
the degrees of freedom (DF) for the parameter. There is one degree of freedom unless the model is not full rank.
the parameter estimate
the standard error, which is the estimate of the standard deviation of the parameter estimate
t Value, the t test that the parameter is 0. This is computed as the parameter estimate divided by the standard error.
the Pr > |t|, the probability that a t statistic would obtain a greater absolute value than that observed given that the true parameter is 0. This is the two-tailed significance probability.
When you do model selection, these p-values are usually liberal because they are not adjusted for the fact that the terms in the model have been selected.
You can request a "Parameter Estimates" table for the model at each step of the selection process by specifying the DETAILS= option in the SELECTION statement.
Timing
The "Timing" table displays the amount of time (in seconds) and the percentage of the time that PROC QTRSELECT required to perform different tasks in the analysis.
OutputCasTables Table
The OutputCasTables table is a special table that has information about each CAS table that is created during a CAS action execution. The information for each CAS table consists of the CAS table name, the caslib in which the table resides, and the number of columns and rows in the CAS table. Because this table is not a typical ODS table that contains analytical results, you cannot include it in the table-spec-list in the DISPLAYOUT statement.