SEVSELECT Procedure
PROC SEVSELECT Statement
PROC SEVSELECT options;
The PROC SEVSELECT statement invokes the procedure. You can specify two types of options in the PROC SEVSELECT statement. One set of options controls input and output. The other set of options controls the model estimation and selection process.
The following options control the input data tables used by PROC SEVSELECT and various forms of output generated by PROC SEVSELECT. The options are listed in alphabetical order.
- COVOUT
specifies that the OUTEST= data table contain the estimate of the covariance structure of the parameters. This option has no effect if you do not specify the OUTEST= option. For more information about how the covariance is reported in the OUTEST= data table, see the section OUTEST= Data Table.
-
CRITERION=criterion-option
CRITERIA=criterion-option
CRIT=criterion-option -
specifies the criterion to report in the model selection table.
If you specify two or more candidate models for estimation, then the one that has the best value for the specified criterion is marked as the best model in the model selection table that is displayed by default or when you specify the PRINT=SELECTION option. If you specify the OUTSTAT= data table, then the observation for the model that has the best value for the specified criterion has a value of 1 for the variable
_SELECTED_.You can specify one of the following criterion-options:
- AD
specifies the Anderson-Darling (AD) statistic value, which is computed by using the empirical distribution function (EDF) estimate, as the selection criterion. A lower value is deemed better.
- AIC
specifies Akaike’s information criterion (AIC) as the selection criterion. A lower value is deemed better.
- AICC
specifies the finite-sample corrected Akaike’s information criterion (AICC) as the selection criterion. A lower value is deemed better.
- CUSTOM
specifies the custom objective function as the selection criterion. You can specify this criterion only if you also specify the OBJECTIVE= option. A lower value is deemed better.
- CVM
specifies the Cramér–von Mises (CvM) statistic value, which is computed by using the empirical distribution function (EDF) estimate, as the selection criterion. A lower value is deemed better.
- KS
specifies the Kolmogorov-Smirnov (KS) statistic value, which is computed by using the empirical distribution function (EDF) estimate, as the selection criterion. A lower value is deemed better.
- LOGLIKELIHOOD | LL
specifies
as the selection criterion, where L is the likelihood of the data. A lower value is deemed better.
- SBC | BIC
specifies the Schwarz Bayesian information criterion (SBC) as the selection criterion. A lower value is deemed better.
If you do not specify the CRITERION= option, then by default, the procedure uses CRITERION=CUSTOM if you specify the OBJECTIVE= option, or CRITERION=LOGLIKELIHOOD if you do not specify the OBJECTIVE= option.
For more information about these criterion-options, see the section Statistics of Fit.
- DATA=libref.data-table
-
names the input data table for PROC SEVSELECT to use. libref.data-table is a two-level name, where
- libref
refers to a collection of information that is defined in the LIBNAME statement and includes the
library, which includes a path to the data, and a session identifier, which defaults to the active session but which can be explicitly defined in the LIBNAME statement. For more information about libref, see the section Using CAS Sessions and CAS Engine Librefs.- data-table
specifies the name of the input data table.
- EDFALPHA=confidence-level
-
specifies the confidence level in the range (0,1) to use for computing the confidence intervals for the EDF estimates. The lower and upper confidence limits that correspond to this level are displayed in the plot that is created when you specify the PLOTS=CDFPERDIST option.
If you do not specify the EDFALPHA= option, then PROC SEVSELECT uses a default value of 0.05.
- INEST=libref.data-table
-
names the input data table that contains the initial values of the parameter estimates to start the optimization process.
libref.data-table is a two-level name, where libref refers to the library, and data-table specifies the name of the input data table. For more information about this two-level name, see the DATA= option and the section Using CAS Sessions and CAS Engine Librefs. The libref must be identical to the libref that you specify in the DATA= option.
The initial values that you specify in the INIT= option in the DIST statement take precedence over any initial values that you specify in the INEST= data table. For more information about the variables in this data table, see the section INEST= Data Table.
- NOCLPRINT<=number>
suppresses the display of the "Class Level Information" table if you do not specify number. If you specify number, the values of the classification variables are displayed for only those variables whose number of levels is less than number. Specifying a number helps reduce the size of the "Class Level Information" table if some classification variables have a large number of levels. This option has no effect if you do not specify the CLASS statement.
- NOPRINT
turns off all displayed and graphical output. If you specify this option, then any value that you specify for the PRINT= option is ignored.
- OUTEST=libref.data-table
-
names the output data table to contain estimates of the parameter values and their standard errors for each model whose parameter estimation process converges.
libref.data-table is a two-level name, where libref refers to the library, and data-table specifies the name of the output data table. For more information about this two-level name, see the DATA= option and the section Using CAS Sessions and CAS Engine Librefs. The libref must be identical to the libref that you specify in the DATA= option.
For more information about the variables in this data table, see the section OUTEST= Data Table.
- OUTMODELINFO=libref.data-table
-
names the output data table to contain the information about each candidate distribution.
libref.data-table is a two-level name, where libref refers to the library, and data-table specifies the name of the output data table. For more information about this two-level name, see the DATA= option and the section Using CAS Sessions and CAS Engine Librefs. The libref must be identical to the libref that you specify in the DATA= option.
For more information about the variables in this data table, see the section OUTMODELINFO= Data Table.
- OUTSTAT=libref.data-table
-
names the output data table to contain the values of statistics of fit for each model whose parameter estimation process converges.
libref.data-table is a two-level name, where libref refers to the library, and data-table specifies the name of the output data table. For more information about this two-level name, see the DATA= option and the section Using CAS Sessions and CAS Engine Librefs. The libref must be identical to the libref that you specify in the DATA= option.
For more information about the variables in this data table, see the section OUTSTAT= Data Table.
- OUTSTORE=libref.data-table
-
names the output data table to contain the context and results of the severity model estimation process in the form of an item store, which has a binary format that cannot be modified. You can specify this item store data table in a subsequent PROC CCDM step by using the SEVERITYSTORE= option.
libref.data-table is a two-level name, where libref refers to the library, and data-table specifies the name of the output data table. For more information about this two-level name, see the DATA= option and the section Using CAS Sessions and CAS Engine Librefs. The libref must be identical to the libref that you specify in the DATA= option.
-
PLOTS <(global-plot-options)> <=plot-request-option>
PLOTS <(global-plot-options)> <=(plot-request-option …plot-request-option)> -
specifies which graphical output to create. If you specify more than one global-plot-option, then separate them with spaces and enclose them in parentheses. If you specify more than one plot-request-option, then separate them with spaces and enclose them in parentheses.
When the active CAS session contains more than one worker node, PROC SEVSELECT prepares the plots by using an aggregation algorithm that merges the plot data from individual worker nodes. Each worker node in turn prepares its plot data by using a sample of the local data to compute the empirical quantiles for a uniform grid of points along the empirical distribution function (EDF) axis; this data sample is the same sample that it uses to compute the EDF-based fit statistics. So the plots are approximate. You can still use them to compare the fits of different distributions visually. However, to get a more accurate visual comparison, it is recommended that you request plots only when you run the PROC SEVSELECT step in a CAS session that uses no more than one worker node.
You can specify the following global-plot-options:
- HISTOGRAM
plots the histogram of the response variable on the probability density function (PDF) plots.
- KERNEL
plots the kernel estimate of the probability density of the response variable on the PDF plots.
You can specify the following plot-request-options:
- ALL
creates all the graphical output.
- CDF
creates a plot that compares the cumulative distribution function (CDF) estimates of all the candidate distribution models to the empirical distribution function (EDF) estimate. The plot does not contain CDF estimates for models whose parameter estimation process does not converge.
- CDFPERDIST
creates a plot of the CDF estimates of each candidate distribution model. A plot is not created for models whose parameter estimation process does not converge.
-
CONDITIONALPDF <(cpdf-options)>
CONDPDF <(cpdf-options)> -
creates a plot that compares the conditional probability density function (PDF) estimates of all the candidate distribution models. The plot does not contain conditional PDF estimates for models whose parameter estimation process does not converge.
A conditional PDF of a loss random variable Y in an interval
is the probability that a specific loss value is observed, given that the loss values belong to that interval. Formally, the conditional PDF of y, denoted by
, for the
interval is defined as
. If
and
denote the PDF and CDF, respectively, at loss value y, then
for the
interval is computed as
. The scaling factor of
ensures that the conditional PDF is a true PDF that integrates to 1 in the
interval.
The conditional PDF comparison plot contains at most three regions (intervals) of mutually exclusive ranges of the loss variable’s value,
where
and
denote the smallest and largest values, respectively, of the loss variable in the DATA= data table and
denotes a small machine-precision constant for a double-precision value.
You can specify the following cpdf-options to control how the values of L and R are computed and which regions are displayed:
-
LEFTQ=number
LEFT=number
L=number specifies a number, which must be between 0 and 1, that sets the CDF value at the end of the left-tail region. The left-tail region always starts at the minimum loss variable value in the DATA= data table. If number is
, then PROC SEVSELECT sets L (the end of the left-tail region) equal to the
th percentile, which is an empirical estimate of
, where Q denotes the quantile function.
-
RIGHTQ=number
RIGHT=number
R=number specifies a number, which must be between 0 and 1, that sets the CDF value at the start of the right-tail region. The right-tail region always ends at the maximum loss variable value in the DATA= data table. If number is
, then PROC SEVSELECT sets R (the start of the right-tail region) equal to the
th percentile, which is an empirical estimate of
, where Q denotes the quantile function.
-
SHOWREGION=(region-options)
SHOW=(region-options) -
specifies one or more regions to display in the plot. You can specify any combination of the following region-options. If you specify more than one region-option, you must enclose the region-options in parentheses.
- CENTER | C
displays the center region of the plot, which is the region between the end of the left-tail region and the beginning of the right-tail region. If you specify this option, you must also specify valid values for both the LEFTQ= and RIGHTQ= options.
- LEFT | L
displays the left-tail region of the plot. If you specify this option, you must also specify a valid value for the LEFTQ= option.
- RIGHT | R
displays the right-tail region of the plot. If you specify this option, you must also specify a valid value for the RIGHTQ= option.
If you do not specify the SHOWREGION option, then PROC SEVSELECT determines the default displayed regions as follows:
If you do not specify either the LEFTQ= or RIGHTQ= option, then this is equivalent to specifying (LEFTQ=0.25 RIGHTQ=0.75), and PROC SEVSELECT displays all three regions (left-tail, center, and right-tail).
If you specify valid values for both the LEFTQ= and RIGHTQ= options, then PROC SEVSELECT displays all three regions (left-tail, center, and right-tail).
If you specify a valid value for the LEFTQ= option but do not specify the RIGHTQ= option, then PROC SEVSELECT displays two regions: the left-tail region and the remaining region that combines the center and right-tail regions.
If you specify a valid value for the RIGHTQ= option but do not specify the LEFTQ= option, then PROC SEVSELECT displays two regions: the right-tail region and the remaining region that combines the center and left-tail regions.
Note: PROC SEVSELECT computes the percentile estimates, which serve as region boundaries L and R, by using the EDF estimates that it computes by using the EDF estimation process as described in the section Empirical Distribution Function Estimation Methods. For large input data tables or distributed mode of execution, the EDF estimates are approximate; therefore, the percentile estimates are also approximate—that is, they are not the same as the percentile estimates that would have been computed if the entire input data table were used.
-
LEFTQ=number
-
CONDITIONALPDFPERDIST <(cpdf-options)>
CONDPDFDIST <(cpdf-options)> -
creates a plot of the conditional PDF estimates of each candidate distribution model. A plot is not created for models whose parameter estimation process does not converge.
The cpdf-options are identical to those listed for the CONDITIONALPDF plot option, except that they are interpreted in the context of each candidate distribution individually. You can specify a different set of values for the cpdf-options in the CONDITIONALPDFPERDIST option than you specify in the CONDITIONALPDF option.
- NONE
creates none of the graphical output. If you specify this option, then it overrides all the other plot-request-options. The default graphical output is also suppressed.
creates a plot that compares the probability density function (PDF) estimates of all the candidate distribution models. The plot does not contain PDF estimates for models whose parameter estimation process does not converge.
- PDFPERDIST
creates a plot of the PDF estimates of each candidate distribution model. A plot is not created for models whose parameter estimation process does not converge.
- PP
creates the probability-probability plot (known as the P-P plot), which compares the CDF estimate of each candidate distribution model to the empirical distribution function (EDF). The data that are shown in this plot are used for computing the EDF-based statistics of fit.
creates the quantile-quantile plot (known as the Q-Q plot), which compares the empirical quantiles to the quantiles of each candidate distribution model.
-
PRINT <(global-display-options)> <=display-option>
PRINT <(global-display-options)> <= (display-option …display-option) > -
specifies which output to display. If you specify more than one display-option, then separate them with spaces and enclose them in parentheses.
You can specify the following global-display-options:
- ONLY
turns off the default displayed output and displays only the requested output.
-
SYMMETRIC=(<symmetric-matrix-print-options>)
SYMM=(<symmetric-matrix-print-options>) -
controls the contents of tables that display a symmetric matrix. You can specify the following symmetric-matrix-print-options:
- SHAPE=shape
-
specifies the shape of the symmetric matrix. You can specify one of the following shape values:
- FULL | F
displays the full matrix.
- LOWER | L
displays the lower triangular matrix.
- UPPER | U
displays the upper triangular matrix.
By default, SHAPE=FULL.
- NOUNITDIAG | ND
turns off the display of the diagonal for symmetric matrices that have a unit diagonal. By default, PROC SEVSELECT displays the unit diagonal.
If you do not specify the SYMMETRIC= option, then the procedure uses SYMMETRIC=(SHAPE=FULL) by default.
You can specify the following display-options:
- ALL
displays all the output.
- ALLFITSTATS
displays the comparison of all the statistics of fit for all the models in one table. The table does not include the models whose parameter estimation process does not converge.
- CONVSTATUS
displays the convergence status of the parameter estimation process.
-
CORRB
CORR displays the estimated correlation matrix of parameters. You can control the display of this matrix by specifying the SHAPE= and NOUNITDIAG suboptions of the SYMMETRIC= global-display-option.
-
COVB
COV displays the estimated variance-covariance matrix of parameters. You can control the display of this matrix by specifying the SHAPE= suboption of the SYMMETRIC= global-display-option.
- DESCSTATS
displays the descriptive statistics for the response variable. If you specify the SCALEMODEL statement, then this option also displays the descriptive statistics for the regression effects that do not contain a CLASS variable.
- DISTINFO
displays the information about each specified distribution. For each distribution, the information includes the name, description, validity status, and number of distribution parameters.
-
ESTIMATES
PARMEST displays the final estimates of parameters. The estimates are not displayed for models whose parameter estimation process does not converge.
- ESTIMATIONDETAILS
displays the details of the estimation process for all the models in one table.
- INITIALVALUES
displays the initial values and bounds used for estimating each model.
- NLOHISTORY
displays the iteration history of the nonlinear optimization process used for estimating the parameters.
- NLOSUMMARY
displays the summary of the nonlinear optimization process used for estimating the parameters.
- NONE
displays none of the output. If you specify this option, then it overrides all other display options. The default displayed output is also suppressed.
-
SELECTION
SELECT displays the model selection table.
-
STATISTICS
FITSTATS displays the statistics of fit for each model. The statistics of fit are not displayed for models whose parameter estimation process does not converge.
If you do not specify the PRINT= option or if you do not specify the ONLY global-display-option, then the default displayed output is equivalent to specifying PRINT=(SELECTION CONVSTATUS NLOSUMMARY STATISTICS ESTIMATES).
- PRINTINTERNALNAMES
- SELECTOUT
-
specifies that only the regression parameters that correspond to the selected effects be written to the OUTEST= data table. If you omit this option, then the special missing value .N is written for such regression parameters. For more information, see the section OUTEST= Data Table.
This option has no effect if you do not specify the SCALEMODEL and SELECTION statements.
- VARDEF=DF | N
-
specifies the denominator to use for computing the covariance estimates. You can specify one of the following values:
- DF
specifies that the number of nonmissing observations minus the model degrees of freedom (number of parameters) be used.
- N
specifies that the number of nonmissing observations be used.
For more information about the covariance estimation, see the section Estimating Covariance and Standard Errors.
- ZEROEST
-
specifies that zero be written to the OUTEST= data table as an estimate of the regression parameter that is not in the final model (because it is either collinear or not selected). If you omit this option, then the special missing value .R or .N is written for such regression parameters. For more information, see the section OUTEST= Data Table.
This option has no effect if you do not specify the SCALEMODEL statement.
The following options control the model estimation and selection process:
-
EMPIRICALCDF=method
EDF=method -
specifies the method to use for computing the nonparametric or empirical estimate of the cumulative distribution function of the data. You can specify one of the following values for method:
- AUTOMATIC | AUTO
-
specifies that the method be chosen automatically based on the data specification.
If you do not specify any censoring or truncation, then the standard empirical estimation method (STANDARD) is chosen. If you specify both right-censoring and left-censoring, then Turnbull’s estimation method (TURNBULL) is chosen. For all other combinations of censoring and truncation, the Kaplan-Meier method (KAPLANMEIER) is chosen.
- KAPLANMEIER | KM
specifies that the product limit estimator proposed by Kaplan and Meier (1958) be used. Specification of this method has no effect when you specify both right-censoring and left-censoring.
- MODIFIEDKM | MKM <(options)>
-
specifies that the modified product limit estimator be used. Specification of this method has no effect when you specify both right-censoring and left-censoring.
This method allows Kaplan-Meier’s product limit estimates to be more robust by ignoring the contributions to the estimate due to small risk-set sizes. The risk set is the set of observations at the risk of failing, where an observation is said to fail if it has not been processed yet and might experience censoring or truncation. You can specify the minimum risk-set size that makes it eligible to be included in the estimation either as an absolute lower bound on the size (RSLB= option) or a relative lower bound determined by the formula
proposed by Lai and Ying (1991). You can specify the values of c and
by using the C= and ALPHA= options, respectively. By default, the relative lower bound is used with values of c = 1 and
= 0.5. However, you can modify the default by using the following options:
- ALPHA | A=number
specifies the value to use for
when the lower bound on the risk set size is defined as
. This value must satisfy
.
- C=number
specifies the value to use for c when the lower bound on the risk set size is defined as
. This value must satisfy
.
- RSLB=number
specifies the absolute lower bound on the risk set size to be included in the estimate.
- NOTURNBULL
-
specifies that the method be chosen automatically based on the data specification and that Turnbull’s method not be used. This option is the default.
This method first replaces each left-censored or interval-censored observation with an uncensored observation. If the resulting set of observations has any truncated or right-censored observations, then the Kaplan-Meier method (KAPLANMEIER) is chosen. Otherwise, the standard empirical estimation method (STANDARD) is chosen. The observations are modified only for the purpose of computing the EDF estimates; the modification does not affect the parameter estimation process.
- STANDARD | STD
specifies that the standard empirical estimation method be used. If you specify both right-censoring and left-censoring, then the specification of this method has no effect. If you specify any other combination of censoring or truncation effects, then this method ignores such effects, and can thus result in estimates that are more biased than those obtained with other methods that are more suitable for censored or truncated data.
- TURNBULL | EM <(options)>
-
specifies that the Turnbull’s method be used. This method is used when you specify both right-censoring and left-censoring. An iterative expectation-maximization (EM) algorithm proposed by Turnbull (1976) is used to compute the empirical estimates. If you also specify truncation, then the modification suggested by Frydman (1994) is used.
This method is used if you specify both right-censoring and left-censoring and if you explicitly specify the EMPIRICALCDF=TURNBULL option.
You can modify the default behavior of the EM algorithm by using the following options:
- ENSUREMLE
specifies that the final EDF estimates be maximum likelihood estimates. The Kuhn-Tucker conditions are computed for the likelihood maximization problem and checked to ensure that EM algorithm converges to maximum likelihood estimates. The method generalizes the method proposed by Gentleman and Geyer (1994) by taking into account any truncation information that you might specify.
- EPS=number
specifies the maximum relative error to be allowed between estimates of two consecutive iterations. This criterion is used to check the convergence of the algorithm. If you do not specify this option, then PROC SEVSELECT uses a default value of 1.0E–8.
- MAXITER=number
specifies the maximum number of iterations to attempt to find the empirical estimates. If you do not specify this option, then PROC SEVSELECT uses a default value of 500.
- ZEROPROB=number
specifies the threshold below which an empirical estimate of the probability is considered zero. This option is used to decide if the final estimate is a maximum likelihood estimate. This option does not have an effect if you do not specify the ENSUREMLE option. If you specify the ENSUREMLE option, but do not specify this option, then PROC SEVSELECT uses a default value of 1.0E–8.
For more information about each of the methods, see the section Empirical Distribution Function Estimation Methods.
-
INITSAMPLE (initsample-option)
INITSAMPLE (initsample-option …initsample-option) -
specifies that a sample of the input data be used for initializing the distribution parameters. If you specify more than one initsample-option, then separate them with spaces.
When you do not specify initial values for the distribution parameters, PROC SEVSELECT needs to compute the empirical distribution function (EDF) estimates as part of the default method for parameter initialization. The EDF estimation process can be expensive, especially when you specify censoring or truncation effects for the loss variable. Furthermore, it is not amenable to parallelism due to the sequential nature of the algorithm for truncation effects. You can use the INITSAMPLE option to specify that only a fraction of the input data be used in order to reduce the time taken to compute the EDF estimates. PROC SEVSELECT uses the uniform random sampling method to select the sample, the size and randomness of which are controlled by the following initsample-options:
- FRACTION=number
specifies the fraction, between 0 and 1, of the input data to be used for sampling.
- SEED=number
specifies the seed to be used for the uniform random number generator. This option enables you to select the same sample from the same input data across different runs of PROC SEVSELECT, which can be useful for replicating the results across different runs. If you do not specify the seed value, PROC SEVSELECT generates a seed that is based on the system clock.
- SIZE=number
-
specifies the size of the sample. If the data are distributed across different nodes, then this size applies to the sample that is prepared at each node. For example, let the input data table of size 100,000 observations be distributed across 10 nodes such that each node has 10,000 observations. If you specify SIZE=1000, then each node computes a local EDF estimate by using a sample of size 1,000 selected randomly from its 10,000 observations.
If you specify both the SIZE= and FRACTION= options, then the value that you specify in the SIZE= option is used and the FRACTION= option is ignored.
If you do not specify the INITSAMPLE option, then a uniform random sample of at most 10,000 observations is used for EDF estimation on each node that has observations.
-
NOBALANCETHREADS
NOBALTHR -
specifies that the observations do not need to be balanced among threads of computation.
Starting with SAS Econometrics 8.5, the parameter estimation phase balances the observations among all threads of computation by default; this helps achieve significantly faster estimation times. However, this balanced processing can cause small numerical differences compared to the prior releases that did not balance observations among threads of computation. You should specify this option only if you want to restore the numerical behavior of releases prior to 8.5. However, use of this option can lead to a significantly slower estimation phase in most cases, and therefore it is strongly discouraged.
-
NOCONSTFITSTATS
NOCONSTSOF -
excludes the constant distribution parameters, if any are specified, from the calculations of likelihood-based fit statistics (AIC, AICC, and SBC) that depend on the number of model parameters.
By default, constant distribution parameters are assumed to be estimable (even if PROC SEVSELECT does not estimate them) and are included in the calculations of the likelihood-based fit statistics.
- OBJECTIVE=symbol-name
-
names the symbol that represents the objective function in the SAS programming statements that you specify. For each model to be estimated, PROC SEVSELECT executes the programming statements to compute the value of this symbol for each observation. The values are added across all observations to obtain the value of the objective function. The optimization algorithm estimates the model parameters such that the objective function value is minimized. A separate optimization problem is solved for each candidate distribution. If you specify a BY statement, then a separate optimization problem is solved for each candidate distribution within each BY group.
For more information about writing SAS programming statements to define your own objective function, see the section Custom Objective Functions.
- SELECTNLOTECH=CONGRA | DBLDOG | NEWRAP | NMSIMP | NRRIDG | QUANEW | TRUREG
-
specifies the nonlinear optimization technique to use for the intermediate steps of the regression effect selection process. The can specify one of the following techniques:
- CONGRA
performs a conjugate-gradient optimization.
- DBLDOG
performs a version of double-dogleg optimization.
- NEWRAP
performs a Newton-Raphson optimization that combines a line-search algorithm with ridging.
- NMSIMP
performs a Nelder-Mead simplex optimization.
- NRRIDG
performs a Newton-Raphson optimization with ridging.
- QUANEW
performs a dual quasi-Newton optimization.
- TRUREG
performs a trust region optimization.
This option has no effect if you do not specify the SELECTION statement.
If you omit this option, then the default optimization technique for the intermediate steps of the selection process is the same technique that PROC SEVSELECT uses to estimate the parameters of the final selected model. You can control that technique by specifying the TECHNIQUE= option in the NLOPTIONS statement.