Descriptions of Model Selection Criteria

The definitions and formulas for the statistics of fit that are available in SAS Visual Forecasting are described below. You can use statistics of fit to measure how well different models fit the data. The statistics of fit for the various forecasting models can be printed or stored in a data set.

The formulas below use the notations described in Notations Used in the Formulas.

For most statistics of fit, a lower number indicates a better fit. If a different measure is used to determine a better fit, it is indicated in the description below. Choose from one of the following options.

Adjusted R-square (ADJRSQ)

The adjusted R2 statistic, 1 negative open . fraction n minus 1 , over n minus k end fraction . close open 1 negative , r squared , close. Click image for alternative formats., where R2 is defined in the RSQUARE statistic .

Better performance is indicated when this measurement is closer to 1.

Akaike information Corrected criterion (AICC)

Akaike’s information criterion with an empirical correction for small sample sizes, eh i c plus open . fraction 2 k open k plus 1 close , over n minus k minus 1 end fraction . close. Click image for alternative formats.

Akaike information criterion (AIC)

Akaike’s information criterion, n natural log of open . fraction s s e , over n end fraction . close plus 2 k. Click image for alternative formats.

Amemiya's adjusted R-square (AADJRSQ)

Amemiya’s adjusted R2, 1 minus open . fraction n plus k , over n minus k end fraction . close open 1 minus , r squared , close. Click image for alternative formats.

Better performance is indicated when this measurement is closer to 1.

Amemiya's prediction criterion (APC)

Amemiya’s prediction criterion, 1 over n s s t open . fraction n plus k , over n minus k end fraction . close open 1 minus , r squared , close equals open . fraction n plus k , over n minus k end fraction . close , 1 over n s s e. Click image for alternative formats.

Geometric mean absolute error percent of standard deviation (GMAPES)

The geometric mean of the absolute error as a percentage of the standard deviation

Geometric mean percent error (GMAPE)

The geometric mean percent error

Geometric mean predictive percent error (GMAPPE)

The geometric mean absolute predictive percent prediction error

Geometric mean relative absolute error (GMRAE)

The geometric mean of the relative absolute errors

Geometric mean symmetric percent error (GMASPE)

The geometric mean of the absolute symmetric percent errors

In-sample mean absolute scaled error (IMASE)

The mean of the in-sample absolute scaled errors (IMASE):

  • For the fit region, normal upper I normal upper M normal upper A normal upper S normal upper E equals normal upper M normal upper A normal upper E divided by normal upper M normal upper R normal upper W normal upper A normal upper E. Click image for alternative formats., where MRWAE is the mean absolute error of the random walk model in the fit region: normal upper M normal upper R normal upper W normal upper A normal upper E equals StartFraction 1 Over n minus 1 EndFraction sigma summation Underscript t equals 2 Overscript n Endscripts StartAbsoluteValue y Subscript t Baseline minus y Subscript t minus 1 Baseline EndAbsoluteValue. Click image for alternative formats..
  • For the forecasta numerical prediction of a future value for a specified time period for each unique combination of BY variable values region, normal upper I normal upper M normal upper A normal upper S normal upper E equals normal upper F normal upper C normal upper S normal upper T normal upper M normal upper A normal upper E divided by normal upper M normal upper R normal upper W normal upper A normal upper E. Click image for alternative formats. where FCSTMAE is the mean absolute error in the forecast region: normal upper F normal upper C normal upper S normal upper T normal upper M normal upper A normal upper E equals StartFraction 1 Over h EndFraction sigma summation Underscript t equals n plus 1 Overscript n plus h Endscripts StartAbsoluteValue y Subscript t Baseline minus ModifyingAbove y With caret Subscript t Baseline EndAbsoluteValue. Click image for alternative formats.
Maximum Absolute Error Percent of Standard Deviation (MAXAPES)

The maximum of the absolute error as a percentage of the standard deviation

Maximum error (MAXERR)

The largest prediction error

Maximum percent error (MAXPE)

The largest percent prediction error, 100 , mehx of open . fraction open , y sub t , minus , y hat sub t , close , over y sub t end fraction . close. Click image for alternative formats.. The summation ignores observations where y sub t , equals 0. Click image for alternative formats..

Maximum predictive percent error (MAXPPE)

The maximum of the predictive percent errors

Maximum relative error (MAXRE)

The maximum of the relative errors

Maximum symmetric percent error (MAXSPE)

The maximum of the symmetric percent errors

Mean absolute error (MAE)

The mean absolute prediction error, 1 over n . cap sigma with subscript t equals 1 , and with superscript n , end sub-superscript . vertical line , y sub t , minus , y hat sub t , vertical line. Click image for alternative formats.

Mean absolute error percent of standard deviation (MAPES)

The mean of the absolute error as a percentage of the standard deviation

Mean absolute percent error (MAPE)

The mean of the absolute percent errors, 100 over N, N over ∑ over i = 1, | y ( i ) − y ^ ( i ) | over | y ( i ) |. Click image for alternative formats.

Mean absolute predictive symmetric percent error (MAPPE)

The mean of the absolute symmetric predictive percent error

Mean absolute scaled error (MASE)

The mean of the absolute scaled errors

  • For the fit region, .. Click image for alternative formats. where MRWAE (mean absolute error of the random walk model in the fit region) is normal upper M normal upper R normal upper W normal upper A normal upper E equals StartFraction 1 Over n minus 1 EndFraction sigma summation Underscript t equals 2 Overscript n Endscripts StartAbsoluteValue y Subscript t Baseline minus y Subscript t minus 1 Baseline EndAbsoluteValue. Click image for alternative formats..
  • For the forecast region, .. Click image for alternative formats., where FCSTMAE (mean absolute error in the forecast region) is normal upper F normal upper C normal upper S normal upper T normal upper M normal upper A normal upper E equals StartFraction 1 Over h EndFraction sigma summation Underscript t equals n plus 1 Overscript n plus h Endscripts StartAbsoluteValue y Subscript t Baseline minus ModifyingAbove y With caret Subscript t Baseline EndAbsoluteValue. Click image for alternative formats. and where FCSTMRWAE (mean absolute error of the random walk model in the forecast region) is .. Click image for alternative formats.
Mean error (ME)

The mean prediction error, 1 over n . cap sigma with subscript t equals 1 , and with superscript n , end sub-superscript . open , y sub t , minus , y hat sub t , close. Click image for alternative formats.

Mean percent error (MPE)

The mean percent prediction error, 1 over n . cap sigma with subscript t equals 1 , and with superscript n , end sub-superscript . fraction open , y sub t , minus , y hat sub t , close , over y sub t end fraction. Click image for alternative formats.. The summation ignores observations where y sub t , equals 0. Click image for alternative formats..

Mean predictive percent error (MPPE)

The mean of the predictive percent error

Mean relative absolute error (MRAE)

The mean of the relative absolute errors

Mean relative error (MRE)

The mean of the relative errors

Mean squared error (MSE)

The mean squared prediction error calculated from the one-step-ahead forecasts, m s e equals , 1 over n s s e. Click image for alternative formats.. This formula enables you to evaluate small holdout samples.

Mean symmetric percent error (MSPE)

The mean of the symmetric percent errors

Mean absolute symmetric percent error (SMAPE)

The symmetric mean of the absolute percent error

Median absolute error percent of standard deviation (MDAPES)

The median of the absolute error as a percentage of the standard deviation

Median absolute percent error (MDAPE)

The median of the percent errors

Median predictive percent error (MDAPPE)

The median of the predictive percent errors

Median relative absolute error (MDRAE)

The median of the relative absolute errors

Median symmetric percent error (MDASPE)

The median of the symmetric percent errors

Minimum Absolute Error Percent of Standard Deviation (MINAPES)

The minimum of the absolute error as a percentage of the standard deviation

Minimum error (MINERR)

The smallest prediction error

Minimum percent error (MINPE)

The smallest percent prediction error, 100 , min of open . fraction open , y sub t , minus , y hat sub t , close , over y sub t end fraction . close. Click image for alternative formats.. The summation ignores observations where y sub t , equals 0. Click image for alternative formats..

Minimum predictive percent error (MINPPE)

The smallest predictive percent error

Minimum relative error (MINRE)

The smallest relative error

Minimum symmetric percent error (MINSPE)

The smallest symmetric percent error

R-square (RSQUARE)

The R2 statistic, r squared , equals 1 minus . fraction s s e , over s s t end fraction. Click image for alternative formats.. If the model fits the series badly, the model sum of squares errors, SSE, might be larger than SST and the R2 statistic is negative.

Better performance is indicated when this measurement is closer to 1.

Random walk R-square (RWRSQ)

The random walk R2 statistic (Harvey’s R2 statistic using the random walk model for comparison), 1 minus open . fraction n minus 1 , over n end fraction . close . fraction s s e , over r w s s e end fraction. Click image for alternative formats., where RWSSE (sum of square errors of the random walk model) is r w s s e equals . cap sigma with subscript t equals 2 , and with superscript n , end sub-superscript . open , y sub t , minus . y sub t minus 1 end sub . minus mu close squared. Click image for alternative formats., and mu equals . fraction 1 , over n minus 1 end fraction . cap sigma with subscript t equals 2 , and with superscript n , end sub-superscript . open , y sub t , minus . y sub t minus 1 end sub . close. Click image for alternative formats.

Better performance is indicated by a higher value.

Root mean squared error (RMSE)

The root mean square error, square root of m s e end root. Click image for alternative formats.

Root mean square scaled error (RMSSE)

The root value of the mean of the square scaled errors

  • For the fit region, normal upper R normal upper M normal upper S normal upper S normal upper E equals StartRoot normal upper M normal upper S normal upper E divided by normal upper M normal upper R normal upper W normal upper S normal upper E EndRoot. Click image for alternative formats. where MRWSE is the mean square error of the random walk model: m r w s e equals . fraction 1 , over n minus 1 end fraction . cap sigma with subscript t equals n plus 1, and with superscript n , end sub-superscript . open , y sub t , minus . y sub t minus 1 end sub . close squared. Click image for alternative formats.
  • For the forecast region, normal upper R normal upper M normal upper S normal upper S normal upper E equals StartRoot normal upper F normal upper C normal upper S normal upper T normal upper M normal upper S normal upper E divided by normal upper M normal upper R normal upper W normal upper S normal upper E EndRoot. Click image for alternative formats. where FCSTMSE is the mean square error in the forecast region: normal upper F normal upper C normal upper S normal upper T normal upper M normal upper S normal upper E equals StartFraction 1 Over h EndFraction sigma summation Underscript t equals n plus 1 Overscript n plus h Endscripts left parenthesis y Subscript t Baseline minus ModifyingAbove y With caret Subscript t Baseline right parenthesis squared. Click image for alternative formats.
Schwarz Bayesian information criterion (SBC)

Schwarz Bayesian information criterion, n times natural log of open . fraction s s e , over n end fraction . close plus k times natural log of open n close. Click image for alternative formats.

Sum of square error (SSE)

The sum of the squared prediction errors, s s e equals . cap sigma with subscript t equals 1 , and with superscript n , end sub-superscript . open , y sub t , minus , y hat sub t , close squared. Click image for alternative formats., where y With caret Subscript t. Click image for alternative formats. is the one-step predicted value

Unbiased mean squared error (UMSE)

The unbiased mean squared error

Unbiased root mean squared error (URMSE)

The unbiased root mean squared error

Corrected Total Sum of Squares (SST)

The total sum of squares for the series corrected for the mean: cap sigma with subscript t equals 1 , and with superscript n , end sub-superscript . open , y sub t , minus , y with macron above , close squared. Click image for alternative formats., where y with macron above. Click image for alternative formats. is the series mean

Note: This statistic is not available for model selection.
Total Sum of Squares (TSS)

The total sum of squares for the series, uncorrected for the mean: cap sigma with subscript t equals 1 , and with superscript n , end sub-superscript . y sub t and super 2. Click image for alternative formats.

Note: This statistic is not available for model selection.
Last updated: March 16, 2026