After the candidate models have been subset by the diagnostics, each model is fit to the data (with the holdout samplethe number of periods of the most recent data that should be excluded from the parameter estimation. The holdout sample can be used to evaluate the forecasting performance of a candidate model. excluded). After model fitting, the one-step-ahead forecasts are made in the fit region (in-sample) or the multistep-ahead forecasts are made in the holdout sample region. The model selection criterion, such as MAPE or RMSE, is used to select the best performing model from the appropriate subset of the candidate models. The model selection criteria are statistics of fit. See Descriptions of Model Selection Criteria for a complete list.
If the length of the time seriesan aggregation of transactional data into specified time intervals and sorted according to unique combinations of the default attributes (BY variables) is short, holdout sample analysis might not be possible due to a lack of data. In this situation, the full range of the data should be used for fitting and evaluation; otherwise, holdout sample analysis is recommended.