Selecting a Champion Pipeline

Summary Tab

The following information is available on the Summary tab for Model Comparison Results. Click at the top of each table or graph to download a CSV file for the data. You can choose to download either the raw data or data formatted for your specific locale.

Model Comparison

• At the top of the Summary tab is the Model Comparison table. The columns are described below.

  Champion - The champion modeling node is marked with . If you determine that another model should be selected as champion, you can manually select the champion model for the pipeline. To select a champion model, right-click the Model Comparison node in the pipeline and select the model from the Select champion model list. If you choose a different champion model, you need to rerun the Model Comparison and Output nodes.

  The following shows the Model Comparison results from a pipeline with four modeling nodes. Two of the modeling nodes are Hierarchical Forecasting. The user applied different transformations on the dependent variable for each node, and renamed them, to compare the results. WMAPE was chosen as the model selection criteria. In this case, the Seasonal Model was selected as champion, based on its performance.

  
• Model Name - Each model in the pipeline or segment is listed.
• Status - Indicates the status of each modeling node in the pipeline or segment. If any models are out of date, rerun the pipeline before selecting the final champion pipeline and working with overrides.
• Weighted metrics - These columns display the selection criteria that are available for comparing different pipelines. Not all columns are shown. To view all of the selection criteria, click .
  Note: For projects using the External Forecast pipeline, if the values are missing from the ACTUAL column, the weighted metrics will have missing values in the Model Comparison results. See Working with External Forecast Projects for more information about the composition of external forecast projects.

  For a complete description, see Weighted Model Comparison Selection Criteria.

MAPE Distribution

For each node in the pipeline, this is a graph showing the MAPE distribution of results.

Execution Summary

For each node in the pipeline, the following summary statistics are provided.

Number of series

the total number of time seriesan aggregation of transactional data into specified time intervals and sorted according to unique combinations of the default attributes (BY variables) accumulated from the data. For a Hierarchical Forecasting modeling node, this corresponds to the number of time series in each level of the hierarchy.

Number of failures

the number of time series where forecasts failed. For a Hierarchical Forecasting modeling node, this corresponds to the number of time series in each level of the hierarchy where forecasts failed.

Number of forecasts equal to zero

the number of time series with at least one forecasta numerical prediction of a future value for a specified time period for each unique combination of BY variable values equal to 0. Retired time series are excluded from this measure.

Number of intermittent series with flat forecasts

the number intermittent time series that have a constant forecast value over the length of the forecast horizonthe number of intervals into the future, beyond a base date, for which analyses and predictions are made. . Retired series and series with at least one zero forecast are excluded from this measure.

Forecasts for a time series are determined to be flat when the forecast range (maximum - minimum predicted value) is less than 0.00001.

Number of seasonal series with flat forecasts

the number in seasonal time series that have a constant forecast value over the length of the forecast horizon. Retired series and series with at least one zero forecast are excluded from this measure.

Forecasts for a time series are determined to be flat when the forecast range (maximum - minimum predicted value) is less than 0.00001.

Number of short series with flat forecasts

the number of short time series that have a constant forecast value over the length of the forecast horizon. Retired series and series with at least one zero forecast are excluded from this measure.

Forecasts for a time series are determined to be flat when the forecast range (maximum - minimum predicted value) is less than 0.00001.

Number of all other series with flat forecasts

the number of all other time series that have a constant forecast value over the length of the forecast horizon. This number does not include short, intermittent, or seasonal time series with flat forecast values. Also, retired series and series with at least one zero forecast are excluded from this measure.

Forecasts for a time series are determined to be flat when the forecast range (maximum - minimum predicted value) is less than 0.00001.

Number of retired series

the number of time series that have been identified as retired. See Retired for a full description of how time series are identified as retired.

Number of forecasts below 70%

the number of time series with forecast values that are below the minimum range of the mean value for that time series. The forecasts for these series can be problematic. The minimum range value is set to 70% by default. If you change this setting using the Forecast exception boundaries in the Project Settings, this number reflects that change.

Number of forecasts above 300%

the number of time series with forecast values that are above the maximum range of the mean value for that time series. The forecasts for these series can be problematic. The maximum range value is set to 300% by default. If you change this setting using the Forecast exception boundaries in the Project Settings, this number reflects that change.

See Also

• Generating Output from SAS Model Studio in SAS Visual Forecasting: Overview

Output Data

The tables on this tab reflect the output generated by the champion modeling node for the pipeline. For a full description of these tables, see Output Data Sets.