Imported Project Does Not Show Out-of-Sample Region

Description

An out-of-sample regionthe number of time periods before the end of the data that are removed when fitting models. After model selection, forecasts are generated in the out-of-sample region and then compared to the actual data to determine accuracy. is specified for a project when Number of periods to exclude from modeling is set to a positive number in Project Settings. When an out-of-sample is specified, you can expect to see the following information when you open the results for a completed modeling node.

• On the Summary tab, you can select to view the MAPE distribution for out-of-sample regions of the project data.
• On the Output tab, the OUTSTAT table shows different model selection statistics for FIT (in-sample) and FORECAST (out-of-sample) regions.

Also, in the Forecast Viewer, the Statistics of Fit window includes the out-of-sample measurement.

For projects that are imported from earlier releases of SAS Visual Forecasting, the out-of-sample measurements might be missing when the out-of-sample region is specified. The MAPE distribution is blank when the out-of-sample region is selected. The FORECAST regions in the OUTSTAT table are not shown.

This issue normally occurs when the project that was imported is using one or more pluggable modeling nodes. Pluggable modeling nodes enable you to open and change the code, customizing it to your own needs. Modeling nodes from previous releases either did not support out-of-sample regions or were not designed for certain boundary cases of the out-of-sample analysis. When the project is imported or upgraded from a previous release, the original modeling node is maintained with no out-of-sample regions or an older method of out-of-sample analysis. This preserves any customizations that were made to the modeling node.

Modeling Nodes Provided by SAS Visual Forecasting explains which modeling nodes are pluggable.

Solution

Add new modeling nodes to your pipelines to replace the existing ones. For example, if your project includes Auto-forecasting, add an Auto-forecasting node to the pipeline from the Nodes panel. The pluggable node that you add includes support for out-of-sample regions. Before you remove the original modeling node, make sure you update the new modeling node to include any customizations that were made to the original.