Time Series Model Package

This example illustrates how to use the RNNSPEC object to fit a recurrent neural network forecasting model to the airline passenger data (see Example 32.1 for more information about the airline series).

The following statements perform forecasting by using the LSTM recurrent neural network on the example data set. Specifying the value 12 for the 'NINPUT' argument means that the length of the input subsequence is 12. The value, 12, of the 'NVALIDATION' argument corresponds to the range of the validation period. The STD normalization is imposed. SetY specifies the AIR variable as the target variable. The number of layers is 4, which means that the neural network consists of one input layer, two hidden layers, and one output layer. The number of neurons in each hidden node is 30. When you specify 'ADAM' in the 'ALGORITHM' optimizer option, the stochastic gradient descent method uses the Adam optimization algorithm.

     data mylib.airtrain;
             set sashelp.air;
     run;

     proc tsmodel data=mylib.airtrain nosummary  outlog=mylib.outlog
                     outobj=( outfor = mylib.outfor(replace=YES)
                             outstat = mylib.outstat(replace=YES)
                             outrnnstat = mylib.outrnnstat(replace=YES)
                             outopt = mylib.outopt(replace=YES)
                             );
             var air;
             id date interval=month;
             require tsm;
             submit;
                     declare object tsmmodel(tsm);
                     declare object rnnspec(rnnspec);
                     declare object outrnnstat(tsmrnnstat);
                                     declare object outstat(tsmstat);
                     declare object outopt(tsmoutopt);
                     declare object outfor(tsmfor);
                     rc = rnnSpec.Open( );
                     rc = rnnSpec.SetOption(
                                     'RNNTYPE','LSTM',
                                     'NINPUT', 12,
                                     'NVALIDATION', 12,
                                     'NLAYER', 4,
                                     'NNEURONH',30,
                                     'NORMALIZE', 'STD',
                                     'POSTTRAIN', 'YES',
                                     'SEED', 12345
                                     );
                     rc = rnnSpec.SetOptimizer(
                                     'ALGORITHM','ADAM',
                                     'LEARNINGRATE', 0.1,
                                     'BETA1', 0.8,
                                     'BETA2', 0.9,
                                     'LEARNINGPOLICY', 'STEP',
                                     'STEPSIZE', 5,
                                     'GAMMA', 0.5,
                                     'STAGNATION', 20,
                                     'MINIBATCHSIZE',8,
                                     'WARMUPEPOCHS',15,
                                     'MAXEPOCHS', 100
                                     );
                     rc = rnnspec.Close();
                     rc = tsmmodel.Initialize(rnnspec);
                     rc = tsmmodel.SetY(air);
                     rc = tsmmodel.SetOption('lead', 12);
                     rc = tsmmodel.SetOption('alpha', 0.05);
                     rc = tsmmodel.Run( );if rc < 0 then stop;
                     rc = outfor.Collect(tsmmodel);if rc < 0 then stop;
                     rc = outstat.Collect(tsmmodel);if rc < 0 then stop;
                     rc = outrnnstat.Collect(tsmmodel);if rc < 0 then stop;
                     rc = outopt.Collect(tsmmodel);if rc < 0 then stop;
             endsubmit;
     print outlog;
     run;

data null;
   set mylib.outrnnstat;
   call symputx('optepoch', OPTEPOCH);
run;

proc sgplot data=mylib.outopt;
      series x=epoch y=trnerror/ LegendLabel="Train Error"
            lineattrs=(color=green) markers
            markerattrs=(color=green symbol=diamond size=5);
      series x=epoch y=vlderror/ LegendLabel="Validation Error"
            lineattrs=(color=red) markers
            markerattrs=(color=red size=5);
      refline  &optepoch/ axis=x ;
      yaxis label="Error";
run;

Figure 15 shows the training and validation errors of the optimization history. The vertical reference line shows where the SGD optimum occurs.

Figure 15: Training and Validation Errors at Each Epoch

The following statements print the output table that is produced by the TSMFOR collect method:

Output 32.6.1 shows part of the forecast value table of the target series, air, that is produced by the LSTM recurrent neural network. The forecast errors and confidence intervals are also shown in the table.

The following statements generate the plot of the actual values and forecast values:

proc sgplot data=mylib.outfor;
   series x=date y=actual /
          legendlabel="Actual Value" name="actual"
          lineattrs=(color=green) markers
          markerattrs=(color=green size=4);
   series x=date y=predict /
          LegendLabel="Forecast Value" name="forecast"
          lineattrs=(color=red) markers
          markerattrs=(color=red size=4 symbol=triangle);
   xaxis values=('01jan49'd to '01dec61'd by month);
   refline  '01dec59'd/ axis=x;
   refline  '01dec60'd/ axis=x;
   keylegend "actual" "forecast";
   yaxis label="Airline Passenger";
run;

Figure 16 shows the plot of forecasts and their actual values in the training and validation periods. It also shows forecasts of the 12 months after the validation period. The first reference line splits the training and validation periods, and the second reference line indicates the end of the validation period.

Figure 16: Forecast Plot of Airline Passenger Data