CSSM Procedure

This example shows how you can use the scoring functionality to monitor a continuous stream of observations. The goal is to detect structural changes in the observation process as early as possible. For illustration purposes, the example uses the quarterly bivariate series (f_KSI, r_KSI), which is discussed in Example 14.1: Bivariate Basic Structural Model . The variable f_KSI represents the quarterly average of the log of the monthly totals of front-seat passengers who are killed or seriously injured in car accidents, and r_KSI represents a similar number for rear-seat passengers. Recall that the seatbelt data set contains 16 years of series values, starting from the first quarter of 1969. This example uses the first 13 years—52 quarters—of data (through the last quarter of 1981) to fit a suitable model for the series and to create a score store that saves the context for later use. It is assumed that the fitted model continues to hold in the subsequent years. Monitoring of the process starts with the first quarter of 1982. The remaining 12 rows of the seatbelt data set are processed one by one and mimic a quarterly streaming data sequence. In real-world applications, the data source can be more general, and the observations could arrive at a much higher frequency, such as every minute.

Assume that the model for = (f_KSI, r_KSI) is

where

This model is the same as the one used in Example 14.1: Bivariate Basic Structural Model , except that in this case no correction is made for the seat-belt law of 1983. In fact, the goal of this exercise is to determine, without the knowledge of this law, whether the level shift in the front-seat passenger series that is caused by this law can be detected during this monitoring.

The following DATA step loads the first 13 years of data into the table mylib.train:

 data mylib.train;
     set seatBelt(obs=52);
 run;

The following statements specify and fit the model, and save the scoring context in the store mylib.inscore:

 proc cssm data=mylib.train opt(tech=activeset);
    id date interval=quarter;
    state error(2) type=WN cov(g);
    component wn1 = error[1];
    component wn2 = error[2];
    state level(2) type=RW cov(rank=1) checkbreak;
    component rw1 = level[1];
    component rw2 = level[2];
    state season(2) type=season(length=4);
    component s1 = season[1];
    component s2 = season[2];
    model f_KSI = rw1 s1  wn1;
    model r_KSI = rw2 s2 wn2;
    output break(alpha=0.01) ao(alpha=0.01);
    score out(10)=mylib.inscore;
 run;

Note that because the OUT(k)= form of the OUT option is used in the SCORE statement (score out(10)=mylib.inscore;), the mylib.inscore store retains the context information from the last 10 quarters (from 1979/3 to 1981/4). The score store also retains the structural break detection settings that are specified in the STATE statement for level, and it retains the significance levels for the additive outlier detection and structural break detection that are specified in the OUTPUT statement.

After the model fitting and the initial score-store creation, the monitoring of streaming data is done using the following steps:

The following macro, %STREAM, performs these three steps:

 %macro STREAM;
   %do i=53 %to 64;
     /* read the next row */
     data mylib.nextObs;
        set seatBelt(firstobs=&i obs=&i);
     run;
     /* score the row that is read */
     proc cssm data=mylib.nextObs;
        ods output AOSummary=ao&i;
        ods output StateElementBreakSummary=ls&i;
        %if &i = 53 %then %do;
           ods output IdInformation=idInfo;
           ods output ResponseInfo=respInfo;
        %end;
        score in=mylib.inscore out=mylib.outscore;
     run;
     /* swap inscore and outscore */
    proc cas;
        table.altertable/name="outscore", rename="tmp";
        table.altertable/name="inscore", rename="outscore";
        table.altertable/name="tmp", rename="inscore";
     quit;
   %end;
 %mend;

 %STREAM;

For each new quarter, the %STREAM macro saves the results, if any, of the additive outlier detection and the structural break detection in SAS data sets (ods output AOSummary=ao&i; and ods output StateElementBreakSummary=ls&i;). For the first quarter, the ID information table and the response information table are also saved. The ID information table for the first quarter (1982/1) is shown in Output 14.20.1. Note that even though the input table (mylib.nextObs) has just one observation (which corresponds to 1982/1), PROC CSSM processed 11 observations (from 1979/3 to 1982/1). This is because, internally, PROC CSSM works with a data set that is created by combining the information from the previous 10 quarters (1979/3 to 1981/4) that is present in the input score store and the one quarter (1982/1) that is present in the current input data table (mylib.nextObs).

For the same reason, Output 14.20.2 also shows the response variable information for 11 quarters.

The outlier detection and the structural break detection, which are performed after the arrival of each new quarterly observation, are also performed on the set of 11 observations that is formed by the current observation and the observations in the 10 previous quarters. It happens that no additive outliers or structural breaks are detected in the four quarters of 1982. However, starting with the first quarter of 1983 (when the seat-belt law is introduced), additive outliers and structural breaks are detected in each quarter. The aggregate summary of this detection activity (obtained by combining the following pairs of data sets: ao57 with ao64 and ls57 with ls64) is shown in two tables: Output 14.20.3 shows the results of break detection for the bivariate state, level, and Output 14.20.4 shows the results of the additive outlier detection. In both tables, an extra column, obs_date, is added to indicate the quarter that is associated with the observation that was scored. In addition, two more columns, zValue (= Estimate/StdErr) and ProbZ (the corresponding p-value), are added to the additive outlier table for ease of comparison with the break detection table. An examination of the two tables shows the following:

In real-world applications, when sufficient evidence for a break is gathered, the monitoring is paused and some appropriate action is taken.