Automatic Time Series Modeling Package
OUTLIER Details
Specifying ARIMA Models
When you specify the 'ARIMA' method for an OUTLIER instance, you can use two methods to add ARIMA models to the instance:
Add predefined models: You can use the SetArima('MODEL', Value) method to add predefined models. Value is a string array in which each element of the array can contain one of the model names from Table 64. For each model name, you can also use the name from the Alias1 or Alias2 column. For example, the following code adds the AIRLINE, ARIMA011100NOINT, and ARIMA200100 models to the OUTLIER instance named "outlier":
array models[3] $ 20 / nosymbols;
models[1] = 'AIRLINE';
models[2] = '_PSD1Q1';
models[3] = '_A5';
declare object outlier(OUTLIER);
outlier.SetOption('METHOD','ARIMA');
outlier.SetArima('MODEL', models);
Add a custom ARIMA model: You can define your own ARIMA model structure by using the ARIMASPEC object in the Time Series Models (TSM) package and add it by using the AddFrom() method. For example, the following code adds a custom ARIMA model:
array ar[2] / nosymbols;
ar[1] = 2;
ar[2] = 24;
array ma[1] / nosymbols;
ma[1] = 1;
array diff[2] / nosymbols;
diff[1] = 1;
diff[2] = 12;
declare object myARIMA(arimaspec);
rc = myARIMA.Open();
rc = myARIMA.AddARPoly(ar);
rc = myARIMA.AddMAPoly(ma);
rc = myARIMA.AddMAPoly(ma,,1);
rc = myARIMA.SetDiff(diff);
rc = myARIMA.SetOption('noint', 1);
rc = myARIMA.Close();
...
declare object outlier(OUTLIER);
...
rc = outlier.addFrom(myARIMA);
Table 64 shows the lists of the predefined ARIMA models.
Table 64: Contents of the Predefined ARIMA Models
| Name | Alias1 | Alias2 | Description |
|---|---|---|---|
| AIRLINE | _D1SQ1S | _A0 | ARIMA: Y D = (1,s) Q = ((1)(1)s) NOINT |
| ARIMA000011NOINT | _DSQS | _A1 | ARIMA: Y D = (s) Q = (1)s NOINT |
| ARIMA011100NOINT | _PSD1Q1 | _A2 | ARIMA: Y P = (1)s D = (1) Q = 1 NOINT |
| ARIMA012011NOINT | _D1SQ12S | _A3 | ARIMA: Y D = (1,s) Q = ((1,2)(1)s) NOINT |
| ARIMA200100 | _P12SINT | _A5 | ARIMA: Y P = ((1,2)(1)s) |
| ARIMA210011NOINT | _P12D1SQ12S | _A6 | ARIMA: Y P = 2 D = (1,s) Q = (1)s NOINT |
| ARIMA212011NOINT | _P12D1SQ12S | _A7 | ARIMA: Y P = 2 D = (1,s) Q = ((1,2)(1)s) NOINT |
'USEBEST' Option for the ARIMA Method
rc=obj.SetArima ('USEBEST',Value);
When you specify multiple ARIMA candidate models for the OUTLIER object, the OUTLIER object fits all those models. You can use the 'USEBEST' option to control which of the successful candidates the OUTLIER object uses to detect and report outliers. The following list describes the behavior for each possible 'USEBEST' setting:
- 0
The OUTLIER object does not select any best model in this case. It detects and reports outliers for all candidates that fit successfully.
- 1
The OUTLIER object selects the best ARIMA model from all successful candidates according to the metric that you specify in the 'CRITERION' option. Then it detects outliers only for the best ARIMA model and reports those outliers.
- 2
For each successful candidate j, the OUTLIER object records the base value of the criterion that you specify in the 'CRITERION' option. Let this base value be denoted by
. Then the OUTLIER object computes
criterion values, where
is the number of outliers detected for candidate j. Each of those
criterion values corresponds to the criterion for an ARIMA model that adds the kth outlier (
) to candidate j. If this model with one regressor succeeds, then let the criterion for that model be denoted by
. If the jth model with the kth outlier fails, then the kth outlier is ignored. Let
denote the number of models that include an outlier as a regressor that succeeds. The final metric for candidate j is computed as
. The metric
measures the average benefit of adding an outlier to candidate j. The OUTLIER object chooses the candidate that has the best
value as the best model and reports outliers only for that model.
Hampel Method of Outlier Detection
The Hampel identifier, also known as the Hampel filter, determines whether an observation is an outlier by comparing its absolute deviation from the median to a configurable multiple of standard deviation, which is computed by using the median absolute deviation (MAD).
MAD is defined as the median of the absolute deviations from the data’s median. Formally, for a data set that contains the observations , let m denote the median. Then, MAD is defined as
The Hampel filter uses a sliding window of configurable size to go over the input data to calculate the MAD for each window. You can set the size as follows:
rc=obj.SetHampel('WIDTH', Value);
Each observation has a window centered on it. The window size is 2 * Value + 1.
The Hampel-based outlier detection method calculates the MAD and uses it as follows to compute a consistent estimate of the standard deviation :
where is a constant scale factor, which depends on the distribution. Five distributions are supported; you can specify which one to use by setting the 'DISTRIBUTION' option in the SetHampel method:
rc=obj.SetHampel('DISTRIBUTION', Value);
The distributions and corresponding scale factors k for each possible Value are shown in Table 65.
Table 65: Hampel Filter Distributions and Scale Factors
| 'DISTRIBUTION' Value | Description | Scale Factor k |
|---|---|---|
| EXP | Exponential distribution | 2.08 |
| LAPLACE | Laplace distribution | 2.04 |
| LOGISTIC | Logistic distribution | 1.65 |
| NORM | Normal distribution | 1.48 |
| UNIFORM | Uniform distribution | 1.16 |
The Hampel filter declares an observation to be an outlier if that observation differs from the window median by more than a configurable number of standard deviations. Let w be the width of the window that is centered around observation . For each observation
, the main steps of the outlier detection algorithm are as follows:
Calculate the absolute deviation
for all the window observations
.
-
Calculate the threshold
to use for detecting outliers. If you specify the 'THRESHOLD' option in the SetHampel method as
rc=obj.SetHampel('THRESHOLD', Value);
If you omit the 'THRESHOLD' option, then the threshold depends on the 'DISTRIBUTION' and 'NUMSTD' options that you specify in the SetHampel() method:
rc=obj.SetHampel('NUMSTD', Value1);
rc=obj.SetHampel('DISTRIBUTION', Value2);
The threshold
, where
is equal to Value1 or the default value of the 'NUMSTD' option, and k is the scale factor that is associated with distribution Value2, as shown in Table 65.
An observation
is an outlier if its absolute deviation from the window median
is more than
standard deviations. The estimate of the standard deviation for a window of observations is
. So an observation is an outlier if
, which is equivalent to
. By setting the threshold
, the outlier determination becomes equivalent to
.
-
If the considered observation differs from the window median by greater than or equal to the threshold
multiplied by the MAD estimate, it is detected as an outlier as follows:
-
Compute the standardized distance,
, for each detected outlier observation
as