SVDD Procedure

This example illustrates the peak criterion method (Kakde et al. 2017) for bandwidth selection. The peak criterion method suggests the values of Gaussian bandwidth s, where the second derivative of the value of the optimal dual objective function (as defined in the section Flexible Data Description) with respect to s first reaches 0. The bandwidth value that is obtained using the peak criterion method provides a data boundary that closely follows the data shape. For certain data sets, the method provides a range of values where this criterion is met. Any value of s within this range provides a good data boundary.

The peak criterion method involves computing the optimal dual objective function value over a range of bandwidth values. These computations can be performed by specifying multiple bandwidth parameter values in the form of BW=n TO m BY p in the KERNEL statement in order to train multiple SVDD models in a single run.

The training data set in this example is star-shaped. This example first computes the bandwidth value by using the peak criterion method. The bandwidth value thus obtained is subsequently used to train an SVDD model. To evaluate the quality of the training results, scoring is performed on a data grid.

The following DATA step creates the star (which is stored in the Work library by default):

	data star;
	infile datalines delimiter=',';
	input x1 x2 ;
	datalines;
	0.07197,6.48334
	0.28467,6.35686
	0.45698,6.35741
	0.57731,6.13851

   ... more lines ...   

The following statements plot variable x1 against variable x2:

 proc sgplot data=star;
 title "Star-shaped Training Data";
 scatter x=x1 y=x2/markerattrs=(size=3 symbol=circlefilled);
 run;

Output 38.5.1 shows the results.

You can load the work.star data set into your CAS session by specifying your CAS engine libref in the following DATA step:

    data mylib.star;
    set star;
    run;

These statements assume that your CAS engine libref is named mylib, as in the section Using CAS Sessions and CAS Engine Librefs, but you can substitute any appropriately defined CAS engine libref.

The following statements execute the SVDD algorithm on the mylib.star data table. The optSummary and modelInfo ODS data sets are stored the in optimization_summary and model_info data sets, respectively.

 ods output optSummary=optimization_summary modelinfo=model_info;
 proc svdd data=mylib.star;
 id x1 x2;
 input x1-x2/level=interval;
 kernel rbf / bw=(0.1 to 5 by 0.1);
 solver actset/stol=1e-10;
 savestate rstore=mylib.scorecat;
 run;

The INPUT statement defines the input variables x1 and x2 as interval variables. The KERNEL statement specifies the kernel function as a radial basis function (RBF), and the BW=(0.1 to 5 by 0.1) option instructs the SVDD procedure to train 50 SVDD models that correspond to bandwidth values from 0.1 to 5. The SAVESTATE statement creates an analytic store for the model and saves it as a binary object in the mylib.scorecat data table.

Output 38.5.2 displays the number of observations in the training data table.

The SVDD procedure is executed with 50 bandwidth values, and it generates the "Model Information," "Training Results," "Optimization Summary," and "Histogram" tables for each bandwidth value. For the sake of brevity, only tables that correspond to one bandwidth value are provided here.

The "Model Information" table in Output 38.5.3 summarizes the key options and the input data variables for the SVDD model that is trained using the option BW=0.1.

The "Training Results" table in Output 38.5.4 shows the number of support vectors and the threshold value for the SVDD model that is trained using the option BW=0.1.

The "Optimization Summary" table in Output 38.5.5 shows whether the solution is optimal, the number of iterations that were required, and the objective function value for the SVDD model that is trained using the option BW=0.1.

The following code uses the optimization_summary and model_info data sets to obtain the optimal objective function and bandwidth values, respectively, for each model. Subsequently, these two data sets are merged to create one data set named peak_input.

 data model_info(keep=model nvalue rename=(nvalue=bw));
 set model_info (where=(rowid="BW"));
 run;
 data optimization_summary(keep=model value rename=(value=obj));
 set optimization_summary (where=(rowid="OBJ"));
 run;
 proc sort data=model_info; by model; run;
 proc sort data=optimization_summary; by model; run;
 data peak_input;
 merge model_info(in=a) optimization_summary(in=b);
 by model;
 if a=b;
 run;

The following code computes the first and second derivative of the optimal objective function value with respect to the bandwidth value s. The sby macro variable specifies the bandwidth step size of 0.1, as used earlier in the SVDD procedure.

 %let sby=0.1;
 data peak_input;
 set peak_input;
 lag_obj=lag(obj);
 lag2_obj=lag2(obj);
 run;

 data peak_input;
 set peak_input;
 if _n_ > 1 then d_obj = (obj - lag_obj)/(&sby);
 if _n_ > 2 then d2_obj = (obj - 2 * lag_obj + lag2_obj) / ( &sby * &sby ) ;
 run;

To decide whether the value of the second derivative is 0, a penalized B-spline is fitted to the second derivative by using the TRANSREG procedure in SAS/STAT software. If the 95% confidence interval of the fitted value of second derivative contains 0, the second derivative value is considered to be approximately 0.

 proc transreg data=peak_input;
 model identity(d2_obj) = pbspline(bw / nknots=20 evenly sbc);
 output out=peak_res predicted cli clm;
 run;

The following code determines the bandwidth value where the second derivative of the optimal objective function value with respect to s first reaches 0:

    data peak_res;
    set peak_res(where=(d2_obj ne .));
    if ((CIUd2_obj ge 0) and (CILd2_obj le 0)) then zero_flg=1;
    else zero_flg=0;
    label d2_obj = "Second Derivative";
    run;

 proc sql noprint;
 select min(bw) into :peak_bw
 from peak_res
 where zero_flg=1;
 quit;

    data peak_res;
    set peak_res;
    if round(bw,0.0001)= &peak_bw then do;
    ss=bw;
    label="Bandwidth="||trim(left(bw));
    end;
    run;

The following code plots the second derivative of the optimal objective function value against bandwidth value s:

 proc template;
 define statgraph bandplot;
 begingraph;
 entrytitle " ";
 layout overlay;
 bandplot x=bw limitupper=CIUd2_obj
 limitlower=CILd2_obj /
 name="band1" modelname="fit"
 legendlabel="95% Confidence Limits";
 scatterplot x=bw y=d2_obj / primary=true;
 seriesplot x=bw y=Pd2_obj / name="fit"
 legendlabel="Fit Line" lineattrs=(thickness=2);;
 referenceline x=ss / curvelabel=label
 lineattrs=(color=gray pattern=dot thickness=1);
 referenceline y=0 /lineattrs=(color=gray pattern=solid );
 discretelegend  "fit" "band1" ;
 endlayout;
 endgraph;
 end;
 run;

 proc sgrender data=peak_res(where=(d2_obj ne .)) template=bandplot;
 run;

Output 38.5.6 shows the results, which indicate that the value of second derivative of the optimal objective function value with respect to s first reaches 0 at s = 0.9. This represents the bandwidth value that is obtained using the peak criterion method.

The following statements execute the SVDD algorithm on the mylib.star data table by using the bandwidth value (0.9) that was obtained using the peak criterion method.

 proc svdd data=mylib.star;
 id x1 x2;
 input x1 x2/level=interval;
 kernel rbf / bw=0.9;
 savestate rstore=mylib.state;
 run;

To evaluate the quality of the training results, scoring is performed on a data grid. The following DATA step creates a data set named scoreds to contain the scoring observations:

    data scoreds;
    do x1=0 to 10 by 0.05;
       do  x2=0 to 10 by 0.05;
          output;
       end;
    end;
    run;

You can load the scoreds data set into your CAS session by specifying your CAS engine libref in the following DATA step:

    data mylib.scoreds;
    set scoreds;
    run;

The following code scores the mylib.scoreds data by using the score analytic store that was created during training:

 proc astore;
 score data=mylib.scoreds
 out=mylib.score_results
 rstore=mylib.state;
 quit;

The following code copies the score_results into the Work library:

    data work.score_results;
    set mylib.score_results;
    run;

The following code plots the scoring results:

 proc sgplot data=score_results(where=(_svddscore_ in (-1,1)));
 styleattrs datacontrastcolors=(ligr black) datasymbols=(circlefilled);
 title "Peak Criterion: Scoring Results";
 scatter x=x1 y=x2/group=_svddscore_ markerattrs=(size=3);
 run;

Output 38.5.7 shows the scatter plot of the scoring data set. The gray area indicates the observations that are identified as outliers, and the black area indicates observations that are identified as inliers. Comparison between Output 38.5.7 and the training data in Output 38.5.1 indicates that the bandwidth that was obtained using the peak criterion method provides an accurate description of the training data.

As outlined in this example, determining the bandwidth that is suggested by the peak criterion method requires that the SVDD solution be computed multiple times for the training data for a list of bandwidth values that lie on a grid. The parameters of the grid such as the starting value, ending value, and increment can affect the quality of results. Also, it is necessary to set the solver tolerance, STOL, to a very low value (such as the value 1E–10 that is used in this example).