ASSESSBIAS Procedure

Note: Input data must be in a CAS table that is accessible in your CAS session. You must refer to this table by using a two-level name. The first level must be a CAS engine libref, and the second level must be the table name. For more information, see the sections Using CAS Sessions and CAS Engine Librefs and Loading a SAS Data Set onto a CAS Server in Chapter 2, Shared Concepts.

This example demonstrates how to use the ASSESSBIAS procedure to measure bias and compute the difference of various statistical metrics based on a scored data set and a sensitive variable.

The following SAS DATA step creates the SAS data table mylib.score, which simulates a credit score data set that is trained by a supervised model such as a decision tree or neural network model, and loads the table into your CAS session:

data mylib.score;
   length credit $4.;
   input ID $ credit $ P_creditgood P_creditbad;
   datalines;
A good 0.83 0.17
A bad  0.65 0.35
A good 0.76 0.24
A good 0.36 0.64
A bad  0.17 0.83
B good 0.55 0.45
B bad  0.22 0.78
B good 0.63 0.37
B good 0.81 0.19
;

These statements assume that your CAS engine libref is named mylib, but you can substitute any appropriately defined CAS engine libref.

In this data table, the variable credit represents the ground truth label with two values, "good" and "bad". The variables P_creditgood and P_creditbad are the predicted probabilities for the target outcomes "good" and "bad", respectively. The "good" event is considered to be the favored condition in the bias measurement. The variable ID is the sensitive variable, with the attribute values A and B.

The following statements call PROC ASSESSBIAS to produce the reports:

proc assessbias data=mylib.score nbins=2;
   var P_creditgood;
   target credit / event="good" level=nominal;
   sensitiveVar ID;
   fitstat pvar=P_creditbad / pevent="bad";
run;

The VAR statement specifies that the variable P_creditgood should be analyzed and that the associated event "good" is the event of interest in the bias metrics. The TARGET statement specifies that the variable credit is the target variable for the classification model. The FITSTAT statement lists the other predicted variable, P_creditbad, and the associated event. The SENSITIVEVAR statement specifies that the variable ID is the variable that the population grouping is based on.

Figure 1 shows the values of demographic parity, predicted parity, equal accuracy, equalized odds, and equal opportunity for two population groups in the sensitive variable ID.

Figure 2 shows the difference of a set of statistical metrics that are related to receiver operation characteristic (ROC) curve analysis, lift analysis, and fit statistics analysis for two population groups in the sensitive variable ID.