Fair AI Tools Action Set
Mitigating Bias of a Support Vector Machine
This section contains PROC CAS code.
Note: Input data must be accessible in your CAS session, either as a CAS table or as a transient-scope table. A CAS table has a two-level name: the first level is your CAS engine libref, and the second level is the table name. You refer to this table in the CAS procedure by specifying only the second level. For more information about two-level names, see Chapter 2, Shared Concepts (SAS Viya: Machine Learning Procedures). A transient-scope table is called directly from the action and exists in memory for the duration of the action. For more information about accessing data, see SAS Viya: System Programming Guide. For more information about PROC CAS and programming in CASL, see SAS Cloud Analytic Services: CASL Programmer’s Guide and SAS Cloud Analytic Services: CASL Reference.
This example trains a support vector machine in order to reduce bias according to the equal opportunity metric by using the home equity data, which are available in the sampsio.hmeq data set. The data set has a binary target named BAD, and the example uses the variable REASON as the sensitive attribute.
You can load the sampsio.hmeq data set into your CAS session by specifying your CAS engine libref in the SET statement in the following DATA step:
data mycas.hmeq;
set sampsio.hmeq;
where reason ne '';
run;
These statements assume that your CAS engine libref is named mycas, but you can substitute any appropriately defined CAS engine libref.
The following statements use the mitigateBiasSVM action to run the exponentiated gradient reduction (EGR) algorithm in order to train a model that has reduced equal opportunity bias, which is calculated on the sensitive attribute REASON. Note that the syntax of the trainOptions parameter here is the same as the syntax of the svm.svmTrain action.
proc cas;
fairAITools.mitigateBiasSVM /
mitigateOptions={biasMetric="EQUALOPPORTUNITY",
bound="75",
event="1",
predictedVariables={"P_BAD1", "P_BAD0"},
responseLevels={"1", "0"},
seed="12345",
sensitiveVariable="REASON"},
trainOptions={c="0.1",
inputs={"CLAGE",
"CLNO",
"DEBTINC",
"LOAN",
"MORTDUE",
"VALUE",
"YOJ",
"DELINQ",
"DEROG",
"JOB",
"NINQ"},
nominals={"DELINQ",
"DEROG",
"JOB",
"NINQ",
"BAD"},
saveState={name="hmeq_svm_astore",
replace="True"},
seed="12345",
table="hmeq",
target="BAD"};
run;
The mitigateOptions parameter includes the following subparameters:
The
biasMetricsubparameter specifies that equal opportunity be used as the bias measurement.The
boundsubparameter specifies 75 as the bound value for the EGR algorithm.The
eventsubparameter specifies the formatted value of the response variable that represents the event of interest.The
predictedVariablessubparameter specifies that the variablesP_BAD0andP_BAD1be used as the model predictions. A predicted variable is given for each level of the target variable.The
responseLevelssubparameter specifies the levels of the target that correspond to each variable that is listed in thepredictedVariablessubparameter. The order of the response levels must match the order of the predicted variables.The
seedsubparameter specifies the seed to use for pseudorandom number generation in the EGR algorithm.The
sensitiveVariablesubparameter specifies that the variableREASONbe used as the sensitive variable in assessing and mitigating bias.
The trainOptions parameter includes the following subparameters that are passed to the svm.svmTrain action call:
The
csubparameter specifies 0.1 as the penalty for training.The
inputssubparameter specifies that the variablesCLAGE,CLNO,DEBTINC,LOAN,MORTDUE,VALUE,YOJ,DELINQ,DEROG,JOB, andNINQbe used as inputs.The
nominalssubparameter specifies that the variablesDELINQ,DEROG,JOB,NINQ, andBADbe used as nominal variables.The
saveStatesubparameter specifies the table to store the generated analytic store model.The
seedsubparameter specifies the seed to use for pseudorandom number generation in the training process.The
tablesubparameter names the input data table for training.The
targetsubparameter specifies that the variableBADbe used as the target variable.
The "Mitigation Information" table that these statements produce is shown in Output 15.16.1. The table shows some key information about the model and values of the key hyperparameters that the EGR algorithm uses.
The "Iteration History" table that these statements produce is shown in Output 15.16.2. The table displays the iteration history of the overall equal opportunity metric, the overall misclassification rate, and the overall multiclass log loss. From the table, you can see that without bias mitigation, the equal opportunity gap between those two sensitive attributes is about 2.4%, and after eight iterations of the EGR algorithm it has been reduced to about 0.5%. Meanwhile, the overall accuracy of the model does not change significantly.
The "Group Metrics" table that these statements produce is shown in Output 15.16.3. The table displays the model assessment statistics and average prediction for each sensitive attribute during each iteration of the EGR algorithm.
Output 15.16.1: Mitigation Information
| Mitigation Information | ||
|---|---|---|
| Description | Character Value | Value |
| Bias Metric | EQUALOPPORTUNITY | . |
| Sensitive Variable | REASON | . |
| Response | BAD | . |
| Bound Value | 75 | 75 |
| Learning Rate | 0.01 | 0.01 |
| Maximum Iteration | 10 | 10 |
| Tolerance | 0.005 | 0.005 |
| Event Level | 1 | . |
Output 15.16.2: Iteration History
| EGR Iteration History | |||
|---|---|---|---|
| Iteration | Equal Opportunity | Misclassification Rate | Multi-Class Log Loss |
| 1 | 0.024344 | 0.1904 | 0.9197 |
| 2 | 0.024344 | 0.1904 | 0.9136 |
| 3 | 0.024344 | 0.1904 | 0.9457 |
| 4 | 0.024344 | 0.1904 | 0.9358 |
| 5 | 0.021660 | 0.1908 | 0.9292 |
| 6 | 0.017633 | 0.1913 | 0.9279 |
| 7 | 0.007759 | 0.1941 | 0.9363 |
| 8 | 0.005074 | 0.1945 | 0.9101 |
Output 15.16.3: Group Metrics History
| Group Metrics | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Iteration | REASON | Number of Observations | True Positive Rate | False Positive Rate | Area Under ROC | Misclassification Rate | Multi-Class Log Loss | Best Kolmogorov-Smirnov Along ROC | Cumulative Lift | Cumulative Response | Proportion Into Event Level | Average Prediction for Event |
| 1 | DebtCon | 3928 | 0.0698 | 0.0041 | 0.5439 | 0.1797 | 0.8789 | 0.0854 | 1.4362 | 14.3624 | 0.0165 | 0.0665 |
| HomeImp | 1780 | 0.0455 | 0.0022 | 0.5514 | 0.2140 | 1.0096 | 0.1146 | 1.7929 | 17.9293 | 0.0118 | 0.0603 | |
| 2 | DebtCon | 3928 | 0.0698 | 0.0041 | 0.5439 | 0.1797 | 0.8732 | 0.0854 | 1.4362 | 14.3624 | 0.0165 | 0.0666 |
| HomeImp | 1780 | 0.0455 | 0.0022 | 0.5514 | 0.2140 | 1.0028 | 0.1146 | 1.7929 | 17.9293 | 0.0118 | 0.0604 | |
| 3 | DebtCon | 3928 | 0.0698 | 0.0041 | 0.5440 | 0.1797 | 0.9035 | 0.0841 | 1.4362 | 14.3624 | 0.0165 | 0.0657 |
| HomeImp | 1780 | 0.0455 | 0.0022 | 0.5514 | 0.2140 | 1.0388 | 0.1121 | 1.7929 | 17.9293 | 0.0118 | 0.0595 | |
| 4 | DebtCon | 3928 | 0.0698 | 0.0041 | 0.5441 | 0.1797 | 0.8941 | 0.0841 | 1.4362 | 14.3624 | 0.0165 | 0.0659 |
| HomeImp | 1780 | 0.0455 | 0.0022 | 0.5514 | 0.2140 | 1.0276 | 0.1146 | 1.7929 | 17.9293 | 0.0118 | 0.0597 | |
| 5 | DebtCon | 3928 | 0.0671 | 0.0041 | 0.5440 | 0.1802 | 0.8879 | 0.0841 | 1.4362 | 14.3624 | 0.0160 | 0.0660 |
| HomeImp | 1780 | 0.0455 | 0.0022 | 0.5514 | 0.2140 | 1.0202 | 0.1146 | 1.7929 | 17.9293 | 0.0118 | 0.0598 | |
| 6 | DebtCon | 3928 | 0.0631 | 0.0041 | 0.5438 | 0.1810 | 0.8867 | 0.0841 | 1.4362 | 14.3624 | 0.0153 | 0.0660 |
| HomeImp | 1780 | 0.0455 | 0.0022 | 0.5514 | 0.2140 | 1.0187 | 0.1146 | 1.7929 | 17.9293 | 0.0118 | 0.0598 | |
| 7 | DebtCon | 3928 | 0.0456 | 0.0041 | 0.5431 | 0.1843 | 0.8947 | 0.0849 | 1.4362 | 14.3624 | 0.0120 | 0.0657 |
| HomeImp | 1780 | 0.0379 | 0.0022 | 0.5512 | 0.2157 | 1.0282 | 0.1189 | 1.7929 | 17.9293 | 0.0101 | 0.0595 | |
| 8 | DebtCon | 3928 | 0.0430 | 0.0041 | 0.5429 | 0.1848 | 0.8700 | 0.0841 | 1.4362 | 14.3624 | 0.0115 | 0.0664 |
| HomeImp | 1780 | 0.0379 | 0.0022 | 0.5512 | 0.2157 | 0.9988 | 0.1146 | 1.7929 | 17.9293 | 0.0101 | 0.0602 | |
Mitigating Bias of a Support Vector Machine
This section contains Lua code for the analysis in the CASL version of this example, which contains details about the results.
For more information about coding in Lua, see Getting Started with SAS Viya for Lua and SAS Viya: System Programming Guide.
This example illustrates the mitigateBiasSVM action by using the home equity data that are described in the CASL version of this example.
The following statements use the mitigateBiasSVM action to run the exponentiated gradient reduction (EGR) algorithm in order to train a model that has reduced equal opportunity bias, which is calculated on the sensitive attribute REASON. Note that the syntax of the trainOptions parameter here is the same as the syntax of the svm.svmTrain action.
mitigateOptions={biasMetric="EQUALOPPORTUNITY",
bound="75",
event="1",
predictedVariables={"P_BAD1", "P_BAD0"},
responseLevels={"1", "0"},
seed=12345,
sensitiveVariable="REASON"},
trainOptions={inputs={"CLAGE",
"CLNO",
"DEBTINC",
"LOAN",
"MORTDUE",
"VALUE",
"YOJ",
"DELINQ",
"DEROG",
"JOB",
"NINQ"},
nominals={"DELINQ",
"DEROG",
"JOB",
"NINQ",
"BAD"},
saveState={
name="hmeq_svm_astore",
replace=True},
seed=12345,
table="hmeq",
target="BAD",
c=0.1
}
The mitigateOptions parameter includes the following subparameters:
The
biasMetricsubparameter specifies that equal opportunity be used as the bias measurement.The
boundsubparameter specifies 75 as the bound value for the EGR algorithm.The
eventsubparameter specifies the formatted value of the response variable that represents the event of interest.The
predictedVariablessubparameter specifies that the variablesP_BAD0andP_BAD1be used as the model predictions. A predicted variable is given for each level of the target variable.The
responseLevelssubparameter specifies the levels of the target that correspond to each variable that is listed in thepredictedVariablessubparameter. The order of the response levels must match the order of the predicted variables.The
seedsubparameter specifies the seed to use for pseudorandom number generation in the EGR algorithm.The
sensitiveVariablesubparameter specifies that the variableREASONbe used as the sensitive variable in assessing and mitigating bias.
The trainOptions parameter includes the following subparameters that are passed to the svm.svmTrain action call:
The
csubparameter specifies 0.1 as the penalty for training.The
inputssubparameter specifies that the variablesCLAGE,CLNO,DEBTINC,LOAN,MORTDUE,VALUE,YOJ,DELINQ,DEROG,JOB, andNINQbe used as inputs.The
nominalssubparameter specifies that the variablesDELINQ,DEROG,JOB,NINQ, andBADbe used as nominal variables.The
saveStatesubparameter specifies the table to store the generated analytic store model.The
seedsubparameter specifies the seed to use for pseudorandom number generation in the training process.The
tablesubparameter names the input data table for training.The
targetsubparameter specifies that the variableBADbe used as the target variable.
Mitigating Bias of a Support Vector Machine
This section contains Python code for the analysis in the CASL version of this example, which contains details about the results.
For more information about coding in Python, see Getting Started with SAS Viya for Python and SAS Viya: System Programming Guide.
This example illustrates the mitigateBiasSVM action by using the home equity data that are described in the CASL version of this example.
The following statements use the mitigateBiasSVM action to run the exponentiated gradient reduction (EGR) algorithm in order to train a model that has reduced equal opportunity bias, which is calculated on the sensitive attribute REASON. Note that the syntax of the trainOptions parameter here is the same as the syntax of the svm.svmTrain action.
s.fairaitools.mitigatebiassvm(
mitigateOptions=dict(biasMetric='EQUALOPPORTUNITY',
bound='75',
event='1',
predictedVariables=['P_BAD1', 'P_BAD0'],
responseLevels=['1', '0'],
seed='12345',
sensitiveVariable='REASON'),
trainOptions=dict(c='0.1',
inputs=['CLAGE',
'CLNO',
'DEBTINC',
'LOAN',
'MORTDUE',
'VALUE',
'YOJ',
'DELINQ',
'DEROG',
'JOB',
'NINQ'],
nominals=['DELINQ',
'DEROG',
'JOB',
'NINQ',
'BAD'],
saveState=dict(name='hmeq_svm_astore',
replace='True'),
seed='12345',
table='hmeq',
target='BAD'))
The mitigateOptions parameter includes the following subparameters:
The
biasMetricsubparameter specifies that equal opportunity be used as the bias measurement.The
boundsubparameter specifies 75 as the bound value for the EGR algorithm.The
eventsubparameter specifies the formatted value of the response variable that represents the event of interest.The
predictedVariablessubparameter specifies that the variablesP_BAD0andP_BAD1be used as the model predictions. A predicted variable is given for each level of the target variable.The
responseLevelssubparameter specifies the levels of the target that correspond to each variable that is listed in thepredictedVariablessubparameter. The order of the response levels must match the order of the predicted variables.The
seedsubparameter specifies the seed to use for pseudorandom number generation in the EGR algorithm.The
sensitiveVariablesubparameter specifies that the variableREASONbe used as the sensitive variable in assessing and mitigating bias.
The trainOptions parameter includes the following subparameters that are passed to the svm.svmTrain action call:
The
csubparameter specifies 0.1 as the penalty for training.The
inputssubparameter specifies that the variablesCLAGE,CLNO,DEBTINC,LOAN,MORTDUE,VALUE,YOJ,DELINQ,DEROG,JOB, andNINQbe used as inputs.The
nominalssubparameter specifies that the variablesDELINQ,DEROG,JOB,NINQ, andBADbe used as nominal variables.The
saveStatesubparameter specifies the table to store the generated analytic store model.The
seedsubparameter specifies the seed to use for pseudorandom number generation in the training process.The
tablesubparameter names the input data table for training.The
targetsubparameter specifies that the variableBADbe used as the target variable.
Mitigating Bias of a Support Vector Machine
This section contains R code for the analysis in the CASL version of this example, which contains details about the results.
For more information about coding in R, see Getting Started with SAS Viya for R and SAS Viya: System Programming Guide.
This example illustrates the mitigateBiasSVM action by using the home equity data that are described in the CASL version of this example.
The following statements use the mitigateBiasSVM action to run the exponentiated gradient reduction (EGR) algorithm in order to train a model that has reduced equal opportunity bias, which is calculated on the sensitive attribute REASON. Note that the syntax of the trainOptions parameter here is the same as the syntax of the svm.svmTrain action.
cas.fairAITools.mitigateBiasSVM(s,
mitigateOptions=list(biasMetric='EQUALOPPORTUNITY',
bound='75',
event='1',
predictedVariables=c('P_BAD1', 'P_BAD0'),
responseLevels=c('1', '0'),
seed='12345',
sensitiveVariable='REASON'),
trainOptions=list(c='0.1',
inputs=c('CLAGE',
'CLNO',
'DEBTINC',
'LOAN',
'MORTDUE',
'VALUE',
'YOJ',
'DELINQ',
'DEROG',
'JOB',
'NINQ'),
nominals=c('DELINQ',
'DEROG',
'JOB',
'NINQ',
'BAD'),
saveState=list(name='hmeq_svm_astore',
replace='True'),
seed='12345',
table='hmeq',
target='BAD'))
The mitigateOptions parameter includes the following subparameters:
The
biasMetricsubparameter specifies that equal opportunity be used as the bias measurement.The
boundsubparameter specifies 75 as the bound value for the EGR algorithm.The
eventsubparameter specifies the formatted value of the response variable that represents the event of interest.The
predictedVariablessubparameter specifies that the variablesP_BAD0andP_BAD1be used as the model predictions. A predicted variable is given for each level of the target variable.The
responseLevelssubparameter specifies the levels of the target that correspond to each variable that is listed in thepredictedVariablessubparameter. The order of the response levels must match the order of the predicted variables.The
seedsubparameter specifies the seed to use for pseudorandom number generation in the EGR algorithm.The
sensitiveVariablesubparameter specifies that the variableREASONbe used as the sensitive variable in assessing and mitigating bias.
The trainOptions parameter includes the following subparameters that are passed to the svm.svmTrain action call:
The
csubparameter specifies 0.1 as the penalty for training.The
inputssubparameter specifies that the variablesCLAGE,CLNO,DEBTINC,LOAN,MORTDUE,VALUE,YOJ,DELINQ,DEROG,JOB, andNINQbe used as inputs.The
nominalssubparameter specifies that the variablesDELINQ,DEROG,JOB,NINQ, andBADbe used as nominal variables.The
saveStatesubparameter specifies the table to store the generated analytic store model.The
seedsubparameter specifies the seed to use for pseudorandom number generation in the training process.The
tablesubparameter names the input data table for training.The
targetsubparameter specifies that the variableBADbe used as the target variable.