Fair AI Tools Action Set
Mitigating Bias of a Gradient Boosting Tree Model
This section contains PROC CAS code.
Note: Input data must be accessible in your CAS session, either as a CAS table or as a transient-scope table. A CAS table has a two-level name: the first level is your CAS engine libref, and the second level is the table name. You refer to this table in the CAS procedure by specifying only the second level. For more information about two-level names, see Chapter 2, Shared Concepts (SAS Viya: Machine Learning Procedures). A transient-scope table is called directly from the action and exists in memory for the duration of the action. For more information about accessing data, see SAS Viya: System Programming Guide. For more information about PROC CAS and programming in CASL, see SAS Cloud Analytic Services: CASL Programmer’s Guide and SAS Cloud Analytic Services: CASL Reference.
This example trains a gradient boosting tree model in order to reduce bias according to the demographic parity metric by using the home equity data, which are available in the sampsio.hmeq data set. The data set has a binary target named BAD, and the example uses the variable REASON as the sensitive attribute.
You can load the sampsio.hmeq data set into your CAS session by specifying your CAS engine libref in the SET statement in the following DATA step:
data mycas.hmeq;
set sampsio.hmeq;
where reason ne '';
run;
These statements assume that your CAS engine libref is named mycas, but you can substitute any appropriately defined CAS engine libref.
The following statements use the mitigateBiasGradientBoostTree action to run the exponentiated gradient reduction (EGR) algorithm in order to train a model that has reduced demographic parity bias, which is calculated on the sensitive attribute REASON. Note that the syntax of the trainOptions parameter here is the same as the syntax of the decisionTree.gbtreeTrain action.
proc cas;
fairAITools.mitigateBiasGradientBoostTree /
mitigateOptions={biasMetric="DEMOGRAPHICPARITY",
predictedVariables={"P_BAD1", "P_BAD0"},
responseLevels={"1", "0"},
seed="12345",
sensitiveVariable="REASON"},
trainOptions={casOut={name="gbtreeOut",
replace="true"},
encodeName="True",
inputs={"CLAGE",
"DEBTINC",
"DEROG",
"JOB",
"LOAN",
"MORTDUE",
"REASON",
"VALUE"},
maxLevel="11",
nBins="20",
nominals={"BAD",
"JOB",
"REASON"},
saveState={name="hmeq_gbt_astore",
replace="True"},
table="hmeq",
target="BAD"};
run;
The mitigateOptions parameter includes the following subparameters:
The
biasMetricsubparameter specifies that the demographic parity be used as the bias measurement.The
predictedVariablessubparameter specifies that the variablesP_BAD0andP_BAD1be used as the model predictions. A predicted variable is given for each level of the target variable.The
responseLevelssubparameter specifies the levels of the target that correspond to each variable that is listed in thepredictedVariablessubparameter. The order of the response levels must match the order of the predicted variables.The
seedsubparameter specifies the seed to use for pseudorandom number generation.The
sensitiveVariablesubparameter specifies that the variableREASONbe used as the sensitive variable in assessing and mitigating bias.
The trainOptions parameter includes the following subparameters that are passed to the decisionTree.gbtreeTrain action call:
The
casOutsubparameter specifies the output table that contains the gradient boosting tree model.The
encodeNamesubparameter specifies that the variable names of the predicted probabilities of the target be encoded.The
inputssubparameter specifies that the variablesCLAGE,DEBTINC,DEROG,JOB,LOAN,MORTDUE,REASON, andVALUEbe used as inputs.The
maxLevelsubparameter specifies the maximum tree level.The
nBinssubparameter specifies the number of bins to use for numeric variables in calculating the decision tree.The
nominalssubparameter specifies that the variablesBAD,JOB, andREASONbe used as nominal variables.The
saveStatesubparameter specifies the table to store the generated analytic store model.The
tablesubparameter names the input data table for training.The
targetsubparameter specifies that the variableBADbe used as the target variable.
The "Mitigation Information" table that these statements produce is shown in Output 15.14.1. The table shows some key information about the model and values of the key hyperparameters that the EGR algorithm uses.
The "Iteration History" table that these statements produce is shown in Output 15.14.2. The table displays the iteration history of the overall demographic parity, the overall misclassification rate, and the overall multiclass log loss. From the table, you can see that without bias mitigation, the demographic parity gap between those two sensitive attributes is about 3.2%, and after 10 iterations of the EGR algorithm it has been reduced to about 1.9%. Meanwhile, the overall accuracy of the model does not change significantly.
The "Group Metrics" table that these statements produce is shown in Output 15.14.3. The table displays the model assessment statistics and average prediction for each sensitive attribute during each iteration of the EGR algorithm.
Output 15.14.1: Mitigation Information
| Mitigation Information | ||
|---|---|---|
| Description | Character Value | Value |
| Bias Metric | DEMOGRAPHICPARITY | . |
| Sensitive Variable | REASON | . |
| Response | BAD | . |
| Bound Value | 100 | 100 |
| Learning Rate | 0.01 | 0.01 |
| Maximum Iteration | 10 | 10 |
| Tolerance | 0.005 | 0.005 |
| Event Level | 1 | . |
Output 15.14.2: Iteration History
| EGR Iteration History | |||
|---|---|---|---|
| Iteration | Demographic Parity (Statistical Parity) | Misclassification Rate | Multi-Class Log Loss |
| 1 | 0.032405 | 0.0363 | 0.1210 |
| 2 | 0.034047 | 0.0359 | 0.1226 |
| 3 | 0.027814 | 0.0366 | 0.1241 |
| 4 | 0.025066 | 0.0380 | 0.1252 |
| 5 | 0.022205 | 0.0387 | 0.1245 |
| 6 | 0.018983 | 0.0399 | 0.1271 |
| 7 | 0.022872 | 0.0382 | 0.1242 |
| 8 | 0.022968 | 0.0399 | 0.1257 |
| 9 | 0.015209 | 0.0399 | 0.1291 |
| 10 | 0.019238 | 0.0443 | 0.1275 |
Output 15.14.3: Group Metrics History
| Group Metrics | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Iteration | REASON | Number of Observations | True Positive Rate | False Positive Rate | Area Under ROC | Misclassification Rate | Multi-Class Log Loss | Best Kolmogorov-Smirnov Along ROC | Cumulative Lift | Cumulative Response | Proportion Into Event Level | Average Prediction for Event |
| 1 | DebtCon | 3928 | 0.8604 | 0.0145 | 0.9908 | 0.0382 | 0.1241 | 0.9004 | 5.2752 | 52.7517 | 0.1749 | 0.1864 |
| HomeImp | 1780 | 0.8939 | 0.0108 | 0.9945 | 0.0320 | 0.1141 | 0.9188 | 4.4949 | 44.9495 | 0.2073 | 0.2170 | |
| 2 | DebtCon | 3928 | 0.8631 | 0.0167 | 0.9907 | 0.0395 | 0.1262 | 0.8955 | 5.2483 | 52.4832 | 0.1772 | 0.1884 |
| HomeImp | 1780 | 0.9116 | 0.0108 | 0.9946 | 0.0281 | 0.1146 | 0.9264 | 4.4949 | 44.9495 | 0.2112 | 0.2164 | |
| 3 | DebtCon | 3928 | 0.8644 | 0.0157 | 0.9902 | 0.0384 | 0.1277 | 0.8961 | 5.2886 | 52.8859 | 0.1767 | 0.1879 |
| HomeImp | 1780 | 0.8864 | 0.0094 | 0.9940 | 0.0326 | 0.1159 | 0.9231 | 4.4949 | 44.9495 | 0.2045 | 0.2137 | |
| 4 | DebtCon | 3928 | 0.8685 | 0.0189 | 0.9898 | 0.0402 | 0.1291 | 0.8883 | 5.2617 | 52.6174 | 0.1800 | 0.1902 |
| HomeImp | 1780 | 0.8864 | 0.0101 | 0.9944 | 0.0331 | 0.1168 | 0.9206 | 4.4949 | 44.9495 | 0.2051 | 0.2137 | |
| 5 | DebtCon | 3928 | 0.8671 | 0.0185 | 0.9901 | 0.0402 | 0.1288 | 0.8956 | 5.2483 | 52.4832 | 0.1795 | 0.1891 |
| HomeImp | 1780 | 0.8737 | 0.0094 | 0.9942 | 0.0354 | 0.1149 | 0.9188 | 4.4949 | 44.9495 | 0.2017 | 0.2106 | |
| 6 | DebtCon | 3928 | 0.8604 | 0.0185 | 0.9899 | 0.0415 | 0.1300 | 0.8918 | 5.2617 | 52.6174 | 0.1782 | 0.1905 |
| HomeImp | 1780 | 0.8611 | 0.0072 | 0.9943 | 0.0365 | 0.1207 | 0.9210 | 4.4949 | 44.9495 | 0.1972 | 0.2098 | |
| 7 | DebtCon | 3928 | 0.8698 | 0.0192 | 0.9903 | 0.0402 | 0.1278 | 0.8958 | 5.2349 | 52.3490 | 0.1805 | 0.1906 |
| HomeImp | 1780 | 0.8813 | 0.0094 | 0.9948 | 0.0337 | 0.1163 | 0.9228 | 4.4949 | 44.9495 | 0.2034 | 0.2108 | |
| 8 | DebtCon | 3928 | 0.8577 | 0.0198 | 0.9897 | 0.0430 | 0.1297 | 0.8924 | 5.2617 | 52.6174 | 0.1787 | 0.1907 |
| HomeImp | 1780 | 0.8788 | 0.0079 | 0.9946 | 0.0331 | 0.1166 | 0.9285 | 4.4949 | 44.9495 | 0.2017 | 0.2080 | |
| 9 | DebtCon | 3928 | 0.8685 | 0.0185 | 0.9900 | 0.0400 | 0.1317 | 0.8921 | 5.2081 | 52.0805 | 0.1797 | 0.1919 |
| HomeImp | 1780 | 0.8485 | 0.0079 | 0.9924 | 0.0399 | 0.1236 | 0.9166 | 4.4949 | 44.9495 | 0.1949 | 0.2040 | |
| 10 | DebtCon | 3928 | 0.8456 | 0.0217 | 0.9894 | 0.0468 | 0.1312 | 0.8912 | 5.2617 | 52.6174 | 0.1780 | 0.1925 |
| HomeImp | 1780 | 0.8561 | 0.0087 | 0.9938 | 0.0388 | 0.1193 | 0.9184 | 4.4949 | 44.9495 | 0.1972 | 0.2072 | |
Mitigating Bias of a Gradient Boosting Tree Model
This section contains Lua code for the analysis in the CASL version of this example, which contains details about the results.
For more information about coding in Lua, see Getting Started with SAS Viya for Lua and SAS Viya: System Programming Guide.
This example illustrates the mitigateBiasGradientBoostTree action by using the home equity data that are described in the CASL version of this example.
The following statements use the mitigateBiasGradientBoostTree action to run the exponentiated gradient reduction (EGR) algorithm in order to train a model that has reduced demographic parity bias, which is calculated on the sensitive attribute REASON. Note that the syntax of the trainOptions parameter here is the same as the syntax of the decisionTree.gbtreeTrain action.
mitigateOptions={biasMetric="DEMOGRAPHICPARITY",
predictedVariables={"P_BAD1", "P_BAD0"},
responseLevels={"1", "0"},
seed=12345,
sensitiveVariable="REASON"},
trainOptions={casOut={
name="gbtreeOut",
replace=true
},
encodeName=True,
inputs={"CLAGE","DEBTINC","DEROG","JOB","LOAN","MORTDUE","REASON","VALUE"},
maxLevel=11,
nBins=20,
nominals={"BAD", "JOB", "REASON"},
saveState={
name="hmeq_gbt_astore",
replace=True},
table="hmeq",
target="BAD"
}
The mitigateOptions parameter includes the following subparameters:
The
biasMetricsubparameter specifies that the demographic parity be used as the bias measurement.The
predictedVariablessubparameter specifies that the variablesP_BAD0andP_BAD1be used as the model predictions. A predicted variable is given for each level of the target variable.The
responseLevelssubparameter specifies the levels of the target that correspond to each variable that is listed in thepredictedVariablessubparameter. The order of the response levels must match the order of the predicted variables.The
seedsubparameter specifies the seed to use for pseudorandom number generation.The
sensitiveVariablesubparameter specifies that the variableREASONbe used as the sensitive variable in assessing and mitigating bias.
The trainOptions parameter includes the following subparameters that are passed to the decisionTree.gbtreeTrain action call:
The
casOutsubparameter specifies the output table that contains the gradient boosting tree model.The
encodeNamesubparameter specifies that the variable names of the predicted probabilities of the target be encoded.The
inputssubparameter specifies that the variablesCLAGE,DEBTINC,DEROG,JOB,LOAN,MORTDUE,REASON, andVALUEbe used as inputs.The
maxLevelsubparameter specifies the maximum tree level.The
nBinssubparameter specifies the number of bins to use for numeric variables in calculating the decision tree.The
nominalssubparameter specifies that the variablesBAD,JOB, andREASONbe used as nominal variables.The
saveStatesubparameter specifies the table to store the generated analytic store model.The
tablesubparameter names the input data table for training.The
targetsubparameter specifies that the variableBADbe used as the target variable.
Mitigating Bias of a Gradient Boosting Tree Model
This section contains Python code for the analysis in the CASL version of this example, which contains details about the results.
For more information about coding in Python, see Getting Started with SAS Viya for Python and SAS Viya: System Programming Guide.
This example illustrates the mitigateBiasGradientBoostTree action by using the home equity data that are described in the CASL version of this example.
The following statements use the mitigateBiasGradientBoostTree action to run the exponentiated gradient reduction (EGR) algorithm in order to train a model that has reduced demographic parity bias, which is calculated on the sensitive attribute REASON. Note that the syntax of the trainOptions parameter here is the same as the syntax of the decisionTree.gbtreeTrain action.
s.fairaitools.mitigatebiasgradientboosttree(
mitigateOptions=dict(biasMetric='DEMOGRAPHICPARITY',
predictedVariables=['P_BAD1', 'P_BAD0'],
responseLevels=['1', '0'],
seed='12345',
sensitiveVariable='REASON'),
trainOptions=dict(casOut=dict(name='gbtreeOut',
replace='true'),
encodeName='True',
inputs=['CLAGE',
'DEBTINC',
'DEROG',
'JOB',
'LOAN',
'MORTDUE',
'REASON',
'VALUE'],
maxLevel='11',
nBins='20',
nominals=['BAD',
'JOB',
'REASON'],
saveState=dict(name='hmeq_gbt_astore',
replace='True'),
table='hmeq',
target='BAD'))
The mitigateOptions parameter includes the following subparameters:
The
biasMetricsubparameter specifies that the demographic parity be used as the bias measurement.The
predictedVariablessubparameter specifies that the variablesP_BAD0andP_BAD1be used as the model predictions. A predicted variable is given for each level of the target variable.The
responseLevelssubparameter specifies the levels of the target that correspond to each variable that is listed in thepredictedVariablessubparameter. The order of the response levels must match the order of the predicted variables.The
seedsubparameter specifies the seed to use for pseudorandom number generation.The
sensitiveVariablesubparameter specifies that the variableREASONbe used as the sensitive variable in assessing and mitigating bias.
The trainOptions parameter includes the following subparameters that are passed to the decisionTree.gbtreeTrain action call:
The
casOutsubparameter specifies the output table that contains the gradient boosting tree model.The
encodeNamesubparameter specifies that the variable names of the predicted probabilities of the target be encoded.The
inputssubparameter specifies that the variablesCLAGE,DEBTINC,DEROG,JOB,LOAN,MORTDUE,REASON, andVALUEbe used as inputs.The
maxLevelsubparameter specifies the maximum tree level.The
nBinssubparameter specifies the number of bins to use for numeric variables in calculating the decision tree.The
nominalssubparameter specifies that the variablesBAD,JOB, andREASONbe used as nominal variables.The
saveStatesubparameter specifies the table to store the generated analytic store model.The
tablesubparameter names the input data table for training.The
targetsubparameter specifies that the variableBADbe used as the target variable.
Mitigating Bias of a Gradient Boosting Tree Model
This section contains R code for the analysis in the CASL version of this example, which contains details about the results.
For more information about coding in R, see Getting Started with SAS Viya for R and SAS Viya: System Programming Guide.
This example illustrates the mitigateBiasGradientBoostTree action by using the home equity data that are described in the CASL version of this example.
The following statements use the mitigateBiasGradientBoostTree action to run the exponentiated gradient reduction (EGR) algorithm in order to train a model that has reduced demographic parity bias, which is calculated on the sensitive attribute REASON. Note that the syntax of the trainOptions parameter here is the same as the syntax of the decisionTree.gbtreeTrain action.
cas.fairAITools.mitigateBiasGradientBoostTree(s,
mitigateOptions=list(biasMetric='DEMOGRAPHICPARITY',
predictedVariables=c('P_BAD1', 'P_BAD0'),
responseLevels=c('1', '0'),
seed='12345',
sensitiveVariable='REASON'),
trainOptions=list(casOut=list(name='gbtreeOut',
replace='true'),
encodeName='True',
inputs=c('CLAGE',
'DEBTINC',
'DEROG',
'JOB',
'LOAN',
'MORTDUE',
'REASON',
'VALUE'),
maxLevel='11',
nBins='20',
nominals=c('BAD',
'JOB',
'REASON'),
saveState=list(name='hmeq_gbt_astore',
replace='True'),
table='hmeq',
target='BAD'))
The mitigateOptions parameter includes the following subparameters:
The
biasMetricsubparameter specifies that the demographic parity be used as the bias measurement.The
predictedVariablessubparameter specifies that the variablesP_BAD0andP_BAD1be used as the model predictions. A predicted variable is given for each level of the target variable.The
responseLevelssubparameter specifies the levels of the target that correspond to each variable that is listed in thepredictedVariablessubparameter. The order of the response levels must match the order of the predicted variables.The
seedsubparameter specifies the seed to use for pseudorandom number generation.The
sensitiveVariablesubparameter specifies that the variableREASONbe used as the sensitive variable in assessing and mitigating bias.
The trainOptions parameter includes the following subparameters that are passed to the decisionTree.gbtreeTrain action call:
The
casOutsubparameter specifies the output table that contains the gradient boosting tree model.The
encodeNamesubparameter specifies that the variable names of the predicted probabilities of the target be encoded.The
inputssubparameter specifies that the variablesCLAGE,DEBTINC,DEROG,JOB,LOAN,MORTDUE,REASON, andVALUEbe used as inputs.The
maxLevelsubparameter specifies the maximum tree level.The
nBinssubparameter specifies the number of bins to use for numeric variables in calculating the decision tree.The
nominalssubparameter specifies that the variablesBAD,JOB, andREASONbe used as nominal variables.The
saveStatesubparameter specifies the table to store the generated analytic store model.The
tablesubparameter names the input data table for training.The
targetsubparameter specifies that the variableBADbe used as the target variable.