examples/example_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Detecting and mitigating age bias on credit decisions \n",
    "\n",
    "The goal of this tutorial is to introduce the basic functionality of AI Fairness 360 to an interested developer who may not have a background in bias detection and mitigation.\n",
    "\n",
    "### Biases and Machine Learning\n",
    "A machine learning model makes predictions of an outcome for a particular instance. (Given an instance of a loan application, predict if the applicant will repay the loan.) The model makes these predictions based on a training dataset, where many other instances (other loan applications) and actual outcomes (whether they repaid) are provided. Thus, a machine learning algorithm will attempt to find patterns, or generalizations, in the training dataset to use when a prediction for a new instance is needed. (For example, one pattern it might discover is \"if a person has salary > USD 40K and has outstanding debt < USD 5, they will repay the loan\".) In many domains this technique, called supervised machine learning, has worked very well.\n",
    "\n",
    "However, sometimes the patterns that are found may not be desirable or may even be illegal. For example, a loan repay model may determine that age plays a significant role in the prediction of repayment because the training dataset happened to have better repayment for one age group than for another. This raises two problems: 1) the training dataset may not be representative of the true population of people of all age groups, and 2) even if it is representative, it is illegal to base any decision on a applicant's age, regardless of whether this is a good prediction based on historical data.\n",
    "\n",
    "AI Fairness 360 is designed to help address this problem with _fairness metrics_ and _bias mitigators_.  Fairness metrics can be used to check for bias in machine learning workflows.  Bias mitigators can be used to overcome bias in the workflow to produce a more fair outcome. \n",
    "\n",
    "The loan scenario describes an intuitive example of illegal bias. However, not all undesirable bias in machine learning is illegal it may also exist in more subtle ways.  For example, a loan company may want a diverse portfolio of customers across all income levels, and thus, will deem it undesirable if they are making more loans to high income levels over low income levels.  Although this is not illegal or unethical, it is undesirable for the company's strategy.\n",
    "\n",
    "As these two examples illustrate, a bias detection and/or mitigation toolkit needs to be tailored to the particular bias of interest.  More specifically, it needs to know the attribute or attributes, called _protected atrributes_, that are of interest: race is one example of a _protected attribute_ and age is a second.\n",
    "\n",
    "### The Machine Learning Workflow\n",
    "To understand how bias can enter a machine learning model, we first review the basics of how a model is created in a supervised machine learning process.  \n",
    "\n",
    "\n",
    "\n",
    "![image](https://github.com/IBM/ensure-loan-fairness-aif360/tree/master/doc/source/images/Complex_NoProc_V3.jpg)\n",
    "\n",
    "\n",
    "\n",
    "   \n",
    "    \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "First, the process starts with a _training dataset_, which contains a sequence of instances, where each instance has two components: the features and the correct prediction for those features.  Next, a machine learning algorithm is trained on this training dataset to produce a machine learning model.  This generated model can be used to make a prediction when given a new instance.  A second dataset with features and correct predictions, called a _test dataset_, is used to assess the accuracy of the model.\n",
    "Since this test dataset is the same format as the training dataset, a set of instances of features and prediction pairs, often these two datasets derive from the same initial dataset.  A random partitioning algorithm is used to split the initial dataset into training and test datasets.\n",
    "\n",
    "Bias can enter the system in any of the three steps above.  The training data set may be biased in that its outcomes may be biased towards particular kinds of instances.  The algorithm that creates the model may be biased in that it may generate models that are weighted towards particular features in the input. The test data set may be biased in that it has expectations on correct answers that may be biased.  These three points in the machine learning process represent points for testing and mitigating bias.  In AI Fairness 360 codebase, we call these points _pre-processing_, _in-processing_, and _post-processing_. \n",
    "\n",
    "### AI Fairness 360\n",
    "We are now ready to utilize AI Fairness 360 (`aif360`) to detect and mitigate bias.  We will use the German credit dataset, splitting it into a training and test dataset.  We will look for bias in the creation of a machine learning model to predict if an applicant should be given credit based on various features from a typical credit application.  The protected attribute will be \"Age\", with \"1\" (older than or equal to 25) and \"0\" (younger than 25) being the values for the privileged and unprivileged groups, respectively.\n",
    "For this first tutorial, we will check for bias in the initial training data, mitigate the bias, and recheck.  More sophisticated machine learning workflows are given in the author tutorials and demo notebooks in the codebase.\n",
    "\n",
    "Here are the steps involved\n",
    "1. Import the aif360 toolkit and install it\n",
    "1. Write import statements\n",
    "1. Set bias detection options, load dataset, and split between train and test\n",
    "1. Compute fairness metric on original training dataset\n",
    "1. Mitigate bias by transforming the original dataset\n",
    "1. Compute fairness metric on transformed training dataset\n",
    "\n",
    "### Step 1\n",
    "We'll install the aif360 toolkit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting aif360\n",
      "  Downloading https://files.pythonhosted.org/packages/cf/ec/6497d4ee752611e80a6d5cd51ae89ae8ded02fe6c48a5424db2db7c252af/aif360-0.2.0-py3-none-any.whl (23.4MB)\n",
      "\u001b[K    100% |████████████████████████████████| 23.4MB 43kB/s  eta 0:00:01\n",
      "\u001b[?25hCollecting numpy<1.16,>=1.14 (from aif360)\n",
      "  Downloading https://files.pythonhosted.org/packages/86/04/bd774106ae0ae1ada68c67efe89f1a16b2aa373cc2db15d974002a9f136d/numpy-1.15.4-cp35-cp35m-manylinux1_x86_64.whl (13.8MB)\n",
      "\u001b[K    100% |████████████████████████████████| 13.8MB 75kB/s  eta 0:00:01\n",
      "\u001b[?25hCollecting pandas==0.23.3 (from aif360)\n",
      "  Downloading https://files.pythonhosted.org/packages/1d/cb/1cc27b57855a7f7c25965ab257e2bf251589bfe450e2cb23820b11a674a6/pandas-0.23.3-cp35-cp35m-manylinux1_x86_64.whl (8.7MB)\n",
      "\u001b[K    100% |████████████████████████████████| 8.7MB 120kB/s eta 0:00:01\n",
      "\u001b[?25hRequirement not upgraded as not directly required: scipy in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from aif360)\n",
      "Requirement not upgraded as not directly required: scikit-learn in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from aif360)\n",
      "Requirement not upgraded as not directly required: pytz>=2011k in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas==0.23.3->aif360)\n",
      "Requirement not upgraded as not directly required: python-dateutil>=2.5.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas==0.23.3->aif360)\n",
      "Requirement not upgraded as not directly required: six>=1.5 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from python-dateutil>=2.5.0->pandas==0.23.3->aif360)\n",
      "Installing collected packages: numpy, pandas, aif360\n",
      "  Found existing installation: numpy 1.13.3\n",
      "    Uninstalling numpy-1.13.3:\n",
      "      Successfully uninstalled numpy-1.13.3\n",
      "  Found existing installation: pandas 0.21.0\n",
      "    Uninstalling pandas-0.21.0:\n",
      "      Successfully uninstalled pandas-0.21.0\n",
      "Successfully installed aif360-0.2.0 numpy-1.15.4 pandas-0.23.3\n"
     ]
    }
   ],
   "source": [
    "!pip install aif360"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2\n",
    "As with any python program, the first step will be to import the necessary packages.  Below we import several components from the `aif360` package.  We import the GermanDataset, metrics to check for bias, and classes related to the algorithm we will use to mitigate bias."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting cvxpy==0.4.11\n",
      "  Downloading https://files.pythonhosted.org/packages/a7/52/d2928100c93e726acdbb793e4a3662d4c65ace58ca0ddd09463a172f7bed/cvxpy-0.4.11.tar.gz (159kB)\n",
      "\u001b[K    100% |████████████████████████████████| 163kB 5.2MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting ecos>=2 (from cvxpy==0.4.11)\n",
      "  Downloading https://files.pythonhosted.org/packages/69/56/e66d3dc602dc317e87e712992997c9a0745a4ddb01714ed6820c1bc2519f/ecos-2.0.7.post1-cp35-cp35m-manylinux1_x86_64.whl (147kB)\n",
      "\u001b[K    100% |████████████████████████████████| 153kB 6.0MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting scs>=1.1.3 (from cvxpy==0.4.11)\n",
      "  Downloading https://files.pythonhosted.org/packages/b3/fd/6e01c4f4a69fcc6c3db130ba55572089e78e77ea8c0921a679f9da1ec04c/scs-2.0.2.tar.gz (133kB)\n",
      "\u001b[K    100% |████████████████████████████████| 143kB 6.7MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting multiprocess (from cvxpy==0.4.11)\n",
      "  Downloading https://files.pythonhosted.org/packages/31/60/6d74caa02b54ca43092e745640c7d98f367f07160441682a01602ce00bc5/multiprocess-0.70.7.tar.gz (1.4MB)\n",
      "\u001b[K    100% |████████████████████████████████| 1.4MB 734kB/s eta 0:00:01\n",
      "\u001b[?25hRequirement not upgraded as not directly required: fastcache in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from cvxpy==0.4.11)\n",
      "Requirement not upgraded as not directly required: six in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from cvxpy==0.4.11)\n",
      "Requirement not upgraded as not directly required: toolz in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from cvxpy==0.4.11)\n",
      "Requirement not upgraded as not directly required: numpy>=1.9 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from cvxpy==0.4.11)\n",
      "Requirement not upgraded as not directly required: scipy>=0.15 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from cvxpy==0.4.11)\n",
      "Collecting CVXcanon>=0.0.22 (from cvxpy==0.4.11)\n",
      "  Downloading https://files.pythonhosted.org/packages/9e/e6/63eb6e7dca5dcb723429e65b456d0e3638976e63f6696b7eb48fee3a491d/CVXcanon-0.1.1.tar.gz (694kB)\n",
      "\u001b[K    100% |████████████████████████████████| 696kB 1.5MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting dill>=0.2.9 (from multiprocess->cvxpy==0.4.11)\n",
      "  Downloading https://files.pythonhosted.org/packages/fe/42/bfe2e0857bc284cbe6a011d93f2a9ad58a22cb894461b199ae72cfef0f29/dill-0.2.9.tar.gz (150kB)\n",
      "\u001b[K    100% |████████████████████████████████| 153kB 6.2MB/s eta 0:00:01\n",
      "\u001b[?25hBuilding wheels for collected packages: cvxpy, scs, multiprocess, CVXcanon, dill\n",
      "  Running setup.py bdist_wheel for cvxpy ... \u001b[?25ldone\n",
      "\u001b[?25h  Stored in directory: /home/dsxuser/.cache/pip/wheels/2d/26/ba/f0a0388a7ab98124caf88ea8b532b81cf4acb1b6a4a990e3b4\n",
      "  Running setup.py bdist_wheel for scs ... \u001b[?25ldone\n",
      "\u001b[?25h  Stored in directory: /home/dsxuser/.cache/pip/wheels/ff/f0/aa/530ccd478d7d9900b4e9ef5bc5a39e895ce110bed3d3ac653e\n",
      "  Running setup.py bdist_wheel for multiprocess ... \u001b[?25ldone\n",
      "\u001b[?25h  Stored in directory: /home/dsxuser/.cache/pip/wheels/3a/ed/51/77c833462c3e757ce50c4b2b68bdf53f5d1745542fe567d740\n",
      "  Running setup.py bdist_wheel for CVXcanon ... \u001b[?25ldone\n",
      "\u001b[?25h  Stored in directory: /home/dsxuser/.cache/pip/wheels/46/a7/aa/2116e25ce5ef04deae6a3c4d01908449e572393351edf65b14\n",
      "  Running setup.py bdist_wheel for dill ... \u001b[?25ldone\n",
      "\u001b[?25h  Stored in directory: /home/dsxuser/.cache/pip/wheels/5b/d7/0f/e58eae695403de585269f4e4a94e0cd6ca60ec0c202936fa4a\n",
      "Successfully built cvxpy scs multiprocess CVXcanon dill\n",
      "Installing collected packages: ecos, scs, dill, multiprocess, CVXcanon, cvxpy\n",
      "  Found existing installation: dill 0.2.7.1\n",
      "    Uninstalling dill-0.2.7.1:\n",
      "      Successfully uninstalled dill-0.2.7.1\n",
      "Successfully installed CVXcanon-0.1.1 cvxpy-0.4.11 dill-0.2.9 ecos-2.0.7.post1 multiprocess-0.70.7 scs-2.0.2\n"
     ]
    }
   ],
   "source": [
    "!pip install cvxpy==0.4.11"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load all necessary packages\n",
    "import sys\n",
    "sys.path.insert(1, \"../\")  \n",
    "\n",
    "import numpy as np\n",
    "np.random.seed(0)\n",
    "\n",
    "from aif360.datasets import GermanDataset\n",
    "from aif360.metrics import BinaryLabelDatasetMetric\n",
    "from aif360.algorithms.preprocessing import Reweighing\n",
    "\n",
    "from IPython.display import Markdown, display"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3 Load dataset, specifying protected attribute, and split dataset into train and test\n",
    "In Step 3 we begin by dowloading the dataset. Then we load the initial dataset, setting the protected attribute to be age.  We then split the original dataset into training and testing datasets.  Note that we use a random seed number for this demonstration, which gives us the same result for each split().\n",
    "Although we will use only  the training dataset in this tutorial, a normal workflow would also use a test dataset for assessing the efficacy (accuracy, fairness, etc.) during the development of a machine learning model.\n",
    "Finally, we set two variables (to be used in Step 3) for the privileged (1) and unprivileged (0) values for the age attribute.  These are key inputs for detecting and mitigating bias, which will be Step 3 and Step 4.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/DSX-Python35/lib/python3.5/site-packages/aif360/data/raw/german\n"
     ]
    }
   ],
   "source": [
    "aif360_location = !python -c \"from distutils.sysconfig import get_python_lib; print(get_python_lib())\"\n",
    "import os\n",
    "install_loc = os.path.join(aif360_location[0], \"aif360/data/raw/german/\")\n",
    "%cd $install_loc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2019-02-26 18:55:32--  ftp://ftp.ics.uci.edu/pub/machine-learning-databases/statlog/german/german.data\n",
      "           => ‘german.data.1’\n",
      "Resolving ftp.ics.uci.edu (ftp.ics.uci.edu)... 128.195.1.14\n",
      "Connecting to ftp.ics.uci.edu (ftp.ics.uci.edu)|128.195.1.14|:21... connected.\n",
      "Logging in as anonymous ... Logged in!\n",
      "==> SYST ... done.    ==> PWD ... done.\n",
      "==> TYPE I ... done.  ==> CWD (1) /pub/machine-learning-databases/statlog/german ... done.\n",
      "==> SIZE german.data ... 79793\n",
      "==> PASV ... done.    ==> RETR german.data ... done.\n",
      "Length: 79793 (78K) (unauthoritative)\n",
      "\n",
      "100%[======================================>] 79,793      --.-K/s   in 0.1s    \n",
      "\n",
      "2019-02-26 18:55:33 (784 KB/s) - ‘german.data.1’ saved [79793]\n",
      "\n",
      "--2019-02-26 18:55:33--  ftp://ftp.ics.uci.edu/pub/machine-learning-databases/statlog/german/german.doc\n",
      "           => ‘german.doc.1’\n",
      "Resolving ftp.ics.uci.edu (ftp.ics.uci.edu)... 128.195.1.14\n",
      "Connecting to ftp.ics.uci.edu (ftp.ics.uci.edu)|128.195.1.14|:21... connected.\n",
      "Logging in as anonymous ... Logged in!\n",
      "==> SYST ... done.    ==> PWD ... done.\n",
      "==> TYPE I ... done.  ==> CWD (1) /pub/machine-learning-databases/statlog/german ... done.\n",
      "==> SIZE german.doc ... 4679\n",
      "==> PASV ... done.    ==> RETR german.doc ... done.\n",
      "Length: 4679 (4.6K) (unauthoritative)\n",
      "\n",
      "100%[======================================>] 4,679       --.-K/s   in 0.008s  \n",
      "\n",
      "2019-02-26 18:55:34 (548 KB/s) - ‘german.doc.1’ saved [4679]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget ftp://ftp.ics.uci.edu/pub/machine-learning-databases/statlog/german/german.data\n",
    "!wget ftp://ftp.ics.uci.edu/pub/machine-learning-databases/statlog/german/german.doc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:'scores' has no well-defined meaning out of range [0, 1].\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/dsxuser/work\n"
     ]
    }
   ],
   "source": [
    "%cd -\n",
    "dataset_orig = GermanDataset(protected_attribute_names=['age'],\n",
    "# this dataset also contains protected\n",
    "# attribute for \"sex\" which we do not\n",
    "# consider in this evaluation\n",
    "privileged_classes=[lambda x: x >= 25],      # age >=25 is considered privileged\n",
    "features_to_drop=['personal_status', 'sex']) # ignore sex-related attributes\n",
    "\n",
    "dataset_orig_train, dataset_orig_test = dataset_orig.split([0.7], shuffle=True)\n",
    "\n",
    "privileged_groups = [{'age': 1}]\n",
    "unprivileged_groups = [{'age': 0}]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4 Compute fairness metric on original training dataset\n",
    "Now that we've identified the protected attribute 'age' and defined privileged and unprivileged values, we can use aif360 to detect bias in the dataset. One simple test is to compare the percentage of favorable results for the privileged and unprivileged groups, subtracting the former percentage from the latter. A negative value indicates less favorable outcomes for the unprivileged groups. This is implemented in the method called mean_difference on the BinaryLabelDatasetMetric class. The code below performs this check and displays the output, showing that the difference is -0.169905"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "#### Original training dataset"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Difference in mean outcomes between unprivileged and privileged groups = -0.169905\n"
     ]
    }
   ],
   "source": [
    "metric_orig_train = BinaryLabelDatasetMetric(dataset_orig_train, \n",
    "                                             unprivileged_groups=unprivileged_groups,\n",
    "                                             privileged_groups=privileged_groups)\n",
    "display(Markdown(\"#### Original training dataset\"))\n",
    "print(\"Difference in mean outcomes between unprivileged and privileged groups = %f\" % metric_orig_train.mean_difference())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5 Mitigate bias by transforming the original dataset\n",
    "The previous step showed that the privileged group was getting 17% more positive outcomes in the training dataset. Since this is not desirable, we are going to try to mitigate this bias in the training dataset. As stated above, this is called pre-processing mitigation because it happens before the creation of the model.\n",
    "\n",
    "AI Fairness 360 implements several pre-processing mitigation algorithms. We will choose the Reweighing algorithm [1], which is implemented in the Reweighing class in the aif360.algorithms.preprocessing package. This algorithm will transform the dataset to have more equity in positive outcomes on the protected attribute for the privileged and unprivileged groups.\n",
    "\n",
    "We then call the fit and transform methods to perform the transformation, producing a newly transformed training dataset (dataset_transf_train).\n",
    "\n",
    "[1] F. Kamiran and T. Calders,  \"Data Preprocessing Techniques for Classification without Discrimination,\" Knowledge and Information Systems, 2012."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "RW = Reweighing(unprivileged_groups=unprivileged_groups,\n",
    "                privileged_groups=privileged_groups)\n",
    "dataset_transf_train = RW.fit_transform(dataset_orig_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 6 Compute fairness metric on transformed dataset\n",
    "Now that we have a transformed dataset, we can check how effective it was in removing bias by using the same metric we used for the original training dataset in Step 3. Once again, we use the function mean_difference in the BinaryLabelDatasetMetric class. We see the mitigation step was very effective, the difference in mean outcomes is now 0.0. So we went from a 17% advantage for the privileged group to equality in terms of mean outcome."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "#### Transformed training dataset"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Difference in mean outcomes between unprivileged and privileged groups = 0.000000\n"
     ]
    }
   ],
   "source": [
    "metric_transf_train = BinaryLabelDatasetMetric(dataset_transf_train, \n",
    "                                               unprivileged_groups=unprivileged_groups,\n",
    "                                               privileged_groups=privileged_groups)\n",
    "display(Markdown(\"#### Transformed training dataset\"))\n",
    "print(\"Difference in mean outcomes between unprivileged and privileged groups = %f\" % metric_transf_train.mean_difference())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary\n",
    "The purpose of this tutorial is to give a new user to bias detection and mitigation a gentle introduction to some of the functionality of AI Fairness 360. A more complete use case would take the next step and see how the transformed dataset impacts the accuracy and fairness of a trained model. This is implemented in the demo notebook in the examples directory of toolkit, called demo_reweighing_preproc.ipynb. I highly encourage readers to view that notebook as it is generalization and extension of this simple tutorial.\n",
    "\n",
    "There are many metrics one can use to detect the presence of bias. AI Fairness 360 provides many of them for your use. Since it is not clear which of these metrics to use, we also provide some guidance. Likewise, there are many different bias mitigation algorithms one can employ, many of which are in AI Fairness 360. Other tutorials will demonstrate the use of some of these metrics and mitigations algorithms.\n",
    "\n",
    "As mentioned earlier, both fairness metrics and mitigation algorithms can be performed at various stages of the machine learning pipeline.  We recommend checking for bias as often as possible, using as many metrics are relevant for the application domain.  We also recommend incorporating bias detection in an automated continouus integration pipeline to ensure bias awareness as a software project evolves."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.5",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}