Skip to content

Commit

Permalink
make getting-started notebook a markdown file showing APIs
Browse files Browse the repository at this point in the history
  • Loading branch information
@imatiach-msft
imatiach-msft committed Mar 3, 2022
1 parent 2dd3799 commit eeba203
Show file tree
Hide file tree
Showing 2 changed files with 114 additions and 47 deletions.
152 changes: 105 additions & 47 deletions notebooks/responsibleaidashboard/getting-started.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,213 +2,271 @@
"cells": [
{
"cell_type": "markdown",
"id": "8bf3a3e1",
"metadata": {},
"source": [
"# Getting Started"
]
},
{
"cell_type": "markdown",
"id": "e1491dd2",
"metadata": {},
"source": [
"This getting started notebook is an overview of the functionality in this repository. Note that this notebook is not runnable, it has a high-level overview of the APIs available and contains links to other notebooks in the repository."
]
},
{
"cell_type": "markdown",
"id": "2bf7fe3a",
"metadata": {},
"source": [
"## Installation"
]
},
{
"cell_type": "markdown",
"id": "07303d17",
"metadata": {},
"source": [
"Use the following `pip` commands to install the Responsible AI Toolbox."
"Use the following `pip` commands to install the Responsible AI Toolbox.\n",
"\n",
"If running in jupyter, please make sure to restart the jupyter kernel after installing."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "756f5f51",
"metadata": {},
"outputs": [],
"source": [
"!pip install raiwidgets"
]
},
{
"cell_type": "markdown",
"id": "60dca68e",
"metadata": {},
"source": [
"## Dependencies"
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "1bf0e821",
"metadata": {},
"source": [
"Please make sure to have the latest version of pandas installed if you are planning to use the error analysis component."
]
},
{
"cell_type": "markdown",
"id": "6f2ae1cd",
"metadata": {},
"outputs": [],
"source": [
"!pip install --upgrade pandas"
]
},
{
"cell_type": "markdown",
"id": "f46ea6a5",
"metadata": {},
"source": [
"## Overview & Setup"
]
},
{
"cell_type": "markdown",
"id": "da3c519c",
"metadata": {},
"source": [
"Responsible AI Toolbox is an interoperable, customizable tool that empowers machine learning practitioners to evaluate their models and data based on their place in the model lifecycle.\n",
"\n",
"Users may select components whose functionality supports their current objectives. First, import the relevant objects."
"Users may select components whose functionality supports their current objectives. First, the RAIInsights and ResponsibleAIDashboard must be imported."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "428f2b2d",
"metadata": {},
"outputs": [],
"source": [
"```Python\n",
"from raiwidgets import ResponsibleAIDashboard\n",
"from responsibleai import RAIInsights"
"from responsibleai import RAIInsights\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "9f152ac8",
"metadata": {},
"source": [
"It is necessary to initialize a RAIInsights object upon which the different components can be loaded. `task_type` holds the string `'regression'` or `'classification'` depending on the developer's purpose."
"Users will need to load a dataset, spit it into train and test datasets, and train a model on the training dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "88a8e0fd",
"metadata": {},
"outputs": [],
"source": [
"rai_insights = RAIInsights(model, train_data, test_data, target_feature, task_type, \n",
" categorical_features=['f1', 'f2', 'f3'])"
"It is necessary to initialize a RAIInsights object upon which the different components can be loaded. `task_type` holds the string `'regression'` or `'classification'` depending on the developer's purpose.\n",
"\n",
"Users can also specify categorical features via the `categorical_features` parameter."
]
},
{
"cell_type": "markdown",
"id": "c9433f32",
"metadata": {},
"source": [
"```Python\n",
"task_type = 'regression'\n",
"rai_insights = RAIInsights(model, train_data, test_data, target_feature, task_type)\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "c360364e",
"metadata": {},
"source": [
"The Interpretability and Error Analysis components can be added to the dashboard without any additional arguments:"
"The Interpretability and Error Analysis components can be added to the dashboard without any additional arguments.\n",
"\n",
"For an example, please see the [census classification model debugging notebook](https://github.com/microsoft/responsible-ai-toolbox/blob/main/notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb)."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "407571b5",
"metadata": {},
"outputs": [],
"source": [
"```Python\n",
"rai_insights.explainer.add()\n",
"rai_insights.error_analysis.add()"
"rai_insights.error_analysis.add()\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "714655b3",
"metadata": {},
"source": [
"The Causal Inferencing component must be added with a specification of the feature that would be changed as a treatment."
"The Causal Inferencing component must be added with a specification of the feature that would be changed as a treatment.\n",
"\n",
"For an example, please see the [diabetes decision making notebook](https://github.com/microsoft/responsible-ai-toolbox/blob/main/notebooks/responsibleaidashboard/responsibleaidashboard-diabetes-decision-making.ipynb)."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "70f6f73e",
"metadata": {},
"outputs": [],
"source": [
"rai_insights.causal.add(treatment_features=['f1', 'f2', 'f3'])"
"```Python\n",
"rai_insights.causal.add(treatment_features=['bmi', 'bp', 's2'])\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "7f5e8f45",
"metadata": {},
"source": [
"The Counterfactuals component takes arguments specifying the number of counterfactuals to generate, the list of columns containing continuous values, and the desired label of the counterfactuals."
]
},
{
"cell_type": "markdown",
"id": "308d93ad",
"metadata": {},
"source": [
"In a classification situation, `desired_class` must specify the classification that the generated counterfactuals would fall into."
"In a classification situation, `desired_class` must specify the classification that the generated counterfactuals would fall into.\n",
"\n",
"For an example, please see the [housing classification model debugging notebook](https://github.com/microsoft/responsible-ai-toolbox/blob/main/notebooks/responsibleaidashboard/responsibleaidashboard-housing-classification-model-debugging.ipynb)."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "c7c244f2",
"metadata": {},
"outputs": [],
"source": [
"rai_insights.counterfactual.add(total_CFs=20, desired_class='opposite', continuous_features=['f1', 'f2', 'f3'])"
"```Python\n",
"rai_insights.counterfactual.add(total_CFs=20, desired_class='opposite')\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "aa9ec639",
"metadata": {},
"source": [
"In a regression situation, `desired_range` must specify the minimum and maximum label that the generated counterfactuals can have."
"In a regression situation, `desired_range` must specify the minimum and maximum label that the generated counterfactuals can have.\n",
"For an example, please see the [diabetes regression model debugging notebook](https://github.com/microsoft/responsible-ai-toolbox/blob/main/notebooks/responsibleaidashboard/responsibleaidashboard-diabetes-regression-model-debugging.ipynb)."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "824449d9",
"metadata": {},
"outputs": [],
"source": [
"rai_insights.counterfactual.add(total_CFs=20, desired_range=[10, 20], continuous_features=['f1', 'f2', 'f3'])"
"```Python\n",
"rai_insights.counterfactual.add(total_CFs=20, desired_range=[50, 120])\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "3031a740",
"metadata": {},
"source": [
"## Computing and Visualizing Insights"
]
},
{
"cell_type": "markdown",
"id": "b3a4aec0",
"metadata": {},
"source": [
"After loading the components into the RAIInsights object, it is necessary to calculate values relevant to them, such as model metrics and counterfactuals."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "c3e534a3",
"metadata": {},
"outputs": [],
"source": [
"rai_insights.compute()"
"```Python\n",
"rai_insights.compute()\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "d2e05195",
"metadata": {},
"source": [
"Once the values for each component have been computed, they can be displayed by loading the RAIInsights object into a ResponsibleAIDashboard."
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"id": "3188e42d",
"metadata": {},
"outputs": [],
"source": [
"ResponsibleAIDashboard(rai_insights)"
"```Python\n",
"ResponsibleAIDashboard(rai_insights)\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "17df1a35",
"metadata": {},
"source": [
"## Learn More"
]
},
{
"cell_type": "markdown",
"id": "10b56cde",
"metadata": {},
"source": [
"Visit the [GitHub](https://github.com/microsoft/responsible-ai-widgets) of Responsible AI Toolbox for more details, and take this [dashboard tour](./tour.ipynb) for an explanation of the different parts of each component."
Expand All @@ -231,7 +289,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
"version": "3.7.11"
}
},
"nbformat": 4,
Expand Down
9 changes: 9 additions & 0 deletions notebooks/test_notebooks.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -255,3 +255,12 @@ def test_responsibleaidashboard_multiclass_dnn_model_debugging():

test_values = {}
assay_one_notebook(nb_path, nb_name, test_values)


@pytest.mark.notebooks
def test_responsibleaidashboard_getting_started():
nb_path = RESPONSIBLEAIDASHBOARD
nb_name = "getting-started"

test_values = {}
assay_one_notebook(nb_path, nb_name, test_values)

0 comments on commit eeba203

Please sign in to comment.