notebooks/3_model_serving_data_validation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Data Validation in Model Serving"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "In this notebook, we will go through the process of validating data in the serving phase of the data science pipeline."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/NatanMish/data_validation/blob/main/notebooks/3_model_serving_data_validation.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Install the required packages and import them to the notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!pip install pydantic pandas hypothesis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Let's remind ourselves the architecture of a modern machine learning model:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "<div>\n",
    "<img src=\"https://raw.githubusercontent.com/NatanMish/data_validation/main/notebooks/serving_diagram.png\" width=\"1000\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "So we should expect to receive the input for our model via a REST API in, and return the prediction or classification accordingly. An object arriving in the REST API can be a JSON object, but we can also receive a CSV file, txt file etc. For the purposes of this tutorial, we will assume that the input is a JSON object, and we receive it in batches. Let's create a few model inputs straight from the test data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "test_house_data = pd.read_csv('https://github.com/NatanMish/data_validation/blob/a77b247b25c6622ce0c8f8cbc505228161c31a3c/data/test.csv?raw=true')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "feature_names = ['YearBuilt', 'LotFrontage', 'GarageArea', 'OverallQual', 'OverallCond', 'MSZoning','TotalBsmtSF']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# we'll choose one record from the test data\n",
    "input_dict = test_house_data[feature_names].loc[1108].to_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "input_dict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "We can already see that one of the features is missing. If our model is not made to handle missing values, it will break and no one wants that."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "<div>\n",
    "<img src=\"https://miro.medium.com/max/959/1*WNd3LXOi5xlDbitxsIARyw.png\" width=\"500\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Pydantic allows data validation and settings management using python type annotations. It enforces type hints at runtime, and provides user-friendly errors when data is invalid. Many great and loved projects are using Pydantic extensively, even the Jupyter project for notebooks we are using right now!\n",
    "Let's see how we can use it to validate our model inputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# The most basic building block of Pydantic is the BaseModel class. We can use it to define our custom models that \n",
    "# define the structure of our objects:\n",
    "class Input(BaseModel):\n",
    "    YearBuilt: int\n",
    "    LotFrontage: int\n",
    "    GarageArea: float\n",
    "    OverallQual: int\n",
    "    OverallCond: int\n",
    "    MSZoning: str\n",
    "    TotalBsmtSF: float"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# We can now use the Input class, and create a Input object from our sample record like so:\n",
    "model_input_object = Input(\n",
    "    YearBuilt=input_dict['YearBuilt'],\n",
    "    LotFrontage=input_dict['LotFrontage'],\n",
    "    GarageArea=input_dict['GarageArea'],\n",
    "    OverallQual=input_dict['OverallQual'],\n",
    "    OverallCond=input_dict['OverallCond'],\n",
    "    MSZoning=input_dict['MSZoning'],\n",
    "    TotalBsmtSF=input_dict['TotalBsmtSF']\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "As we can see, the Input object raised an error while validating because one of the features is missing.\n",
    "Let's see how we can adjust the model, so it can handle missing values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# What's cool about Pydantic is that it allows leveraging the built-in Python typing definitions.\n",
    "from typing import Optional, Any\n",
    "\n",
    "class Input(BaseModel):\n",
    "    YearBuilt: int\n",
    "    LotFrontage: Optional[Any]\n",
    "    GarageArea: float\n",
    "    OverallQual: int\n",
    "    OverallCond: int\n",
    "    MSZoning: str\n",
    "    TotalBsmtSF: float"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Instead of specifying each field one by one, we can use the **kwargs syntax to specify a list of fields:\n",
    "input_object = Input(**input_dict)\n",
    "inpout_object_2 = Input.parse_obj(input_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# And this is how our object looks like:\n",
    "input_object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Here are a few other cool features of Pydantic:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### 1. Recursive models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Pydantic supports recursive models. This means that we can define a model that contains other models. For example, we can define a model that contains a batch of Input objects:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from typing import List\n",
    "\n",
    "class InputBatch(BaseModel):\n",
    "    inputs: List[Input]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# We'll grab a sample of the data:\n",
    "data_sample_dict = test_house_data[feature_names].sample(n=5).to_dict('index')\n",
    "data_sample_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# And we can create a InputBatch object from our sample data:\n",
    "inputs_list = [Input(**input_dict) for input_dict in data_sample_dict.values()]\n",
    "input_batch = InputBatch(inputs=inputs_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# And we can see that our object looks like this:\n",
    "input_batch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### 2. Enums"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Pydantic supports enumerations. This means that we can define a model that has a set of predefined values. For example, we can define a model that has a set of values for the MSZoning feature:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from enum import Enum\n",
    "\n",
    "class MSZoning(str, Enum):\n",
    "    C = 'C (all)'\n",
    "    FV = 'FV'\n",
    "    RH = 'RH'\n",
    "    RL = 'RL'\n",
    "    RM = 'RM'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# We can then include the enum in our model:\n",
    "class Input(BaseModel):\n",
    "    YearBuilt: int\n",
    "    LotFrontage: Optional[float]\n",
    "    GarageArea: float\n",
    "    OverallQual: int\n",
    "    OverallCond: int\n",
    "    MSZoning: MSZoning\n",
    "    TotalBsmtSF: float"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### 3. Custom base models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "When developing a model you might end up referencing the model's field names many times in many different places using its literal name, for example:\n",
    "`year_built = model_input_object.YearBuilt`\n",
    "in the case that the field name changes in the future, you might have to update all the places that use the literal name which could be a pain.\n",
    "Luckily, we can define an extended base model that allows us to use the field name directly. Then we can create an enum class that holds the field names and use that whenever we set or get a specific field. If the field name changes in the source, we only have to update the enum class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "input_batch.inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "class ExtendedBaseModel(BaseModel):\n",
    "    def __getitem__(self, item):\n",
    "        return getattr(self, item)\n",
    "\n",
    "    def __setitem__(self, item, value):\n",
    "        return setattr(self, item, value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# by using the extended base model, we can use a bracket notation to access the field values:\n",
    "class Input(ExtendedBaseModel):\n",
    "    YearBuilt: int\n",
    "    LotFrontage: float\n",
    "    GarageArea: float\n",
    "    OverallQual: int\n",
    "    OverallCond: int\n",
    "    MSZoning: MSZoning\n",
    "    TotalBsmtSF: float\n",
    "\n",
    "input_object = Input(**input_dict)\n",
    "# now we can use a bracket notation to access the field values:\n",
    "input_object['YearBuilt']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Let's create an enum class that holds the field names:\n",
    "class InputFieldNames(str, Enum):\n",
    "    YearBuilt = 'YearBuilt'\n",
    "    LotFrontage = 'LotFrontage'\n",
    "    GarageArea = 'GarageArea'\n",
    "    OverallQual = 'OverallQual'\n",
    "    OverallCond = 'OverallCond'\n",
    "    MSZoning = 'MSZoning'\n",
    "    TotalBsmtSF = 'TotalBsmtSF'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# and then we can access the field like so:\n",
    "input_object[InputFieldNames.YearBuilt.value]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### 4. Catch invalid data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "In the case that an invalid model input arrives, we can catch it, display a warning message, isolate the record and then continue without breaking the model. This can be done using a try except block. This is a common and valid practice in Python called EAFP - \"easier to ask for forgiveness than permission\" which might not be as well recieved in other languages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from pydantic import ValidationError\n",
    "\n",
    "class Input(ExtendedBaseModel):\n",
    "    YearBuilt: int\n",
    "    LotFrontage: int\n",
    "    GarageArea: float\n",
    "    OverallQual: int\n",
    "    OverallCond: int\n",
    "    MSZoning: MSZoning\n",
    "    TotalBsmtSF: float\n",
    "\n",
    "try:\n",
    "    model_input_object = Input(**input_dict)\n",
    "except ValidationError as e:\n",
    "    print(f'Record: {input_dict}')\n",
    "    print(f'Bad data: {e.errors()}')\n",
    "\n",
    "    # isolate the record and then continue"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### 5. Custom validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from pydantic import validator\n",
    "# Let's create a model input class with a custom validation function that checks that the garage area is greater than 1000\n",
    "class Input(ExtendedBaseModel):\n",
    "    YearBuilt: int\n",
    "    LotFrontage: Optional[float]\n",
    "    GarageArea: float\n",
    "    OverallQual: int\n",
    "    OverallCond: int\n",
    "    MSZoning: MSZoning\n",
    "    TotalBsmtSF: float\n",
    "\n",
    "    @validator('GarageArea')\n",
    "    def check_garage_area(v):\n",
    "        if v % 5 != 0:\n",
    "            raise ValueError('Garage area must be divisible by 5')\n",
    "        return v\n",
    "    \n",
    "    @validator(\n",
    "        \"LotFrontage\",\n",
    "        \"GarageArea\",\n",
    "        pre=False,\n",
    "        each_item=True,\n",
    "    )\n",
    "    def set_metrics_precision(cls, v):\n",
    "        \"\"\"Round all figures to 2 decimal places\"\"\"\n",
    "        return round(v, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# As we can see by the value, the validator should alert us for this record:\n",
    "input_dict['GarageArea']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "input_object = Input(**input_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "#### Exercise 1 - Pydantic models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Create a model class that represents the model input. Not all the checks requested were shown above, for some of them you'll need to have a quick search in the Pydantic documentation. The model class should have the following fields:\n",
    "1. YearBuilt - int - required - non nullable\n",
    "2. Fireplaces - int - optional - nullable\n",
    "3. FireplaceQu - FireplaceQu - optional - nullable - enum values of 'Ex', 'Gd', 'TA', 'Fa', 'Po'. If Fireplaces is greater than 0, FireplaceQu is required.\n",
    "4. Make the input model object immutable so that once it is created, it cannot be changed.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "class FireplaceQu(str, Enum):\n",
    "    <YOUR CODE HERE>\n",
    "\n",
    "class Input(BaseModel):\n",
    "    <YOUR CODE HERE>\n",
    "\n",
    "    @validator(\"FireplaceQu\")\n",
    "    @classmethod\n",
    "    def validate_fire_places_quality_field(cls, field_value, values):\n",
    "        <YOUR CODE HERE>\n",
    "\n",
    "    class Config:\n",
    "        <YOUR CODE HERE>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Let's create a few records and see if the validation works, this is a valid record:\n",
    "input_dict = {\n",
    "    'YearBuilt': '1901',\n",
    "    'Fireplaces': '0',\n",
    "    'FireplaceQu': 'Ex'\n",
    "}\n",
    "input_object = Input(**input_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# and this is an invalid record:\n",
    "input_dict = {\n",
    "    'YearBuilt': '1901',\n",
    "    'Fireplaces': '1',\n",
    "    'FireplaceQu': None\n",
    "}\n",
    "input_object = Input(**input_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "*Exercise solutions can be found in the exercise solutions file in the current directory.*\n",
    "\n",
    "### 6. JSON Serialization\n",
    "After we have received an object, and our model generated an output, we need to serialize it to JSON so we can send it back to the client. We'll start by creating an output model class to define how our output will look like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "from typing import Tuple\n",
    "from pydantic import confloat, Field\n",
    "from uuid import UUID, uuid4\n",
    "\n",
    "class ImportantFeature(BaseModel):\n",
    "    FeatureName: str\n",
    "    FeatureValue: Any\n",
    "    ImportanceScore: confloat(ge=0, le=1)\n",
    "\n",
    "class HousePricePrediction(BaseModel):\n",
    "    PredictionId: UUID = Field(default_factory=uuid4)\n",
    "    HousePrice: confloat(ge=0)\n",
    "    PredictionGenerationTime: datetime = Field(default_factory=datetime.now)\n",
    "    Explanation: Optional[List[ImportantFeature]]\n",
    "    ConfidenceInterval: Optional[Tuple[float, float]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# Let's create an output object:\n",
    "output_object = HousePricePrediction(\n",
    "    HousePrice=12345,\n",
    "    Explanation=[\n",
    "        ImportantFeature(FeatureName='YearBuilt', FeatureValue=1901, ImportanceScore=0.5),\n",
    "        ImportantFeature(FeatureName='Fireplaces', FeatureValue=0, ImportanceScore=0.5),\n",
    "        ImportantFeature(FeatureName='FireplaceQu', FeatureValue='Ex', ImportanceScore=0.5)\n",
    "    ],\n",
    "    ConfidenceInterval=(12000, 13000)\n",
    ")\n",
    "output_object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "When we want to send this object over an API to the client, we need to serialize it to JSON. We can do this using the `json()` method. This method is super useful because it can detect and handle different fields types and convert them to a JSON friendly format, usually better than just using `json.dumps(output_object)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "output_object.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "We can even define custom rules for serialization using the `json_encoders` parameter. This is a dictionary that maps the field names to functions that will be used to serialize the field."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "def encrypt_feature_value(feature: ImportantFeature, encryption_key=\"some_secret_key\"):\n",
    "    feature.FeatureValue = feature.FeatureValue.encode(encryption_key)\n",
    "    return feature\n",
    "\n",
    "class HousePricePrediction(BaseModel):\n",
    "    PredictionId: UUID = Field(default_factory=uuid4)\n",
    "    HousePrice: confloat(ge=0)\n",
    "    PredictionGenerationTime: datetime = Field(default_factory=datetime.now)\n",
    "    Explanation: Optional[List[ImportantFeature]]\n",
    "    ConfidenceInterval: Optional[Tuple[float, float]]\n",
    "\n",
    "    class Config:\n",
    "        json_encoders = {\n",
    "            'prediction_generation_time': datetime.isoformat,\n",
    "            'explanation': lambda important_features: [encrypt_feature_value(feature_explanation) for feature_explanation in important_features]\n",
    "        }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 7. Hypothesis plugin\n",
    "\n",
    "Hypothesis is a Python package used for property based testing. We can use it in combination with Pydantic as a tool for generating random data and testing the validity of a data model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from hypothesis import given, strategies as st\n",
    "\n",
    "@given(st.builds(HousePricePrediction))\n",
    "def test_property(instance):\n",
    "    # Hypothesis calls this test function many times with varied Models,\n",
    "    # so you can write a test that should pass given *any* instance.\n",
    "    assert len(str(instance.PredictionId)) == 36\n",
    "    assert instance.HousePrice >= 0\n",
    "    assert instance.PredictionGenerationTime is not None\n",
    "\n",
    "test_property()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### 8. Base settings\n",
    "One of the most useful features in Pydantic is the `BaseSettings` class. This class allows for centralising config keys and variables that are used throughout the project. The feature allows three different ways to parse environment variables into the object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['ENCRYPTION_KEY'] = \"some_secret_key\"\n",
    "os.environ['MY_API_KEY'] = \"my_api_key\"\n",
    "os.environ['REDIS_URL'] = \"redis://redis:6379\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from pydantic import BaseSettings, RedisDsn\n",
    "class ProjectConfig(BaseSettings):\n",
    "    RedisUrl: RedisDsn\n",
    "    ApiKey: str = Field(..., env='MY_API_KEY')\n",
    "    ENCRYPTION_KEY: str\n",
    "\n",
    "    class Config:\n",
    "        fields = {\n",
    "            'RedisUrl': {\n",
    "                'env': ['REDIS_URL']\n",
    "            }\n",
    "        }\n",
    "        case_sensitive = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# This is how the config object looks like:\n",
    "ProjectConfig().dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Exercise 2 - Constrained and Strict Field Types\n",
    "In this exercise we will explore what the constraints and strict field types in Pydantic offer us.\n",
    "Create a `Input` model class with the following constraints:\n",
    "1. Amenities is a list of strings, with no more than 5 items.\n",
    "2. HousePrice is a non-coercable(strict) integer between 0 and 10000000, and a multiple of 100.\n",
    "3. PoolBool is a strict boolean, i.e. it must be either `True` or `False`.\n",
    "4. MSZoning is string with max of 20 characters, turns all letters to lower case and must have the word `zone` in it (hint: use regex)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from pydantic import <YOUR CODE HERE>\n",
    "class Input(BaseModel):\n",
    "    <YOUR CODE HERE>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# This Input model class is valid:\n",
    "Input(\n",
    "  Amenities=['Amenity1', 'Amenity2', 'Amenity3', 'Amenity4', 'Amenity5'],\n",
    "  HousePrice=125000,\n",
    "  PoolBool=True,\n",
    "  MSZoning='1ZONE!'\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# This Input model class is invalid:\n",
    "Input(\n",
    "  Amenities=['Amenity1', 'Amenity2', 'Amenity3', 'Amenity4', 'Amenity5', 'Amenity6'],\n",
    "  HousePrice=1250,\n",
    "  PoolBool=1,\n",
    "  MSZoning='1Zne'\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.9"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}