08-model-deployment-gradio.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7e97510a-f65b-4edd-b102-ea6cb0f673b9",
   "metadata": {},
   "source": [
    "# Deploy Machine Learning Model using Gradio\n",
    "\n",
    "![gradio-icon](https://github.com/gradio-app/gradio/raw/main/readme_files/gradio.svg)\n",
    "\n",
    "[Gradio](https://gradio.app/) is a Python library that simplifies the process of creating customizable and interactive interfaces for machine learning models. It provides a user-friendly way to build web-based UIs, allowing users to input data, visualize results, and interact with models in real-time. With Gradio, you can create interfaces for a wide range of machine learning tasks, such as image classification, text generation, sentiment analysis, and more. It supports popular deep learning frameworks like TensorFlow and PyTorch, as well as other machine learning libraries.\n",
    "\n",
    "Gradio handles the integration with the model and provides an easy-to-use API to link the interface components with the underlying machine learning code. It takes care of processing user inputs, passing them to the model, and displaying the results back to the user. The resulting interface is automatically hosted as a web application that can be accessed locally or deployed to a server for wider use. Gradio simplifies the process of sharing and showcasing machine learning models, making it easier for users to understand and interact with the underlying AI technology.\n",
    "\n",
    "Once you understand with Gradio, you could deploy and share your own app from a cloud hosting or use [HuggingFace Space](https://huggingface.co/spaces)!\n",
    "\n",
    "Read the full [documentation](https://gradio.app/docs/) and [examples](https://gradio.app/quickstart) on how to use Gradio with your app."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0f718421-758a-415d-b47e-d3816a38e021",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n",
      "\u001b[33mWARNING: You are using pip version 21.2.4; however, version 23.1.2 is available.\n",
      "You should consider upgrading via the '/usr/bin/python -m pip install --upgrade pip' command.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install -q gradio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "aff41c00-0602-45d2-88df-cf53a1231e77",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "ROOT_DIR = os.path.abspath('')\n",
    "PRETRAINED_MODEL = 'pretrained/simple-lightning-epoch100/resnet18_epoch99.ckpt'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1651dca-5f9d-470c-9184-29615d92244e",
   "metadata": {},
   "source": [
    "## Launching Simple Interface\n",
    "\n",
    "Before we integrate our model to the system, we need to understand how it works. In the example below, we're building a simple application to classify image. The [`Interface`](https://gradio.app/docs/#interface) will take `image_classifier` as a `Callable` function that takes an input type of [`Image`](https://gradio.app/docs/#image) and will return an output type of [`Label`](https://gradio.app/docs/#label) generated from the `dict`.\n",
    "\n",
    "You could also open the local URL to view it from a browser."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0f1513b4-3a17-4799-954e-4bcbfbd20e2c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on local URL:  http://127.0.0.1:7860\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7860/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import gradio as gr\n",
    "\n",
    "def image_classifier(inp):\n",
    "    return {'cat': 0.3, 'dog': 0.7}\n",
    "\n",
    "demo = gr.Interface(fn=image_classifier, inputs=\"image\", outputs=\"label\")\n",
    "demo.launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6dd0cee-55b5-4e25-9d74-0e559c3744f8",
   "metadata": {},
   "source": [
    "## Integrating the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3fcfce3-7068-4851-bbc4-211e9708c590",
   "metadata": {},
   "source": [
    "### Loading Checkpoint\n",
    "\n",
    "We've trained the ResNet18 model from the previous section using PyTorch Lightning. The checkpoint will contain all of the information from the training process, so we have to extract the weights for the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "83fc221f-6b50-4dd4-bc98-a3fc8b1ec9c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from src.models import ResNet18, BasicBlock\n",
    "\n",
    "model = ResNet18(3, 10)\n",
    "\n",
    "checkpoint = torch.load(PRETRAINED_MODEL)\n",
    "\n",
    "# The state dict will contains net.layer_name\n",
    "# Our model doesn't contains `net.` so we have to rename it\n",
    "state_dict = checkpoint['state_dict']\n",
    "for key in list(state_dict.keys()):\n",
    "    if 'net.' in key:\n",
    "        state_dict[key.replace('net.', '')] = state_dict[key]\n",
    "        del state_dict[key]\n",
    "\n",
    "model.load_state_dict(state_dict)\n",
    "model.eval()\n",
    "\n",
    "class_names = ['apple_pie', 'bibimbap', 'cannoli', 'edamame', 'falafel', 'french_toast', 'ice_cream', 'ramen', 'sushi', 'tiramisu']\n",
    "class_names.sort()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8cb00e79-e6b1-4e43-ad96-f61f47ea780f",
   "metadata": {},
   "source": [
    "### Launching a Classification Model\n",
    "\n",
    "Now we integrate the application with our model by adding the preprocessing, inference, and postprocessing steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "1d90e572-d5a6-4afa-9a75-b6e91c86b1fa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on local URL:  http://127.0.0.1:7861\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import gradio as gr\n",
    "\n",
    "import numpy as np\n",
    "import cv2\n",
    "from src.dataset import RGB_MEAN, RGB_STD, INPUT_SIZE\n",
    "from torchvision.transforms import transforms\n",
    "\n",
    "transformation_pipeline = transforms.Compose([\n",
    "    transforms.ToPILImage(),\n",
    "    transforms.CenterCrop(INPUT_SIZE),\n",
    "    transforms.ToTensor(),\n",
    "    transforms.Normalize(mean=RGB_MEAN, std=RGB_STD)\n",
    "])\n",
    "\n",
    "\n",
    "def preprocess_image(image: np.ndarray):\n",
    "    \"\"\"Preprocess the input image.\n",
    "\n",
    "    Note that the input image is in RGB mode.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    image: np.ndarray\n",
    "        Input image from callback.\n",
    "    \"\"\"\n",
    "\n",
    "    image = transformation_pipeline(image)\n",
    "    image = torch.unsqueeze(image, 0)\n",
    "\n",
    "    return image\n",
    "    \n",
    "\n",
    "def image_classifier(inp):\n",
    "    \"\"\"Image Classifier Function.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    inp: Optional[np.ndarray] = None\n",
    "        Input image from callback\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    Dict\n",
    "        A dictionary class names and its probability\n",
    "    \"\"\"\n",
    "\n",
    "    # If input not valid, return dummy data or raise error\n",
    "    if inp is None:\n",
    "        return {'cat': 0.3, 'dog': 0.7}\n",
    "\n",
    "    # preprocess\n",
    "    image = preprocess_image(inp)\n",
    "    image = image.to(dtype=torch.float32)\n",
    "\n",
    "    # inference\n",
    "    result = model(image)\n",
    "\n",
    "    # postprocess\n",
    "    result = torch.nn.functional.softmax(result, dim=1) # apply softmax\n",
    "    result = result[0].detach().numpy().tolist() # take the first batch\n",
    "    labeled_result = {name:score for name, score in zip(class_names, result)}\n",
    "\n",
    "    return labeled_result\n",
    "\n",
    "demo = gr.Interface(fn=image_classifier, inputs=\"image\", outputs=\"label\")\n",
    "demo.launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ecb7062-11ac-4391-b4f7-4f2985ef6345",
   "metadata": {},
   "source": [
    "## Custom Gradio App Block\n",
    "\n",
    "[Blocks](https://gradio.app/docs/#blocks) is Gradio's low-level API that allows you to create more custom web applications and demos than Interfaces (yet still entirely in Python).\n",
    "\n",
    "Let's build the same pipeline as before but using `Block` instead of an `Interface`. This application also contains example of images to quickly try the app without uploading an image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "11f36afc-3b13-446d-a685-2a0b206faa9e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "sample_files = os.listdir('samples')\n",
    "sample_files = [os.path.join('samples', path) for path in sample_files]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b21f455b-db72-4b1e-bb98-71d9354739ed",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on local URL:  http://127.0.0.1:7862\n",
      "\n",
      "To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7862/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import gradio as gr\n",
    "def update(name):\n",
    "    return f\"Welcome to Gradio, {name}!\"\n",
    "\n",
    "with gr.Blocks() as demo:\n",
    "    with gr.Row(): # Build a row\n",
    "        with gr.Column(): # build a column section as the first item\n",
    "            inp = gr.Image(label=\"image\", image_mode=\"RGB\") # build an image as the first column item\n",
    "            with gr.Row(): # build a row section as the second item\n",
    "                clear_btn = gr.Button(\"Clear\")\n",
    "                submit_btn = gr.Button(\"Submit\")\n",
    "\n",
    "        # build a label as the second item\n",
    "        out = gr.Label(label=\"prediction\", num_top_classes=3)\n",
    "\n",
    "        # Define buttons functionalities\n",
    "        submit_btn.click(fn=image_classifier, inputs=inp, outputs=out)\n",
    "        clear_btn.click(\n",
    "            lambda: (\n",
    "                gr.update(value=None),\n",
    "                gr.update(value=None),\n",
    "            ),\n",
    "            inputs=None,\n",
    "            outputs=[inp, out]\n",
    "        )\n",
    "\n",
    "    # Add examples\n",
    "    gr.Markdown(\"## Image Examples\")\n",
    "    gr.Examples(sample_files, inputs=[inp], label=\"Image Examples\")\n",
    "\n",
    "demo.launch()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "208f1c61-af3b-49e2-92c2-866a034f1a7f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}