use-cases/nlp/classification/keras/code/keras_nb.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text classification using Keras with Neptune tracking\n",
    "\n",
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/neptune-ai/examples/blob/main/use-cases/nlp/classification/keras/code/keras_nb.ipynb\">\n",
    "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\"/>\n",
    "</a><a target=\"_blank\" href=\"https://github.com/neptune-ai/examples/blob/main/use-cases/nlp/classification/keras/code/keras_nb.ipynb\">\n",
    "  <img alt=\"Open in GitHub\" src=\"https://img.shields.io/badge/Open_in_GitHub-blue?logo=github&labelColor=black\">\n",
    "</a><a target=\"_blank\" href=\"https://app.neptune.ai/o/showcase/org/project-text-classification/runs/details?viewId=9827345e-70b8-49ba-bf0d-c67946b18c78&detailsTab=dashboard&dashboardId=98274451-09a3-49e0-8204-4750716207bc&shortId=TXTCLF-394\"> \n",
    "  <img alt=\"Explore in Neptune\" src=\"https://neptune.ai/wp-content/uploads/2024/01/neptune-badge.svg\">\n",
    "</a>\n",
    "\n",
    "Notebook inspired from https://keras.io/examples/nlp/text_classification_from_scratch/\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (Neptune) Install the neptune-notebooks widget (optional)\n",
    "The neptune-notebooks jupyter extension lets you version, manage and share notebook checkpoints in your projects, without leaving your notebook.  \n",
    "[Read the docs](https://docs.neptune.ai/integrations-and-supported-tools/ide-and-notebooks/jupyter-lab-and-jupyter-notebook)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install requirements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: neptune[tensorflow-keras] in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (1.3.1)\n",
      "Requirement already satisfied: numpy in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (1.23.5)\n",
      "Collecting numpy\n",
      "  Using cached numpy-1.24.3-cp38-cp38-win_amd64.whl (14.9 MB)\n",
      "Requirement already satisfied: pydot in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (1.4.2)\n",
      "Requirement already satisfied: tensorflow in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (2.12.0)\n",
      "Requirement already satisfied: graphviz in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (0.20.1)\n",
      "Requirement already satisfied: GitPython>=2.0.8 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (3.1.31)\n",
      "Requirement already satisfied: Pillow>=1.1.6 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (9.4.0)\n",
      "Requirement already satisfied: PyJWT in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (2.6.0)\n",
      "Requirement already satisfied: boto3>=1.16.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (1.24.59)\n",
      "Requirement already satisfied: bravado<12.0.0,>=11.0.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (11.0.3)\n",
      "Requirement already satisfied: click>=7.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (8.1.3)\n",
      "Requirement already satisfied: future>=0.17.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (0.18.2)\n",
      "Requirement already satisfied: oauthlib>=2.1.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (3.2.2)\n",
      "Requirement already satisfied: packaging in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (22.0)\n",
      "Requirement already satisfied: pandas in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (1.5.2)\n",
      "Requirement already satisfied: pipreqs>=0.4 in c:\\users\\siddh\\appdata\\roaming\\python\\python38\\site-packages (from neptune[tensorflow-keras]) (0.4.13)\n",
      "Requirement already satisfied: psutil in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (5.9.0)\n",
      "Requirement already satisfied: requests>=2.20.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (2.28.1)\n",
      "Requirement already satisfied: requests-oauthlib>=1.0.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (1.3.1)\n",
      "Requirement already satisfied: six>=1.12.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (1.16.0)\n",
      "Requirement already satisfied: swagger-spec-validator>=2.7.4 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (3.0.3)\n",
      "Requirement already satisfied: urllib3 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (1.26.13)\n",
      "Requirement already satisfied: websocket-client!=1.0.0,>=0.35.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (1.4.2)\n",
      "Requirement already satisfied: neptune-tensorflow-keras in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from neptune[tensorflow-keras]) (2.1.1)\n",
      "Requirement already satisfied: pyparsing>=2.1.4 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from pydot) (3.0.9)\n",
      "Requirement already satisfied: tensorflow-intel==2.12.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow) (2.12.0)\n",
      "Requirement already satisfied: absl-py>=1.0.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (1.3.0)\n",
      "Requirement already satisfied: astunparse>=1.6.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (1.6.3)\n",
      "Requirement already satisfied: flatbuffers>=2.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (22.12.6)\n",
      "Requirement already satisfied: gast<=0.4.0,>=0.2.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (0.4.0)\n",
      "Requirement already satisfied: google-pasta>=0.1.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (0.2.0)\n",
      "Requirement already satisfied: h5py>=2.9.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (3.7.0)\n",
      "Requirement already satisfied: jax>=0.3.15 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (0.4.12)\n",
      "Requirement already satisfied: libclang>=13.0.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (14.0.6)\n",
      "Requirement already satisfied: opt-einsum>=2.3.2 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (3.3.0)\n",
      "Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (4.23.3)\n",
      "Requirement already satisfied: setuptools in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (67.8.0)\n",
      "Requirement already satisfied: termcolor>=1.1.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (2.1.1)\n",
      "Requirement already satisfied: typing-extensions>=3.6.6 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (4.4.0)\n",
      "Requirement already satisfied: wrapt<1.15,>=1.11.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (1.14.1)\n",
      "Requirement already satisfied: grpcio<2.0,>=1.24.3 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (1.51.1)\n",
      "Requirement already satisfied: tensorboard<2.13,>=2.12 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (2.12.3)\n",
      "Requirement already satisfied: tensorflow-estimator<2.13,>=2.12.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (2.12.0)\n",
      "Requirement already satisfied: keras<2.13,>=2.12.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (2.12.0)\n",
      "Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorflow-intel==2.12.0->tensorflow) (0.29.0)\n",
      "Requirement already satisfied: botocore<1.28.0,>=1.27.59 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from boto3>=1.16.0->neptune[tensorflow-keras]) (1.27.59)\n",
      "Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from boto3>=1.16.0->neptune[tensorflow-keras]) (0.10.0)\n",
      "Requirement already satisfied: s3transfer<0.7.0,>=0.6.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from boto3>=1.16.0->neptune[tensorflow-keras]) (0.6.0)\n",
      "Requirement already satisfied: bravado-core>=5.16.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from bravado<12.0.0,>=11.0.0->neptune[tensorflow-keras]) (5.17.1)\n",
      "Requirement already satisfied: msgpack in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from bravado<12.0.0,>=11.0.0->neptune[tensorflow-keras]) (1.0.4)\n",
      "Requirement already satisfied: python-dateutil in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from bravado<12.0.0,>=11.0.0->neptune[tensorflow-keras]) (2.8.2)\n",
      "Requirement already satisfied: pyyaml in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from bravado<12.0.0,>=11.0.0->neptune[tensorflow-keras]) (6.0)\n",
      "Requirement already satisfied: simplejson in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from bravado<12.0.0,>=11.0.0->neptune[tensorflow-keras]) (3.18.4)\n",
      "Requirement already satisfied: monotonic in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from bravado<12.0.0,>=11.0.0->neptune[tensorflow-keras]) (1.6)\n",
      "Requirement already satisfied: colorama in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from click>=7.0->neptune[tensorflow-keras]) (0.4.5)\n",
      "Requirement already satisfied: gitdb<5,>=4.0.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from GitPython>=2.0.8->neptune[tensorflow-keras]) (4.0.10)\n",
      "Requirement already satisfied: docopt in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from pipreqs>=0.4->neptune[tensorflow-keras]) (0.6.2)\n",
      "Requirement already satisfied: yarg in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from pipreqs>=0.4->neptune[tensorflow-keras]) (0.1.9)\n",
      "Requirement already satisfied: charset-normalizer<3,>=2 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from requests>=2.20.0->neptune[tensorflow-keras]) (2.1.1)\n",
      "Requirement already satisfied: idna<4,>=2.5 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from requests>=2.20.0->neptune[tensorflow-keras]) (3.4)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from requests>=2.20.0->neptune[tensorflow-keras]) (2022.12.7)\n",
      "Requirement already satisfied: jsonschema in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (4.17.3)\n",
      "Requirement already satisfied: pytz>=2020.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from pandas->neptune[tensorflow-keras]) (2022.6)\n",
      "Requirement already satisfied: wheel<1.0,>=0.23.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from astunparse>=1.6.0->tensorflow-intel==2.12.0->tensorflow) (0.38.4)\n",
      "Requirement already satisfied: jsonref in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from bravado-core>=5.16.1->bravado<12.0.0,>=11.0.0->neptune[tensorflow-keras]) (1.0.1)\n",
      "Requirement already satisfied: smmap<6,>=3.0.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from gitdb<5,>=4.0.1->GitPython>=2.0.8->neptune[tensorflow-keras]) (5.0.0)\n",
      "Requirement already satisfied: ml-dtypes>=0.1.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jax>=0.3.15->tensorflow-intel==2.12.0->tensorflow) (0.2.0)\n",
      "Requirement already satisfied: scipy>=1.7 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jax>=0.3.15->tensorflow-intel==2.12.0->tensorflow) (1.10.1)\n",
      "Requirement already satisfied: importlib-metadata>=4.6 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jax>=0.3.15->tensorflow-intel==2.12.0->tensorflow) (4.13.0)\n",
      "Requirement already satisfied: google-auth<3,>=1.6.3 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (2.15.0)\n",
      "Requirement already satisfied: google-auth-oauthlib<1.1,>=0.5 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (1.0.0)\n",
      "Requirement already satisfied: markdown>=2.6.8 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (3.3.7)\n",
      "Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (0.7.1)\n",
      "Requirement already satisfied: werkzeug>=1.0.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (2.3.4)\n",
      "Requirement already satisfied: attrs>=17.4.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (22.1.0)\n",
      "Requirement already satisfied: importlib-resources>=1.4.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (5.10.1)\n",
      "Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (1.3.10)\n",
      "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (0.19.2)\n",
      "Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (4.2.4)\n",
      "Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (0.2.8)\n",
      "Requirement already satisfied: rsa<5,>=3.1.4 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (4.9)\n",
      "Requirement already satisfied: zipp>=0.5 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from importlib-metadata>=4.6->jax>=0.3.15->tensorflow-intel==2.12.0->tensorflow) (3.11.0)\n",
      "Requirement already satisfied: fqdn in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (1.5.1)\n",
      "Requirement already satisfied: isoduration in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (20.11.0)\n",
      "Requirement already satisfied: jsonpointer>1.13 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (2.3)\n",
      "Requirement already satisfied: rfc3339-validator in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (0.1.4)\n",
      "Requirement already satisfied: rfc3987 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (1.3.8)\n",
      "Requirement already satisfied: uri-template in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (1.2.0)\n",
      "Requirement already satisfied: webcolors>=1.11 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (1.12)\n",
      "Requirement already satisfied: MarkupSafe>=2.1.1 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from werkzeug>=1.0.1->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (2.1.1)\n",
      "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.13,>=2.12->tensorflow-intel==2.12.0->tensorflow) (0.4.8)\n",
      "Requirement already satisfied: arrow>=0.15.0 in c:\\users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages (from isoduration->jsonschema->swagger-spec-validator>=2.7.4->neptune[tensorflow-keras]) (1.2.3)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "! pip install -U neptune[tensorflow-keras] numpy pydot tensorflow graphviz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create utility functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def extract_files(source: str, destination: str) -> None:\n",
    "    \"\"\"Extracts files from the source archive to the destination path\n",
    "\n",
    "    Args:\n",
    "        source (str): Archive file path\n",
    "        destination (str): Extract destination path\n",
    "    \"\"\"\n",
    "\n",
    "    import tarfile\n",
    "\n",
    "    print(\"Extracting data...\")\n",
    "    with tarfile.open(source) as f:\n",
    "        f.extractall(destination)\n",
    "\n",
    "\n",
    "def prep_data(imdb_folder: str, dest_path: str) -> None:\n",
    "    \"\"\"Removes unnecessary folders/files and renames source folder\n",
    "\n",
    "    Args:\n",
    "        imdb_folder (str): Path of the aclImdb folder\n",
    "        dest_name (str): Destination folder to which the aclImdb folder has to be renamed to\n",
    "    \"\"\"\n",
    "    import os\n",
    "    import shutil\n",
    "\n",
    "    shutil.rmtree(f\"{imdb_folder}/train/unsup\")\n",
    "    os.remove(f\"{imdb_folder.rsplit('/', maxsplit=1)[0]}/aclImdb_v1.tar.gz\")\n",
    "\n",
    "    if os.path.exists(dest_path):\n",
    "        shutil.rmtree(dest_path)\n",
    "\n",
    "    os.rename(imdb_folder, dest_path)\n",
    "    print(f\"{imdb_folder} renamed to {dest_path}\")\n",
    "\n",
    "\n",
    "def custom_standardization(input_data):\n",
    "    import string\n",
    "    import re\n",
    "\n",
    "    lowercase = tf.strings.lower(input_data)\n",
    "    stripped_html = tf.strings.regex_replace(lowercase, \"<br />\", \" \")\n",
    "    return tf.strings.regex_replace(stripped_html, f\"[{re.escape(string.punctuation)}]\", \"\")\n",
    "\n",
    "\n",
    "def vectorize_text(text, label):\n",
    "    text = tf.expand_dims(text, -1)\n",
    "    return vectorize_layer(text), label\n",
    "\n",
    "\n",
    "def build_model(model_params: dict, data_params: dict):\n",
    "    \"\"\"Accepts model and data parameters to build and compile a keras model\n",
    "\n",
    "    Args:\n",
    "        model_params (dict): Model parameters\n",
    "        data_params (dict): Data parameters\n",
    "\n",
    "    Returns:\n",
    "        A compiled keras model\n",
    "    \"\"\"\n",
    "\n",
    "    import tensorflow as tf\n",
    "    from tensorflow.keras import layers\n",
    "\n",
    "    # A integer input for vocab indices.\n",
    "    inputs = tf.keras.Input(shape=(None,), dtype=\"int64\")\n",
    "\n",
    "    # Next, we add a layer to map those vocab indices into a space of dimensionality\n",
    "    # 'embedding_dim'.\n",
    "    x = layers.Embedding(data_params[\"max_features\"], data_params[\"embedding_dim\"])(inputs)\n",
    "    x = layers.Dropout(model_params[\"dropout\"])(x)\n",
    "\n",
    "    # Conv1D + global max pooling\n",
    "    x = layers.Conv1D(\n",
    "        data_params[\"embedding_dim\"],\n",
    "        model_params[\"kernel_size\"],\n",
    "        padding=\"valid\",\n",
    "        activation=model_params[\"activation\"],\n",
    "        strides=model_params[\"strides\"],\n",
    "    )(x)\n",
    "    x = layers.Conv1D(\n",
    "        data_params[\"embedding_dim\"],\n",
    "        model_params[\"kernel_size\"],\n",
    "        padding=\"valid\",\n",
    "        activation=model_params[\"activation\"],\n",
    "        strides=model_params[\"strides\"],\n",
    "    )(x)\n",
    "    x = layers.GlobalMaxPooling1D()(x)\n",
    "\n",
    "    # We add a vanilla hidden layer:\n",
    "    x = layers.Dense(data_params[\"embedding_dim\"], activation=model_params[\"activation\"])(x)\n",
    "    x = layers.Dropout(model_params[\"dropout\"])(x)\n",
    "\n",
    "    # We project onto a single unit output layer, and squash it with a sigmoid:\n",
    "    predictions = layers.Dense(1, activation=\"sigmoid\", name=\"predictions\")(x)\n",
    "\n",
    "    keras_model = tf.keras.Model(inputs, predictions)\n",
    "\n",
    "    # Compile the model with binary crossentropy loss and an adam optimizer.\n",
    "    keras_model.compile(\n",
    "        loss=model_params[\"loss\"],\n",
    "        optimizer=model_params[\"optimizer\"],\n",
    "        metrics=model_params[\"metrics\"],\n",
    "    )\n",
    "\n",
    "    return keras_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) Import Neptune and initialize a project\n",
    "**A project is a collection of runs, models, and other metadata created by project members.** Typically you should create one project per machine learning task, to make it easy to compare runs that are connected to building certain kinds of ML model.  \n",
    "[Read the docs](https://docs.neptune.ai/you-should-know/core-concepts#project)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.environ[\"NEPTUNE_PROJECT\"] = \"common/project-text-classification\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You need a Neptune API token to be able to log to Neptune.\n",
    "Read how to get and use one [here](https://docs.neptune.ai/setup/setting_api_token/#setting-your-api-token).\n",
    "\n",
    "**or** \n",
    "\n",
    "If you don't have an API token, you can use the `neptune.ANONYMOUS_API_TOKEN` to log to a public project.  \n",
    "To log anonymously to a public project, set it as your environment variable as below:\n",
    "\n",
    "```python\n",
    "os.environ[\"NEPTUNE_API_TOKEN\"] = neptune.ANONYMOUS_API_TOKEN\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://app.neptune.ai/showcase/project-text-classification/\n"
     ]
    }
   ],
   "source": [
    "import neptune\n",
    "\n",
    "project = neptune.init_project()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data preparation\n",
    "We are using the IMDB sentiment analysis data available at https://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz. For the purposes of this demo, we've uploaded this data to S3 at https://neptune-examples.s3.us-east-2.amazonaws.com/data/text-classification/aclImdb_v1.tar.gz and will be downloading it from there."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) Track datasets using Neptune\n",
    "Neptune lets you track pointers to datasets, models, and other artifacts stored locally or in S3.  \n",
    "To use this, you will need to have your `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables set.  \n",
    "[Read the docs](https://docs.neptune.ai/how-to-guides/data-versioning)\n",
    "\n",
    "Since this dataset will be used among all the runs in the project, we track it at the project level.\n",
    "Read more about logging project-level metadata [here](https://docs.neptune.ai/logging/project_metadata/#logging-project-level-metadata).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "project[\"keras/data/files\"].track_files(\n",
    "    \"s3://neptune-examples/data/text-classification/aclImdb_v1.tar.gz\", wait=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) Download files from S3 using Neptune\n",
    "You can also download tracked files from S3 using Neptune, without having to write boilerplate boto3 code.\n",
    "Read the artifact API reference to know more: https://docs.neptune.ai/api/field_types/#download"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data...\n"
     ]
    }
   ],
   "source": [
    "print(\"Downloading data...\")\n",
    "project[\"keras/data/files\"].download(\"..\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "extract_files(source=\"../aclImdb_v1.tar.gz\", destination=\"..\")\n",
    "prep_data(\n",
    "    imdb_folder=\"../aclImdb\", dest_path=\"../data\"\n",
    ")  # If you get a permission error here, you can manually rename the `aclImdb` folder to `data`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (Neptune) Upload dataset sample to Neptune project\n",
    "In addition to tracking external files, you can also upload them directly to Neptune.\n",
    "Such uploaded files can be visualized directly in the Neptune app.  \n",
    "[Read more here](https://docs.neptune.ai/logging/files/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "base_namespace = \"keras/data/sample/\"\n",
    "\n",
    "project[base_namespace][\"train/pos\"].upload(\n",
    "    f\"../data/train/pos/{random.choice(os.listdir('../data/train/pos'))}\"\n",
    ")\n",
    "project[base_namespace][\"train/neg\"].upload(\n",
    "    f\"../data/train/neg/{random.choice(os.listdir('../data/train/neg'))}\"\n",
    ")\n",
    "project[base_namespace][\"test/pos\"].upload(\n",
    "    f\"../data/test/pos/{random.choice(os.listdir('../data/test/pos'))}\"\n",
    ")\n",
    "project[base_namespace][\"test/neg\"].upload(\n",
    "    f\"../data/test/neg/{random.choice(os.listdir('../data/test/neg'))}\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) Initialize a run\n",
    "**A run is a namespace inside a project where you log metadata.** Typically, you create a run every time you execute a script that does model training, re-training, or inference.  \n",
    "[Read the docs](https://docs.neptune.ai/you-should-know/core-concepts#run)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\siddh\\miniconda3\\envs\\py38\\lib\\site-packages\\neptune\\common\\warnings.py:62: NeptuneWarning: To avoid unintended consumption of logging hours during interactive sessions, the following monitoring options are disabled unless set to 'True' when initializing the run: 'capture_stdout', 'capture_stderr', and 'capture_hardware_metrics'.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://app.neptune.ai/showcase/project-text-classification/e/TXTCLF-394\n"
     ]
    }
   ],
   "source": [
    "run = neptune.init_run(\n",
    "    name=\"Keras text classification\",\n",
    "    tags=[\"keras\", \"notebook\"],\n",
    "    dependencies=\"requirements.txt\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (Neptune) Log data metadata to run\n",
    "You can log nested dictionaries to create custom nested namespaces.  \n",
    "[Read the docs](https://docs.neptune.ai/logging/methods/#essential-logging-methods)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_params = {\n",
    "    \"batch_size\": 16,\n",
    "    \"validation_split\": 0.2,\n",
    "    \"max_features\": 1500,\n",
    "    \"embedding_dim\": 128,\n",
    "    \"sequence_length\": 1000,\n",
    "    \"seed\": 42,\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run[\"data/params\"] = data_params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### # (Neptune) Track dataset at the run-level\n",
    "We can fetch the dataset from the project metadata and track it at the run level using the `fetch()` method.  \n",
    "[`fetch()` API reference](https://docs.neptune.ai/api/field_types/#fetch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run[\"data/files\"] = project[\"keras/data/files\"].fetch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate training, validation, and test datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 25000 files belonging to 2 classes.\n",
      "Using 20000 files for training.\n",
      "Using 5000 files for validation.\n",
      "Found 25000 files belonging to 2 classes.\n",
      "Number of batches in raw_train_ds: 1250\n",
      "Number of batches in raw_val_ds: 313\n",
      "Number of batches in raw_test_ds: 1563\n"
     ]
    }
   ],
   "source": [
    "raw_train_ds, raw_val_ds = tf.keras.preprocessing.text_dataset_from_directory(\n",
    "    \"../data/train\",\n",
    "    batch_size=data_params[\"batch_size\"],\n",
    "    validation_split=data_params[\"validation_split\"],\n",
    "    subset=\"both\",\n",
    "    seed=data_params[\"seed\"],\n",
    ")\n",
    "\n",
    "raw_test_ds = tf.keras.preprocessing.text_dataset_from_directory(\n",
    "    \"../data/test\", batch_size=data_params[\"batch_size\"]\n",
    ")\n",
    "\n",
    "print(f\"Number of batches in raw_train_ds: {raw_train_ds.cardinality()}\")\n",
    "print(f\"Number of batches in raw_val_ds: {raw_val_ds.cardinality()}\")\n",
    "print(f\"Number of batches in raw_test_ds: {raw_test_ds.cardinality()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clean data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.layers import TextVectorization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vectorize_layer = TextVectorization(\n",
    "    standardize=custom_standardization,\n",
    "    max_tokens=data_params[\"max_features\"],\n",
    "    output_mode=\"int\",\n",
    "    output_sequence_length=data_params[\"sequence_length\"],\n",
    ")\n",
    "\n",
    "text_ds = raw_train_ds.map(lambda x, y: x)\n",
    "vectorize_layer.adapt(text_ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vectorize data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Vectorize the data.\n",
    "train_ds = raw_train_ds.map(vectorize_text)\n",
    "val_ds = raw_val_ds.map(vectorize_text)\n",
    "test_ds = raw_test_ds.map(vectorize_text)\n",
    "\n",
    "# Do async prefetching / buffering of the data for best performance on GPU.\n",
    "train_ds = train_ds.cache().prefetch(buffer_size=10)\n",
    "val_ds = val_ds.cache().prefetch(buffer_size=10)\n",
    "test_ds = test_ds.cache().prefetch(buffer_size=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modelling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) Register a model and create a new model version\n",
    "With Neptune's model registry, you can store your ML models in a central location and collaboratively manage their lifecycle.  \n",
    "[Read the docs](https://docs.neptune.ai/how-to-guides/model-registry)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://app.neptune.ai/showcase/project-text-classification/m/TXTCLF-KER/v/TXTCLF-KER-18\n"
     ]
    }
   ],
   "source": [
    "from neptune.exceptions import NeptuneModelKeyAlreadyExistsError\n",
    "\n",
    "project_key = project[\"sys/id\"].fetch()\n",
    "model_key = \"KER\"\n",
    "\n",
    "try:\n",
    "    model = neptune.init_model(name=\"keras\", key=model_key)\n",
    "    model.stop()\n",
    "except NeptuneModelKeyAlreadyExistsError:\n",
    "    # If it already exists, we don't have to do anything.\n",
    "    pass\n",
    "\n",
    "model_version = neptune.init_model_version(model=f\"{project_key}-{model_key}\", name=\"keras\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build a model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_params = {\n",
    "    \"dropout\": 0.4,\n",
    "    \"strides\": 3,\n",
    "    \"activation\": \"relu\",\n",
    "    \"kernel_size\": 3,\n",
    "    \"loss\": \"binary_crossentropy\",\n",
    "    \"optimizer\": \"adam\",\n",
    "    \"metrics\": [\"accuracy\"],\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from neptune.utils import stringify_unsupported\n",
    "\n",
    "\n",
    "model_version[\"params/model\"] = run[\"training/model/params\"] = stringify_unsupported(model_params)\n",
    "model_version[\"params/data\"] = data_params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "keras_model = build_model(model_params, data_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (Neptune) Initialize the Neptune callback\n",
    "The Neptune–Keras integration logs the following metadata automatically:\n",
    "\n",
    "* Model summary\n",
    "* Parameters of the optimizer used for training the model\n",
    "* Parameters passed to Model.fit during the training\n",
    "* Current learning rate at every epoch\n",
    "* Hardware consumption and stdout/stderr output during training\n",
    "* Training code and Git information\n",
    "\n",
    "Read more about the Neptune–Keras integration here: https://docs.neptune.ai/integrations/keras/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from neptune.integrations.tensorflow_keras import NeptuneCallback\n",
    "\n",
    "neptune_callback = NeptuneCallback(run=run, log_model_diagram=False, log_on_batch=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_params = {\n",
    "    \"epochs\": 2,\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/2\n",
      "1250/1250 [==============================] - 57s 44ms/step - loss: 0.4888 - accuracy: 0.7413 - val_loss: 0.3748 - val_accuracy: 0.8292\n",
      "Epoch 2/2\n",
      "1250/1250 [==============================] - 50s 40ms/step - loss: 0.3276 - accuracy: 0.8630 - val_loss: 0.3354 - val_accuracy: 0.8526\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x132a9e9d700>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fit the model using the train and test datasets.\n",
    "keras_model.fit(\n",
    "    train_ds, validation_data=val_ds, epochs=training_params[\"epochs\"], callbacks=neptune_callback\n",
    ")\n",
    "# Training parameters are logged automatically to Neptune"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1563/1563 [==============================] - 30s 19ms/step - loss: 0.3240 - accuracy: 0.8592\n"
     ]
    }
   ],
   "source": [
    "# We save the accuracy of the  model to be able to evaluate it against the champion model in production later in the notebook\n",
    "_, curr_model_acc = keras_model.evaluate(test_ds, callbacks=neptune_callback)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) Associate run with model and vice-versa\n",
    "We can fetch metadata from the run's `sys` namespace and add those to the model_version to be able to link model versions with the runs that created them, and vice-versa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'id': 'TXTCLF-394', 'name': 'Keras text classification', 'url': 'https://app.neptune.ai/showcase/project-text-classification/e/TXTCLF-394'}\n"
     ]
    }
   ],
   "source": [
    "run_meta = {\n",
    "    \"id\": run[\"sys/id\"].fetch(),\n",
    "    \"name\": run[\"sys/name\"].fetch(),\n",
    "    \"url\": run.get_url(),\n",
    "}\n",
    "\n",
    "print(run_meta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_version[\"run\"] = run_meta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'id': 'TXTCLF-KER-18', 'name': 'keras', 'url': 'https://app.neptune.ai/showcase/project-text-classification/m/TXTCLF-KER/v/TXTCLF-KER-18'}\n"
     ]
    }
   ],
   "source": [
    "model_version_meta = {\n",
    "    \"id\": model_version[\"sys/id\"].fetch(),\n",
    "    \"name\": model_version[\"sys/name\"].fetch(),\n",
    "    \"url\": model_version.get_url(),\n",
    "}\n",
    "\n",
    "print(model_version_meta)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run[\"training/model/meta\"] = model_version_meta"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) Upload serialized model and model weights to Neptune"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_version[\"serialized_model\"] = keras_model.to_json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "keras_model.save_weights(\"model_weights.h5\")\n",
    "model_version[\"model_weights\"].upload(\"model_weights.h5\", wait=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) Update model stage\n",
    "We can update the model stage both in the app and through the API.  \n",
    "[Read the docs](https://docs.neptune.ai/model_registry/managing_stage/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_version.change_stage(\"staging\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (Neptune) Promote best model to production"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) Fetch current champion model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://app.neptune.ai/showcase/project-text-classification/m/TXTCLF-KER\n",
      "Shutting down background jobs, please wait a moment...\n",
      "Done!\n",
      "All 0 operations synced, thanks for waiting!\n",
      "Explore the metadata in the Neptune app:\n",
      "https://app.neptune.ai/showcase/project-text-classification/m/TXTCLF-KER/metadata\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sys/creation_time</th>\n",
       "      <th>sys/id</th>\n",
       "      <th>sys/model_id</th>\n",
       "      <th>sys/modification_time</th>\n",
       "      <th>sys/monitoring_time</th>\n",
       "      <th>sys/name</th>\n",
       "      <th>sys/owner</th>\n",
       "      <th>sys/ping_time</th>\n",
       "      <th>sys/running_time</th>\n",
       "      <th>sys/size</th>\n",
       "      <th>...</th>\n",
       "      <th>params/model/loss</th>\n",
       "      <th>params/model/metrics</th>\n",
       "      <th>params/model/optimizer</th>\n",
       "      <th>params/model/strides</th>\n",
       "      <th>params/optimizer</th>\n",
       "      <th>params/strides</th>\n",
       "      <th>run/id</th>\n",
       "      <th>run/name</th>\n",
       "      <th>run/url</th>\n",
       "      <th>serialized_model</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2023-06-21 09:15:31.867000+00:00</td>\n",
       "      <td>TXTCLF-KER-18</td>\n",
       "      <td>TXTCLF-KER</td>\n",
       "      <td>2023-06-21 09:19:11.183000+00:00</td>\n",
       "      <td>189</td>\n",
       "      <td>keras</td>\n",
       "      <td>siddhant.sadangi</td>\n",
       "      <td>2023-06-21 09:19:11.183000+00:00</td>\n",
       "      <td>219.303</td>\n",
       "      <td>1259443.0</td>\n",
       "      <td>...</td>\n",
       "      <td>binary_crossentropy</td>\n",
       "      <td>['accuracy']</td>\n",
       "      <td>adam</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>TXTCLF-394</td>\n",
       "      <td>Keras text classification</td>\n",
       "      <td>https://app.neptune.ai/showcase/project-text-c...</td>\n",
       "      <td>{\"class_name\": \"Functional\", \"config\": {\"name\"...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2023-06-21 08:53:28.305000+00:00</td>\n",
       "      <td>TXTCLF-KER-17</td>\n",
       "      <td>TXTCLF-KER</td>\n",
       "      <td>2023-06-21 09:02:31.250000+00:00</td>\n",
       "      <td>236</td>\n",
       "      <td>keras</td>\n",
       "      <td>siddhant.sadangi</td>\n",
       "      <td>2023-06-21 09:02:31.250000+00:00</td>\n",
       "      <td>542.915</td>\n",
       "      <td>1521870.0</td>\n",
       "      <td>...</td>\n",
       "      <td>binary_crossentropy</td>\n",
       "      <td>['accuracy']</td>\n",
       "      <td>adam</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>TXTCLF-392</td>\n",
       "      <td>Keras text classification</td>\n",
       "      <td>https://app.neptune.ai/showcase/project-text-c...</td>\n",
       "      <td>{\"class_name\": \"Functional\", \"config\": {\"name\"...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2023-06-21 08:48:18.085000+00:00</td>\n",
       "      <td>TXTCLF-KER-16</td>\n",
       "      <td>TXTCLF-KER</td>\n",
       "      <td>2023-06-21 09:02:31.179000+00:00</td>\n",
       "      <td>94</td>\n",
       "      <td>Untitled</td>\n",
       "      <td>siddhant.sadangi</td>\n",
       "      <td>2023-06-21 09:02:31.179000+00:00</td>\n",
       "      <td>375.367</td>\n",
       "      <td>4966326.0</td>\n",
       "      <td>...</td>\n",
       "      <td>binary_crossentropy</td>\n",
       "      <td>['accuracy']</td>\n",
       "      <td>adam</td>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>TXTCLF-390</td>\n",
       "      <td>Keras text classification</td>\n",
       "      <td>https://app.neptune.ai/showcase/project-text-c...</td>\n",
       "      <td>{\"class_name\": \"Functional\", \"config\": {\"name\"...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2023-06-21 08:28:08.005000+00:00</td>\n",
       "      <td>TXTCLF-KER-15</td>\n",
       "      <td>TXTCLF-KER</td>\n",
       "      <td>2023-06-21 08:50:22.446000+00:00</td>\n",
       "      <td>221</td>\n",
       "      <td>Untitled</td>\n",
       "      <td>siddhant.sadangi</td>\n",
       "      <td>2023-06-21 08:50:22.446000+00:00</td>\n",
       "      <td>649.209</td>\n",
       "      <td>2033591.0</td>\n",
       "      <td>...</td>\n",
       "      <td>binary_crossentropy</td>\n",
       "      <td>['accuracy']</td>\n",
       "      <td>adam</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>TXTCLF-388</td>\n",
       "      <td>Keras text classification</td>\n",
       "      <td>https://app.neptune.ai/showcase/project-text-c...</td>\n",
       "      <td>{\"class_name\": \"Functional\", \"config\": {\"name\"...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2023-03-07 13:12:09.772000+00:00</td>\n",
       "      <td>TXTCLF-KER-14</td>\n",
       "      <td>TXTCLF-KER</td>\n",
       "      <td>2023-03-07 13:13:38.372000+00:00</td>\n",
       "      <td>66</td>\n",
       "      <td>keras</td>\n",
       "      <td>siddhant.sadangi</td>\n",
       "      <td>2023-03-07 13:13:38.372000+00:00</td>\n",
       "      <td>88.596</td>\n",
       "      <td>2039449.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>adam</td>\n",
       "      <td>5.0</td>\n",
       "      <td>TXTCLF-385</td>\n",
       "      <td>Keras text classification</td>\n",
       "      <td>https://app.neptune.ai/showcase/project-text-c...</td>\n",
       "      <td>{\"class_name\": \"Functional\", \"config\": {\"name\"...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2023-03-07 12:46:03.101000+00:00</td>\n",
       "      <td>TXTCLF-KER-13</td>\n",
       "      <td>TXTCLF-KER</td>\n",
       "      <td>2023-03-07 12:52:19.437000+00:00</td>\n",
       "      <td>168</td>\n",
       "      <td>keras</td>\n",
       "      <td>siddhant.sadangi</td>\n",
       "      <td>2023-03-07 12:52:19.437000+00:00</td>\n",
       "      <td>376.314</td>\n",
       "      <td>1777305.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>adam</td>\n",
       "      <td>3.0</td>\n",
       "      <td>TXTCLF-383</td>\n",
       "      <td>Keras text classification</td>\n",
       "      <td>https://app.neptune.ai/showcase/project-text-c...</td>\n",
       "      <td>{\"class_name\": \"Functional\", \"config\": {\"name\"...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2023-03-07 12:29:02.563000+00:00</td>\n",
       "      <td>TXTCLF-KER-12</td>\n",
       "      <td>TXTCLF-KER</td>\n",
       "      <td>2023-06-21 08:38:04.570000+00:00</td>\n",
       "      <td>171</td>\n",
       "      <td>Untitled</td>\n",
       "      <td>siddhant.sadangi</td>\n",
       "      <td>2023-06-21 09:05:05.019000+00:00</td>\n",
       "      <td>2373.426</td>\n",
       "      <td>1777270.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>adam</td>\n",
       "      <td>2.0</td>\n",
       "      <td>TXTCLF-381</td>\n",
       "      <td>Keras text classification</td>\n",
       "      <td>https://app.neptune.ai/showcase/project-text-c...</td>\n",
       "      <td>{\"class_name\": \"Functional\", \"config\": {\"name\"...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2023-03-06 16:34:05.166000+00:00</td>\n",
       "      <td>TXTCLF-KER-11</td>\n",
       "      <td>TXTCLF-KER</td>\n",
       "      <td>2023-03-06 17:31:56.266000+00:00</td>\n",
       "      <td>658</td>\n",
       "      <td>keras</td>\n",
       "      <td>siddhant.sadangi</td>\n",
       "      <td>2023-03-06 17:31:56.266000+00:00</td>\n",
       "      <td>3470.926</td>\n",
       "      <td>10993269.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>adam</td>\n",
       "      <td>2.0</td>\n",
       "      <td>TXTCLF-379</td>\n",
       "      <td>Keras text classification</td>\n",
       "      <td>https://app.neptune.ai/showcase/project-text-c...</td>\n",
       "      <td>{\"class_name\": \"Functional\", \"config\": {\"name\"...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 sys/creation_time         sys/id sys/model_id  \\\n",
       "0 2023-06-21 09:15:31.867000+00:00  TXTCLF-KER-18   TXTCLF-KER   \n",
       "1 2023-06-21 08:53:28.305000+00:00  TXTCLF-KER-17   TXTCLF-KER   \n",
       "2 2023-06-21 08:48:18.085000+00:00  TXTCLF-KER-16   TXTCLF-KER   \n",
       "3 2023-06-21 08:28:08.005000+00:00  TXTCLF-KER-15   TXTCLF-KER   \n",
       "4 2023-03-07 13:12:09.772000+00:00  TXTCLF-KER-14   TXTCLF-KER   \n",
       "5 2023-03-07 12:46:03.101000+00:00  TXTCLF-KER-13   TXTCLF-KER   \n",
       "6 2023-03-07 12:29:02.563000+00:00  TXTCLF-KER-12   TXTCLF-KER   \n",
       "7 2023-03-06 16:34:05.166000+00:00  TXTCLF-KER-11   TXTCLF-KER   \n",
       "\n",
       "             sys/modification_time  sys/monitoring_time  sys/name  \\\n",
       "0 2023-06-21 09:19:11.183000+00:00                  189     keras   \n",
       "1 2023-06-21 09:02:31.250000+00:00                  236     keras   \n",
       "2 2023-06-21 09:02:31.179000+00:00                   94  Untitled   \n",
       "3 2023-06-21 08:50:22.446000+00:00                  221  Untitled   \n",
       "4 2023-03-07 13:13:38.372000+00:00                   66     keras   \n",
       "5 2023-03-07 12:52:19.437000+00:00                  168     keras   \n",
       "6 2023-06-21 08:38:04.570000+00:00                  171  Untitled   \n",
       "7 2023-03-06 17:31:56.266000+00:00                  658     keras   \n",
       "\n",
       "          sys/owner                    sys/ping_time  sys/running_time  \\\n",
       "0  siddhant.sadangi 2023-06-21 09:19:11.183000+00:00           219.303   \n",
       "1  siddhant.sadangi 2023-06-21 09:02:31.250000+00:00           542.915   \n",
       "2  siddhant.sadangi 2023-06-21 09:02:31.179000+00:00           375.367   \n",
       "3  siddhant.sadangi 2023-06-21 08:50:22.446000+00:00           649.209   \n",
       "4  siddhant.sadangi 2023-03-07 13:13:38.372000+00:00            88.596   \n",
       "5  siddhant.sadangi 2023-03-07 12:52:19.437000+00:00           376.314   \n",
       "6  siddhant.sadangi 2023-06-21 09:05:05.019000+00:00          2373.426   \n",
       "7  siddhant.sadangi 2023-03-06 17:31:56.266000+00:00          3470.926   \n",
       "\n",
       "     sys/size  ...    params/model/loss params/model/metrics  \\\n",
       "0   1259443.0  ...  binary_crossentropy         ['accuracy']   \n",
       "1   1521870.0  ...  binary_crossentropy         ['accuracy']   \n",
       "2   4966326.0  ...  binary_crossentropy         ['accuracy']   \n",
       "3   2033591.0  ...  binary_crossentropy         ['accuracy']   \n",
       "4   2039449.0  ...                  NaN                  NaN   \n",
       "5   1777305.0  ...                  NaN                  NaN   \n",
       "6   1777270.0  ...                  NaN                  NaN   \n",
       "7  10993269.0  ...                  NaN                  NaN   \n",
       "\n",
       "  params/model/optimizer  params/model/strides params/optimizer  \\\n",
       "0                   adam                   3.0              NaN   \n",
       "1                   adam                   3.0              NaN   \n",
       "2                   adam                   5.0              NaN   \n",
       "3                   adam                   3.0              NaN   \n",
       "4                    NaN                   NaN             adam   \n",
       "5                    NaN                   NaN             adam   \n",
       "6                    NaN                   NaN             adam   \n",
       "7                    NaN                   NaN             adam   \n",
       "\n",
       "   params/strides      run/id                   run/name  \\\n",
       "0             NaN  TXTCLF-394  Keras text classification   \n",
       "1             NaN  TXTCLF-392  Keras text classification   \n",
       "2             NaN  TXTCLF-390  Keras text classification   \n",
       "3             NaN  TXTCLF-388  Keras text classification   \n",
       "4             5.0  TXTCLF-385  Keras text classification   \n",
       "5             3.0  TXTCLF-383  Keras text classification   \n",
       "6             2.0  TXTCLF-381  Keras text classification   \n",
       "7             2.0  TXTCLF-379  Keras text classification   \n",
       "\n",
       "                                             run/url  \\\n",
       "0  https://app.neptune.ai/showcase/project-text-c...   \n",
       "1  https://app.neptune.ai/showcase/project-text-c...   \n",
       "2  https://app.neptune.ai/showcase/project-text-c...   \n",
       "3  https://app.neptune.ai/showcase/project-text-c...   \n",
       "4  https://app.neptune.ai/showcase/project-text-c...   \n",
       "5  https://app.neptune.ai/showcase/project-text-c...   \n",
       "6  https://app.neptune.ai/showcase/project-text-c...   \n",
       "7  https://app.neptune.ai/showcase/project-text-c...   \n",
       "\n",
       "                                    serialized_model  \n",
       "0  {\"class_name\": \"Functional\", \"config\": {\"name\"...  \n",
       "1  {\"class_name\": \"Functional\", \"config\": {\"name\"...  \n",
       "2  {\"class_name\": \"Functional\", \"config\": {\"name\"...  \n",
       "3  {\"class_name\": \"Functional\", \"config\": {\"name\"...  \n",
       "4  {\"class_name\": \"Functional\", \"config\": {\"name\"...  \n",
       "5  {\"class_name\": \"Functional\", \"config\": {\"name\"...  \n",
       "6  {\"class_name\": \"Functional\", \"config\": {\"name\"...  \n",
       "7  {\"class_name\": \"Functional\", \"config\": {\"name\"...  \n",
       "\n",
       "[8 rows x 38 columns]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with neptune.init_model(with_id=f\"{project_key}-KER\") as model:\n",
    "    model_versions_df = model.fetch_model_versions_table().to_pandas()\n",
    "model_versions_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "production_models = model_versions_df[model_versions_df[\"sys/stage\"] == \"production\"][\"sys/id\"]\n",
    "# assert (\n",
    "#     len(production_models) == 1\n",
    "# ), f\"Multiple model versions found in production: {production_models.values}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current champion model: TXTCLF-KER-17\n"
     ]
    }
   ],
   "source": [
    "prod_model_id = production_models.values[0]\n",
    "print(f\"Current champion model: {prod_model_id}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://app.neptune.ai/showcase/project-text-classification/m/TXTCLF-KER/v/TXTCLF-KER-17\n"
     ]
    }
   ],
   "source": [
    "npt_prod_model = neptune.init_model_version(with_id=prod_model_id)\n",
    "npt_prod_model_params = npt_prod_model[\"params/model\"].fetch()\n",
    "prod_model = tf.keras.models.model_from_json(npt_prod_model[\"serialized_model\"].fetch())\n",
    "\n",
    "npt_prod_model[\"model_weights\"].download()\n",
    "prod_model.load_weights(\"model_weights.h5\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) Evaluate current model on lastest test data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### (Neptune) Fetch data parameters from the current champion model to preserve data preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'batch_size': 32, 'embedding_dim': 128, 'max_features': 1500, 'seed': 42, 'sequence_length': 1000, 'validation_split': 0.2}\n"
     ]
    }
   ],
   "source": [
    "prod_data_params = npt_prod_model[\"params/data\"].fetch()\n",
    "\n",
    "print(prod_data_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Preparing test data according to fetched data parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 25000 files belonging to 2 classes.\n",
      "Number of batches in raw_test_ds: 782\n"
     ]
    }
   ],
   "source": [
    "raw_test_ds = tf.keras.preprocessing.text_dataset_from_directory(\n",
    "    \"../data/test\", batch_size=prod_data_params[\"batch_size\"]\n",
    ")\n",
    "\n",
    "\n",
    "print(f\"Number of batches in raw_test_ds: {raw_test_ds.cardinality()}\")\n",
    "\n",
    "vectorize_layer = TextVectorization(\n",
    "    standardize=custom_standardization,\n",
    "    max_tokens=prod_data_params[\"max_features\"],\n",
    "    output_mode=\"int\",\n",
    "    output_sequence_length=prod_data_params[\"sequence_length\"],\n",
    ")\n",
    "\n",
    "text_ds = raw_train_ds.map(lambda x, y: x)\n",
    "vectorize_layer.adapt(text_ds)\n",
    "\n",
    "# Vectorize the data.\n",
    "test_ds = raw_test_ds.map(vectorize_text)\n",
    "\n",
    "# Do async prefetching / buffering of the data for best performance on GPU.\n",
    "test_ds = test_ds.cache().prefetch(buffer_size=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "782/782 [==============================] - 31s 39ms/step - loss: 0.3125 - accuracy: 0.8681\n"
     ]
    }
   ],
   "source": [
    "# Evaluate champion model using the model's original loss and optimizer, but the current metric\n",
    "prod_model.compile(\n",
    "    loss=npt_prod_model_params[\"loss\"],\n",
    "    optimizer=npt_prod_model_params[\"optimizer\"],\n",
    "    metrics=model_params[\"metrics\"],\n",
    ")\n",
    "\n",
    "_, prod_model_acc = prod_model.evaluate(test_ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Neptune) If challenger model outperforms production model, promote it to production"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Champion model accuracy: 0.8680800199508667\n",
      "Challenger model accuracy: 0.8591600060462952\n",
      "Archiving challenger model\n"
     ]
    }
   ],
   "source": [
    "print(f\"Champion model accuracy: {prod_model_acc}\\nChallenger model accuracy: {curr_model_acc}\")\n",
    "\n",
    "if curr_model_acc > prod_model_acc:\n",
    "    print(\"Promoting challenger to champion\")\n",
    "    npt_prod_model.change_stage(\"archived\")\n",
    "    model_version.change_stage(\"production\")\n",
    "else:\n",
    "    print(\"Archiving challenger model\")\n",
    "    model_version.change_stage(\"archived\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (Neptune) Stop tracking\n",
    "When working in an interactive notebook environment, we need to explicitly stop all initialized neptune objects to prevent unnecessary monitoring.  \n",
    "Read more about stopping neptune objects here: https://docs.neptune.ai/usage/best_practices/#stopping-runs-and-other-objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shutting down background jobs, please wait a moment...\n",
      "Done!\n",
      "All 0 operations synced, thanks for waiting!\n",
      "Explore the metadata in the Neptune app:\n",
      "https://app.neptune.ai/showcase/project-text-classification/m/TXTCLF-KER/v/TXTCLF-KER-17/metadata\n",
      "Shutting down background jobs, please wait a moment...\n",
      "Done!\n",
      "All 0 operations synced, thanks for waiting!\n",
      "Explore the metadata in the Neptune app:\n",
      "https://app.neptune.ai/showcase/project-text-classification/m/TXTCLF-KER/v/TXTCLF-KER-18/metadata\n",
      "Shutting down background jobs, please wait a moment...\n",
      "Done!\n",
      "All 0 operations synced, thanks for waiting!\n",
      "Explore the metadata in the Neptune app:\n",
      "https://app.neptune.ai/showcase/project-text-classification/e/TXTCLF-394/metadata\n",
      "Shutting down background jobs, please wait a moment...\n",
      "Done!\n",
      "All 0 operations synced, thanks for waiting!\n",
      "Explore the metadata in the Neptune app:\n",
      "https://app.neptune.ai/showcase/project-text-classification/metadata\n"
     ]
    }
   ],
   "source": [
    "npt_prod_model.stop()\n",
    "model_version.stop()\n",
    "run.stop()\n",
    "project.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore the project in Neptune\n",
    "\n",
    "<a target=\"_blank\" href=\"https://app.neptune.ai/o/showcase/org/project-text-classification/runs/details?viewId=9827345e-70b8-49ba-bf0d-c67946b18c78&detailsTab=dashboard&dashboardId=98274451-09a3-49e0-8204-4750716207bc&shortId=TXTCLF-394\"> \n",
    "  <img alt=\"Explore in Neptune\" src=\"https://neptune.ai/wp-content/uploads/2024/01/neptune-badge.svg\">\n",
    "</a>"
   ]
  }
 ],
 "metadata": {
  "keep_output": true,
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.15"
  },
  "neptune": {
   "notebookId": "98a177ec-a21b-404a-ad14-def08e56f560",
   "projectVersion": 2
  },
  "vscode": {
   "interpreter": {
    "hash": "a9715cf0b0024f6e1c62cb31a4f1f43970eb41991212681878768b4bfe53050a"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}