06-advanced-training-pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4e25f765-87ee-4cc8-b37c-648956dc46e9",
   "metadata": {},
   "source": [
    "# Advanced Training Pipeline\n",
    "\n",
    "Once we have set up our pipeline, we can focus on trying out different combinations of data and models. However, to do this, we need to make sure that our pipeline can be easily adjusted for the specific task. This means that we should be able to run the pipeline using different models, settings, and other components. Imagine if we want to run 100 experiments with various combinations, it would be difficult and time-consuming to write separate scripts for each experiment. It would also be challenging to maintain and manage multiple pipelines or configurations if we cannot reuse the same pipeline.\n",
    "\n",
    "<img src=\"https://raw.githubusercontent.com/facebookresearch/hydra/master/website/static/img/Hydra-Readme-logo2.svg\" alt=\"logo\" width=\"40%\" />\n",
    "\n",
    "In this section, I will demonstrate how we can create complex configurations and run multiple experiments with just one command using [Hydra](https://github.com/facebookresearch/hydra), a tool developed by Facebook Research. Hydra simplifies the process by providing a convenient way to manage configurations. We will begin by running a basic experiment with the default settings and then proceed to run multiple experiments effortlessly using Hydra. Finally, we will combine Hydra with the [Optuna Framework](https://optuna.org/) to perform a Hyperparameter Sweep, allowing us to run thousands of experiments to find the best configuration for our model.\n",
    "\n",
    "This section is inspired from [ashleve/lightning-hydra-template](https://github.com/ashleve/lightning-hydra-template). Please read the repository the fully understand Lightning+Hydra setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "18edd3dc-640b-4d0c-abb8-51f5984f28dc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n",
      "\u001b[33mWARNING: You are using pip version 21.2.4; however, version 23.1.2 is available.\n",
      "You should consider upgrading via the '/usr/bin/python -m pip install --upgrade pip' command.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install -q hydra-core hydra-colorlog timm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe612121-9b2e-4059-b663-678dd4a7f48e",
   "metadata": {},
   "source": [
    "Instantiate the necessary paths and lets move to the root directory to accesss `train.py`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2e71f56f-5827-4a8e-a66c-c6eb4a19b3ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "DATA_DIR = 'data/food-101-tiny'\n",
    "\n",
    "TRAIN_DATA_PATH = os.path.join(DATA_DIR, 'train')\n",
    "VAL_DATA_PATH = os.path.join(DATA_DIR, 'valid')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4e551ce-3a44-4f68-9b5a-a5b754e342fa",
   "metadata": {},
   "source": [
    "## Hydra Configuration Pipeline\n",
    "\n",
    "In Hydra, you can create separate configuration files for different parts of your project or experiment. Each file contains settings and parameters specific to that part. For example, you might have one file for model configuration, another for dataset configuration, and so on. Hydra allows you to define a hierarchical structure for these configuration files. You can specify relationships and dependencies between them. For instance, you can have a base configuration file that contains common settings shared by all other configurations, and then have specialized configuration files that override or extend the base settings.\n",
    "\n",
    "When you run your experiment or application, Hydra intelligently combines these configuration files to create a single, cohesive configuration. It merges the settings from different files based on their hierarchy and resolves any conflicts or inconsistencies. The resulting configuration is a composition of all the specified settings, providing a comprehensive configuration that captures the specific requirements of your experiment. This modular approach makes it easy to manage and reuse configurations across different experiments or projects, but it will be **challenging** to manage the software since it's build using the OOP principles.\n",
    "\n",
    "For more details on how to use hydra, please read the [documentation](https://hydra.cc/docs/intro/)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae6d46c0-2eb8-452c-a221-ed89ec7d65ac",
   "metadata": {},
   "source": [
    "### Python Scripts\n",
    "\n",
    "We're going to wrap the training pipeline in `src/train_pipeline.py`. Before that, we're going to copy all of the lightning functions that we developed from the previous section in to `src/`.\n",
    "\n",
    "<details>\n",
    "<summary><b>Training pipeline code sinppet</b></summary>\n",
    "\n",
    "```python\n",
    "def train(cfg: DictConfig) -> Tuple[dict, dict]:\n",
    "    \"\"\"Trains the model. Can additionally evaluate on a testset, using best weights obtained during\n",
    "    training.\n",
    "\n",
    "    This method is wrapped in optional @task_wrapper decorator, that controls the behavior during\n",
    "    failure. Useful for multiruns, saving info about the crash, etc.\n",
    "\n",
    "    Args:\n",
    "        cfg (DictConfig): Configuration composed by Hydra.\n",
    "\n",
    "    Returns:\n",
    "        Tuple[dict, dict]: Dict with metrics and dict with all instantiated objects.\n",
    "    \"\"\"\n",
    "\n",
    "    # set seed for random number generators in pytorch, numpy and python.random\n",
    "    if cfg.get(\"seed\"):\n",
    "        L.seed_everything(cfg.seed, workers=True)\n",
    "\n",
    "    log.info(f\"Instantiating datamodule <{cfg.data._target_}>\")\n",
    "    datamodule: LightningDataModule = hydra.utils.instantiate(cfg.data)\n",
    "\n",
    "    log.info(f\"Instantiating model <{cfg.model._target_}>\")\n",
    "    model: LightningModule = hydra.utils.instantiate(cfg.model)\n",
    "\n",
    "    log.info(\"Instantiating callbacks...\")\n",
    "    callbacks: List[Callback] = utils.instantiate_callbacks(cfg.get(\"callbacks\"))\n",
    "\n",
    "    log.info(\"Instantiating loggers...\")\n",
    "    logger: List[Logger] = utils.instantiate_loggers(cfg.get(\"logger\"))\n",
    "\n",
    "    log.info(f\"Instantiating trainer <{cfg.trainer._target_}>\")\n",
    "    trainer: Trainer = hydra.utils.instantiate(cfg.trainer, callbacks=callbacks, logger=logger)\n",
    "\n",
    "    object_dict = {\n",
    "        \"cfg\": cfg,\n",
    "        \"datamodule\": datamodule,\n",
    "        \"model\": model,\n",
    "        \"callbacks\": callbacks,\n",
    "        \"logger\": logger,\n",
    "        \"trainer\": trainer,\n",
    "    }\n",
    "\n",
    "    if logger:\n",
    "        log.info(\"Logging hyperparameters!\")\n",
    "        utils.log_hyperparameters(object_dict)\n",
    "\n",
    "    if cfg.get(\"compile\"):\n",
    "        log.info(\"Compiling model!\")\n",
    "        model = torch.compile(model)\n",
    "\n",
    "    if cfg.get(\"train\"):\n",
    "        log.info(\"Starting training!\")\n",
    "        trainer.fit(model=model, datamodule=datamodule, ckpt_path=cfg.get(\"ckpt_path\"))\n",
    "\n",
    "    train_metrics = trainer.callback_metrics\n",
    "\n",
    "    if cfg.get(\"test\"):\n",
    "        log.info(\"Starting testing!\")\n",
    "        ckpt_path = trainer.checkpoint_callback.best_model_path\n",
    "        if ckpt_path == \"\":\n",
    "            log.warning(\"Best ckpt not found! Using current weights for testing...\")\n",
    "            ckpt_path = None\n",
    "        trainer.test(model=model, datamodule=datamodule, ckpt_path=ckpt_path)\n",
    "        log.info(f\"Best ckpt path: {ckpt_path}\")\n",
    "\n",
    "    test_metrics = trainer.callback_metrics\n",
    "\n",
    "    # merge train and test metrics\n",
    "    metric_dict = {**train_metrics, **test_metrics}\n",
    "\n",
    "    return metric_dict, object_dict\n",
    "\n",
    "```\n",
    "</details>\n",
    "<br>\n",
    "\n",
    "The function above will construct the required modules to execute the training pipeline, e.g. `datamodule`, `model`, `callbacks`, `logger`, and `trainer`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b0d09f1-52c6-4213-85d2-24c8752e5ca5",
   "metadata": {},
   "source": [
    "### Configuration Files\n",
    "\n",
    "Next, we're going to define the configurations for every modules or functions that we're going to use in `configs/`.\n",
    "\n",
    "```yaml\n",
    "configs/\n",
    "  train.yaml            -> Main configuration file\n",
    "  trainer/              -> Trainer configs\n",
    "  model/                -> Model configs\n",
    "  logger/               -> Logger configs\n",
    "  hparams_search/       -> Hyperparameter search configs\n",
    "  experiment/           -> Experiment configs\n",
    "  data/                 -> Data configs\n",
    "  callbacks/            -> Callback configs\n",
    "```\n",
    "\n",
    "### How It Works\n",
    "\n",
    "All PyTorch Lightning modules are dynamically instantiated from module paths specified in config. Example model config:\n",
    "\n",
    "```yaml\n",
    "_target_: src.models.ClassificationLightningModule\n",
    "num_classes: 10\n",
    "lr: 0.0001\n",
    "net:\n",
    "  _target_: src.models.ResNet18\n",
    "  input_channels: 3\n",
    "  num_classes: ${..num_classes}\n",
    "```\n",
    "\n",
    "Using this config we can instantiate the object with the following line:\n",
    "\n",
    "```python\n",
    "model = hydra.utils.instantiate(config.model)\n",
    "```\n",
    "\n",
    "This allows you to easily iterate over new models! Every time you create a new one, just specify its module path and parameters in appropriate config file. <br>\n",
    "\n",
    "Switch between models and datamodules with command line arguments:\n",
    "\n",
    "```bash\n",
    "python train.py model=timm\n",
    "```\n",
    "\n",
    "### Main Config\n",
    "\n",
    "Location: [configs/train.yaml](../configs/train.yaml) <br>\n",
    "Main project config contains default training configuration.<br>\n",
    "It determines how config is composed when simply executing command `python train.py`.<br>\n",
    "\n",
    "<details>\n",
    "<summary><b>Show main project config</b></summary>\n",
    "\n",
    "```yaml\n",
    "# order of defaults determines the order in which configs override each other\n",
    "defaults:\n",
    "  - _self_\n",
    "  - data: food101.yaml\n",
    "  - model: resnet18.yaml\n",
    "  - callbacks: default.yaml\n",
    "  - logger: null # set logger here or use command line (e.g. `python train.py logger=csv`)\n",
    "  - trainer: default.yaml\n",
    "\n",
    "  # experiment configs allow for version control of specific hyperparameters\n",
    "  # e.g. best hyperparameters for given model and datamodule\n",
    "  - experiment: null\n",
    "\n",
    "  # config for hyperparameter optimization\n",
    "  - hparams_search: null\n",
    "\n",
    "work_dir: ${hydra:runtime.cwd}\n",
    "\n",
    "# task name, determines output directory path\n",
    "task_name: \"train\"\n",
    "\n",
    "# path to data directory\n",
    "data_dir: data/\n",
    "\n",
    "# tags to help you identify your experiments\n",
    "# you can overwrite this in experiment configs\n",
    "# overwrite from command line with `python train.py tags=\"[first_tag, second_tag]\"`\n",
    "tags: [\"dev\"]\n",
    "\n",
    "# set False to skip model training\n",
    "train: True\n",
    "\n",
    "# evaluate on test set, using best model weights achieved during training\n",
    "# lightning chooses best weights based on the metric specified in checkpoint callback\n",
    "test: True\n",
    "\n",
    "# simply provide checkpoint path to resume training\n",
    "ckpt_path: null\n",
    "\n",
    "# seed for random number generators in pytorch, numpy and python.random\n",
    "seed: null\n",
    "\n",
    "# disable python warnings if they annoy you\n",
    "ignore_warnings: True\n",
    "\n",
    "# pretty print config tree at the start of the run using Rich library\n",
    "print_config: True\n",
    "```\n",
    "\n",
    "</details>\n",
    "\n",
    "### Experiment Config\n",
    "\n",
    "Location: [configs/experiment](../configs/experiment)<br>\n",
    "Experiment configs allow you to overwrite parameters from main config.<br>\n",
    "For example, you can use them to version control best hyperparameters for each combination of model and dataset.\n",
    "\n",
    "<details>\n",
    "<summary><b>Show example experiment config</b></summary>\n",
    "\n",
    "```yaml\n",
    "# @package _global_\n",
    "\n",
    "# to execute this experiment run:\n",
    "# python train.py experiment=example\n",
    "\n",
    "defaults:\n",
    "  - override /data: food101.yaml\n",
    "  - override /model: resnet18.yaml\n",
    "  - override /callbacks: default.yaml\n",
    "  - override /trainer: gpu.yaml\n",
    "\n",
    "# all parameters below will be merged with parameters from default configurations set above\n",
    "# this allows you to overwrite only specified parameters\n",
    "\n",
    "tags: [\"resnet18\", \"food101-tiny\"]\n",
    "\n",
    "seed: 12345\n",
    "\n",
    "trainer:\n",
    "  min_epochs: 1\n",
    "  max_epochs: 10\n",
    "  gradient_clip_val: 0.5\n",
    "  precision: 16\n",
    "\n",
    "model:\n",
    "  num_classes: 10\n",
    "\n",
    "data:\n",
    "  batch_size: 16\n",
    "\n",
    "logger:\n",
    "  wandb:\n",
    "    tags: ${tags}\n",
    "    group: \"resnet\"\n",
    "```\n",
    "\n",
    "</details>\n",
    "\n",
    "<br>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41e59df7-9ebb-4654-a133-638e2a96d082",
   "metadata": {},
   "source": [
    "### Open a Configuration File\n",
    "\n",
    "This example shows how to use hydra using python, but the same method also works with `cli`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "29cc6458-228c-4d54-8801-89c3abc8bed8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "work_dir: ${hydra:runtime.cwd}\n",
      "task_name: train\n",
      "data_dir: data/\n",
      "tags:\n",
      "- dev\n",
      "train: true\n",
      "test: true\n",
      "ckpt_path: null\n",
      "seed: null\n",
      "ignore_warnings: true\n",
      "print_config: true\n",
      "data:\n",
      "  _target_: src.dataset.Food101LitDatamodule\n",
      "  data_dir: ${data_dir}\n",
      "  input_size:\n",
      "  - 384\n",
      "  - 384\n",
      "  batch_size: 16\n",
      "  num_workers: 4\n",
      "  pin_memory: false\n",
      "model:\n",
      "  _target_: src.models.ClassificationLightningModule\n",
      "  num_classes: 10\n",
      "  lr: 0.0001\n",
      "  net:\n",
      "    _target_: src.models.ResNet18\n",
      "    input_channels: 3\n",
      "    num_classes: ${..num_classes}\n",
      "callbacks:\n",
      "  model_checkpoint:\n",
      "    _target_: lightning.pytorch.callbacks.ModelCheckpoint\n",
      "    dirpath: null\n",
      "    filename: epoch_{epoch:03d}\n",
      "    monitor: val/acc\n",
      "    mode: max\n",
      "    save_last: true\n",
      "    save_top_k: 1\n",
      "    auto_insert_metric_name: false\n",
      "  early_stopping:\n",
      "    _target_: lightning.pytorch.callbacks.EarlyStopping\n",
      "    monitor: val/acc\n",
      "    patience: 100\n",
      "    mode: max\n",
      "trainer:\n",
      "  _target_: lightning.pytorch.trainer.Trainer\n",
      "  default_root_dir: null\n",
      "  min_epochs: 1\n",
      "  max_epochs: 10\n",
      "  accelerator: cpu\n",
      "  devices: 1\n",
      "  check_val_every_n_epoch: 1\n",
      "  deterministic: false\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from hydra import initialize, initialize_config_module, initialize_config_dir, compose\n",
    "from omegaconf import OmegaConf\n",
    "\n",
    "with initialize(version_base=None, config_path=\"configs/\"):\n",
    "    cfg = compose(config_name=\"train.yaml\")\n",
    "    print(OmegaConf.to_yaml(cfg))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d545d373-e0e0-43dc-9318-3c3b5b305cf5",
   "metadata": {},
   "source": [
    "The configuration above"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b23457c8-7f7c-4215-bbf0-d1c11d266d86",
   "metadata": {},
   "source": [
    "### Running an Experiment\n",
    "\n",
    "Using `hydra` is simple, just run the following command.\n",
    "\n",
    "Equivalent CLI command:\n",
    "```\n",
    "python3 train.py -m experiment=food101\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "055bf98c-2d7d-4b08-9cdd-242971507111",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Global seed set to 12345\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/pytorch/utilities/parsing.py:196: UserWarning: Attribute 'net' is an instance of `nn.Module` and is already saved during checkpointing. It is recommended to ignore them using `self.save_hyperparameters(ignore=['net'])`.\n",
      "  rank_zero_warn(\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/fabric/connector.py:555: UserWarning: 16 is supported for historical reasons but its usage is discouraged. Please set your precision to 16-mixed instead!\n",
      "  rank_zero_warn(\n",
      "Using 16bit Automatic Mixed Precision (AMP)\n",
      "GPU available: True (cuda), used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/pytorch/callbacks/model_checkpoint.py:615: UserWarning: Checkpoint directory /home/haritsahm/Documents/Getting Started/checkpoints exists and is not empty.\n",
      "  rank_zero_warn(f\"Checkpoint directory {dirpath} exists and is not empty.\")\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "\n",
      "  | Name         | Type               | Params\n",
      "----------------------------------------------------\n",
      "0 | net          | ResNet18           | 12.6 M\n",
      "1 | criterion    | CrossEntropyLoss   | 0     \n",
      "2 | train_acc    | MulticlassAccuracy | 0     \n",
      "3 | val_metrics  | MetricCollection   | 0     \n",
      "4 | test_metrics | MetricCollection   | 0     \n",
      "----------------------------------------------------\n",
      "12.6 M    Trainable params\n",
      "0         Non-trainable params\n",
      "12.6 M    Total params\n",
      "50.251    Total estimated model params size (MB)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Sanity Checking: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.8/dist-packages/torchmetrics/utilities/prints.py:36: UserWarning: No negative samples in targets, false positive value should be meaningless. Returning zero tensor in false positive score\n",
      "  warnings.warn(*args, **kwargs)\n",
      "/usr/local/lib/python3.8/dist-packages/torchmetrics/utilities/prints.py:36: UserWarning: No positive samples in targets, true positive value should be meaningless. Returning zero tensor in true positive score\n",
      "  warnings.warn(*args, **kwargs)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bb2f3ddbe3ba47928fc81359561a8443",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Training: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "`Trainer.fit` stopped: `max_epochs=10` reached.\n",
      "Restoring states from the checkpoint path at /home/haritsahm/Documents/Getting Started/checkpoints/epoch_008-v1.ckpt\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "Loaded model weights from the checkpoint at /home/haritsahm/Documents/Getting Started/checkpoints/epoch_008-v1.ckpt\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0b866aa9c30d458096672a95f3de5262",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Testing: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\">        Test metric        </span>┃<span style=\"font-weight: bold\">       DataLoader 0        </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/acc          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.44200000166893005    </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">        test/auroc         </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.04702285677194595    </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">          test/f1          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.44200000166893005    </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/loss         </span>│<span style=\"color: #800080; text-decoration-color: #800080\">     1.609273076057434     </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/prec         </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.44200000166893005    </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/rec          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.44200000166893005    </span>│\n",
       "└───────────────────────────┴───────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/acc         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.44200000166893005   \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m       test/auroc        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.04702285677194595   \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m         test/f1         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.44200000166893005   \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/loss        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m    1.609273076057434    \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/prec        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.44200000166893005   \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/rec         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.44200000166893005   \u001b[0m\u001b[35m \u001b[0m│\n",
       "└───────────────────────────┴───────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from src.train_pipeline import train\n",
    "\n",
    "with initialize(version_base=None, config_path=\"configs/\"):\n",
    "    cfg = compose(\n",
    "        config_name=\"train.yaml\",\n",
    "        overrides=[f\"data_dir={DATA_DIR}\", \"experiment=food101\"]\n",
    "    )\n",
    "    train(cfg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bcd51db-8f2d-40cb-8981-8f63cbd8d6b8",
   "metadata": {},
   "source": [
    "### Running an Experiment with Overriden Parameters\n",
    "\n",
    "We can override existing configuration to execute it with different configurations.\n",
    "\n",
    "Equivalent CLI command:\n",
    "```\n",
    "python3 train.py -m experiment=food101 model=resnet18 model.lr=0.001 data.batch_size=8\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7cc0242b-7603-47fc-b334-d3f1b5b80222",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Global seed set to 12345\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/pytorch/utilities/parsing.py:196: UserWarning: Attribute 'net' is an instance of `nn.Module` and is already saved during checkpointing. It is recommended to ignore them using `self.save_hyperparameters(ignore=['net'])`.\n",
      "  rank_zero_warn(\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/fabric/connector.py:555: UserWarning: 16 is supported for historical reasons but its usage is discouraged. Please set your precision to 16-mixed instead!\n",
      "  rank_zero_warn(\n",
      "Using 16bit Automatic Mixed Precision (AMP)\n",
      "GPU available: True (cuda), used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "\n",
      "  | Name         | Type               | Params\n",
      "----------------------------------------------------\n",
      "0 | net          | ResNet18           | 12.6 M\n",
      "1 | criterion    | CrossEntropyLoss   | 0     \n",
      "2 | train_acc    | MulticlassAccuracy | 0     \n",
      "3 | val_metrics  | MetricCollection   | 0     \n",
      "4 | test_metrics | MetricCollection   | 0     \n",
      "----------------------------------------------------\n",
      "12.6 M    Trainable params\n",
      "0         Non-trainable params\n",
      "12.6 M    Total params\n",
      "50.251    Total estimated model params size (MB)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Sanity Checking: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2ec69fdf570a41228561e6c9061ef011",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Training: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "`Trainer.fit` stopped: `max_epochs=10` reached.\n",
      "Restoring states from the checkpoint path at /home/haritsahm/Documents/Getting Started/checkpoints/epoch_008-v2.ckpt\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "Loaded model weights from the checkpoint at /home/haritsahm/Documents/Getting Started/checkpoints/epoch_008-v2.ckpt\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d449fcf9dbaa40c9a7d6659bd9a70701",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Testing: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\">        Test metric        </span>┃<span style=\"font-weight: bold\">       DataLoader 0        </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/acc          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.30000001192092896    </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">        test/auroc         </span>│<span style=\"color: #800080; text-decoration-color: #800080\">   0.016333334147930145    </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">          test/f1          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.30000001192092896    </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/loss         </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    1.9568886756896973     </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/prec         </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.30000001192092896    </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/rec          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.30000001192092896    </span>│\n",
       "└───────────────────────────┴───────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/acc         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.30000001192092896   \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m       test/auroc        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m  0.016333334147930145   \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m         test/f1         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.30000001192092896   \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/loss        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   1.9568886756896973    \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/prec        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.30000001192092896   \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/rec         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.30000001192092896   \u001b[0m\u001b[35m \u001b[0m│\n",
       "└───────────────────────────┴───────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "overrides = [\n",
    "    f\"data_dir={DATA_DIR}\",\n",
    "    \"experiment=food101\",\n",
    "    \"model=resnet18\",\n",
    "    \"model.lr=0.001\",\n",
    "    \"data.batch_size=8\",\n",
    "]\n",
    "\n",
    "with initialize(version_base=None, config_path=\"configs/\"):\n",
    "    cfg = compose(\n",
    "        config_name=\"train.yaml\",\n",
    "        overrides=overrides,\n",
    "    )\n",
    "    train(cfg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d74d5c10-0008-4249-9eaf-2d1b8272581f",
   "metadata": {},
   "source": [
    "Another example to use `timm` based models\n",
    "\n",
    "Equivalent CLI command:\n",
    "```\n",
    "python3 train.py -m experiment=food101-timm model=timm model.net.model_name=resnetv2_50 model.lr=0.001 data.batch_size=8\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "7d2c5113-9762-4074-a804-0b03435246d0",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Global seed set to 12345\n",
      "Using 16bit Automatic Mixed Precision (AMP)\n",
      "GPU available: True (cuda), used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "\n",
      "  | Name         | Type               | Params\n",
      "----------------------------------------------------\n",
      "0 | net          | ResNetV2           | 23.5 M\n",
      "1 | criterion    | CrossEntropyLoss   | 0     \n",
      "2 | train_acc    | MulticlassAccuracy | 0     \n",
      "3 | val_metrics  | MetricCollection   | 0     \n",
      "4 | test_metrics | MetricCollection   | 0     \n",
      "----------------------------------------------------\n",
      "23.5 M    Trainable params\n",
      "0         Non-trainable params\n",
      "23.5 M    Total params\n",
      "94.083    Total estimated model params size (MB)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Sanity Checking: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "31d0dbedb3954a02b59e56be9592a3fa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Training: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "`Trainer.fit` stopped: `max_epochs=10` reached.\n",
      "Restoring states from the checkpoint path at /home/haritsahm/Documents/Getting Started/checkpoints/epoch_007.ckpt\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "Loaded model weights from the checkpoint at /home/haritsahm/Documents/Getting Started/checkpoints/epoch_007.ckpt\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "53729de8c22c480daa9a4f68d3438f77",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Testing: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\">        Test metric        </span>┃<span style=\"font-weight: bold\">       DataLoader 0        </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/acc          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.3659999966621399     </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">        test/auroc         </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.01939999870955944    </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">          test/f1          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.3659999966621399     </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/loss         </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    1.7680712938308716     </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/prec         </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.3659999966621399     </span>│\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test/rec          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">    0.3659999966621399     </span>│\n",
       "└───────────────────────────┴───────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/acc         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.3659999966621399    \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m       test/auroc        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.01939999870955944   \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m         test/f1         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.3659999966621399    \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/loss        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   1.7680712938308716    \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/prec        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.3659999966621399    \u001b[0m\u001b[35m \u001b[0m│\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test/rec         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.3659999966621399    \u001b[0m\u001b[35m \u001b[0m│\n",
       "└───────────────────────────┴───────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "overrides = [\n",
    "    f\"data_dir={DATA_DIR}\",\n",
    "    \"experiment=food101\",\n",
    "    \"model=timm\",\n",
    "    \"model.net.model_name=resnetv2_50\",\n",
    "    \"model.lr=0.001\",\n",
    "    \"data.batch_size=8\",\n",
    "]\n",
    "\n",
    "with initialize(version_base=None, config_path=\"configs/\"):\n",
    "    cfg = compose(\n",
    "        config_name=\"train.yaml\",\n",
    "        overrides=overrides,\n",
    "    )\n",
    "    train(cfg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "603043bb-639f-4909-9371-66ad5cdf0592",
   "metadata": {},
   "source": [
    "### Running Multiple Experiments\n",
    "\n",
    "`hydra` is capable to run multiple experiments with different configurations. We need to add `--multirun` or `-m` to the python command and execute the main file. <br>\n",
    "This command will run 4 experiments using `timm` model with lr-bs pairs of: `(0.001,8)`, `(0.001,16)`, `(0.0001,8)`, `(0.0001,16)`.\n",
    "\n",
    "Please read the [documentation](https://hydra.cc/docs/tutorials/basic/running_your_app/multi-run/).\n",
    "\n",
    "Unfortunately, `multirun` method is not executable via notebook cell like the previous cells, it must be executed with python command.\n",
    "From the `Root Directory`, run the following command to run multiple experiments:\n",
    "```\n",
    "python3 train.py -m experiment=food101 model=timm trainer.max_epochs=5 model.lr=0.001,0.0001 data.batch_size=8,16 logger=wandb\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ebe53715-874d-420b-9ba3-9c84b94b90c4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2023-06-16 16:01:23,348][HYDRA] Launching 4 jobs locally\n",
      "[2023-06-16 16:01:23,348][HYDRA] \t#0 : experiment=food101 model=timm trainer.max_epochs=5 model.lr=0.001 data.batch_size=8 logger=wandb data_dir=data/food-101-tiny/\n",
      "Global seed set to 12345\n",
      "[2023-06-16 16:01:23,576][src.train_pipeline][INFO] - Instantiating datamodule <src.dataset.Food101LitDatamodule>\n",
      "[2023-06-16 16:01:23,608][src.train_pipeline][INFO] - Instantiating model <src.models.ClassificationLightningModule>\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/pytorch/utilities/parsing.py:196: UserWarning: Attribute 'net' is an instance of `nn.Module` and is already saved during checkpointing. It is recommended to ignore them using `self.save_hyperparameters(ignore=['net'])`.\n",
      "  rank_zero_warn(\n",
      "[2023-06-16 16:01:23,813][src.train_pipeline][INFO] - Instantiating callbacks...\n",
      "[2023-06-16 16:01:23,813][src.utils][INFO] - Instantiating callback <lightning.pytorch.callbacks.ModelCheckpoint>\n",
      "[2023-06-16 16:01:23,814][src.utils][INFO] - Instantiating callback <lightning.pytorch.callbacks.EarlyStopping>\n",
      "[2023-06-16 16:01:23,815][src.train_pipeline][INFO] - Instantiating loggers...\n",
      "[2023-06-16 16:01:23,815][src.utils][INFO] - Instantiating logger <lightning.pytorch.loggers.wandb.WandbLogger>\n",
      "[2023-06-16 16:01:23,956][wandb.sdk.lib.git][ERROR] - git root error: Cmd('git') failed due to: exit code(128)\n",
      "  cmdline: git rev-parse --show-toplevel\n",
      "  stderr: 'fatal: detected dubious ownership in repository at '/home/haritsahm/Documents/Getting Started'\n",
      "To add an exception for this directory, call:\n",
      "\n",
      "\tgit config --global --add safe.directory '/home/haritsahm/Documents/Getting Started''\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mharitsahm\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: wandb version 0.15.4 is available!  To upgrade, please run:\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:  $ pip install wandb --upgrade\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Tracking run with wandb version 0.15.3\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Run data is saved locally in \u001b[35m\u001b[1m/home/haritsahm/Documents/Getting Started/wandb/run-20230616_160125-colh09s6\u001b[0m\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Run \u001b[1m`wandb offline`\u001b[0m to turn off syncing.\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Syncing run \u001b[33mdivine-firebrand-3\u001b[0m\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: ⭐️ View project at \u001b[34m\u001b[4mhttps://wandb.ai/haritsahm/getting-started-dlcv\u001b[0m\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: 🚀 View run at \u001b[34m\u001b[4mhttps://wandb.ai/haritsahm/getting-started-dlcv/runs/colh09s6\u001b[0m\n",
      "[2023-06-16 16:01:27,137][src.train_pipeline][INFO] - Instantiating trainer <lightning.pytorch.trainer.Trainer>\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/fabric/connector.py:555: UserWarning: 16 is supported for historical reasons but its usage is discouraged. Please set your precision to 16-mixed instead!\n",
      "  rank_zero_warn(\n",
      "Using 16bit Automatic Mixed Precision (AMP)\n",
      "GPU available: True (cuda), used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "[2023-06-16 16:01:27,333][src.train_pipeline][INFO] - Logging hyperparameters!\n",
      "[2023-06-16 16:01:27,337][src.train_pipeline][INFO] - Starting training!\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "\n",
      "  | Name         | Type               | Params\n",
      "----------------------------------------------------\n",
      "0 | net          | EfficientNet       | 2.2 M \n",
      "1 | criterion    | CrossEntropyLoss   | 0     \n",
      "2 | train_acc    | MulticlassAccuracy | 0     \n",
      "3 | val_metrics  | MetricCollection   | 0     \n",
      "4 | test_metrics | MetricCollection   | 0     \n",
      "----------------------------------------------------\n",
      "2.2 M     Trainable params\n",
      "0         Non-trainable params\n",
      "2.2 M     Total params\n",
      "8.947     Total estimated model params size (MB)\n",
      "Sanity Checking DataLoader 0:   0%|                       | 0/2 [00:00<?, ?it/s]/usr/local/lib/python3.8/dist-packages/torchmetrics/utilities/prints.py:36: UserWarning: No negative samples in targets, false positive value should be meaningless. Returning zero tensor in false positive score\n",
      "  warnings.warn(*args, **kwargs)\n",
      "/usr/local/lib/python3.8/dist-packages/torchmetrics/utilities/prints.py:36: UserWarning: No positive samples in targets, true positive value should be meaningless. Returning zero tensor in true positive score\n",
      "  warnings.warn(*args, **kwargs)\n",
      "Epoch 0: 100%|████████████████████| 188/188 [00:43<00:00,  4.34it/s, v_num=09s6]\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                  | 1/63 [00:00<00:04, 14.10it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 2/63 [00:00<00:04, 14.09it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                  | 3/63 [00:00<00:04, 14.05it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 4/63 [00:00<00:04, 14.45it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▌                 | 5/63 [00:00<00:03, 14.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▊                 | 6/63 [00:00<00:03, 14.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|██                 | 7/63 [00:00<00:03, 14.76it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▍                | 8/63 [00:00<00:03, 14.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▋                | 9/63 [00:00<00:03, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▊               | 10/63 [00:00<00:03, 15.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|███▏              | 11/63 [00:00<00:03, 15.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▍              | 12/63 [00:00<00:03, 15.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▋              | 13/63 [00:00<00:03, 15.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████              | 14/63 [00:00<00:03, 15.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████▎             | 15/63 [00:00<00:03, 15.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▌             | 16/63 [00:01<00:03, 15.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▊             | 17/63 [00:01<00:03, 15.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|█████▏            | 18/63 [00:01<00:02, 15.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████▍            | 19/63 [00:01<00:02, 15.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▋            | 20/63 [00:01<00:02, 15.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|██████            | 21/63 [00:01<00:02, 15.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|██████▎           | 22/63 [00:01<00:02, 15.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▌           | 23/63 [00:01<00:02, 15.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 24/63 [00:01<00:02, 15.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|███████▏          | 25/63 [00:01<00:02, 15.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▍          | 26/63 [00:01<00:02, 15.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▋          | 27/63 [00:01<00:02, 15.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|████████          | 28/63 [00:01<00:02, 15.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|████████▎         | 29/63 [00:01<00:02, 15.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▌         | 30/63 [00:01<00:02, 15.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▊         | 31/63 [00:02<00:02, 15.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|█████████▏        | 32/63 [00:02<00:02, 15.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|█████████▍        | 33/63 [00:02<00:01, 15.41it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▋        | 34/63 [00:02<00:01, 15.41it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████        | 35/63 [00:02<00:01, 15.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|██████████▎       | 36/63 [00:02<00:01, 15.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▌       | 37/63 [00:02<00:01, 15.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▊       | 38/63 [00:02<00:01, 15.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▏      | 39/63 [00:02<00:01, 15.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|███████████▍      | 40/63 [00:02<00:01, 15.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|███████████▋      | 41/63 [00:02<00:01, 15.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|████████████      | 42/63 [00:02<00:01, 15.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|████████████▎     | 43/63 [00:02<00:01, 15.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|████████████▌     | 44/63 [00:02<00:01, 15.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|████████████▊     | 45/63 [00:02<00:01, 15.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|█████████████▏    | 46/63 [00:02<00:01, 15.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▍    | 47/63 [00:03<00:01, 15.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|█████████████▋    | 48/63 [00:03<00:00, 15.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 49/63 [00:03<00:00, 15.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|██████████████▎   | 50/63 [00:03<00:00, 15.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▌   | 51/63 [00:03<00:00, 15.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|██████████████▊   | 52/63 [00:03<00:00, 15.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 53/63 [00:03<00:00, 15.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|███████████████▍  | 54/63 [00:03<00:00, 15.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|███████████████▋  | 55/63 [00:03<00:00, 15.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|████████████████  | 56/63 [00:03<00:00, 15.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|████████████████▎ | 57/63 [00:03<00:00, 15.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|████████████████▌ | 58/63 [00:03<00:00, 15.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▊ | 59/63 [00:03<00:00, 15.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|█████████████████▏| 60/63 [00:03<00:00, 15.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 61/63 [00:03<00:00, 15.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|█████████████████▋| 62/63 [00:03<00:00, 15.51it/s]\u001b[A\n",
      "Epoch 0: 100%|█| 188/188 [00:47<00:00,  3.95it/s, v_num=09s6, val/loss=3.500, va\u001b[A\n",
      "Epoch 1: 100%|█| 188/188 [00:43<00:00,  4.36it/s, v_num=09s6, val/loss=3.500, va\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                  | 1/63 [00:00<00:04, 13.82it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 2/63 [00:00<00:04, 13.13it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                  | 3/63 [00:00<00:04, 13.12it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 4/63 [00:00<00:04, 13.71it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▌                 | 5/63 [00:00<00:04, 14.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▊                 | 6/63 [00:00<00:04, 14.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|██                 | 7/63 [00:00<00:03, 14.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▍                | 8/63 [00:00<00:03, 14.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▋                | 9/63 [00:00<00:03, 14.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▊               | 10/63 [00:00<00:03, 14.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|███▏              | 11/63 [00:00<00:03, 14.60it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▍              | 12/63 [00:00<00:03, 14.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▋              | 13/63 [00:00<00:03, 14.76it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████              | 14/63 [00:00<00:03, 14.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████▎             | 15/63 [00:01<00:03, 14.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▌             | 16/63 [00:01<00:03, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▊             | 17/63 [00:01<00:03, 14.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|█████▏            | 18/63 [00:01<00:03, 14.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████▍            | 19/63 [00:01<00:02, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▋            | 20/63 [00:01<00:02, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|██████            | 21/63 [00:01<00:02, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|██████▎           | 22/63 [00:01<00:02, 15.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▌           | 23/63 [00:01<00:02, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 24/63 [00:01<00:02, 15.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|███████▏          | 25/63 [00:01<00:02, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▍          | 26/63 [00:01<00:02, 15.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▋          | 27/63 [00:01<00:02, 15.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|████████          | 28/63 [00:01<00:02, 15.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|████████▎         | 29/63 [00:01<00:02, 15.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▌         | 30/63 [00:01<00:02, 15.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▊         | 31/63 [00:02<00:02, 15.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|█████████▏        | 32/63 [00:02<00:02, 15.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|█████████▍        | 33/63 [00:02<00:01, 15.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▋        | 34/63 [00:02<00:01, 15.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████        | 35/63 [00:02<00:01, 15.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|██████████▎       | 36/63 [00:02<00:01, 15.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▌       | 37/63 [00:02<00:01, 15.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▊       | 38/63 [00:02<00:01, 15.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▏      | 39/63 [00:02<00:01, 15.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|███████████▍      | 40/63 [00:02<00:01, 15.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|███████████▋      | 41/63 [00:02<00:01, 15.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|████████████      | 42/63 [00:02<00:01, 15.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|████████████▎     | 43/63 [00:02<00:01, 15.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|████████████▌     | 44/63 [00:02<00:01, 15.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|████████████▊     | 45/63 [00:02<00:01, 15.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|█████████████▏    | 46/63 [00:03<00:01, 15.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▍    | 47/63 [00:03<00:01, 15.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|█████████████▋    | 48/63 [00:03<00:00, 15.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 49/63 [00:03<00:00, 15.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|██████████████▎   | 50/63 [00:03<00:00, 15.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▌   | 51/63 [00:03<00:00, 15.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|██████████████▊   | 52/63 [00:03<00:00, 15.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 53/63 [00:03<00:00, 15.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|███████████████▍  | 54/63 [00:03<00:00, 15.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|███████████████▋  | 55/63 [00:03<00:00, 15.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|████████████████  | 56/63 [00:03<00:00, 15.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|████████████████▎ | 57/63 [00:03<00:00, 15.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|████████████████▌ | 58/63 [00:03<00:00, 15.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▊ | 59/63 [00:03<00:00, 15.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|█████████████████▏| 60/63 [00:03<00:00, 15.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 61/63 [00:03<00:00, 15.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|█████████████████▋| 62/63 [00:04<00:00, 15.35it/s]\u001b[A\n",
      "Epoch 1: 100%|█| 188/188 [00:47<00:00,  3.96it/s, v_num=09s6, val/loss=2.430, va\u001b[A\n",
      "Epoch 2: 100%|█| 188/188 [00:43<00:00,  4.35it/s, v_num=09s6, val/loss=2.430, va\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                  | 1/63 [00:00<00:04, 13.26it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 2/63 [00:00<00:04, 13.13it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                  | 3/63 [00:00<00:04, 13.64it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 4/63 [00:00<00:04, 14.03it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▌                 | 5/63 [00:00<00:04, 14.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▊                 | 6/63 [00:00<00:03, 14.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|██                 | 7/63 [00:00<00:03, 14.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▍                | 8/63 [00:00<00:03, 14.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▋                | 9/63 [00:00<00:03, 14.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▊               | 10/63 [00:00<00:03, 14.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|███▏              | 11/63 [00:00<00:03, 14.81it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▍              | 12/63 [00:00<00:03, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▋              | 13/63 [00:00<00:03, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████              | 14/63 [00:00<00:03, 14.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████▎             | 15/63 [00:01<00:03, 14.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▌             | 16/63 [00:01<00:03, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▊             | 17/63 [00:01<00:03, 15.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|█████▏            | 18/63 [00:01<00:02, 15.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████▍            | 19/63 [00:01<00:02, 15.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▋            | 20/63 [00:01<00:02, 15.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|██████            | 21/63 [00:01<00:02, 15.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|██████▎           | 22/63 [00:01<00:02, 15.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▌           | 23/63 [00:01<00:02, 15.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 24/63 [00:01<00:02, 15.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|███████▏          | 25/63 [00:01<00:02, 15.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▍          | 26/63 [00:01<00:02, 15.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▋          | 27/63 [00:01<00:02, 15.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|████████          | 28/63 [00:01<00:02, 15.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|████████▎         | 29/63 [00:01<00:02, 15.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▌         | 30/63 [00:01<00:02, 15.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▊         | 31/63 [00:02<00:02, 15.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|█████████▏        | 32/63 [00:02<00:02, 15.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|█████████▍        | 33/63 [00:02<00:01, 15.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▋        | 34/63 [00:02<00:01, 15.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████        | 35/63 [00:02<00:01, 15.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|██████████▎       | 36/63 [00:02<00:01, 15.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▌       | 37/63 [00:02<00:01, 15.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▊       | 38/63 [00:02<00:01, 15.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▏      | 39/63 [00:02<00:01, 15.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|███████████▍      | 40/63 [00:02<00:01, 15.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|███████████▋      | 41/63 [00:02<00:01, 15.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|████████████      | 42/63 [00:02<00:01, 15.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|████████████▎     | 43/63 [00:02<00:01, 15.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|████████████▌     | 44/63 [00:02<00:01, 15.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|████████████▊     | 45/63 [00:02<00:01, 15.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|█████████████▏    | 46/63 [00:03<00:01, 15.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▍    | 47/63 [00:03<00:01, 15.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|█████████████▋    | 48/63 [00:03<00:00, 15.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 49/63 [00:03<00:00, 15.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|██████████████▎   | 50/63 [00:03<00:00, 15.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▌   | 51/63 [00:03<00:00, 15.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|██████████████▊   | 52/63 [00:03<00:00, 15.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 53/63 [00:03<00:00, 15.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|███████████████▍  | 54/63 [00:03<00:00, 15.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|███████████████▋  | 55/63 [00:03<00:00, 15.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|████████████████  | 56/63 [00:03<00:00, 15.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|████████████████▎ | 57/63 [00:03<00:00, 15.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|████████████████▌ | 58/63 [00:03<00:00, 15.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▊ | 59/63 [00:03<00:00, 15.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|█████████████████▏| 60/63 [00:03<00:00, 15.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 61/63 [00:03<00:00, 15.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|█████████████████▋| 62/63 [00:04<00:00, 15.33it/s]\u001b[A\n",
      "Epoch 2: 100%|█| 188/188 [00:47<00:00,  3.96it/s, v_num=09s6, val/loss=2.320, va\u001b[A\n",
      "Epoch 3: 100%|█| 188/188 [00:43<00:00,  4.35it/s, v_num=09s6, val/loss=2.320, va\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                  | 1/63 [00:00<00:04, 13.42it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 2/63 [00:00<00:04, 12.70it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                  | 3/63 [00:00<00:04, 13.06it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 4/63 [00:00<00:04, 13.58it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▌                 | 5/63 [00:00<00:04, 13.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▊                 | 6/63 [00:00<00:04, 14.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|██                 | 7/63 [00:00<00:03, 14.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▍                | 8/63 [00:00<00:03, 14.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▋                | 9/63 [00:00<00:03, 14.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▊               | 10/63 [00:00<00:03, 14.71it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|███▏              | 11/63 [00:00<00:03, 14.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▍              | 12/63 [00:00<00:03, 14.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▋              | 13/63 [00:00<00:03, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████              | 14/63 [00:00<00:03, 14.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████▎             | 15/63 [00:01<00:03, 14.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▌             | 16/63 [00:01<00:03, 14.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▊             | 17/63 [00:01<00:03, 15.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|█████▏            | 18/63 [00:01<00:02, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████▍            | 19/63 [00:01<00:02, 15.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▋            | 20/63 [00:01<00:02, 15.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|██████            | 21/63 [00:01<00:02, 15.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|██████▎           | 22/63 [00:01<00:02, 15.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▌           | 23/63 [00:01<00:02, 15.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 24/63 [00:01<00:02, 15.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|███████▏          | 25/63 [00:01<00:02, 15.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▍          | 26/63 [00:01<00:02, 15.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▋          | 27/63 [00:01<00:02, 15.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|████████          | 28/63 [00:01<00:02, 15.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|████████▎         | 29/63 [00:01<00:02, 15.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▌         | 30/63 [00:01<00:02, 15.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▊         | 31/63 [00:02<00:02, 15.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|█████████▏        | 32/63 [00:02<00:02, 15.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|█████████▍        | 33/63 [00:02<00:01, 15.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▋        | 34/63 [00:02<00:01, 15.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████        | 35/63 [00:02<00:01, 15.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|██████████▎       | 36/63 [00:02<00:01, 15.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▌       | 37/63 [00:02<00:01, 15.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▊       | 38/63 [00:02<00:01, 15.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▏      | 39/63 [00:02<00:01, 15.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|███████████▍      | 40/63 [00:02<00:01, 15.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|███████████▋      | 41/63 [00:02<00:01, 15.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|████████████      | 42/63 [00:02<00:01, 15.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|████████████▎     | 43/63 [00:02<00:01, 15.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|████████████▌     | 44/63 [00:02<00:01, 15.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|████████████▊     | 45/63 [00:02<00:01, 15.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|█████████████▏    | 46/63 [00:02<00:01, 15.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▍    | 47/63 [00:03<00:01, 15.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|█████████████▋    | 48/63 [00:03<00:00, 15.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 49/63 [00:03<00:00, 15.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|██████████████▎   | 50/63 [00:03<00:00, 15.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▌   | 51/63 [00:03<00:00, 15.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|██████████████▊   | 52/63 [00:03<00:00, 15.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 53/63 [00:03<00:00, 15.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|███████████████▍  | 54/63 [00:03<00:00, 15.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|███████████████▋  | 55/63 [00:03<00:00, 15.41it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|████████████████  | 56/63 [00:03<00:00, 15.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|████████████████▎ | 57/63 [00:03<00:00, 15.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|████████████████▌ | 58/63 [00:03<00:00, 15.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▊ | 59/63 [00:03<00:00, 15.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|█████████████████▏| 60/63 [00:03<00:00, 15.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 61/63 [00:03<00:00, 15.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|█████████████████▋| 62/63 [00:04<00:00, 15.43it/s]\u001b[A\n",
      "Epoch 3: 100%|█| 188/188 [00:47<00:00,  3.96it/s, v_num=09s6, val/loss=2.060, va\u001b[A\n",
      "Epoch 4: 100%|█| 188/188 [00:43<00:00,  4.34it/s, v_num=09s6, val/loss=2.060, va\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                  | 1/63 [00:00<00:05, 10.98it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 2/63 [00:00<00:05, 11.28it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                  | 3/63 [00:00<00:05, 11.89it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 4/63 [00:00<00:04, 12.58it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▌                 | 5/63 [00:00<00:04, 13.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▊                 | 6/63 [00:00<00:04, 13.41it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|██                 | 7/63 [00:00<00:04, 13.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▍                | 8/63 [00:00<00:03, 13.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▋                | 9/63 [00:00<00:03, 14.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▊               | 10/63 [00:00<00:03, 14.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|███▏              | 11/63 [00:00<00:03, 14.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▍              | 12/63 [00:00<00:03, 14.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▋              | 13/63 [00:00<00:03, 14.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████              | 14/63 [00:00<00:03, 14.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████▎             | 15/63 [00:01<00:03, 14.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▌             | 16/63 [00:01<00:03, 14.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▊             | 17/63 [00:01<00:03, 14.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|█████▏            | 18/63 [00:01<00:03, 14.58it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████▍            | 19/63 [00:01<00:03, 14.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▋            | 20/63 [00:01<00:02, 14.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|██████            | 21/63 [00:01<00:02, 14.71it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|██████▎           | 22/63 [00:01<00:02, 14.74it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▌           | 23/63 [00:01<00:02, 14.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 24/63 [00:01<00:02, 14.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|███████▏          | 25/63 [00:01<00:02, 14.76it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▍          | 26/63 [00:01<00:02, 14.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▋          | 27/63 [00:01<00:02, 14.81it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|████████          | 28/63 [00:01<00:02, 14.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|████████▎         | 29/63 [00:01<00:02, 14.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▌         | 30/63 [00:02<00:02, 14.77it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▊         | 31/63 [00:02<00:02, 14.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|█████████▏        | 32/63 [00:02<00:02, 14.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|█████████▍        | 33/63 [00:02<00:02, 14.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▋        | 34/63 [00:02<00:01, 14.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████        | 35/63 [00:02<00:01, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|██████████▎       | 36/63 [00:02<00:01, 14.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▌       | 37/63 [00:02<00:01, 14.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▊       | 38/63 [00:02<00:01, 14.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▏      | 39/63 [00:02<00:01, 14.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|███████████▍      | 40/63 [00:02<00:01, 14.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|███████████▋      | 41/63 [00:02<00:01, 14.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|████████████      | 42/63 [00:02<00:01, 14.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|████████████▎     | 43/63 [00:02<00:01, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|████████████▌     | 44/63 [00:02<00:01, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|████████████▊     | 45/63 [00:03<00:01, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|█████████████▏    | 46/63 [00:03<00:01, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▍    | 47/63 [00:03<00:01, 14.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|█████████████▋    | 48/63 [00:03<00:01, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 49/63 [00:03<00:00, 15.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|██████████████▎   | 50/63 [00:03<00:00, 15.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▌   | 51/63 [00:03<00:00, 15.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|██████████████▊   | 52/63 [00:03<00:00, 15.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 53/63 [00:03<00:00, 15.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|███████████████▍  | 54/63 [00:03<00:00, 15.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|███████████████▋  | 55/63 [00:03<00:00, 15.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|████████████████  | 56/63 [00:03<00:00, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|████████████████▎ | 57/63 [00:03<00:00, 15.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|████████████████▌ | 58/63 [00:03<00:00, 15.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▊ | 59/63 [00:03<00:00, 15.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|█████████████████▏| 60/63 [00:03<00:00, 15.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 61/63 [00:04<00:00, 15.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|█████████████████▋| 62/63 [00:04<00:00, 15.12it/s]\u001b[A\n",
      "Epoch 4: 100%|█| 188/188 [00:47<00:00,  3.95it/s, v_num=09s6, val/loss=1.940, va\u001b[A\n",
      "Epoch 4: 100%|█| 188/188 [00:47<00:00,  3.95it/s, v_num=09s6, val/loss=1.940, va\u001b[A`Trainer.fit` stopped: `max_epochs=5` reached.\n",
      "Epoch 4: 100%|█| 188/188 [00:47<00:00,  3.93it/s, v_num=09s6, val/loss=1.940, va\n",
      "[2023-06-16 16:05:27,947][src.train_pipeline][INFO] - Starting testing!\n",
      "Restoring states from the checkpoint path at /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003.ckpt\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "Loaded model weights from the checkpoint at /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003.ckpt\n",
      "Testing DataLoader 0: 100%|█████████████████████| 63/63 [00:04<00:00, 15.64it/s]\n",
      "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
      "┃\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m┃\n",
      "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/acc         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2680000066757202    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m       test/auroc        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.01810000091791153   \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m         test/f1         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2680000066757202    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/loss        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m    2.056892156600952    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/prec        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2680000066757202    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/rec         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2680000066757202    \u001b[0m\u001b[35m \u001b[0m│\n",
      "└───────────────────────────┴───────────────────────────┘\n",
      "[2023-06-16 16:05:32,730][src.train_pipeline][INFO] - Best ckpt path: /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003.ckpt\n",
      "[2023-06-16 16:05:32,731][src.utils][INFO] - Metric name is None! Skipping metric value retrieval...\n",
      "[2023-06-16 16:05:32,734][HYDRA] \t#1 : experiment=food101 model=timm trainer.max_epochs=5 model.lr=0.001 data.batch_size=16 logger=wandb data_dir=data/food-101-tiny/\n",
      "Global seed set to 12345\n",
      "[2023-06-16 16:05:32,879][src.train_pipeline][INFO] - Instantiating datamodule <src.dataset.Food101LitDatamodule>\n",
      "[2023-06-16 16:05:32,881][src.train_pipeline][INFO] - Instantiating model <src.models.ClassificationLightningModule>\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/pytorch/utilities/parsing.py:196: UserWarning: Attribute 'net' is an instance of `nn.Module` and is already saved during checkpointing. It is recommended to ignore them using `self.save_hyperparameters(ignore=['net'])`.\n",
      "  rank_zero_warn(\n",
      "[2023-06-16 16:05:32,960][src.train_pipeline][INFO] - Instantiating callbacks...\n",
      "[2023-06-16 16:05:32,960][src.utils][INFO] - Instantiating callback <lightning.pytorch.callbacks.ModelCheckpoint>\n",
      "[2023-06-16 16:05:32,961][src.utils][INFO] - Instantiating callback <lightning.pytorch.callbacks.EarlyStopping>\n",
      "[2023-06-16 16:05:32,962][src.train_pipeline][INFO] - Instantiating loggers...\n",
      "[2023-06-16 16:05:32,962][src.utils][INFO] - Instantiating logger <lightning.pytorch.loggers.wandb.WandbLogger>\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/pytorch/loggers/wandb.py:396: UserWarning: There is a wandb run already in progress and newly created instances of `WandbLogger` will reuse this run. If this is not desired, call `wandb.finish()` before instantiating `WandbLogger`.\n",
      "  rank_zero_warn(\n",
      "[2023-06-16 16:05:32,965][src.train_pipeline][INFO] - Instantiating trainer <lightning.pytorch.trainer.Trainer>\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/fabric/connector.py:555: UserWarning: 16 is supported for historical reasons but its usage is discouraged. Please set your precision to 16-mixed instead!\n",
      "  rank_zero_warn(\n",
      "Using 16bit Automatic Mixed Precision (AMP)\n",
      "GPU available: True (cuda), used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "[2023-06-16 16:05:33,016][src.train_pipeline][INFO] - Logging hyperparameters!\n",
      "[2023-06-16 16:05:33,018][src.train_pipeline][INFO] - Starting training!\n",
      "/usr/local/lib/python3.8/dist-packages/lightning/pytorch/callbacks/model_checkpoint.py:615: UserWarning: Checkpoint directory /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints exists and is not empty.\n",
      "  rank_zero_warn(f\"Checkpoint directory {dirpath} exists and is not empty.\")\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "\n",
      "  | Name         | Type               | Params\n",
      "----------------------------------------------------\n",
      "0 | net          | EfficientNet       | 2.2 M \n",
      "1 | criterion    | CrossEntropyLoss   | 0     \n",
      "2 | train_acc    | MulticlassAccuracy | 0     \n",
      "3 | val_metrics  | MetricCollection   | 0     \n",
      "4 | test_metrics | MetricCollection   | 0     \n",
      "----------------------------------------------------\n",
      "2.2 M     Trainable params\n",
      "0         Non-trainable params\n",
      "2.2 M     Total params\n",
      "8.947     Total estimated model params size (MB)\n",
      "Epoch 0: 100%|██████████████████████| 94/94 [00:42<00:00,  2.23it/s, v_num=09s6]\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 1/32 [00:00<00:04,  7.06it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 2/32 [00:00<00:04,  7.34it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▊                 | 3/32 [00:00<00:03,  7.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██▍                | 4/32 [00:00<00:03,  7.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▉                | 5/32 [00:00<00:03,  8.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▌               | 6/32 [00:00<00:03,  8.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████▏              | 7/32 [00:00<00:03,  8.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▊              | 8/32 [00:00<00:02,  8.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|█████▎             | 9/32 [00:01<00:02,  8.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▋            | 10/32 [00:01<00:02,  8.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|██████▏           | 11/32 [00:01<00:02,  8.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 12/32 [00:01<00:02,  8.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▎          | 13/32 [00:01<00:02,  8.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▉          | 14/32 [00:01<00:02,  8.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████▍         | 15/32 [00:01<00:01,  8.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|█████████         | 16/32 [00:01<00:01,  8.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|█████████▌        | 17/32 [00:01<00:01,  8.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████▏       | 18/32 [00:02<00:01,  8.58it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▋       | 19/32 [00:02<00:01,  8.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▎      | 20/32 [00:02<00:01,  8.60it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|███████████▊      | 21/32 [00:02<00:01,  8.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|████████████▍     | 22/32 [00:02<00:01,  8.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|████████████▉     | 23/32 [00:02<00:01,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▌    | 24/32 [00:02<00:00,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 25/32 [00:02<00:00,  8.66it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▋   | 26/32 [00:02<00:00,  8.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 27/32 [00:03<00:00,  8.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|███████████████▊  | 28/32 [00:03<00:00,  8.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|████████████████▎ | 29/32 [00:03<00:00,  8.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▉ | 30/32 [00:03<00:00,  8.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 31/32 [00:03<00:00,  8.73it/s]\u001b[A\n",
      "Epoch 0: 100%|█| 94/94 [00:46<00:00,  2.04it/s, v_num=09s6, val/loss=3.540, val/\u001b[A\n",
      "Epoch 1: 100%|█| 94/94 [00:42<00:00,  2.23it/s, v_num=09s6, val/loss=3.540, val/\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 1/32 [00:00<00:03,  8.31it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 2/32 [00:00<00:03,  8.00it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▊                 | 3/32 [00:00<00:03,  7.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██▍                | 4/32 [00:00<00:03,  8.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▉                | 5/32 [00:00<00:03,  8.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▌               | 6/32 [00:00<00:03,  8.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████▏              | 7/32 [00:00<00:02,  8.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▊              | 8/32 [00:00<00:02,  8.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|█████▎             | 9/32 [00:01<00:02,  8.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▋            | 10/32 [00:01<00:02,  8.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|██████▏           | 11/32 [00:01<00:02,  8.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 12/32 [00:01<00:02,  8.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▎          | 13/32 [00:01<00:02,  8.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▉          | 14/32 [00:01<00:02,  8.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████▍         | 15/32 [00:01<00:01,  8.58it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|█████████         | 16/32 [00:01<00:01,  8.60it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|█████████▌        | 17/32 [00:01<00:01,  8.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████▏       | 18/32 [00:02<00:01,  8.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▋       | 19/32 [00:02<00:01,  8.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▎      | 20/32 [00:02<00:01,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|███████████▊      | 21/32 [00:02<00:01,  8.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|████████████▍     | 22/32 [00:02<00:01,  8.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|████████████▉     | 23/32 [00:02<00:01,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▌    | 24/32 [00:02<00:00,  8.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 25/32 [00:02<00:00,  8.66it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▋   | 26/32 [00:02<00:00,  8.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 27/32 [00:03<00:00,  8.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|███████████████▊  | 28/32 [00:03<00:00,  8.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|████████████████▎ | 29/32 [00:03<00:00,  8.71it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▉ | 30/32 [00:03<00:00,  8.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 31/32 [00:03<00:00,  8.73it/s]\u001b[A\n",
      "Epoch 1: 100%|█| 94/94 [00:46<00:00,  2.04it/s, v_num=09s6, val/loss=3.260, val/\u001b[A\n",
      "Epoch 2: 100%|█| 94/94 [00:42<00:00,  2.23it/s, v_num=09s6, val/loss=3.260, val/\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 1/32 [00:00<00:03,  8.54it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 2/32 [00:00<00:03,  8.14it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▊                 | 3/32 [00:00<00:03,  8.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██▍                | 4/32 [00:00<00:03,  8.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▉                | 5/32 [00:00<00:03,  8.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▌               | 6/32 [00:00<00:03,  8.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████▏              | 7/32 [00:00<00:02,  8.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▊              | 8/32 [00:00<00:02,  8.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|█████▎             | 9/32 [00:01<00:02,  8.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▋            | 10/32 [00:01<00:02,  8.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|██████▏           | 11/32 [00:01<00:02,  8.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 12/32 [00:01<00:02,  8.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▎          | 13/32 [00:01<00:02,  8.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▉          | 14/32 [00:01<00:02,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████▍         | 15/32 [00:01<00:01,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|█████████         | 16/32 [00:01<00:01,  8.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|█████████▌        | 17/32 [00:01<00:01,  8.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████▏       | 18/32 [00:02<00:01,  8.66it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▋       | 19/32 [00:02<00:01,  8.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▎      | 20/32 [00:02<00:01,  8.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|███████████▊      | 21/32 [00:02<00:01,  8.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|████████████▍     | 22/32 [00:02<00:01,  8.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|████████████▉     | 23/32 [00:02<00:01,  8.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▌    | 24/32 [00:02<00:00,  8.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 25/32 [00:02<00:00,  8.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▋   | 26/32 [00:02<00:00,  8.71it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 27/32 [00:03<00:00,  8.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|███████████████▊  | 28/32 [00:03<00:00,  8.73it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|████████████████▎ | 29/32 [00:03<00:00,  8.74it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▉ | 30/32 [00:03<00:00,  8.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 31/32 [00:03<00:00,  8.75it/s]\u001b[A\n",
      "Epoch 2: 100%|█| 94/94 [00:46<00:00,  2.04it/s, v_num=09s6, val/loss=2.370, val/\u001b[A\n",
      "Epoch 3: 100%|█| 94/94 [00:42<00:00,  2.23it/s, v_num=09s6, val/loss=2.370, val/\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 1/32 [00:00<00:03,  8.08it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 2/32 [00:00<00:03,  7.91it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▊                 | 3/32 [00:00<00:03,  8.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██▍                | 4/32 [00:00<00:03,  8.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▉                | 5/32 [00:00<00:03,  8.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▌               | 6/32 [00:00<00:03,  8.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████▏              | 7/32 [00:00<00:02,  8.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▊              | 8/32 [00:00<00:02,  8.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|█████▎             | 9/32 [00:01<00:02,  8.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▋            | 10/32 [00:01<00:02,  8.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|██████▏           | 11/32 [00:01<00:02,  8.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 12/32 [00:01<00:02,  8.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▎          | 13/32 [00:01<00:02,  8.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▉          | 14/32 [00:01<00:02,  8.58it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████▍         | 15/32 [00:01<00:01,  8.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|█████████         | 16/32 [00:01<00:01,  8.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|█████████▌        | 17/32 [00:01<00:01,  8.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████▏       | 18/32 [00:02<00:01,  8.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▋       | 19/32 [00:02<00:01,  8.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▎      | 20/32 [00:02<00:01,  8.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|███████████▊      | 21/32 [00:02<00:01,  8.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|████████████▍     | 22/32 [00:02<00:01,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|████████████▉     | 23/32 [00:02<00:01,  8.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▌    | 24/32 [00:02<00:00,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 25/32 [00:02<00:00,  8.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▋   | 26/32 [00:03<00:00,  8.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 27/32 [00:03<00:00,  8.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|███████████████▊  | 28/32 [00:03<00:00,  8.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|████████████████▎ | 29/32 [00:03<00:00,  8.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▉ | 30/32 [00:03<00:00,  8.71it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 31/32 [00:03<00:00,  8.71it/s]\u001b[A\n",
      "Epoch 3: 100%|█| 94/94 [00:46<00:00,  2.04it/s, v_num=09s6, val/loss=2.390, val/\u001b[A\n",
      "Epoch 4: 100%|█| 94/94 [00:42<00:00,  2.23it/s, v_num=09s6, val/loss=2.390, val/\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 1/32 [00:00<00:04,  7.51it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 2/32 [00:00<00:03,  7.55it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▊                 | 3/32 [00:00<00:03,  7.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██▍                | 4/32 [00:00<00:03,  7.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▉                | 5/32 [00:00<00:03,  8.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▌               | 6/32 [00:00<00:03,  8.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████▏              | 7/32 [00:00<00:03,  8.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▊              | 8/32 [00:00<00:02,  8.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|█████▎             | 9/32 [00:01<00:02,  8.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▋            | 10/32 [00:01<00:02,  8.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|██████▏           | 11/32 [00:01<00:02,  8.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 12/32 [00:01<00:02,  8.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▎          | 13/32 [00:01<00:02,  8.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▉          | 14/32 [00:01<00:02,  8.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████▍         | 15/32 [00:01<00:02,  8.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|█████████         | 16/32 [00:01<00:01,  8.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|█████████▌        | 17/32 [00:02<00:01,  8.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████▏       | 18/32 [00:02<00:01,  8.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▋       | 19/32 [00:02<00:01,  8.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▎      | 20/32 [00:02<00:01,  8.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|███████████▊      | 21/32 [00:02<00:01,  8.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|████████████▍     | 22/32 [00:02<00:01,  8.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|████████████▉     | 23/32 [00:02<00:01,  8.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▌    | 24/32 [00:02<00:00,  8.58it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 25/32 [00:02<00:00,  8.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▋   | 26/32 [00:03<00:00,  8.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 27/32 [00:03<00:00,  8.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|███████████████▊  | 28/32 [00:03<00:00,  8.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|████████████████▎ | 29/32 [00:03<00:00,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▉ | 30/32 [00:03<00:00,  8.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 31/32 [00:03<00:00,  8.66it/s]\u001b[A\n",
      "Epoch 4: 100%|█| 94/94 [00:46<00:00,  2.04it/s, v_num=09s6, val/loss=2.390, val/\u001b[A\n",
      "Epoch 4: 100%|█| 94/94 [00:46<00:00,  2.04it/s, v_num=09s6, val/loss=2.390, val/\u001b[A`Trainer.fit` stopped: `max_epochs=5` reached.\n",
      "Epoch 4: 100%|█| 94/94 [00:46<00:00,  2.02it/s, v_num=09s6, val/loss=2.390, val/\n",
      "[2023-06-16 16:09:26,582][src.train_pipeline][INFO] - Starting testing!\n",
      "Restoring states from the checkpoint path at /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003-v1.ckpt\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "Loaded model weights from the checkpoint at /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003-v1.ckpt\n",
      "Testing DataLoader 0: 100%|█████████████████████| 32/32 [00:03<00:00,  8.97it/s]\n",
      "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
      "┃\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m┃\n",
      "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/acc         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2540000081062317    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m       test/auroc        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m  0.038327619433403015   \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m         test/f1         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2540000081062317    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/loss        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m    2.392517566680908    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/prec        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2540000081062317    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/rec         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2540000081062317    \u001b[0m\u001b[35m \u001b[0m│\n",
      "└───────────────────────────┴───────────────────────────┘\n",
      "[2023-06-16 16:09:31,066][src.train_pipeline][INFO] - Best ckpt path: /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003-v1.ckpt\n",
      "[2023-06-16 16:09:31,067][src.utils][INFO] - Metric name is None! Skipping metric value retrieval...\n",
      "[2023-06-16 16:09:31,069][HYDRA] \t#2 : experiment=food101 model=timm trainer.max_epochs=5 model.lr=0.0001 data.batch_size=8 logger=wandb data_dir=data/food-101-tiny/\n",
      "Global seed set to 12345\n",
      "[2023-06-16 16:09:31,217][src.train_pipeline][INFO] - Instantiating datamodule <src.dataset.Food101LitDatamodule>\n",
      "[2023-06-16 16:09:31,218][src.train_pipeline][INFO] - Instantiating model <src.models.ClassificationLightningModule>\n",
      "[2023-06-16 16:09:31,292][src.train_pipeline][INFO] - Instantiating callbacks...\n",
      "[2023-06-16 16:09:31,292][src.utils][INFO] - Instantiating callback <lightning.pytorch.callbacks.ModelCheckpoint>\n",
      "[2023-06-16 16:09:31,294][src.utils][INFO] - Instantiating callback <lightning.pytorch.callbacks.EarlyStopping>\n",
      "[2023-06-16 16:09:31,294][src.train_pipeline][INFO] - Instantiating loggers...\n",
      "[2023-06-16 16:09:31,295][src.utils][INFO] - Instantiating logger <lightning.pytorch.loggers.wandb.WandbLogger>\n",
      "[2023-06-16 16:09:31,297][src.train_pipeline][INFO] - Instantiating trainer <lightning.pytorch.trainer.Trainer>\n",
      "Using 16bit Automatic Mixed Precision (AMP)\n",
      "GPU available: True (cuda), used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "[2023-06-16 16:09:31,346][src.train_pipeline][INFO] - Logging hyperparameters!\n",
      "[2023-06-16 16:09:31,348][src.train_pipeline][INFO] - Starting training!\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "\n",
      "  | Name         | Type               | Params\n",
      "----------------------------------------------------\n",
      "0 | net          | EfficientNet       | 2.2 M \n",
      "1 | criterion    | CrossEntropyLoss   | 0     \n",
      "2 | train_acc    | MulticlassAccuracy | 0     \n",
      "3 | val_metrics  | MetricCollection   | 0     \n",
      "4 | test_metrics | MetricCollection   | 0     \n",
      "----------------------------------------------------\n",
      "2.2 M     Trainable params\n",
      "0         Non-trainable params\n",
      "2.2 M     Total params\n",
      "8.947     Total estimated model params size (MB)\n",
      "Sanity Checking: 0it [00:00, ?it/s]Traceback (most recent call last):\n",
      "  File \"/usr/lib/python3.8/multiprocessing/util.py\", line 300, in _run_finalizers\n",
      "    finalizer()\n",
      "  File \"/usr/lib/python3.8/multiprocessing/util.py\", line 224, in __call__\n",
      "    res = self._callback(*self._args, **self._kwargs)\n",
      "  File \"/usr/lib/python3.8/multiprocessing/util.py\", line 133, in _remove_temp_dir\n",
      "    rmtree(tempdir)\n",
      "  File \"/usr/lib/python3.8/shutil.py\", line 722, in rmtree\n",
      "    onerror(os.rmdir, path, sys.exc_info())\n",
      "  File \"/usr/lib/python3.8/shutil.py\", line 720, in rmtree\n",
      "    os.rmdir(path)\n",
      "OSError: [Errno 39] Directory not empty: '/tmp/pymp-49l7kc6w'\n",
      "Epoch 0: 100%|████████████████████| 188/188 [00:43<00:00,  4.34it/s, v_num=09s6]\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                  | 1/63 [00:00<00:04, 13.60it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 2/63 [00:00<00:04, 13.60it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                  | 3/63 [00:00<00:04, 13.83it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 4/63 [00:00<00:04, 14.07it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▌                 | 5/63 [00:00<00:04, 14.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▊                 | 6/63 [00:00<00:03, 14.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|██                 | 7/63 [00:00<00:03, 14.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▍                | 8/63 [00:00<00:03, 14.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▋                | 9/63 [00:00<00:03, 14.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▊               | 10/63 [00:00<00:03, 14.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|███▏              | 11/63 [00:00<00:03, 14.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▍              | 12/63 [00:00<00:03, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▋              | 13/63 [00:00<00:03, 15.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████              | 14/63 [00:00<00:03, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████▎             | 15/63 [00:00<00:03, 15.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▌             | 16/63 [00:01<00:03, 15.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▊             | 17/63 [00:01<00:03, 15.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|█████▏            | 18/63 [00:01<00:02, 15.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████▍            | 19/63 [00:01<00:02, 15.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▋            | 20/63 [00:01<00:02, 15.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|██████            | 21/63 [00:01<00:02, 15.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|██████▎           | 22/63 [00:01<00:02, 15.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▌           | 23/63 [00:01<00:02, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 24/63 [00:01<00:02, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|███████▏          | 25/63 [00:01<00:02, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▍          | 26/63 [00:01<00:02, 14.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▋          | 27/63 [00:01<00:02, 14.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|████████          | 28/63 [00:01<00:02, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|████████▎         | 29/63 [00:01<00:02, 14.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▌         | 30/63 [00:02<00:02, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▊         | 31/63 [00:02<00:02, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|█████████▏        | 32/63 [00:02<00:02, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|█████████▍        | 33/63 [00:02<00:02, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▋        | 34/63 [00:02<00:01, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████        | 35/63 [00:02<00:01, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|██████████▎       | 36/63 [00:02<00:01, 14.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▌       | 37/63 [00:02<00:01, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▊       | 38/63 [00:02<00:01, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▏      | 39/63 [00:02<00:01, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|███████████▍      | 40/63 [00:02<00:01, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|███████████▋      | 41/63 [00:02<00:01, 14.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|████████████      | 42/63 [00:02<00:01, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|████████████▎     | 43/63 [00:02<00:01, 14.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|████████████▌     | 44/63 [00:02<00:01, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|████████████▊     | 45/63 [00:03<00:01, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|█████████████▏    | 46/63 [00:03<00:01, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▍    | 47/63 [00:03<00:01, 14.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|█████████████▋    | 48/63 [00:03<00:01, 14.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 49/63 [00:03<00:00, 14.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|██████████████▎   | 50/63 [00:03<00:00, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▌   | 51/63 [00:03<00:00, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|██████████████▊   | 52/63 [00:03<00:00, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 53/63 [00:03<00:00, 15.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|███████████████▍  | 54/63 [00:03<00:00, 15.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|███████████████▋  | 55/63 [00:03<00:00, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|████████████████  | 56/63 [00:03<00:00, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|████████████████▎ | 57/63 [00:03<00:00, 15.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|████████████████▌ | 58/63 [00:03<00:00, 15.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▊ | 59/63 [00:03<00:00, 15.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|█████████████████▏| 60/63 [00:03<00:00, 15.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 61/63 [00:04<00:00, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|█████████████████▋| 62/63 [00:04<00:00, 15.07it/s]\u001b[A\n",
      "Epoch 0: 100%|█| 188/188 [00:47<00:00,  3.94it/s, v_num=09s6, val/loss=2.150, va\u001b[A\n",
      "Epoch 1: 100%|█| 188/188 [00:43<00:00,  4.33it/s, v_num=09s6, val/loss=2.150, va\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                  | 1/63 [00:00<00:04, 13.08it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 2/63 [00:00<00:04, 12.64it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                  | 3/63 [00:00<00:04, 12.98it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 4/63 [00:00<00:04, 13.18it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▌                 | 5/63 [00:00<00:04, 13.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▊                 | 6/63 [00:00<00:04, 13.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|██                 | 7/63 [00:00<00:04, 13.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▍                | 8/63 [00:00<00:03, 14.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▋                | 9/63 [00:00<00:03, 14.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▊               | 10/63 [00:00<00:03, 14.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|███▏              | 11/63 [00:00<00:03, 14.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▍              | 12/63 [00:00<00:03, 14.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▋              | 13/63 [00:00<00:03, 14.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████              | 14/63 [00:00<00:03, 14.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████▎             | 15/63 [00:01<00:03, 14.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▌             | 16/63 [00:01<00:03, 14.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▊             | 17/63 [00:01<00:03, 14.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|█████▏            | 18/63 [00:01<00:03, 14.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████▍            | 19/63 [00:01<00:02, 14.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▋            | 20/63 [00:01<00:02, 14.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|██████            | 21/63 [00:01<00:02, 14.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|██████▎           | 22/63 [00:01<00:02, 14.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▌           | 23/63 [00:01<00:02, 14.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 24/63 [00:01<00:02, 14.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|███████▏          | 25/63 [00:01<00:02, 14.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▍          | 26/63 [00:01<00:02, 14.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▋          | 27/63 [00:01<00:02, 14.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|████████          | 28/63 [00:01<00:02, 14.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|████████▎         | 29/63 [00:01<00:02, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▌         | 30/63 [00:02<00:02, 14.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▊         | 31/63 [00:02<00:02, 14.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|█████████▏        | 32/63 [00:02<00:02, 14.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|█████████▍        | 33/63 [00:02<00:02, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▋        | 34/63 [00:02<00:01, 14.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████        | 35/63 [00:02<00:01, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|██████████▎       | 36/63 [00:02<00:01, 14.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▌       | 37/63 [00:02<00:01, 14.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▊       | 38/63 [00:02<00:01, 14.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▏      | 39/63 [00:02<00:01, 14.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|███████████▍      | 40/63 [00:02<00:01, 14.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|███████████▋      | 41/63 [00:02<00:01, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|████████████      | 42/63 [00:02<00:01, 14.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|████████████▎     | 43/63 [00:02<00:01, 14.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|████████████▌     | 44/63 [00:02<00:01, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|████████████▊     | 45/63 [00:03<00:01, 14.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|█████████████▏    | 46/63 [00:03<00:01, 14.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▍    | 47/63 [00:03<00:01, 14.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|█████████████▋    | 48/63 [00:03<00:01, 14.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 49/63 [00:03<00:00, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|██████████████▎   | 50/63 [00:03<00:00, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▌   | 51/63 [00:03<00:00, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|██████████████▊   | 52/63 [00:03<00:00, 14.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 53/63 [00:03<00:00, 14.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|███████████████▍  | 54/63 [00:03<00:00, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|███████████████▋  | 55/63 [00:03<00:00, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|████████████████  | 56/63 [00:03<00:00, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|████████████████▎ | 57/63 [00:03<00:00, 14.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|████████████████▌ | 58/63 [00:03<00:00, 14.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▊ | 59/63 [00:03<00:00, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|█████████████████▏| 60/63 [00:03<00:00, 15.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 61/63 [00:04<00:00, 15.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|█████████████████▋| 62/63 [00:04<00:00, 15.04it/s]\u001b[A\n",
      "Epoch 1: 100%|█| 188/188 [00:47<00:00,  3.93it/s, v_num=09s6, val/loss=2.060, va\u001b[A\n",
      "Epoch 2: 100%|█| 188/188 [00:43<00:00,  4.33it/s, v_num=09s6, val/loss=2.060, va\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                  | 1/63 [00:00<00:05, 11.67it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 2/63 [00:00<00:04, 12.54it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                  | 3/63 [00:00<00:04, 13.19it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 4/63 [00:00<00:04, 13.68it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▌                 | 5/63 [00:00<00:04, 13.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▊                 | 6/63 [00:00<00:04, 14.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|██                 | 7/63 [00:00<00:03, 14.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▍                | 8/63 [00:00<00:03, 14.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▋                | 9/63 [00:00<00:03, 14.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▊               | 10/63 [00:00<00:03, 14.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|███▏              | 11/63 [00:00<00:03, 14.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▍              | 12/63 [00:00<00:03, 14.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▋              | 13/63 [00:00<00:03, 14.76it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████              | 14/63 [00:00<00:03, 14.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████▎             | 15/63 [00:01<00:03, 14.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▌             | 16/63 [00:01<00:03, 14.77it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▊             | 17/63 [00:01<00:03, 14.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|█████▏            | 18/63 [00:01<00:03, 14.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████▍            | 19/63 [00:01<00:02, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▋            | 20/63 [00:01<00:02, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|██████            | 21/63 [00:01<00:02, 14.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|██████▎           | 22/63 [00:01<00:02, 14.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▌           | 23/63 [00:01<00:02, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 24/63 [00:01<00:02, 14.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|███████▏          | 25/63 [00:01<00:02, 15.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▍          | 26/63 [00:01<00:02, 14.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▋          | 27/63 [00:01<00:02, 14.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|████████          | 28/63 [00:01<00:02, 15.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|████████▎         | 29/63 [00:01<00:02, 15.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▌         | 30/63 [00:01<00:02, 15.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▊         | 31/63 [00:02<00:02, 15.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|█████████▏        | 32/63 [00:02<00:02, 15.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|█████████▍        | 33/63 [00:02<00:01, 15.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▋        | 34/63 [00:02<00:01, 15.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████        | 35/63 [00:02<00:01, 15.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|██████████▎       | 36/63 [00:02<00:01, 15.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▌       | 37/63 [00:02<00:01, 15.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▊       | 38/63 [00:02<00:01, 15.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▏      | 39/63 [00:02<00:01, 15.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|███████████▍      | 40/63 [00:02<00:01, 15.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|███████████▋      | 41/63 [00:02<00:01, 15.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|████████████      | 42/63 [00:02<00:01, 15.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|████████████▎     | 43/63 [00:02<00:01, 15.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|████████████▌     | 44/63 [00:02<00:01, 15.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|████████████▊     | 45/63 [00:02<00:01, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|█████████████▏    | 46/63 [00:03<00:01, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▍    | 47/63 [00:03<00:01, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|█████████████▋    | 48/63 [00:03<00:00, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 49/63 [00:03<00:00, 15.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|██████████████▎   | 50/63 [00:03<00:00, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▌   | 51/63 [00:03<00:00, 15.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|██████████████▊   | 52/63 [00:03<00:00, 15.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 53/63 [00:03<00:00, 15.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|███████████████▍  | 54/63 [00:03<00:00, 15.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|███████████████▋  | 55/63 [00:03<00:00, 15.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|████████████████  | 56/63 [00:03<00:00, 15.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|████████████████▎ | 57/63 [00:03<00:00, 15.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|████████████████▌ | 58/63 [00:03<00:00, 15.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▊ | 59/63 [00:03<00:00, 15.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|█████████████████▏| 60/63 [00:03<00:00, 15.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 61/63 [00:04<00:00, 15.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|█████████████████▋| 62/63 [00:04<00:00, 15.14it/s]\u001b[A\n",
      "Epoch 2: 100%|█| 188/188 [00:47<00:00,  3.93it/s, v_num=09s6, val/loss=2.090, va\u001b[A\n",
      "Epoch 3: 100%|█| 188/188 [00:43<00:00,  4.33it/s, v_num=09s6, val/loss=2.090, va\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                  | 1/63 [00:00<00:04, 13.25it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 2/63 [00:00<00:04, 13.06it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                  | 3/63 [00:00<00:04, 13.61it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 4/63 [00:00<00:04, 13.67it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▌                 | 5/63 [00:00<00:04, 13.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▊                 | 6/63 [00:00<00:04, 13.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|██                 | 7/63 [00:00<00:04, 14.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▍                | 8/63 [00:00<00:03, 14.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▋                | 9/63 [00:00<00:03, 14.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▊               | 10/63 [00:00<00:03, 14.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|███▏              | 11/63 [00:00<00:03, 14.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▍              | 12/63 [00:00<00:03, 14.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▋              | 13/63 [00:00<00:03, 14.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████              | 14/63 [00:00<00:03, 14.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████▎             | 15/63 [00:01<00:03, 14.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▌             | 16/63 [00:01<00:03, 14.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▊             | 17/63 [00:01<00:03, 14.66it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|█████▏            | 18/63 [00:01<00:03, 14.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████▍            | 19/63 [00:01<00:02, 14.73it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▋            | 20/63 [00:01<00:02, 14.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|██████            | 21/63 [00:01<00:02, 14.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|██████▎           | 22/63 [00:01<00:02, 14.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▌           | 23/63 [00:01<00:02, 14.77it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 24/63 [00:01<00:02, 14.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|███████▏          | 25/63 [00:01<00:02, 14.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▍          | 26/63 [00:01<00:02, 14.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▋          | 27/63 [00:01<00:02, 14.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|████████          | 28/63 [00:01<00:02, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|████████▎         | 29/63 [00:01<00:02, 14.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▌         | 30/63 [00:02<00:02, 14.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▊         | 31/63 [00:02<00:02, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|█████████▏        | 32/63 [00:02<00:02, 14.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|█████████▍        | 33/63 [00:02<00:02, 14.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▋        | 34/63 [00:02<00:01, 14.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████        | 35/63 [00:02<00:01, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|██████████▎       | 36/63 [00:02<00:01, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▌       | 37/63 [00:02<00:01, 14.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▊       | 38/63 [00:02<00:01, 14.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▏      | 39/63 [00:02<00:01, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|███████████▍      | 40/63 [00:02<00:01, 14.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|███████████▋      | 41/63 [00:02<00:01, 14.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|████████████      | 42/63 [00:02<00:01, 14.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|████████████▎     | 43/63 [00:02<00:01, 14.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|████████████▌     | 44/63 [00:02<00:01, 14.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|████████████▊     | 45/63 [00:03<00:01, 14.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|█████████████▏    | 46/63 [00:03<00:01, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▍    | 47/63 [00:03<00:01, 14.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|█████████████▋    | 48/63 [00:03<00:01, 14.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 49/63 [00:03<00:00, 14.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|██████████████▎   | 50/63 [00:03<00:00, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▌   | 51/63 [00:03<00:00, 14.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|██████████████▊   | 52/63 [00:03<00:00, 14.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 53/63 [00:03<00:00, 14.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|███████████████▍  | 54/63 [00:03<00:00, 14.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|███████████████▋  | 55/63 [00:03<00:00, 14.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|████████████████  | 56/63 [00:03<00:00, 14.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|████████████████▎ | 57/63 [00:03<00:00, 14.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|████████████████▌ | 58/63 [00:03<00:00, 14.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▊ | 59/63 [00:03<00:00, 14.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|█████████████████▏| 60/63 [00:03<00:00, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 61/63 [00:04<00:00, 15.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|█████████████████▋| 62/63 [00:04<00:00, 15.01it/s]\u001b[A\n",
      "Epoch 3: 100%|█| 188/188 [00:47<00:00,  3.93it/s, v_num=09s6, val/loss=2.050, va\u001b[A\n",
      "Epoch 4: 100%|█| 188/188 [00:43<00:00,  4.33it/s, v_num=09s6, val/loss=2.050, va\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/63 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                  | 1/63 [00:00<00:05, 11.24it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 2/63 [00:00<00:05, 11.58it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                  | 3/63 [00:00<00:04, 12.18it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 4/63 [00:00<00:04, 12.84it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▌                 | 5/63 [00:00<00:04, 13.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▊                 | 6/63 [00:00<00:04, 13.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|██                 | 7/63 [00:00<00:04, 13.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▍                | 8/63 [00:00<00:04, 13.73it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▋                | 9/63 [00:00<00:03, 13.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▊               | 10/63 [00:00<00:03, 13.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|███▏              | 11/63 [00:00<00:03, 14.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▍              | 12/63 [00:00<00:03, 14.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▋              | 13/63 [00:00<00:03, 14.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████              | 14/63 [00:00<00:03, 14.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████▎             | 15/63 [00:01<00:03, 14.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▌             | 16/63 [00:01<00:03, 14.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▊             | 17/63 [00:01<00:03, 14.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|█████▏            | 18/63 [00:01<00:03, 14.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████▍            | 19/63 [00:01<00:03, 14.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▋            | 20/63 [00:01<00:02, 14.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|██████            | 21/63 [00:01<00:02, 14.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|██████▎           | 22/63 [00:01<00:02, 14.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▌           | 23/63 [00:01<00:02, 14.73it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 24/63 [00:01<00:02, 14.76it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|███████▏          | 25/63 [00:01<00:02, 14.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▍          | 26/63 [00:01<00:02, 14.81it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▋          | 27/63 [00:01<00:02, 14.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|████████          | 28/63 [00:01<00:02, 14.74it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|████████▎         | 29/63 [00:01<00:02, 14.73it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▌         | 30/63 [00:02<00:02, 14.74it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▊         | 31/63 [00:02<00:02, 14.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|█████████▏        | 32/63 [00:02<00:02, 14.77it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|█████████▍        | 33/63 [00:02<00:02, 14.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▋        | 34/63 [00:02<00:01, 14.81it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████        | 35/63 [00:02<00:01, 14.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|██████████▎       | 36/63 [00:02<00:01, 14.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▌       | 37/63 [00:02<00:01, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▊       | 38/63 [00:02<00:01, 14.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▏      | 39/63 [00:02<00:01, 14.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|███████████▍      | 40/63 [00:02<00:01, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|███████████▋      | 41/63 [00:02<00:01, 14.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|████████████      | 42/63 [00:02<00:01, 14.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|████████████▎     | 43/63 [00:02<00:01, 14.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|████████████▌     | 44/63 [00:02<00:01, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|████████████▊     | 45/63 [00:03<00:01, 14.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|█████████████▏    | 46/63 [00:03<00:01, 14.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▍    | 47/63 [00:03<00:01, 14.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|█████████████▋    | 48/63 [00:03<00:01, 14.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 49/63 [00:03<00:00, 14.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|██████████████▎   | 50/63 [00:03<00:00, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▌   | 51/63 [00:03<00:00, 14.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|██████████████▊   | 52/63 [00:03<00:00, 14.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 53/63 [00:03<00:00, 14.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|███████████████▍  | 54/63 [00:03<00:00, 14.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|███████████████▋  | 55/63 [00:03<00:00, 14.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|████████████████  | 56/63 [00:03<00:00, 14.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|████████████████▎ | 57/63 [00:03<00:00, 14.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|████████████████▌ | 58/63 [00:03<00:00, 14.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▊ | 59/63 [00:03<00:00, 14.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|█████████████████▏| 60/63 [00:04<00:00, 14.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 61/63 [00:04<00:00, 14.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|█████████████████▋| 62/63 [00:04<00:00, 14.94it/s]\u001b[A\n",
      "Epoch 4: 100%|█| 188/188 [00:47<00:00,  3.93it/s, v_num=09s6, val/loss=2.050, va\u001b[A\n",
      "Epoch 4: 100%|█| 188/188 [00:47<00:00,  3.93it/s, v_num=09s6, val/loss=2.050, va\u001b[A`Trainer.fit` stopped: `max_epochs=5` reached.\n",
      "Epoch 4: 100%|█| 188/188 [00:48<00:00,  3.91it/s, v_num=09s6, val/loss=2.050, va\n",
      "[2023-06-16 16:13:33,124][src.train_pipeline][INFO] - Starting testing!\n",
      "Restoring states from the checkpoint path at /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003-v2.ckpt\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "Loaded model weights from the checkpoint at /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003-v2.ckpt\n",
      "Testing DataLoader 0: 100%|█████████████████████| 63/63 [00:04<00:00, 15.38it/s]\n",
      "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
      "┃\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m┃\n",
      "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/acc         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.27799999713897705   \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m       test/auroc        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m  0.014866666868329048   \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m         test/f1         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.27799999713897705   \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/loss        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m    2.050128936767578    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/prec        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.27799999713897705   \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/rec         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.27799999713897705   \u001b[0m\u001b[35m \u001b[0m│\n",
      "└───────────────────────────┴───────────────────────────┘\n",
      "[2023-06-16 16:13:37,902][src.train_pipeline][INFO] - Best ckpt path: /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003-v2.ckpt\n",
      "[2023-06-16 16:13:37,902][src.utils][INFO] - Metric name is None! Skipping metric value retrieval...\n",
      "[2023-06-16 16:13:37,905][HYDRA] \t#3 : experiment=food101 model=timm trainer.max_epochs=5 model.lr=0.0001 data.batch_size=16 logger=wandb data_dir=data/food-101-tiny/\n",
      "Global seed set to 12345\n",
      "[2023-06-16 16:13:38,056][src.train_pipeline][INFO] - Instantiating datamodule <src.dataset.Food101LitDatamodule>\n",
      "[2023-06-16 16:13:38,058][src.train_pipeline][INFO] - Instantiating model <src.models.ClassificationLightningModule>\n",
      "[2023-06-16 16:13:38,141][src.train_pipeline][INFO] - Instantiating callbacks...\n",
      "[2023-06-16 16:13:38,142][src.utils][INFO] - Instantiating callback <lightning.pytorch.callbacks.ModelCheckpoint>\n",
      "[2023-06-16 16:13:38,143][src.utils][INFO] - Instantiating callback <lightning.pytorch.callbacks.EarlyStopping>\n",
      "[2023-06-16 16:13:38,144][src.train_pipeline][INFO] - Instantiating loggers...\n",
      "[2023-06-16 16:13:38,144][src.utils][INFO] - Instantiating logger <lightning.pytorch.loggers.wandb.WandbLogger>\n",
      "[2023-06-16 16:13:38,147][src.train_pipeline][INFO] - Instantiating trainer <lightning.pytorch.trainer.Trainer>\n",
      "Using 16bit Automatic Mixed Precision (AMP)\n",
      "GPU available: True (cuda), used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n",
      "[2023-06-16 16:13:38,201][src.train_pipeline][INFO] - Logging hyperparameters!\n",
      "[2023-06-16 16:13:38,204][src.train_pipeline][INFO] - Starting training!\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "\n",
      "  | Name         | Type               | Params\n",
      "----------------------------------------------------\n",
      "0 | net          | EfficientNet       | 2.2 M \n",
      "1 | criterion    | CrossEntropyLoss   | 0     \n",
      "2 | train_acc    | MulticlassAccuracy | 0     \n",
      "3 | val_metrics  | MetricCollection   | 0     \n",
      "4 | test_metrics | MetricCollection   | 0     \n",
      "----------------------------------------------------\n",
      "2.2 M     Trainable params\n",
      "0         Non-trainable params\n",
      "2.2 M     Total params\n",
      "8.947     Total estimated model params size (MB)\n",
      "Epoch 0: 100%|██████████████████████| 94/94 [00:42<00:00,  2.23it/s, v_num=09s6]\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 1/32 [00:00<00:03,  8.08it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 2/32 [00:00<00:03,  7.76it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▊                 | 3/32 [00:00<00:03,  7.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██▍                | 4/32 [00:00<00:03,  8.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▉                | 5/32 [00:00<00:03,  8.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▌               | 6/32 [00:00<00:03,  8.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████▏              | 7/32 [00:00<00:03,  8.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▊              | 8/32 [00:00<00:02,  8.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|█████▎             | 9/32 [00:01<00:02,  8.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▋            | 10/32 [00:01<00:02,  8.41it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|██████▏           | 11/32 [00:01<00:02,  8.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 12/32 [00:01<00:02,  8.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▎          | 13/32 [00:01<00:02,  8.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▉          | 14/32 [00:01<00:02,  8.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████▍         | 15/32 [00:01<00:02,  8.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|█████████         | 16/32 [00:01<00:01,  8.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|█████████▌        | 17/32 [00:01<00:01,  8.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████▏       | 18/32 [00:02<00:01,  8.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▋       | 19/32 [00:02<00:01,  8.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▎      | 20/32 [00:02<00:01,  8.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|███████████▊      | 21/32 [00:02<00:01,  8.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|████████████▍     | 22/32 [00:02<00:01,  8.58it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|████████████▉     | 23/32 [00:02<00:01,  8.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▌    | 24/32 [00:02<00:00,  8.60it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 25/32 [00:02<00:00,  8.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▋   | 26/32 [00:03<00:00,  8.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 27/32 [00:03<00:00,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|███████████████▊  | 28/32 [00:03<00:00,  8.66it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|████████████████▎ | 29/32 [00:03<00:00,  8.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▉ | 30/32 [00:03<00:00,  8.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 31/32 [00:03<00:00,  8.68it/s]\u001b[A\n",
      "Epoch 0: 100%|█| 94/94 [00:46<00:00,  2.04it/s, v_num=09s6, val/loss=2.200, val/\u001b[A\n",
      "Epoch 1: 100%|█| 94/94 [00:42<00:00,  2.23it/s, v_num=09s6, val/loss=2.200, val/\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 1/32 [00:00<00:03,  7.92it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 2/32 [00:00<00:03,  7.59it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▊                 | 3/32 [00:00<00:03,  7.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██▍                | 4/32 [00:00<00:03,  7.77it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▉                | 5/32 [00:00<00:03,  7.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▌               | 6/32 [00:00<00:03,  7.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████▏              | 7/32 [00:00<00:03,  8.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▊              | 8/32 [00:00<00:02,  8.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|█████▎             | 9/32 [00:01<00:02,  8.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▋            | 10/32 [00:01<00:02,  8.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|██████▏           | 11/32 [00:01<00:02,  8.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 12/32 [00:01<00:02,  8.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▎          | 13/32 [00:01<00:02,  8.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▉          | 14/32 [00:01<00:02,  8.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████▍         | 15/32 [00:01<00:02,  8.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|█████████         | 16/32 [00:01<00:01,  8.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|█████████▌        | 17/32 [00:02<00:01,  8.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████▏       | 18/32 [00:02<00:01,  8.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▋       | 19/32 [00:02<00:01,  8.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▎      | 20/32 [00:02<00:01,  8.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|███████████▊      | 21/32 [00:02<00:01,  8.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|████████████▍     | 22/32 [00:02<00:01,  8.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|████████████▉     | 23/32 [00:02<00:01,  8.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▌    | 24/32 [00:02<00:00,  8.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 25/32 [00:02<00:00,  8.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▋   | 26/32 [00:03<00:00,  8.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 27/32 [00:03<00:00,  8.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|███████████████▊  | 28/32 [00:03<00:00,  8.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|████████████████▎ | 29/32 [00:03<00:00,  8.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▉ | 30/32 [00:03<00:00,  8.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 31/32 [00:03<00:00,  8.57it/s]\u001b[A\n",
      "Epoch 1: 100%|█| 94/94 [00:46<00:00,  2.04it/s, v_num=09s6, val/loss=2.070, val/\u001b[A\n",
      "Epoch 2: 100%|█| 94/94 [00:42<00:00,  2.23it/s, v_num=09s6, val/loss=2.070, val/\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 1/32 [00:00<00:03,  8.74it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 2/32 [00:00<00:03,  8.16it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▊                 | 3/32 [00:00<00:03,  8.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██▍                | 4/32 [00:00<00:03,  7.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▉                | 5/32 [00:00<00:03,  8.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▌               | 6/32 [00:00<00:03,  8.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████▏              | 7/32 [00:00<00:03,  8.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▊              | 8/32 [00:00<00:02,  8.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|█████▎             | 9/32 [00:01<00:02,  8.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▋            | 10/32 [00:01<00:02,  8.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|██████▏           | 11/32 [00:01<00:02,  8.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 12/32 [00:01<00:02,  8.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▎          | 13/32 [00:01<00:02,  8.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▉          | 14/32 [00:01<00:02,  8.41it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████▍         | 15/32 [00:01<00:02,  8.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|█████████         | 16/32 [00:01<00:01,  8.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|█████████▌        | 17/32 [00:02<00:01,  8.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████▏       | 18/32 [00:02<00:01,  8.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▋       | 19/32 [00:02<00:01,  8.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▎      | 20/32 [00:02<00:01,  8.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|███████████▊      | 21/32 [00:02<00:01,  8.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|████████████▍     | 22/32 [00:02<00:01,  8.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|████████████▉     | 23/32 [00:02<00:01,  8.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▌    | 24/32 [00:02<00:00,  8.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 25/32 [00:02<00:00,  8.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▋   | 26/32 [00:03<00:00,  8.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 27/32 [00:03<00:00,  8.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|███████████████▊  | 28/32 [00:03<00:00,  8.58it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|████████████████▎ | 29/32 [00:03<00:00,  8.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▉ | 30/32 [00:03<00:00,  8.60it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 31/32 [00:03<00:00,  8.61it/s]\u001b[A\n",
      "Epoch 2: 100%|█| 94/94 [00:46<00:00,  2.03it/s, v_num=09s6, val/loss=2.080, val/\u001b[A\n",
      "Epoch 3: 100%|█| 94/94 [00:42<00:00,  2.23it/s, v_num=09s6, val/loss=2.080, val/\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 1/32 [00:00<00:03,  8.58it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 2/32 [00:00<00:03,  8.10it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▊                 | 3/32 [00:00<00:03,  8.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██▍                | 4/32 [00:00<00:03,  7.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▉                | 5/32 [00:00<00:03,  8.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▌               | 6/32 [00:00<00:03,  8.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████▏              | 7/32 [00:00<00:03,  8.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▊              | 8/32 [00:00<00:02,  8.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|█████▎             | 9/32 [00:01<00:02,  8.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▋            | 10/32 [00:01<00:02,  8.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|██████▏           | 11/32 [00:01<00:02,  8.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 12/32 [00:01<00:02,  8.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▎          | 13/32 [00:01<00:02,  8.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▉          | 14/32 [00:01<00:02,  8.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████▍         | 15/32 [00:01<00:02,  8.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|█████████         | 16/32 [00:01<00:01,  8.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|█████████▌        | 17/32 [00:01<00:01,  8.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████▏       | 18/32 [00:02<00:01,  8.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▋       | 19/32 [00:02<00:01,  8.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▎      | 20/32 [00:02<00:01,  8.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|███████████▊      | 21/32 [00:02<00:01,  8.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|████████████▍     | 22/32 [00:02<00:01,  8.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|████████████▉     | 23/32 [00:02<00:01,  8.58it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▌    | 24/32 [00:02<00:00,  8.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 25/32 [00:02<00:00,  8.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▋   | 26/32 [00:03<00:00,  8.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 27/32 [00:03<00:00,  8.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|███████████████▊  | 28/32 [00:03<00:00,  8.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|████████████████▎ | 29/32 [00:03<00:00,  8.66it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▉ | 30/32 [00:03<00:00,  8.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 31/32 [00:03<00:00,  8.68it/s]\u001b[A\n",
      "Epoch 3: 100%|█| 94/94 [00:46<00:00,  2.03it/s, v_num=09s6, val/loss=2.000, val/\u001b[A\n",
      "Epoch 4: 100%|█| 94/94 [00:42<00:00,  2.23it/s, v_num=09s6, val/loss=2.000, val/\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                        | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                           | 0/32 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                  | 1/32 [00:00<00:03,  7.89it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█▏                 | 2/32 [00:00<00:03,  7.73it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▊                 | 3/32 [00:00<00:03,  7.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██▍                | 4/32 [00:00<00:03,  7.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▉                | 5/32 [00:00<00:03,  7.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▌               | 6/32 [00:00<00:03,  7.81it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|████▏              | 7/32 [00:00<00:03,  7.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▊              | 8/32 [00:01<00:03,  7.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|█████▎             | 9/32 [00:01<00:02,  8.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▋            | 10/32 [00:01<00:02,  8.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|██████▏           | 11/32 [00:01<00:02,  8.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▊           | 12/32 [00:01<00:02,  8.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████▎          | 13/32 [00:01<00:02,  8.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▉          | 14/32 [00:01<00:02,  8.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████▍         | 15/32 [00:01<00:02,  8.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|█████████         | 16/32 [00:01<00:01,  8.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|█████████▌        | 17/32 [00:02<00:01,  8.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|██████████▏       | 18/32 [00:02<00:01,  8.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████▋       | 19/32 [00:02<00:01,  8.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|███████████▎      | 20/32 [00:02<00:01,  8.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|███████████▊      | 21/32 [00:02<00:01,  8.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|████████████▍     | 22/32 [00:02<00:01,  8.41it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|████████████▉     | 23/32 [00:02<00:01,  8.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|█████████████▌    | 24/32 [00:02<00:00,  8.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|██████████████    | 25/32 [00:02<00:00,  8.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|██████████████▋   | 26/32 [00:03<00:00,  8.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|███████████████▏  | 27/32 [00:03<00:00,  8.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|███████████████▊  | 28/32 [00:03<00:00,  8.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|████████████████▎ | 29/32 [00:03<00:00,  8.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|████████████████▉ | 30/32 [00:03<00:00,  8.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|█████████████████▍| 31/32 [00:03<00:00,  8.55it/s]\u001b[A\n",
      "Epoch 4: 100%|█| 94/94 [00:46<00:00,  2.03it/s, v_num=09s6, val/loss=1.990, val/\u001b[A\n",
      "Epoch 4: 100%|█| 94/94 [00:46<00:00,  2.03it/s, v_num=09s6, val/loss=1.990, val/\u001b[A`Trainer.fit` stopped: `max_epochs=5` reached.\n",
      "Epoch 4: 100%|█| 94/94 [00:46<00:00,  2.02it/s, v_num=09s6, val/loss=1.990, val/\n",
      "[2023-06-16 16:17:32,221][src.train_pipeline][INFO] - Starting testing!\n",
      "Restoring states from the checkpoint path at /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003-v3.ckpt\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "Loaded model weights from the checkpoint at /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003-v3.ckpt\n",
      "Testing DataLoader 0: 100%|█████████████████████| 32/32 [00:03<00:00,  8.80it/s]\n",
      "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
      "┃\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m┃\n",
      "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/acc         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2840000092983246    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m       test/auroc        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m  0.040650952607393265   \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m         test/f1         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2840000092983246    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/loss        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   1.9966589212417603    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/prec        \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2840000092983246    \u001b[0m\u001b[35m \u001b[0m│\n",
      "│\u001b[36m \u001b[0m\u001b[36m        test/rec         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m   0.2840000092983246    \u001b[0m\u001b[35m \u001b[0m│\n",
      "└───────────────────────────┴───────────────────────────┘\n",
      "[2023-06-16 16:17:36,779][src.train_pipeline][INFO] - Best ckpt path: /home/haritsahm/Documents/Getting Started/getting-started-dlcv/colh09s6/checkpoints/epoch_003-v3.ckpt\n",
      "[2023-06-16 16:17:36,780][src.utils][INFO] - Metric name is None! Skipping metric value retrieval...\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Waiting for W&B process to finish... \u001b[32m(success).\u001b[0m\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Run history:\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:               epoch ▁▁▂▂▄▄▅▅▇▇▁▁▂▂▄▄▅▅▇▇▁▁▂▂▄▄▅▅▇▇▁▁▂▂▄▄▅▅▇█\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:            test/acc ▄▁▇█\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:          test/auroc ▂▇▁█\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:             test/f1 ▄▁▇█\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:           test/loss ▂█▂▁\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:           test/prec ▄▁▇█\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:            test/rec ▄▁▇█\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:           train/acc ▃▃▅▇█▄▅▆▇▆▂▅▇▆█▁▆▇██\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:          train/loss ▇▅▃▂▁▆▄▃▃▂▆▂▂▂▁█▂▂▁▁\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: trainer/global_step ▂▂▃▃▅▅▆▆██▁▁▂▂▃▃▃▃▄▄▂▂▃▃▅▅▆▆██▁▁▂▂▃▃▃▃▄▄\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:             val/acc ▂▄▃▇▆▁▁▄▆▅▄▆▆█▇▆▆▆██\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:           val/auroc ▁▂▂▂▂▆▆▇▇█▂▂▂▂▂▆▇▇██\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:              val/f1 ▂▄▃▇▆▁▁▄▆▅▄▆▆█▇▆▆▆██\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:            val/loss █▃▃▁▁█▇▃▃▃▂▁▂▁▁▂▂▂▁▁\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:            val/prec ▂▄▃▇▆▁▁▄▆▅▄▆▆█▇▆▆▆██\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:             val/rec ▂▄▃▇▆▁▁▄▆▅▄▆▆█▇▆▆▆██\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Run summary:\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:               epoch 5\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:            test/acc 0.284\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:          test/auroc 0.04065\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:             test/f1 0.284\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:           test/loss 1.99666\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:           test/prec 0.284\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:            test/rec 0.284\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:           train/acc 0.254\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:          train/loss 2.01887\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: trainer/global_step 470\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:             val/acc 0.282\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:           val/auroc 0.04136\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:              val/f1 0.282\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:            val/loss 1.98687\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:            val/prec 0.282\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m:             val/rec 0.282\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: 🚀 View run \u001b[33mdivine-firebrand-3\u001b[0m at: \u001b[34m\u001b[4mhttps://wandb.ai/haritsahm/getting-started-dlcv/runs/colh09s6\u001b[0m\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Find logs at: \u001b[35m\u001b[1m./wandb/run-20230616_160125-colh09s6/logs\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!python3 train.py -m experiment=food101 model=timm trainer.max_epochs=5 model.lr=0.001,0.0001 data.batch_size=8,16 logger=wandb data_dir=data/food-101-tiny/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "beb6ffa3-1a9c-4266-bbbb-077ef05c447f",
   "metadata": {},
   "source": [
    "## Hyperparameter Sweeps using Optuna\n",
    "\n",
    "<img src=\"https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png\" alt=\"logo\" width=\"40%\" />\n",
    "\n",
    "[Optuna](https://optuna.org/) automates the process of searching for the optimal combination of hyperparameters by intelligently exploring the hyperparameter space. It employs state-of-the-art algorithms, including tree-structured Parzen estimators (TPE), genetic algorithms, and particle swarm optimization, to efficiently navigate the search space and find the best set of hyperparameters.\n",
    "\n",
    "To use Optuna, you define an objective function that evaluates the performance of your model using a specific set of hyperparameters. Optuna then iteratively samples different hyperparameter configurations, evaluates their performance by calling the objective function, and updates its search strategy based on the collected results. This process continues for a specified number of iterations or until a convergence criterion is met.\n",
    "\n",
    "To combine the capabilities of Hydra and Optuna, there is an [Optuna Sweeper plugin](https://hydra.cc/docs/plugins/optuna_sweeper/) available for Hydra. This plugin integrates Optuna's hyperparameter optimization capabilities into Hydra's configuration management. With the Optuna Sweeper plugin, you can define a search space for hyperparameters in your Hydra configuration files. During the hyperparameter sweep, Optuna will sample different combinations of hyperparameters and run the experiments accordingly.\n",
    "\n",
    "By leveraging the Optuna Sweeper plugin in Hydra, you can easily perform hyperparameter optimization and explore different configurations without the need for writing separate scripts or managing multiple pipelines. It simplifies the process of finding the best hyperparameters for your machine learning models.\n",
    "\n",
    "\n",
    "Install the Optuna Sweeper plugin for Hydra.\n",
    "```shell\n",
    "pip install hydra-optuna-sweeper\n",
    "```\n",
    "\n",
    "The following command will run 20 experiments to find the best configuration based on the `val/acc` metric. It will try to find the best `timm` model by experimenting with different `batch_size`, `lr`, and `model_name` parameters.\n",
    "\n",
    "Unfortunately, `multirun` is not available in notebook.\n",
    "From the `02_Model_Development/`, run the following command to run multiple experiments:\n",
    "```\n",
    "python3 train.py -m experiment=food101 hparams_search=food101_optuna model=timm trainer.max_epochs=5 logger=wandb\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "030c390a-8175-4007-87f8-25c21c916ece",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -q hydra-optuna-sweeper"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "52825f0c-0652-4f22-bed8-3d6003d26a01",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 train.py -m experiment=food101 hparams_search=food101_optuna model=timm trainer.max_epochs=5 logger=wandb"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}