Laboratoire 5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Rappel Google Colab\n",
    "\n",
    "Tout d'abord, sélectionnez l'option GPU de Colab avec *Edit > Notebook settings* et sélectionner GPU comme Hardware accelerator. \n",
    "Installer ensuite deeplib avec la commande suivante:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install git+https://github.com/ulaval-damas/glo4030-labs.git"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Laboratoire 5: CNN\n",
    "\n",
    "Dans ce laboratoire, nous allons implémenter avec l'aide de PyTorch divers réseaux à convolutions. La [documentation de PyTorch](https://pytorch.org/docs/stable/index.html) va donc vous être d'un grand secours. Entre autres, la documentation de [`torch`](https://pytorch.org/docs/stable/torch.html), [`torch.nn`](https://pytorch.org/docs/stable/nn.html) (que nous avons importé sous le nom de `nn`) ainsi que [`torch.nn.functional`](https://pytorch.org/docs/stable/nn.functional.html) (que nous avons importé sous le nom de `F`). En particulier, voici une liste de classes ou de fonctions dont vous êtes susceptible d'utiliser au cours de ce laboratoire.\n",
    "- [`nn.Conv2d`](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html#torch.nn.Conv2d)\n",
    "- [`nn.BatchNorm2d`](https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.html#torch.nn.BatchNorm2d)\n",
    "- [`F.max_pool2d`](https://pytorch.org/docs/stable/nn.functional.html#torch.nn.functional.max_pool2d) ou [`nn.MaxPool2d`](https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html#torch.nn.MaxPool2d)\n",
    "- [`F.relu`](https://pytorch.org/docs/stable/nn.functional.html#torch.nn.functional.relu) ou [`nn.ReLU`](https://pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU)\n",
    "- [`torch.flatten`](https://pytorch.org/docs/stable/generated/torch.flatten.html#torch.flatten) ou [`nn.Flatten`](https://pytorch.org/docs/stable/generated/torch.nn.Flatten.html#torch.nn.Flatten)\n",
    "- [`nn.Linear`](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear)\n",
    "- [`nn.Sequential`](https://pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential)\n",
    "- [`nn.ModuleList`](https://pytorch.org/docs/stable/generated/torch.nn.ModuleList.html#torch.nn.ModuleList)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "\n",
    "from deeplib.training import train, test\n",
    "import torch.optim as optim\n",
    "import torch\n",
    "import numpy as np\n",
    "from deeplib.datasets import load_cifar10, load_mnist\n",
    "from deeplib.visualization import view_filters\n",
    "import torch.nn.functional as F\n",
    "import torch.nn as nn\n",
    "from random import randrange\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['figure.dpi'] = 150\n",
    "\n",
    "cifar_train, cifar_test = load_cifar10()\n",
    "mnist_train, mnist_test = load_mnist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Filtres de convolution\n",
    "\n",
    "Lors de l'entraînement, le réseau apprend les bons paramètres à utiliser. Par contre, autrefois, il fallait utiliser des filtres faits à la main comme [les filtres de Gabor](https://en.wikipedia.org/wiki/Gabor_filter).\n",
    "\n",
    "#### Exercice\n",
    "\n",
    "- Le réseau suivant contient une seule couche à convolution. Créez manuellement quelques filtres que vous utiliserez pour faire de la classification sur CIFAR10. Par la suite, figez les poids de la couche de convolution et entraînez le réseau. Tentez d'obtenir les meilleurs résultats possibles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.conv1 = nn.Conv2d(1, 6, 3, padding=1)\n",
    "        self.fc = nn.Linear(6 * 14 * 14, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = F.max_pool2d(F.relu(self.conv1(x)), 2)\n",
    "        x = x.flatten(1)\n",
    "        x = self.fc(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modifier les paramètres des filtres. Essayez de faire des filtres permettant d'extraire des caractéristiques de bas niveau (ligne, coin, etc...). Vous pouvez consulter [ceci](http://lodev.org/cgtutor/filtering.html) pour avoir des idées."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filters = []\n",
    "filters.append([[[0, 0, 0],\n",
    "                 [0, 1, 0],\n",
    "                 [0, 0, 0]]]) # Ce filtre retourne l'image original\n",
    "\n",
    "filters.append([[[0, 0, 0],\n",
    "                 [0, 0, 0],\n",
    "                 [0, 0, 0]]])\n",
    "\n",
    "filters.append([[[0, 0, 0],\n",
    "                 [0, 0, 0],\n",
    "                 [0, 0, 0]]])\n",
    "\n",
    "filters.append([[[0, 0, 0],\n",
    "                 [0, 0, 0],\n",
    "                 [0, 0, 0]]])\n",
    "\n",
    "filters.append([[[0, 0, 0],\n",
    "                 [0, 0, 0],\n",
    "                 [0, 0, 0]]])\n",
    "\n",
    "filters.append([[[0, 0, 0],\n",
    "                 [0, 0, 0],\n",
    "                 [0, 0, 0]]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# On crée le réseau, remplace les paramètres par les filtres précédents et fige les poids.\n",
    "\n",
    "net = Net()\n",
    "filters = np.asarray(filters, dtype=np.float32)\n",
    "net.conv1.weight.data = torch.from_numpy(filters)\n",
    "for param in net.conv1.parameters():\n",
    "    param.requires_grad = False # Ceci fige chacune des couches\n",
    "net = net.cuda()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Vous pouvez utiliser cette cellule pour visualiser l'effet de vos filtres sur des images du dataset.\n",
    "for i in range(3):\n",
    "    image, label = mnist_train[randrange(0, len(mnist_train))]\n",
    "    view_filters(net, image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = 0.001\n",
    "n_epoch = 5\n",
    "batch_size = 32"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "optimizer = optim.Adam(net.fc.parameters(), lr=lr) # On optimise uniquement la couche pleinement connectée.\n",
    "history = train(net, optimizer, mnist_train, n_epoch, batch_size)\n",
    "history.display()\n",
    "print('Précision en test: {:.2f}'.format(test(net, mnist_test, batch_size)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Entraînons maintenant le même réseau mais cette fois-ci, laissez l'optimiseur Adam faire son travail d'optimiser les couches à convolutions. Nous pourrons en même temps comparer si nos filtres faits à la main compétitionne avec les filtres entraînés. Remarquez cependant qu'ici, nous entraînons seulement avec 6 filtres et 1 seule couche alors qu'il est possible d'entraîner un réseau avec des milliers de filtres et plusieurs dizaines de couches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "net = Net()\n",
    "net.cuda()\n",
    "\n",
    "# On optimise autant la couche à convolution que la couche de classification.\n",
    "optimizer = optim.Adam(net.parameters(), lr=lr)\n",
    "\n",
    "history = train(net, optimizer, mnist_train, n_epoch, batch_size)\n",
    "history.display()\n",
    "print('Précision en test: {:.2f}'.format(test(net, mnist_test, batch_size)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualisons les filtres qui ont été appris par le réseau."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(3):\n",
    "    image, label = mnist_train[randrange(0, len(mnist_train))]\n",
    "    view_filters(net, image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Utilisation des filtres à convolutions\n",
    "\n",
    "Comme on peut le constater dans l'interface de la classe [`nn.Conv2d`](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html#torch.nn.Conv2d) de PyTorch, les couches à convolutions ont des hyperparamètres ayant tous divers effets. Dans cette section, nous explorerons les plus importants.\n",
    "\n",
    "Voici donc l'interface simplifiée de la classe:\n",
    "\n",
    "`nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0)`\n",
    "\n",
    "Lorsqu'on utilise la classe, elle s'attend à recevoir en entrée un tenseur de taille `(taille de la batch, canaux, hauteur, largeur)`. Par exemple dans le cas de CIFAR10, on aurait en tenseur de taille `(taille de la batch, 3, 32, 32)`. Créons-nous un tenseur de taille similaire avec une taille de batch de 8 afin de tester la classe `nn.Conv2d` de PyTorch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_batch = torch.randn(8, 3, 32, 32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notez que le premier paramètre `in_channels` de `nn.Conv2d` doit correspondre au nombre de canaux (*channels*) du tenseur donné en entrée de la couche. Par exemple, la couche à convolution de 100 filtres 3x3 que l'on crée dans **le code suivant va nous donner une erreur** étant donné qu'on spécifie 2 canaux au lieu de 3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "conv = nn.Conv2d(2, 100, 3)\n",
    "conv(image_batch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Créons maintenant une couche à convolution de 100 filtres 3x3 avec le bon nombre de canaux en entrée et regardons la shape du tenseur résultant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "conv = nn.Conv2d(3, 100, 3)\n",
    "conv(image_batch).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On remarque que la taille du tenseur de sortie a 100 canaux étant donné qu'on a demandé 100 filtres à convolution. On remarque aussi que la hauteur et la largeur ont diminué de 2 dû à la convolution. Pour pallier à ceci, ajoutons un padding de 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "conv = nn.Conv2d(3, 100, 3, padding=1)\n",
    "conv(image_batch).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercice\n",
    "\n",
    "- Quand vient le temps d'empiler les couches à convolution, il faut tenir compte du nombre de canaux en sortie de la couche précédente pour la couche suivante. Corrigez le code ci-dessous pour que l'appel des couches à convolution s'effectue sans erreur."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO Corrigez le code suivant\n",
    "conv1 = nn.Conv2d(3, 100, 3, padding=1)\n",
    "conv2 = nn.Conv2d(3, 100, 3, padding=1)\n",
    "conv2(conv1(image_batch)).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercice\n",
    "\n",
    "- Testons maintenant avec des filtres de tailles différentes. On remarquera que la hauteur et la largeur ont changé avec des filtres plus gros. Ajustez le padding pour que la hauteur et la largeur restent les mêmes, c'est-à-dire 32x32."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "conv = nn.Conv2d(3, 100, 5, padding=1)\n",
    "print(\"Filtre 5x5:\", conv(image_batch).shape)\n",
    "\n",
    "conv = nn.Conv2d(3, 100, 7, padding=1)\n",
    "print(\"Filtre 7x7:\", conv(image_batch).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mettons maintenant un *stride* de 2. Ceci va faire en sorte que les filtres à convolutions auront des déplacements de taille 2 au lieu de 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "conv = nn.Conv2d(3, 100, 3, padding=1, stride=2)\n",
    "conv(image_batch).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On remarque que la hauteur et la largeur du tenseur résultant ont été divisées par 2."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Architecture de base\n",
    "\n",
    "#### Exercice\n",
    "\n",
    "- Implémentez une architecture de base de réseau de neurones à convolution pour CIFAR10 ayant les caractéristiques suivantes.\n",
    "\n",
    "    1. 3 couches à convolution (voir [`nn.Conv2d`](https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html#torch.nn.Conv2d)).\n",
    "    2. Toutes les couches ont 100 filtres de taille 3x3 et 1 de padding.\n",
    "    3. Batch normalization après chaque couche (voir [`nn.BatchNorm2d`](https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.html#torch.nn.BatchNorm2d)).\n",
    "    4. Utiliser la ReLU comme fonction d'activation (Voir [`F.relu`](https://pytorch.org/docs/stable/nn.functional.html#torch.nn.functional.relu) ou [`nn.ReLU`](https://pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU)).\n",
    "    5. Maxpooling avec un noyau de taille 2 après les 2 premières couches. (Voir [`F.max_pool2d`](https://pytorch.org/docs/stable/nn.functional.html#torch.nn.functional.max_pool2d) ou [`nn.MaxPool2d`](https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html#torch.nn.MaxPool2d)).\n",
    "    6. 1 seule couche linéaire pour la classification (aucune activation nécessaire) (Voir [`nn.Linear`](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear)).\n",
    "\n",
    "Ne pas oublier d'utiliser [`torch.flatten`](https://pytorch.org/docs/stable/generated/torch.flatten.html#torch.flatten) ou [`nn.Flatten`](https://pytorch.org/docs/stable/generated/torch.nn.Flatten.html#torch.nn.Flatten) avant la couche de classification sinon vous allez obtenir des erreurs de *shape*.\n",
    "\n",
    "Essentiellement, chaque couche devrait consister en `Conv2d -> BatchNorm2d -> ReLU` suivi de `MaxPool2d` pour les deux premières couches. Notez que le maxpooling et la ReLU pourraient être échangés étant donné que tous les deux sont des opérations de maximum. Pour vous faciliter les prochains exercices, on vous demande de suivre l'ordre donné ici.\n",
    "\n",
    "Notez que la taille des images de CIFAR10 est de 3x32x32 (images en couleur). Il faut donc tenir en compte de cela lors de l'instanciation de la première couche à convolutions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ConvNet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        pass\n",
    "\n",
    "    def forward(self, x):\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = 0.001\n",
    "n_epoch = 5\n",
    "batch_size = 32"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = ConvNet()\n",
    "model.cuda()\n",
    "\n",
    "optimizer = optim.Adam(model.parameters(), lr=lr)\n",
    "history = train(model, optimizer, cifar_train, n_epoch, batch_size)\n",
    "history.display()\n",
    "print('Précision en test: {:.2f}'.format(test(model, cifar_test, batch_size)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Architecture profonde\n",
    "\n",
    "#### Exercice\n",
    "\n",
    "- Ajoutez 2 couches de convolution de 100 filtres dans le réseau précédent (n'oubliez pas la batch normalization et le padding). Mettez du maxpooling après la couche 1 et 3. Comparez les résultats."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ConvNet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        pass\n",
    "\n",
    "    def forward(self, x):\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = ConvNet()\n",
    "model.cuda()\n",
    "\n",
    "optimizer = optim.Adam(model.parameters(), lr=lr)\n",
    "history = train(model, optimizer, cifar_train, n_epoch, batch_size)\n",
    "history.display()\n",
    "print('Précision en test: {:.2f}'.format(test(model, cifar_test, batch_size)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Question\n",
    "- Que remarquez-vous par rapport au performance du réseau précédent?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercice\n",
    "\n",
    "- Codez le réseau ci-dessous qui prend en paramètre le nombre de couches désiré. Vous allez devoir utiliser la classe [`nn.ModuleList`](https://pytorch.org/docs/stable/generated/torch.nn.ModuleList.html#torch.nn.ModuleList) pour pouvoir avoir un nombre variable de couches. Encore une fois, chaque couche va consister en 100 filtres à convolution 3x3 avec padding de 1 suivi d'une batch normalization et d'une ReLU. Mettez du maxpooling après les couches 3 et 5. Vous pouvez supposer que le nombres de couche demandé est plus de 5. Entrainez ensuite le réseau avec 51 couches.\n",
    "\n",
    "    L'entraînement risque de prendre un certain temps. Vous avez certainement le temps d'aller prendre un bon café."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ConvNet(nn.Module):\n",
    "    def __init__(self, num_layers):\n",
    "        super().__init__()\n",
    "        assert num_layers > 5\n",
    "        pass\n",
    "\n",
    "    def forward(self, x):\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = ConvNet(51)\n",
    "model.cuda()\n",
    "\n",
    "optimizer = optim.Adam(model.parameters(), lr=lr)\n",
    "history = train(model, optimizer, cifar_train, n_epoch, batch_size)\n",
    "history.display()\n",
    "print('Précision en test: {:.2f}'.format(test(model, cifar_test, batch_size)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Question\n",
    "\n",
    "- Que remarquez-vous par rapport au performance du réseau précédent?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Un petit défi (exercice facultatif)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch fournit l'interface de programmation [`nn.Sequential`](https://pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential) permettant de programmer et de visualiser plus facilement les réseaux programmables sous forme de couches. Au lieu d'hériter de la classe [`nn.Module`](https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module), on instancie un objet `nn.Sequential` avec les couches désirés en paramètre. L'objet `nn.Sequential` se chargera d'appeler dans l'ordre fourni chaque couche avec en entrée la sortie de la couche précédente."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prenons l'exemple ci-dessous qui contient seulement une couche à convolutions, quelques activations et une couche linéaire."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 100, 3, padding=1)\n",
    "        self.fc = nn.Linear(100 * 16 * 16, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = F.max_pool2d(F.relu(self.conv1(x)), 2)\n",
    "        x = x.flatten(1)\n",
    "        x = self.fc(x)\n",
    "        return x\n",
    "\n",
    "net = Net()\n",
    "print(net)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "En utilisant l'interface `nn.Sequential`, On peut simplifier le code comme ci-dessous. On remarquera également qu'avec l'affichage d'un réseau utilisant `nn.Sequential`, les opérations exécutées par le réseau sont beaucoup clairs et explicites qu'avec `nn.Module`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_sequential_network():\n",
    "    return nn.Sequential(\n",
    "        nn.Conv2d(3, 100, 3, padding=1),\n",
    "        nn.ReLU(),\n",
    "        nn.MaxPool2d(2),\n",
    "        nn.Flatten(),\n",
    "        nn.Linear(100 * 16 * 16, 10)\n",
    "    )\n",
    "\n",
    "net = create_sequential_network()\n",
    "print(net)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On vous soumet donc ici un défi. Le but est de réimplémenter le réseau que l'on vient de coder en utilisant l'interface `nn.Sequential`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_sequential_conv_net(num_layers):\n",
    "    assert num_layers > 5\n",
    "    layers = []\n",
    "\n",
    "    # TODO Ajoutez les couches à la liste \"layers\" ici.\n",
    "\n",
    "    return nn.Sequential(*layers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Il n'est pas nécessaire d'effectuer l'entraînement au complet, juste de confirmer que votre implémentation a des performances similaires à votre implémentation précédente."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = create_sequential_conv_net(51)\n",
    "model.cuda()\n",
    "\n",
    "optimizer = optim.Adam(model.parameters(), lr=lr)\n",
    "history = train(model, optimizer, cifar_train, n_epoch, batch_size)\n",
    "history.display()\n",
    "print('Précision en test: {:.2f}'.format(test(model, cifar_test, batch_size)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connexion résiduelle\n",
    "\n",
    "Comme vous avez pu le constater, ajouter de plus en plus de couches augmente aussi la difficulté avec laquelle le gradient peut se propager dans le réseau. Une des solutions suggérées est d'utiliser une connexion résiduelle permettant au gradient de _sauter_ des couches. Dans l'article présentant cette connexion, elle est définie comme suit:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoUAAAFACAYAAADK5AZOAAAABHNCSVQICAgIfAhkiAAAIABJREFUeF7snQtYlNe19//5TsuQRoQEZBo0oFFGTxTSaLBV4CQS+CJKYxSSaCGVSNt46Qlqq9hUtGpivbQop4r2OYHYyMSkguaCl8jU1IBJA0lah7SRIakOJsShmjBiI8Pp+eZb+51hmCsMMMBc1s4zgXnf/e699m8P+Gftvda+yUgFXJgAE2ACTIAJMAEmwAQCmsD/CejR8+CZABNgAkyACTABJsAEJAIsCvmDwASYABNgAkyACTABJsCikD8DTIAJMAEmwASYABNgAuwp5M8AE2ACTIAJMAEmwASYABHg5WP+GDABJsAEmAATYAJMgAmwKOTPABNgAkNL4KOPPoJOpxvaTrk3JsAEmAAT6JUAewp7RcQVmAAT8BSBiooKTJ8+HRMmTMAnn3ziqWa5HSbABJgAE/AAARaFHoDITTABJuAeAeEl7CqNjY3uPcS1mAATYAJMYEgIsCgcEszcCRNgAkyACTABJsAEvJvA17zbPLaOCTABicD5PfjuD47galAQgmQyyEBfgzrR2QkYDAb62omQB3fijcJp3cCuVOKJh/ei0fKM+RbVb6f6iZtPYGeKjAEzASbABJgAE5AIsCjkDwIT8AkC4RgdHi5Z2tn+OVqukhq0lCCERN2OiJAgu5EI8Xgd7VetL1Ndakd6sR6048VvmQATYAKBTYBFYWDPP4/eVwhMWoT9ry4yWXvtHax/LB+vtnQZT17CuBUoXhlnO5qIDOw/Ho71D1Pdq0EYm/UM9q+9H6NZDPrKrLOdTIAJMIEhJcB7CocUN3fGBDxAYOQMFJYU4N6Q7rZa3liP/DINDDbNt+CVgk0kCIGoBzfg+UIWhB6gz00wASbABPyWAItCv51aHpg/E5CNy0TxrgUYa1kx7oS6+CmsOdnlPryK2sJlKHz7C4TcW4D9W/4vIvwZCI+NCTABJsAEBkyAReGAEXIDTGB4CIxMWIf967+DbofhF3izcA12qa/i3O6nkP/a5wia+H3s25WJcbxkPDyTxL0yASbABHyIAItCH5osNpUJ2BMYPe9ZFGfHUCyyuXQ2oSxvHpY834TOqLnYWfJj3D3S/il+zwSYABNgAkzAkQCLQkcmfIUJ+BCBECSsLUbhrNu6baZ0M52IwZJd65DCa8Y+NJdsKhNgAkxgeAmwKBxe/tw7E/AAgSg8vH0nciZap6TR4tXianzmgda5CSbABJgAEwgMAiwKA2OeeZR+TsDQ8iHOt1jnLgSuvr0NS3e8j2t+PnYeHhNgAkyACXiGAItCz3DkVpjA8BH47BTW5+3Ce+23ITF7Lrodhp24qFyDtYctCQ2Hz0bumQkwASbABLyeAItCr58iNpAJ9EDg2vvYvnwzTl4dgfj8/0Lx2o0o3nIfTGefiOeu4+yOfGyub++hEb7FBJgAE2ACTABgUcifAibgqwQMGhxctQblF0GnlTyLfUsUdKwdMHo2nVzyRKxVRLIWh6ne7y/46kDZbibABJgAExgKAjcZqQxFR9wHE2ACniTQghMrl2Dtm18gfNYGKHdnYLRN81dxeu0S5L/xefdVSlGz7+WNSBrGFDU///nPUVxcjJtuugl33XUXws3nOfeFzJUrV/Dll19CLpdj1KhR+NrXvoavf/3r0teu7+3fW1//xje+AfEKDg6Wvt588829vu+LfVyXCTABJuCrBFgU+urMsd2BS8BAgvCZfKx/TYvOsQtR/vvVuNtZcmryJJbmLcHuhu4AlJB7V0FZsmjYklnPmzcPp0+f9rm5Cw0NxciRIyG+hoWFISQkRPq+61rXV3Httttuw6233oqoqChJtHJhAkyACfgKARaFvjJTbGdAE7h2ehuW7W+AwXAdLS2fo92i84IQMjYDO36/Dkk2wtCA+h3LsP3Ni2hsuW7DLiQqFuPCReUY5O3eOKS5DH/0ox/h0KFDATWXY8aMgXgJz+btt98uiUXxdfTo0fjmN78p3RPeSi5MgAkwgeEm8LXhNoD7ZwJMoHcCnVcv4vwFLSALgizkNimQpJOSVMPQifaWq3AWRnKlRYPGK0BQyAhpr2FQkCmPYWf75zh/RSS4Bj4z9N63J2vccccdEDtWxPLxxo0b8e1vf7tfzYslZOGV+5//+R/861//cutrV93r16/jxo0bltdXX30lfS++dn3fdb+93RnZvpn86aefQrx6KsKzOG7cOMTExEhfo6OjMXbsWOk1fvz4nh7le0yACTABjxFgT6HHUHJDTIAJ9EZgy5Yt2LFjh1Tt5Zdfxpw5c3p7xCvut7a2Qq/XS69r165Zvhfvhcj84osvpOtd9/7xj39ACFdx3RNFeBOtReO///u/IzY2FpMmTfJE89wGE2ACTEAiwJ5C/iAwASbABHohEBkZCfHqTxGCUojEq1evout7IRi7hKO49ve//11676p0eRtramocqigUCkycONHhJYJouDABJsAE+kKARWFfaHFdJsAEmEAfCbgrKMWS9cWLF6XXhQsXoNVqJbHYda2jo8NpzxqNBuL1+uuv29wX3kXhUUxISMA999wjfe1PtLfTTvkiE2ACfkmARaFfTisPigkwAV8jIIJNhIgTL2fl8uXLkkBsbGyUXh999JH09dKlS86qS/sYxau6utpyXwjFLoE4depUTJs2DSNGjHD6PF9kAkwg8AiwKAy8OecRMwEm4IMERKSyeH3nO9+xsV4ExwhP4fnz56WXEIri68cff+wwyi6haO1VjIuLQ1JSkvRKTk6W0ulwYQJMIDAJsCgMzHnnUTMBJuAnBMTewW9961vSy7688847eO+991BfX4+6ujp89tln9lXQ0NAgvfbt2yfdE57KLoH4H//xH7zk7ECMLzAB/yXAotB/55ZHxgSYQIATmDFjBsSrq4hgFmuh+MEHH0hpeKyLWJYWr//+7/+WLotAli5P4v33388JuQP8M8XD928CLAr9e355dEyACTABCwFxwspDDz0kvbqK8CSKqGbxEoJRpNixLl2BLGVlZdLlyZMnIyMjQ3o5807aPMxvmAAT8CkCnKfQp6aLjWUCvk3AV/MU+jb1vln//vvv24jEnhJ4i9NZ5s6dKwnElJSUvnXEtZkAE/A6AiwKvW5K2CAm4L8EWBT63tyKJWZrT6JI0O2siPOf09LS8N3vfhcPPvggRzU7g8TXmICXE/g/Xm4fm8cEmAATYALDSECkrsnPz0dFRYUUqHL8+HEsW7ZMOrPZugixWFlZidzcXOls58cffxwnTpwYRsu5aybABPpKgEVhX4lxfSbABJhAABMQaWvEUYUiGKW2thbr1q3DXXfd5UDklVdewaOPPiodzyfqiAhnLkyACXg3ARaF3j0/bB0TYAJMwGsJ3H333fj5z3+Od999F3/5y19QUFAgiUDrIo7027t3L2bOnCnlWPzNb34jnQvNhQkwAe8jwKLQ++aELWICTIAJ+ByB8ePHY/369VCr1Th9+jSefPJJRERE2Izjr3/9K55++mlJOC5evBh/+tOffG6cbDAT8GcCLAr9eXZ5bEyACTCBYSAgzln+1a9+JZ3hfOTIETz22GO45ZZbbCwR10VgikiQ/dJLLw2DldwlE2AC9gRYFNoT4fdMgAkMiMCNGzdw6NChfrUhcuSxQOgXOq99SAi/5557ThKIItehiEy2Ln/+85/xwx/+EMLT+Mtf/pKXlr12JtmwQCDAojAQZpnHyASGkMDNN98sHasm9pv97Gc/k6JVXaUxEWZ9/vnnUmTrihUrMGXKFPzrX/8aQmu5q6EiID4XjzzyiDTX4lzmtWvX2hyh19raiq1bt0pLyz/5yU/Q0tIyVKZxP0yACZgJcJ5C/igwASbgcQLiH/SJEyfatBsXF4e2tjZcunRJui4iVkVi5K734lpMTAw+/PBDj9vDDXovgeeff14KRGlsbHQwUngQf/rTn0IkyebCBJjA4BNgUTj4jLkHJhCQBFavXm05P9ddAPv370d2dra71bmeHxGorq7Gnj17pCAV+8Li0J4Iv2cCg0OAReHgcOVWmUDAE7h8+TJiY2Pd5iCSIYvcd1wCm8C5c+ekZWSx7cC+/OAHP8CaNWvYc2gPht8zAQ8R4D2FHgLJzTABJmBL4Jvf/CbEP+LuFpHvjgsTEHtRX375ZSkxdnp6ug0QEbAitiWI4xK/+uorhsUEmICHCbCn0MNAuTkmwAS6CbjrLWQvIX9qXBEQnsNnnnkGJ0+etKkyevRoyaO4YMECV4/ydSbABPpIgD2FfQTG1ZkAE3CfgPAW5uXl9foAewl7RRSwFYTn8PDhwzhz5oxNOhtxDrNIgC1S3jgLUglYYDxwJjAAAuwpHAA8fpQJMIHeCYjoYmdn43Y9yV7C3hlyjW4CYln5qaeeQlNTkw0WcYJKYWEhQkNDGRcTYAL9JMCewn6C48eYABNwj8Add9yBJUuWuKwsjj3jwgTcJZCUlIQPPvgAzz77rM0pKb/97W/xrW99y2mAirttcz0mEOgE2FMY6J8AHj8TGAICrryF7CUcAvh+3IVOp0NBQQEqKyttRvn4449jx44dGDFihB+PnofGBDxPgD2FnmfKLTIBJmBHQHgLn3jiCQcu7CV0QMIX+kBALpfjwIEDOHHihE36o4MHD2L69OlSBDMXJsAE3CfAnkL3WXFNJsAEBkDA3lvIXsIBwORHnRLYsGEDdu3aZXNPHJ+4bds2p/X5IhNgArYE2FPInwgmwASGhIDwFubm5lr6Eucic2ECniSwefNmiJNRxGetq4gj9O69916I1DZcmAAT6JkAewp75sN3mQAT8CCBLm8hewk9CJWbciBw/fp16cxkpVJpc6+kpARivyEXJsAEnBNgUeicC1/1cgJ6vV467kqIDC6+RUCkEgkJCYHIYcjFdwjcdNNNmDFjhpT2xVeKOCrvRz/6EcTvi64iUtf86le/8pUhsJ1MYEgJsCgcUtzcmacIHD16FN///vc91Ry3wwSYgJsExPnUwtPrK+XTTz/FY489BrVabTFZpLV58cUXceutt/rKMNhOJjAkBHhP4ZBg5k48TYDPPfU0UW6PCbhHQCzN+lIRAvYPf/gD5s+fbzFbRCULYcgnofjSTLKtQ0Hga0PRCffBBAaTgIgsFBGGXJgAExgcAiKdUEVFxeA0PgStBgcH44UXXkBxcTHWr18v9djc3Iz77rsPv/vd72yOzxsCc7gLJuC1BNhT6LVTw4YxASbABJiAJwnk5+fjtddek/a0ivLPf/4TWVlZDsmvPdknt8UEfIkAi0Jfmi22lQkwASbABAZEYNasWaipqcGdd95paUekSnr99dcH1C4/zAT8gQCLQn+YRR4DE2ACTIAJuE1g/PjxOHPmDBISEizPfO9732Nh6DZBruivBFgU+uvM8riYABNgAkzAJYGwsDAcO3ZM2lfYVYQwfOONN1w+wzeYgL8TYFHo7zPM42MCTIAJMAGnBG6++WZUVVVh7ty5lvtij+Gbb77ptD5fZAL+ToBFob/PMI+PCTABJsAEeiTw0ksv4dFHH7XUEd/X1dX1+AzfZAL+SIBFoT/OKo+JCTABJsAE+kSgtLQUS5culZ7p6OiQopI/+eSTPrXBlZmArxNgUejrM8j2MwEmwASYgEcI7Ny5E2JfoShffvklvvvd7+LKlSseaZsbYQK+QIBFoS/MEtvIBJgAE2ACQ0Lgt7/9Le6//36pL3G2+sMPP4wbN24MSd/cCRMYbgIsCod7Brh/JsAEmAAT8CoCYo/hXXfdJdl07tw55OTkeJV9bAwTGCwCLAoHiyy3ywSYABNgAj5J4JZbbpFyFt5+++2S/adOncLGjRt9cixsNBPoCwEWhX2hxXWZABOQCGirS1FUVITyOt0wEdGhrryIbChFjXaYTOBu/ZpAZGSkJAxFPkNRxOe9urrar8fMg2MCLAr5M8AE/JqAEE97JfFUre3w0Eh1UBWtlDwnK0vqPdRmX5upR9HKjWTDSmytGS5h2lebub6vEZg4cSIOHjxoMfuJJ55AS0uLrw2D7WUCbhNgUeg2Kq7IBHyQQM1WLFq2ThJPT2ys8cEBsMlMYHgJiKCT1atXS0bo9XosWrRoeA3i3pnAIBJgUTiIcLlpJjDsBGTBCO4yItjy3bCb1aGpwIb8fOQXVYNXf4d9OtiAXghs2rQJ06ZNk2p98MEHWLduXS9P8G0m4JsEvuabZrPVTIAJuEVg+iYcPZ4MtV6G+KRktx4Zikq66hLsKqOlZ5keaavTEDMUnXIfTGAABJRKJb797W9L3sK9e/ciIyMDSUlJA2iRH2UC3keAPYXeNydsERPwIIFgKJIzkJWRBoVpv7wH2+ammEDgEBg9ejTEqSdd5cknn+T8hYEz/QEzUhaFATPVPNAhIUDHY/Va3KnSSzttOg3UdDZrnVoLnTvt9WoUVejQQaNWQ63Ros2NNp012aZRSzb193lnbQ7sWgd0WrKpxg1WvTDvtqMDPeIhjlo19Venhkbb1rv5dv22CXvFs2482nvjXMOTBB588EF8//vfl5psbm7GL37xC082z20xgWEnwKJw2KeADfAXArrSdESMGoWQiPkodxoQq0N5egRCRoUgIr0UTqugDvmTQzCK2plZpLFD0wZ1eT7SJ4/BHROmIfGBB/BA4hRMGDMGMxdSugyn0cXVWDqG2guJwMIK5ypDW12EhVPHkF0TMC0xEYnTpuAOsjEkxOoVMRUb1M5nqqNDi6oNCzFzfATumJYo2XTHmMlIL6iy2y/YhqqF1A+1O2WdOWrZUIlFVv1EpOyF/aid99rT1Q5oa0qRP38mxkeMwoQpZNMcM6tRYzCVWNkHLHfPXQpKe9jk2FaxEGNCRmEUzbE9zg5tNYpyqc8xEzAlkfp7IBHTptyBiMnpyC9Xwxl97d4U6TMzJreaztvVoDx3qsle8ewdM+HwEehp2HxvSAg8++yzuO2226S+SkpKUF8/XBH4QzJc7iTACLAoDLAJ5+EOHgF5QgKiRfOGGlRWO5EAbXS93iAZYKivgrMqqKuEqlnUiERCksLKWC0qckmwLStDbbMeobFJmJuZiblJsQg16NFwbCMWpCxCuYOgMZi9dgboO0x9WzUKLYmclAUbcaxJD1lkHFLnzkVqQjRkVpVkMhkoXsV50VWjYE4iFu06hobWYERGRyJU1DQ0o7ZkEVKWVluJIVeNWDXdowvOuQn2V3UVuUicsxJlqga0EsfYuCTa+5WA2EgxKj2aiNWclHzUWE2RPJmW1yW767G30pUs1aFKqaIWqMSnIsFqOb6tbgPmJC7AxkpTn3Gpc5GZSXWiZYSiFmXLUjCnoMZBGBoMbRCz0qHX0PymY1llk/TeVPRwx9Foqc7fDAkBkbdQ5CzsKj/84Q+HpF/uhAkMCQEjFybggwTKy8uNI0aMkF579uzxkhG8a1x7l8mm8MdeN35pZ9WNU4uNo802jxgRbnzsdfsaRuO5wntM47rzSeNbN7obaPz1LGO49OxdxsUHG41Wt4w3Gg8bF3f1O/s540Wbfl83PhZusmn2wcu2Fn15yrh4tOneXU++brS+e/mttcYZ0nN3Gp88ZW/nZeNzs03PmeYg3HjPkweN57qqXX7LWDgr3Dw/9xgLzzlOz8U9s0z3wxcbX7cejGNVF1dcj0u0Pfqeh42FB981XrZp+4bx3J7Z5jkIN9ryuGjcM8s8pnsKjU5MNhovHzTOlpjQs89Z0RIc7zQ9O3pWofEtG8xfGt8qJHvMc/eU9aTSyBp/bZ7v8HBpfsPvedJ48Nxl45dfXjSeO/WWsbFfbFwgG8Dl3Nxcy8/bRx99NICW/OfRefPmWZhs3rzZfwbGIwloAuwpHBLpzZ0EBoHpyEiNlIZqqK1AvZ3XS11VY/IymWqgtqLebm+aFjWqJuluaFomEroca21V2FpUL3mQYlcdwv4cRXeaGboWrMjC7t2Z5BMT/e6FS0eX1HJ36ahXQiXcXrIk/GxTBuRW9+TJm7ApQ/j8WlFRUuXg4equGoqETcdxen8O4rs8Z/JkbN6/HLFSpSaoejtyxA0HopVpvX4bs+I0Pv3gKDbnTIfcpu1gxK/YhGyTOxcNKrUV/xhkLUkyeUibKqF0slSuq1JCcvTKEpCd0U1LU7YRla10PXQudh/ajGRrkAhD8ub9WB0nzG5G5V6Vc5YGAwzR2Th0Yj9y4uV0ikYM4tOSofAwm17hcQW3CYgI5GBzmqcdO3bg4sWLbj/LFZmAtxJgUeitM8N2+SSBhKw0SZxBX4MqG1WoRlWNUA6RtOybIIkPfU0VbHYj6VSoahAPhyI1M9ki/DpIYFZJa5YJyMuLtxGE4qooYclZSJDWfJtQ7ebRc/pmnUmkyuOhsBEyop1gyGkpWBSDrpWkofMiS92GI6unk/SxK4oMpEniC9BppPVwLykKxJnt0ut1NqJcnpGHZIlhMy0T19nZS0vHJOIlTZich25NqKHr0qQhMmOF1XXrxxXIyJJUIfT11S72TMZi+fO7keYwD9bt8PfeREBEI69du9ZikshlyIUJ+DoBzlPo6zPI9nsVgeCETKSFKqHUt0JVRe6m5Okm+7Q1qBVOwNBk5DydgNbKetS3kggUVeJNVdqqzSIxNAlZyd0uIg1Fokr7zGhvn0FTjWqnW960MIhHqKJOI0JYelcXoaGhkjg16LRoFV5NG69UB0VXmqUg1ZP2CToroWGOglCqJ0e0MIH0oEHvgY2Czvruw7WONh10wg7afynpa1HszQpLRR55R1WVejRXKFGzaTos06CrglJyE5Jgz0vtHnMHRQqb5yOUwmpqqu0bNXWl79rrSUK0mapMt/cARiYj0+Fil6H81VsJ/Od//if27duHf/zjH6ioqMBPf/pTTJ482VvNZbuYQK8EWBT2iogrMIE+EAhOpiXkUCiFsCCRp94+HULztdWoIFYkZQmU8FYRD03cOtQ3NNNyMSmKeBHi0IHaKrMnKikLSVaiQddqFmeGWmxcUNsHY3quGkz5C5Nlx6AyqLCVjsBL2p5sETsddVuxVXJPypCQkeqGxLTvq/skFecyyb6+h99TRHR1aRFKK1WoVTdD3x290UNHYUjNzkBkpRKtrUIEbkeyWRVqSSSaNGEqclKt/KI6HVrNbTcpl2GBsofmpVtuGdJbI3zfSwiI5WNxBviPf/xjyaKCggJUVVV5iXVsBhPoOwEWhX1nxk8wgR4IBCMpg/amVR6DoUkFlXYz4mNI8KlMgi8hQwgvOVJTo7GRRGFDFR3ztlqBmA5abq41ibBkEmE2y7FdqkoWh8zlqaYIZ6cWkNdOTlHLWWbXo9M6VhflWeQZWyd5xhpK5mBKNUXpxscgmFKj1JC9QuzIYvOwKc/HzhvRViB3zlJUNkuLvYiMTUCqIhryMKG0RbqaSorgdg4nODkbGZFKlJEQr1bWoIOikoPJA1hFnl1RIjOykWrv5TM3FZ26HJlx1nHbdn1Q1KqcltUzXDzv3CK+6u0EFi9ejOLiYjQ1NeHMmTN48803MWvWLG83m+1jAk4JsCh0ioUvMoH+EwhLzTJ74BpQpdJhdZ4G1TVC8MUiwxyFEE/CL3JXGVrVKsqZtwIxWkpRI2nCBGSm2e7QC5WLxVu6GaxA9ubNSOu/aTZPdqi3YgMJQoTGISmaUsg01FJqGrMnUkbiMvtpbNuW57jU6aH+B6cZLUqfMAvCSAr8OLofeZYIGNFjByoWVpEodOGxI09vXlY0ykqa0UqBJTUdaUijpWOTJoxERnb3Xk/Jfqul9VAShZtX+JiAHpxJCLhWt27dikceeUQat0hozaIw4D4CfjNgDjTxm6nkgXgNgTDaH2aK+oCaPIE6dTWkGJPoNFo6NlsZT4EYQutRTsMqSpgnIpNFFVlClkOwgSJOYYqKpVx2DU73E/Zv5PUlSgpLIam6+gBOvP0p/vHxh3j/7B/wh7Mf4uNPP6GIYhKEDhEk/evL9VPkBvXk+jLlgqww54JMeLrYThC6tsL6TnxenilyWq+CUtUGjYg6FhWis5BttddTeiZMAXJCSqW5vsGjQzG1yv/3BQKzZ8/GjBkzJFM/+OAD/OlPf/IFs9lGJuBAgEWhAxK+wAQGSkCOtCxThLGBliqVFbWS+ApNTpX2F0rFvPdQ7DGrrSpDhTkVjfAg2oeIhInnpFXJBjp7tc5DwoOOf9OZQi5a6Qg4DQmzYHkMFPHTMZ2WkG1TuXQZ7bmvwcHmZVYK/DCb4ZnG9XrLrr1QCsxxKLQ0rnblJeyqrMhEXoJ4o4eqnPYlKk3RxbGZ2TCHDVk1G4/UZFOUtl5Vigrnx9Q4mMEX/I9AYWGhZVC7d+/2vwHyiAKCAIvCgJhmHuRQE5CnZpgEIHkCt5YIP5MMSakJVgG+Yu+hOTVN1VaUSLqD9gxmOFl+jMnG6gyT8GimU0JySRi2OQxI7JUrx4YNFS5Sntg/EIxkSnsjLUyrlmEaHZU3fvxkTJ46FTNnpiAlfT4W5uajqJxSqHjSk2c2Q66IMaXuIR+cUqkxC90OtOkcR2ZveY/v5QpEm0Ol65VKW9t1NSiaPx+7TBqvh2YoZ2G2KWeh/tgu89zQ0n+Os72awUhbvQLSVkLyLK6cT4EGTo4b7NBRSqK9BdhrfYxKDxbwLd8jkJycjLg4U+qhY8eOQaPxoFvf93CwxT5KgEWhj04cm+3lBGIyKOhA2EhJiaXtawlIS7Zdiw2j6F9plVkkLhZV4mjp2IkmpDVKZOw+gCWxonIrjq18gM47nioJt/kkcuanzMTkMaMwZc4y7KJE0+7+UyTPeR77M01iU6RqaW1tRjNtlm9ooHQ5tSocqyzDxmULMI3O7t2r9rAyFKl7pK4NqN84DRPGT8XU8XSmc3qRFKXd7xKchtWUKVrSaLXrMHNaConbpchdmEKpQuZgY60BcXSMX29FTgElppyF5ppxOaCc4c6LYjUObKc9onTX0FCCRVPGYPLMdNPczE9HytTxGDOBjgJcV0IRza4yPjpvmq/6FoH8/HyLwSKSOct4AAAgAElEQVT4hAsT8DUCLAp9bcbYXh8hEIO0bJMnUBgsUtGk2q8Ly1ORZd57KOrEZWWYzt91NkLap1h8+jT2iehj8oQZ9E2ScFOp6EV72Zr1MkQnzMWq/U8j1eZ5mdk7Kb7aLqfqKpYiXxzFQVHNS458iEuXPsbHH9K+wvfP4uzxI9i3KRNx0qEmtViXu9VKrImzkE1t2bdp27W5jrNoWxJv2w6sQpJ0HjEJuNYmNFG4c2iMwmH53BkO4Xk1Nes4rvjVR3F0FXGipg3N9SRulag8Vg8d8V516CyOLjeJRpfnOYsO5RkUmW0WzPQ2ITvT9dzQfUXeUZw+son+EIgkywxopqAdaW5UtahvIsahsUjK3obt2fbK0gzHfDKG87HyVV8h8Nhjj2HUqFGSuS+88IKUv5ALE/AlAjeJQ/58yWC2lQkIAkpaGly6dKkEY9u2bVixYoVXghFJk/UGGULlYba5oS3WiiVT2gcnCzWnTHFnGLQfkJamtG1i/xy1HUaJoqNjIGVccVY6ROJmGeRkQ3e3NVhKnjNlK+Uh3PY+TruImm0rn48Jy1TUTxy2ffg2LNU62qhNEnFyuYtxUU9SHdJDLscurBFjUUNLe/GkPY10tIqrYTgMzdm4rCtR/2IJz2QDBYQouuegg/ILdoTScXI9dKbdOxNT1tFaMx0DuPuvJ5BnL+odDDJfaNNCTQnE9eQBllEfcnk0Ylxu0jTNP0Hq0RZXXQ3V9SeeeEJKzixKfX09Jk2aNFRd+1w/u3btom0cGyS7V61aBToX2efGwAYHLgFOSRO4c88jHwICwSTYetAdZEEw5RbsuYajmXQEHSXAdlejkNpyDByhk1HqpJXMaCQlO12zlroNDuvyLlIyamszgynnXm9mu1OHxi9X0BnF9g40x0E7XnE2Luta1L8InHHWdHBPYlZqQ01BPabNh7IEyl3oNmx6VJxbPN01U9uB9Gf+HVHwFe8h8IMf/AC//OUvcePGDTz//PMsCr1natgSNwjw8rEbkLgKE/A7AiSYTJquGaoKtfOI5rY6bN0qvIRU4uj0k74IIx8H1lFTCqUIGSdPbHJehvsC3MfHzeYPnEBISAjEMrIobW1teO211wbeKLfABIaIAHsKhwg0d8MEvIqASLuSVISVdIpKw65ETFalUiBMHBTSEnMbtOp6OtWEUumIrDWhCdi0e7lTj5tXjcljxnRAVVop5Y2ELBXZ1sfaeawPbsifCSxatAgHDhyQhvjiiy/ioYce8ufh8tj8iACLQj+aTB4KE3CfQAzyjh6HjM5q/aWylgIjKFEzvWxLKGJTs/H09k3IUvS2Vux+z15fs60KpdK5z6SHM3LAmtDrZ8zrDJw5cybGjh2LixcvQqSn+fLLL3Hrrbd6nZ1sEBOwJ8Ci0J4Iv2cCgUIgOB45xSeQs10EZDRA20zBEW0ii3UYxT1EQxEXj5hBP9HEC2FTpPfPDjyPzI5QxKem2Z5D7YXmskneSeB73/sebb/YKhn38ssvWwLjvNNatooJmAiwKORPAhMIdAJSQEYyvQIdRNf45ZiekeXk9BLmwwTcJ7Bw4UKLKLTOluB+C1yTCQw9AQ40GXrm3CMTYAJMgAn4OYFx48bR6UAzpVH+5S9/kZaSuTABbyfAotDbZ4jtYwJMgAkwAZ8ksGDBAovdJ06c8MkxsNGBRYBFYWDNN4+WCTABJsAEhoiAddTx8ePHh6hX7oYJ9J8Ai8L+s+MnmQATYAJMgAm4JHD77bdbTn/54x//iK+++splXb7BBLyBAItCb5gFtoEJMAEmwAT8kkB6erplXG+88YZfjpEH5T8EWBT6z1zySJgAE2ACTMDLCMyePdtiES8he9nksDkOBFgUOiDhC0yACTABJsAEPENARCCPHDlSaoxFoWeYciuDR4BF4eCx5ZaZABNgAkyACSAlJUWicO3aNfz5z39mIkzAawlw8mqvnRo2zF0C69atg3hxYQJMgAl4I4EZM2bglVdekUx7++23cc8993ijmWwTEwB7CvlD4JMEvvGNb/ik3YFutNFotCCw/j7QufjS+EeMGOFL5nqFrV1JrIUx77zzjlfYxEYwAWcE2FPojApf83oCYjlm0aJFuHTpktfbygZ2E9BqtZY5u+uuuxAeHs54fITATTfdBOHxGjNmjI9Y7D1mfutb34JMJoPBYEBNTY33GMaWMAE7AjfRX+vdf7ozHibABJjAIBLYsmULduzYIfXw8ssvY86cOYPYGzfNBLyHwLx583D69GnJILGvcMKECd5jHFvCBMwEePmYPwpMgAkwASbABAaZgPUSsthXyIUJeCMBFoXeOCtsExNgAkyACfgVgcTERMt4eF+hX02tXw2GRaFfTScPhgkwASbABLyRwLRp0yxm/fWvf/VGE9kmJsDRx/wZYAJMgAkwASYw2ARuvvlmjB49Wurm448/HuzuuH0m0C8C7CnsFzZ3HzLgwsnf4ZXz7tb3QD2DBq+U/REXDB5oi5tgAkyACTABjxGIjY2V2mpvb4dOp/NYu9wQE/AUARaFniLp0E476ncswdLDQZg0zuHm4F2QxWASXsLSvD2ovzZ43XDLTCDwCNRhw8zxGBMxBlPzq9EReAB4xAMk0CUKRTNNTU0DbI0fZwKeJxCYeQoN7bhypR0DcqbJQhAREQKZ0zkx4Nzupcg/G4cdBxdhkvNKTp8c+EUZJi3ZiWc+W4L85SEoK108xP0PfATcAhPwTgKt0GhaoadfHHqNThKFwd5pKFvlpQSs09CIJeSkpCQvtZTNClQCgSMKrzTgldeO4NVXz+C9i9c9M99BtyE++1nsWzkNpuPOTc1eObkZ+UoZlv1+HZKsb3imVzdaCUFC4U4syyFhWBgD5Y77EeHGU1yFCTABJsAEBo8AewoHjy237BkCAbB8LPb1/RzfTc9DYfExG0EYFBQE8bItpmtd96y/OiDv/ALqN87ggvWNC4dIiJ3B6PwNeHwol40djBuLx7f8EFFvkkAtv+hwly8wASbABJjA0BKw9xQObe/cGxPonYDfewoN6r1YWnAGmPl97CORlBRhv5Z7EaWPPordjQLWFKw9XobHTQFiTugZcO0zLc7uX4O1r30OhExBzvrFuNtSswW/f2Yv1FELcThnrJPnh/jSuEUoyD6CR/YX4ZXZ/4WH2V04xBPA3TEBJsAEugmMG9ftKeAIZP5keCMBvxeFiFqA4j+swCQHMWiejmsfor7L1Rc1BQkuBaGoL8PI0QqkL1mFK/fKkDRrBsZZLQ9fO70XJe8FYdZ22sfnJbM9ackKzKpYi907/ogUWkYeltVsL2HBZjABJsAEhpvAxIkT0djYSPtTNcNtCvfPBBwI+P3ysSxirGtBSDgMDe9A3WniEhJHIs8BkZML4+7H4/NsBSE1hH07q3E1KgN5s0OcPDRMl0bej7yHbsfVN3Zhn3pAoTXDNADulgkwASbgPwSs9xV+8skn/jMwHolfEPB7UdjbLJ0/+yHapUpBuDdlioto4t5aAa69+QJebQHGPrjAajm59+eGosbdjyzAWHyOV8vOgLPUDAVx7mPwCXSgbm8u5qfPR26pWooE7tBWYcPCmRg/ZgzGjJ+K9IJqOM0E16ZGRVE+FqZMxeTxYxAxZjwmT03Bwvy9qNK09d90TTmWzk9H+vxc7K3rIWEN2Vmw0FSvqGYA/fXfUn5yGAnwvsJhhM9d90ogwEVhC9Tv0d5AUYKmImVmfz18LThZ/g6Jy1g8PG9sr9CHvMK4+/DwREqYevYlnLwy5L1zh0xgEAjo0VBVCVWtCpVVauhp7/D8xEXYdawBrXo99K1NqK2ogv0Cna66AClTEvHExjIcq29Cc6seBn0rmpvqcaxsHRbRUWTpRXXol1TTVKFCVYtaVSWqGvSux9xM9Y6Z6lWoe6jnugW+48MEWBT68OQFgOmBLQqvfYCz5v2EQfH39T99zGdn8GoDrUGPnYFEt9afh/qTNRb33ns70PkhXj1J7kwuTMCfCOiqsHTROtQKfRUZh9TMbMxNikVoaDBCrcbZVpOP9AUlqBf1QuOQuel5HPnDWZw9exyHdi9BUqQIQmtF7cYFeKJc60+EeCxeRMB6+ZiDTbxoYtgUiYD/B5r0MNHX3jtj2U84afaMfufyu/L2GYiT7MLj3dyT2INNg3VrUuJUhCiP4fybH+BKTlS/xzpY9nG7TKDfBBqOQUUPhyZtw/FDKxAf5qSljjr8cmkZpDMkQpOw+/RR5Cm6U0/HxycjIyMDS1MWQNmsh2rdRlRnHUAaZ6d2ApMvDYRAVFSU5fGWFv4jfSAs+VnPEwhgUWjAeyffN+8nnILZM7t/UPuG2YB62pfYSXsSJ92r6HVP4mflSylFjEbKjyiTCc8E5UWkpzvpfBUDxYF0dnYicfMJ7EzpTp1jOP1zzNrwTvczIrUiOSYN9EBnpwJrX+k93Yxs4hRMBOVpVJMQNmTAqvm+DZdrMwFvJBC7BIdcCUKyt01VRGJPGE5ZAzbttxGEluHI07DpZ0moWFZLS8oqKFUdSMtgVeiN0+3LNo0c2Z0D4to13uXty3Ppj7YHrig0fIDTb5tPNplIS8c9pqLpaeo1UDeK8OXbMW6cG3sSQ8IRFR4uHbHX3qLFVXPks6mHEYgaeztC7FMpipvt13HV2oygEQinY/bCQ1wdtWdnc4QCk0j3vteiQT3lZEyJ72lMfI8J+BKBWCzfvx3JzjyE0jA6UF9RC2n3niwB2RkxLgcnn54GBWrRQLXVdbQjMYN/UFzC4hv9InDbbbdZnmtr69fu1X71yw8xAXcIBKwoNKhPodYUdoyJs+9zLxWNM6LXtDgvrQDEYOJYZxVsr42e9ywq5pmuGdR7kJ33AiRNaS7j6NziDYm2qlCW8izePBiOrMdfwkUSn7M2F+MZCmjpW87BKIwTztCWz3HhAg083g0B2/twuAYTGH4CsngkT+/Jo6dBTVdAR3AwBaVUo9qV1fruvYStrU5jl109ydeZgNsEbrnlFvzzn/8EewrdRsYVh4hAgIpCA9Qn37F43q6c3Ial9fbH3dnPwFjkbF/tGIzSchFSQC95ACP6ptIgi/8x9m/RIotOXDF5Aa/j7DP52BxVhg0JVqLtyh+xvkAIwhFIXF+MnSQInTkT7S22fS88lKZ15wsXPhcG91yd7zIBvyGgR2ureTC0LLxugdiB2Hvp6CGrTO9Pcw0m4JqAWEJmUeiaD98ZPgKBKQpp6fjVN76wUL/aSFHIPc5BEKIo/+AkJ6LPQJ43yVFIS8L9OUUuYvYz2N+yDNnFYl8ilU4tDq9ag0kHi/HoOJJ+1xqwffl6nGwJwsQnilH8SH8EoWlwERHh9M3nuNJisxDd48j5JhPwfQIdUh5DqciiEZcQbROV7Di+YAQHy5Gal+B4i68wAQ8QCA0Nxeeffw5ePvYATG7CowQCUhReO1uF09LSMQmtrFXIiXfhd2v/EKffM+DuhxYjJ8W5GDO0t5vEnCy8n743GSbRkvHO80uQ/4bw4FFp/wBblm9D1HOL0bIhH+W0vhz1UDH2r4zrh4ew+/Miixhhbr9dSmLtRON2V+bvmIDfEAhFaNePeGgqtp8oRvIgj43PDhpkwD7evBCFXeX69esYMcL8u9nHx8Xm+z6BABSF7Tj72hlT1HHIfVien4kUl+ooAw/n9DzJnZ3mYJXeVp97bCYcKVt2Ym3LUuxoMLfXcgz5D1dL0cjhMzegbEv/U+Z0dR0UZPqXsbPTLGR7tIlvMgF/ISCHQsSWiCVkfTO0tLffdVDKAMYsC4PY2SgEYQcl0AbkA2iMH/VnAtYRyHr6rLAo9OfZ9q2xBV7y6iun8PuzpsiOqIcWItGlIHRvIkUaGanIKMWMe484ryVT4PHdz2KeCAYxFyEIET4Xz+zOQL+Do616kzLgiEI2syfDCgx/6+cEYhAfF2n+7NfTKSiDtFmQvD9d/p9mtbZ7ydqGbhtqKuokfcolcAlYewo52CRwPwfeOPKAE4UXXnsJ70maMBY52QNbjhWtyEgMSsUgMg0OsETMQGHJKsRbex2vnsG+w5qBty1MtAhY2cAE7ACHyY8zgaEmkJydhWipUz0qN2xET0cT99s2RRwU5j+89ColVA7ZRkgQbliARWVSCm0uAUzAWhQKTyEXJuAtBAJLFBrex0Hz8VUhsxZjngfcbyIJtVTIq2eVWabf83uFgl4+s2noOtQ712D96YEHh3R2mlShsJljj/s9RfygLxKYvhqb5pq9hU0lWJCyFKU19t68NmjrKrB3aTomT16K6r46FMMykJ1s9hXqK7F0wQZUaXTooDBmnboKG+YnYv6ueuhlofxHmS9+hjxoM3sKPQiTm/IogYAShdfefAknJW11G2Zn3+eRQAsZJY+WZGH7VSl4YyDl2llKjVNI6WlCpuCRrClWwu1znCxYg9LzA/NFXrti2q8oo6TXA1rqHsgg+VkmMCwE5MjafwCrEkyiTd+gxMo5UzAqYgzGjx9PrzGICLkDUx54AuuUtWjWtfXDOx+GrO0/g7kL6Ot3YdG0CRg1ahQmJC7CLlUzDHTE3qbj25DKP4DD8inwlk75VBNvmQm2w55AAInCFrxa/o4UYBIUtxh5CZ75rSyLMqeiIVFozoVtz9it94bzv8OygiO4SEmwc3YVY0NhMXZkWZ280Pkhdi/fjBNSUsT+lfYrJm9j1Ljb+9cAP8UEvIYAbYEINv8MU0Jqt36aw5Kx+fhZHNmUibhI8xMGkcOwlV56kwiURSIuNRvbDm2H4wl3MimQRBQZ9em0KFbgyJHdyIztji41P4Ho1FU48v4JrJ4ut7TjohWnTfNF/yFg7SnktDT+M6/+MJKAiT421L+A8gaxLjsCKUs8E7ghfQCiFBCxIS0kCluEYOtPssLPqrBm+V6o28mDuX0/CsyJq5MKd6Lw4hJsec8ckXy1GuuX346ogz/G3W79K2j9ESX7pDP1ghARxaLQmgx/74sEwpBz9FOk6fSQhcrh8oQ7+6EFxyBt9QF6daBNo0GDVge9njzwFCQSHaOAQtEt2OwfBdJw4NNL2E71Q+Wu64VNz8OBD7KxXaOGWtMKAy0XR8cnIF7eJQG72qFu5W5b7mgOX/FZArx87LNT5/eG+5UovEL7dg4e/hDjlqzDw+Os5+4qTpRVmZJMTyQvYYoHd9SNjDGfKXzRdNxdX0XhtXewmXISvnl1BO5dsx/PzBYJprvKWDy6ayfOP56PwxdNGw07G19AfoECyt3/t48RyS04f0G0ezsdx9dnRen3Pwg8QF8kEAy5RWj11f5ghCnoeDxFH58LDoN7XZJtiulIc9W+2+300T6u7hMEvv71r1vslLJMcGECXkLAT5aPr+L0lkfw4OObUfbaEWx55hAk/dNVzh9C6dviB+82zMunk0k8Cl+BhIliV+EXuNDYt2AQw5V3sP0HayTBF/XQsyjOcZIge+Q0bChZh0QrHXv1zc3I393Qtz1PVzS4IMwLIXttBLNHYXBjTIAJMAEm0AuB//f//p+lxr/927/1UptvM4GhI+AfotCgwYk3tJbo3873duGJvD04fUHs8ruKV4rFXr0gjM1+Fmut1ZVHOMsQP2uKFGzSWN+7UKvfvQRZj2bju+kpmPmA6bQSYUbLyZ/jkbWnTOco29ilQekzR/BZV5SzdK8Tjc/n46FHlyA7h15rK21FsJNxGSiquZGuB8XdB1cHuDh5jC8xgT4TENG2AyniTFguTMCfCfzrX/+yDO9rX/OrBTt/nraAGJt/fBplM7ByzXdQv+FPJAFN5ep7tMz68AukgoRcG4HEfArcWBLnkYhj+09GRGIaJgV9AHXDBziP+3G3fQXLewPaL5LHjtyYIr+hLPw2ijCmVDbkxDQYrktnEjvGF7eTB/JD2rNIaWRCxFFIQaYh0XPtLRpcofyIaJ/q5DlbI86f/ZACYYKQOHvaoDBwOWS+EXAEgikIY9GiRZg3bx7S09Npu5590IVzJP/4xz/w4osvQqfTYevWrc4r8VUm4AcE/vd//9cyCvYU+sGE+tEQ/EMU0oSMnvdfUIYUYcvOIzjb0rVHYwQmJi5AXv5ipI/z4D5C+w9AxP2YF78L6vfO4OT51bjb5fq0DCm7a/G+/fM9vp+GZ07X4Zke6/R2U4PTZz8nPfkdzJ5lvWext+f4PhPoHwEhCrOzs6WH7777biQmJiI5ORmXLl2yNPi3v/0NX3zxBc6ePYt3330XTU2mpM5//etf+9cpP8UEfIQAi0IfmagANNNvRKGYu9Epq7E/ZQWuffY5WjqDEBUVhZFDElMRTnkPZ2A3icLTr2lQMMnV7vJh+oRRAM7Ji3Ri3oOLkD7AY/2GaQTcrY8ReOihhzBx4kQ0Njbi3Llz0qukpMRmFJs2bXIYVW5uLqKjTWePONzkC0zATwiwKPSTifTDYfjHnkKbiZFh5OixmDRuqAShqfORid/HvLFib+Ah1DuuAQ/jR8eA+sOn0BIUi8eXzXAvn9swWstd+w8BZ6Kvt9EVFBT0VoXvMwGfJ2C9p5CXj31+Ov1qAH7lKRzWmZHFYdnauTixvBolJ3+M5+d5yTLtlSqy5wspujmHo46H9SMSaJ3PnTsXcXFxaGhocGvoP/jBDzBmzBi36nIlJuDLBPw7+vgdrE/5OU7Tal1QCCWZp4kSW/ulzDsGAwziSNioxSj7/WKrTCAUUPkoHT3Z0v2MNL/m+rIHd+KNwmm+POU+Y7sfegqHj/3IxB+jYFYQ3tv/3zjnFd5C8hLuewHvye5DQf409hIO30cjYHvesGGD22Nft26d23W5IhPwZQL+vXwchHA66Ss8PIQCKdvR0vI5Ll6kLV30teXqF2gPEvdG2P17RKoxiIIn27/AVVHP/BLnQchC6NSw0SJglMtQEGBR6FHK4UhfsxqJ7VVYv0/Ta0SwR7t20phBvRebX2tH4ppVSOG9hE4I8aXBJjB79mzJW9hb+dGPfkSJqOW9VeP7TMAvCPh3SpppWFV+GK+/Sq8T5dgx6zbbOQuZgQ3bM2G7cDUWeeWvYstMkWGD0unGfR9lf6jB+6dP4o0Th6GkzCFchoYAi0JPcx6dgR3b09Cp3IQS9TC6Cw0N2F14BHhoJ3bMi/L0KLk9JuA2AXf2Fq5du9bt9rgiE/B1Av7tKbSenSikb9+JJdIBD6bSefElOpXLPreuAefL1mDH29cRNHYBikt+jISIIYkS9fWPksftZ1HocaTAyMR12J8fgsMr1+PEZ4PQQa9NtuBEwRqciFiF/6J9GOwk7BUYVxhEAmlpabj33ntd9rB8+XL2Erqkwzf8kUDgiEKaPdpvv6rkGVif4Nr+9nY8teUdXDNP7mevrsfSYsqlG34fnqETvBL4H61h+9izKBwU9DKMyylG2ZJ27F5VhPquT/6g9GXfaDvqd6zBbsNi7C+xd9Hb1+X3TGBoCBQWFjrtSCaTgb2ETtHwRT8mEHDRx5TL95mSFYi32hp4seLnyC/X4LOz27D0mTO4GjIVa0k8po/244n3gaGxKBy0SZJhUs5+KNco0N63I5EHaNFVtE9cDeW+RZjE3vcBsuTHPUUgJSXFqbdQ7CUMD/eSSH1PDZbbYQK9EAgoT6GZhWzSYhRvn4vuzUzX8d7OJXhopTiGNhY5u3bicf5Hq5dPzuDfZlE4yIwjEjKQMqSpYMYiZd40RAzyuLh5JtBXAr/4xS9sHhFewp/85Cd9bYbrMwGfJ+DfKWlcT09ECm2tWjOVopK7ijjmNQjx+TtRkDCIp465Nonv2BFgUcgfCSbABIaEwH333WfjLRR7CdlLOCTouRMvIxCInkLTFIitVRT8+NDtVjPSCbXyv3Fa5J/hMuwEWBQO+xSwAUwgcAhs2bJFGuwtt9yC/Pz8wBk4j5QJWBG4fv265d2IEaY0LAED6NpF1Dfa7alqOYY1K3+H88OYsCNg+PcyUD7RpBdAfNt7CRw8eBCXLl3yXgPZMqcExNnGo0ePxm9/+1un9/midxK46aabkJCQgNTUVO800Iesunq1WxTddptdHj8fGkefTTVcxO9X5aOssRNRDy5EfMNLONliaqWzYS/yC2Og3HE/b3/qM1jPPcCi0HMsuaUhJKBSqSCWH7n4HgGj0QitVot33nnH94xni6W5CyghMwhz/sUXX1haDZwtFFdxujAfW967jvCZG1C2IwMRF2Jw5fHteK/dhKPlDUpNM6mMklUr+ASuQfjcudMkLx+7Q4nreB0BnU7ndTaxQe4REB4n8eLimwRaW1t903AvstraU/jNb37TiywbLFPacW7HU1jzxucIotNKindnQGSekY3LRPGuBRhr6bYTjcVU7+SQpuwYrEH7ZLvsKfTJaWOjhbepq/zwhz/E/PnzGQoTYAKDRGD79u04c+bMILUeeM12eQoDYz+hARcOr8EyZRM6o+Zi3+4f426rdGkjE9bhv9a3IPuZP9FJyaJ8gTcLn8KuqDKsiue8akP908GicKiJc38eIWDtaRo/fjySk5M90i43wgSYgCOBsrIyx4t8pd8EujyF/r8M347zhzcj/5kP0B40hZJTb0SSk3xp4x55FsUXlmCpUotOQbWzCWV0Itikgzs5mXW/P2X9e5BFYf+48VNMgAkwASbABPpFoCv62C/3E35WhTUFh3ChvRNXrmpx1bxfEJ0fYnfeElzZLTyAttiunN6L3e91SvsIJVEoytUzWP/4IyiNCpGux+fvo1yG7DnswjNYX1kUDhZZbpcJMAEmwASYgB2By5cvW674paew8youNGpxAUGQyW6jXKQ03E5KUm3ohKG9pVskWnO5qsF5kaYmaARCSPcFBZnOw+uktloaPwc9inDeZjgkP0ssCocEM3fCBJgAE2ACTIAcYFbpaPzSUzhuMSrqF/dpqiMeKcP7j/TpEa48SAQ4+niQwHKzTIAJMAEmwATsCVino/FLT6H9gPm9TxFgUehT08XGMgEmwASYgC8TuHKl+ztLnNYAACAASURBVDw3v/QU+vLksO1gUcgfAibABJgAE2ACQ0QgMBNXDxFc7mbABFgUDhghN8AEmAATYAJMwD0Cfr+n0D0MXMtLCbAo9NKJYbOYABNgAkzA/wjwnkL/m1N/GhGLQn+aTR4LE2ACTIAJeDUB9hR69fQEvHEsCgP+I8AAmAATYAJMYKgIWItCjj4eKurcj7sEOE+hu6S4HhNgAhYC2upSVDboEZmUjZzp8mEgo0NduRK1raFIyMxDcswwmMBdMoF+EOBAk35A40eGjACLwiFDzR0xgeEgIMRTBYmnYMRlZiMtJtgDRuigKlqJjbWALFNBojDDA232tYl6FK3ciGMGICkyAydihkOY9tVmrs8EgE8//VTCcOuttyI42BM/j0yVCXiOAItCz7HklpiA9xGo2YpFy8rQSpaFqqPx6YE077ORLWICAURAp9NJox03blwAjZqH6isEeE+hr8wU28kE+kNAFgyLL8KLvBIdmgpsyM9HflE1tP0ZFz/DBHyQwN/+9jeL1SwKfXACA8Bk9hQGwCTzEAOYwPRNOHo8GWq9DPFJyV4DQlddgl1l9bT+rEfa6jTwlkCvmRo2ZBAJ/P3vf7e0zqJwEEFz0/0mwKKw3+j4QSbgCwSCoUjOgMIXTGUbmYCfE7hw4YJlhCwK/XyyfXR4LAp9dOLYbC8l0NEB2j3es3FUpXtN13nVDmqnp03obToNmrVt6AiWI0YRA/nAuwQ6dNBodNRmKKKjYxDWS5vOLG/TqKHpCIWCbOrP887aHNi1Dui0GmibCXpoL6zcmTvJmA76z2pZ3t5A4qjVaKHrCEaYPBqKmDD7Grbv7fpt0xJD2nYWpoiHopdHe26Y73obAWtReOedd3qbeWwPE+Czj/kzwAQ8RUBXmo6IUaMQEjEf5aa95HZN61CeHoGQUSGISC+F0yqoQ/7kEIyidmYWaeyeb4O6PB/pk8fgjgnTkPjAA3ggcQomjBmDmQuLUK0VatO+VGPpGGovJAILK9rsb0rvtdVFWDh1DNk1AdMSE5E4bQruIBtDQqxeEVOxQe30cXR0aFG1YSFmjo/AHdMSJZvuGDMZ6QVVdvsF21C1kPqhdqeso6VjUQyVWGTVT0TKXtiP2nmvPV3tgLamFPnzZ2J8xChMmEI2zTGzGjUGU4lVjR387rlLQWkPmxzbKhZiTMgojKI5tsfZoa1GUS71OWYCpiRSfw8kYtqUOxAxOR355Wo4o6/dmyJ9ZsbkVhNHDcpzp5rsFc/eMRMOH4Gehs33vJ4ALx97/RQFvIEcaBLwHwEG4CkC8oQERIvGDDWorHYiAdroej3lUBFV6qvgrArqKqFqFjUikZBkveirRUUuCTaKJK5t1iM0NglzMzMxNykWoQY9Go5txIKURSh3EDQGtEla0QB9h6lvyQBz0ZLISVlAqV2a9JBFxiF17lykJkRDZlVHJpOB4lWcF101CuYkYtGuY2igtDeR0ZEIFTUNzagtWYSUpdVWYshVI1ZNO9O1znt2eVVXkYvEOStRpmqgqOtIxMYlISkpAbGRYlR6NBGrOSn5qLGaInlymmmJ3VCPvZWuZKkOVUoVtUAlPhUJVl68troNmJO4ABsrTX3Gpc5FZibViZYRilqULUvBnIIaB2FoMLTRzJDvUa+h+U3Hssom6b2p6EHOYC5+RMDaUzh69Gg/GhkPxW8IGLkwAR8kUF5ebhwxYoT02rNnj5eM4F3j2rtMNoU/9rrxSzurbpxabBxttnnEiHDjY6/b1zAazxXeYxrXnU8a37rR3UDjr2cZw6Vn7zIuPthotLplvNF42Li4q9/Zzxkv2vT7uvGxcJNNsw9etrXoy1PGxaNN9+568nWj9d3Lb601zpCeu9P45Cl7Oy8bn5ttes40B+HGe548aDzXVe3yW8bCWeHm+bnHWHjOcXou7plluh++2Pi69WAcq7q44npcou3R9zxsLDz4rvGyTds3jOf2zDbPQbjRlsdF455Z5jHdU2h0YrLRePmgcbbEhJ59zoqW4Hin6dnRswqNb9lg/tL4ViHZY567p6wnlUbW+GvzfIeHS/Mbfs+TxoPnLhu//PKi8dypt4yN/WLjAtkALufm5lp+3j766KMBtBTYj3b9zpo6dWpgg+DRey0B9hT6jbzngQw/genISI2UzDDUVqDezuulrqoxeZlMNVBbUU9706yLFjWqJulCaFomEroca21V2FpUL3mQYlcdwv4chc2WxGBFFnbvziSfmOh3L1w6umz6Iu9UvRIq4faSJeFnmzJgnf5ZnrwJmzKEz68VFSVVDh6u7qboRJFNx3F6fw7iuzxn8mRs3r8csVKlJqhqHNyXtpa44UC0M73HtzErTuPTD45ic850u72WwYhfsQnZJncuGlRqK/4xyFqSZPKQNlVC6WSpXFelhOTolSUgO6OblqZsIyqlRJBzsfvQZiTb5NEOQ/Lm/VgdJ0xuRuVelXOWBgMM0dk4dGI/cuLlCAuLQXxaMhQeZtMjOL45qATOnz9vaX/SpEmD2hc3zgT6S4BFYX/J8XNMwAmBhKw0SZxBX4MqG1WoRlWNUA6RtOybIIkPfU0VzDvrTC3pVKhqEN+GIjUz2SL8OkhgVklrlgnIy4t3GqMSlpyFBGnNtwnVdc53K5o66f6/vllnEqlyCmhwOBAkGHJaChbFoGuVkl87K7LUbTiyejoc4iEUGUiTxBeg00jr4V5SFIgz26XXU1CNlVXyDDouT2LYTMvEdXb20tIxiXhJEybnoVsTaui6NGmIzFhhdd36cQUysiRVCH19tYs9k7FY/vxupDnMg3U7/L0vE1Cru//SiI+P9+WhsO1+TICjj/14cnloQ08gOCETaaFKKPWtUFXRPwLJ001GaGtQK5yAocnIeToBrZX1qG8lESiqmP99aKs2i8TQJGQld7uINHVq0z4z2ttn0FSj2umWNy0M4hFSLToRumrj93POITQ0VBKnBp0WrUId2XilOtDcbJaCVE/aJ+ishIY5CkKpnhzRQuCQHjToPbBR0FnffbjW0aaDTthB+y8lfS2KvVlhqcgj76iqUo/mCiVqNk2HZRp0VVBKbkIS7Hmp3WPuUKPOPB+hFFZTU23fqKkrfZezlISoCISebu8BjExGpsPFLkP5qz8QOHfunGUYLAr9YUb9cwwsCv1zXnlUw0UgOJmWkEOhFMKCRJ56+3QIzddWo4LwE8gSMpBEqUY0cetQ39BMy8WkKOJFQEkHaqvMnqikLCRZiQZdq1mcGWqxcQEdOOyhEkz5C5Nlx6AyqLB1Yw2StidbxE5H3VZsldyTMiRkpLohMe2N6k7Z4lwm2df38HuKiK4uLUJppQq16mboHWNsnHQYhtTsDERWKtHaKkTgdiSbVaGWRKJJE6YiJ9XKL0pHlrWa225SLsMCpZNmbS65ZUhvjfB9HyRgLQrvvvtuHxwBmxwIBFgUBsIs8xiHkEAwkjJob1rlMRiaVFBpNyM+hgSfyiT4EjKE8JIjNTUaG0kUNlTRMW+rFYjpoOXmWpMISyYRZrMc26WqZHHIXJ5qinB2OiLy2skpajnLzaUpeRZ5xtZJnrGGkjmYUk1RuvExCKbUKDVkrxA7stg8bMrzsfNGtBXInbMUlc3SYi8iYxOQqoiGXEqcKNLVVFIEt1OACE7ORkakEmUkxKuVNeigqORg8gBWkWdXlMiMbKTae/nMTUWnLkdmnHXctl0fYTTztKye4eJ55xbxVX8h8Oc//1kaysiRIxEVFeUvw+Jx+BkBFoV+NqE8nOEnEJaaZfbANaBKpcPqPA2qa4Tgi0WGOQohnoRf5K4ytKpVlDNvBWK0lKJG0oQJyEyz3aEXKheLt3QzWIHszZuR5qEhdqi3YgMJQoTGISmaUsg01FJqGrMnUkbiMvtpbNuW57jU6aH+B6cZLUqfMAvCSAr8OLofeZYIGNFjByoWVpEodOGxI09vXlY0ykqa0UqBJTUdaUijpWOTJoxERnb3Xk/Jfqul9VAShZtX+JiAHpxJ4FbtCFy6dAnXrl2Trk6bNo35MAGvJcCBJl47NWyYzxIIo/1hpqgPqMkTqFNXQ4oxiU6jpWPzqOIpEENoPcppWEUJ80RksqgiS8hyCDZQxClMUbGUy67B6X7C/pGqL1FSWApJ1dUHcOLtT/GPjz/E+2f/gD+c/RAff/oJRRSTIHSIIOlfX66fIjeoJ9eXKRdkhTkXZMLTxXaC0LUV1nfi8/JMkdN6FZSqNmhE1LGoEJ2FbKu9ntIzYQqQE1IqzfUNHh2KqVX+vz8Q8P6l4zoUpU/F5MlTkb6BPOT+AJ3H0C8CLAr7hY0fYgI9EZAjLcsUYWygpUplRa0kvkKTU6X9hVIx7z0UkSG1VWWoMKeiER5E+wDUMPGctCrZgNLSOg/9wqbj33SmkIvWmjo6mo5MksdAET8d02kJubdj87qG0d+vwcHmZVYK/DCb0d+mbJ/T6y3Jn0MpMMeh0NK42pWXsKuyIhN5CeKNHqpy2peoNEUXx2Zmwxw2ZNVsPFKTTVHaelUpKtwL/HYwiy/4NwHryGOP7ifUlCI3ZSamzqRTcyp6Sf3UE+KOZtTVN1FwWROtGGg99Dumpw6H5p62gk6AmjkVM1NyUe7BP6iHxvrh6YVF4fBw5179nIA8NcMkAMkTuLVE+JlkSEpNsArwFXsPzalpqraiRNIdtGcww8nyY0w2VmeYhEcznRKSS8LQ8aALsVeuHBs2VLhIeWIPPBjJlPZGWphWLcM0Oipv/PjJmDyVfoHOTEFK+nwszM1HUTmlUBkEt4GczkY2jageSqXG/I9QB9p0jiOzt7zH93IFos2h0vVKpa3tuhoUzZ+PXSaN10MzlLMw25SzUH9sl3luaOk/x9lezWCkrV4BaSsheRZXzi9AlZPjBjt0lJJobwH2Wh+j0oMFfMu/CPzpT3+yDOjb3/62xwanLqe8pOShbqKtH2WlKhdHZ3qsOx9rSAtVCZ0A1dCEhvpKFFWxKnRnAlkUukOJ6zCBvhKIyaCgA/EQJSWWtq8lIC3Zdi02jKJ/pVVmkbhYVImjpWMnmpDWKJGx+wCWxIrKrTi28gE673iqJNzmk8iZT56CyWNGYcqcZdhFiabd/dUnz3ke+zNN0kykamltbUZzE/0CbaB0ObUqHKssw8ZlCzCNzu7dq/awMhSpe6SuDajfOA0Txk/F1PF0pnN6kRSl3e8SnIbVlCla0mi16zBzWgqJ26XIXZhCS2NzsLHWgDg6xq+3IqeAElPOQnPNuBxQznDnRbEaB7bTHlG6a2gowaIpYzCZPDfS3MxPR8rU8RgzgY4CXFdCEc2uMj46b5qv+geBN998UxrI2LFjcccdd3hsUKHy7t8poaFy16mjPNajFzREaaBKly6kn61cFFT15B2lVFrS0ZaiyCCn/b8ORVOO/IX0c7ownz2JZjgsCh0+JXyBCXiCQAzSsk2eQNGaSEWTar8uLE9FlnnvoagTl5VhOn/XWfe0T7H49GnsE9HH9LvNoG+ShJtKRS/yFDTrZYhOmItV+59Gqs3zMrN3Uny1XU7VVSxFvjiKg6Kalxz5EJcufYyPP6R9he+fxdnjR7BvUybixO/R1lqsy91qJdbEWcimtuzbtO3aXMdZtC2Jt20HViHJ/Etb39qEJgp3Do1ROCyfO8MhfsmbmnUcV/zqozi6ijhR94bmehK3SlQeq4eOeK86dBZHl5tEo8vznEWH8gyKzDYLZnqbkJ3pem7oviLvKE4f2UR/CESSZQY0k+dGmhtVLeqbiHFoLJKyt2F7tr2yNMMJdgbJ+cj5qm8RqKmpsRh83333edT4mLzn8YdD+7Dv+SM4vj/DaWJ7j3boDY2Rx1+ppFRaKtqa06P3LwxZxadx6Pl9eP7QcRzIs/8FTNuZ1dTGMfo5PVaGSvUAVym8gY0HbODoYw9A5CaYgDMCCnHcWjadGmKQQfxF7/jPfgzyTnyKTNpUZ5CFmlOmOGvJfC0sHjnbj9KL9gNqNNC2if1z1HYYJYqOjoGUccWhpOHApx9jO4lGuZVXAZQCZ2PBMfI7Uh7CTYdQ3OWiJMeD9KuTtEs8pWPJiNRjwjKVlF6nRkqvI26GIefQx0ij5H+hcsdftCYT5Mg7+jEyRHCzdb9W9oUlb8aJT56msaihpb140p5GOlrF6TDcHZdUT05Hyx3FX5+mIBHiJPYshtKyskLRNQcv4dNkOs2EPCuu42jCyKNIY5NEcxKys5y6cG2sikkjjyG90KaFmhKI68kDLKM+5PJoxLjYpKlYfRqXsiUDe7DFYfB8wYcIdHkJhcmeFoUIjsF02nLiuNfVhwANpql0WlOGuym6BtMOH2qbRaEPTRab6nsEgkmw9Sxygim3YM81HEdNR9BRAmxXcsyhfjAJE/su6GSUOmklMxpJya4FT3BYl3eRklFbtxFMOffs27Tv2J06REeuoDOK7R1o9m05e+9sXNb1qH8ROOOs6WASsz2bT0tUpabNh7IEyl3oNmwyQJxbPN01U9uh9Gf+ncHga95K4I9//KPFtPvvv99bzWS7mIBEgJeP+YPABAKRAAkmkyhqhqpC7TzasK0OW7eSl1DwiaPTT/oijHycaUdNKZQiZJw8qcl5Ge4LcB8fN5vvWQL//Oc/UV9vSnw+adIkjBo1yrMdUH7TgoXpSKf9dUW9BDG1qSuwITcdMyfTHlcRWDY1HbkiMK1P24VFQFspCqzaGUMBajPTc7GhvKaHQJcO1O3NxXzaB51b2vX7Roe68gIsTJmK8WSPaCdlfj4FYzkP4ReRxAsXLsTClaWWfdP66l+aronr9MrdUGVjQ1tNEXJpX2/6QgoAszSrQ/WGXKn+oiLzEaI0K+qiJ2zayq8w7c7WlC8lu2mPcO4GVDs3rXtOpT2KpvnYW9cnsJ79XAygNfYUDgAeP8oEfJaASLuSVISVdIpKw65ETFalUiBMHBTSUm8btOp6OtWEUumIrDWhCdi0e7lTj5vPjr9HwzugKq2U8kZClops62PtenyObzIBWwJnzpyxXPiP//gPz+NprkbFsVrps6pPfhqr7YLZTB22oa5oERZtNNWzGKGvReWuWlRVqbD/QFLvtokAj9xFKDjWbEn7ZHqIfodQkFpDbSVKirKxnxLGO+62oDpVlaBfKbRPJBPbU5tRsGgpyhpMabG62qlXlaGe9gpWbTqOo6vjrbz5bahXloG2/9mWVtozfMzqUm0Y8jZ3/xHXWl+OSindlwaK1dvNHn81Kssqccy6a6rR2kDBdVaZCWTIwvYsWmfQ1JHdoo1a6BLykOYyQX0HavZuRNkxMRuR1F/PaxF2I/GatywKvWYq2BAmMJQEaD/j0eOQFRTgl8paCoygRM30si2hiE3NxtPbNyFL4Zu/4PpFtK0KpdK5z6SHM3LAmrBfFPkhIlBVVWXhMHfu3GFhoi1/AgtIEJo+0eKkomxkUxb9Dg2JQmUFBUIp8URKBWQuDvkxGU0nBc2fI/0RKbzn0Ul5WL2CsidQ1FuHToNa5VYUVTZBL9qaT/uj36Zzw139ytAqKbdiLWrFdt3IBDolKJNWIXRQi8ARlRCcetRuzMXGpLexfXpXI2FIXr4JS6Ip2lhPJ0XREUNCesli6Y+25O5sAsHxWZTYq7eSgOxNyyGjjAoGDf3eM595GZ1ER1gquoPxYuhIUtG7IicHcbs2UpZYyhSrrIRmxWrnfyB3UHqtKulPSXOi+97s8NL7Ri5MwAcJlJeXG0eMGCG99uzZ44Mj8CKTb3xpbDz3lvHU64eNhw8eNB4+/Lrx1FvnjBe/9CIbh9SUy8Z3icXBw6eM5wKWgS3w3Nxcy8/bRx99NKSz4aud3bhxwxgZGSlxo1Q0gzOMt54y3mn+PTjj142OfXx5yrj4TtPvyREj7jI+eeqybZ0bjcbDi+8xhpvbkH6nPnzQaP+xv3zwYeNoc517njrlcN9ovGFs3DPL0o6jLZeNz83uskN8DTfetfiw8eINa3NuGN8tnGFpY/TiU9Sqk3Jxj3GW2ZbRT55yUqH7UuOv7zF/bu80PvWuY9Ubhx+29PfwYftRd9W/aHxuVri5nXuMhecc2xFXbrz1pGUu7lrropLzR73qKu8p9FKxzmYxgSEjIAVkJCMtIwtZ9FdxFqXGSUuOR4zr0NwhM214OpJTRGcWcrLSYHNs8vAYw736KIFjtK751VdfSdZnZWUNyyja6JSdLudVZPZu7E6z2xhM56lnHTiBA9k95e7UYG+RyuRpjF6C4u1pTiLlg6FYsQ155mYaKnrOlxqdeQCnD2QhxsabGIzpqymlljmdoF5d43bO1cGFG4OMJcnmhF5NqKI92I6Flo6V1aYtJ3RIZma2s0T3jk954xUWhd44K2wTE2ACTIAJ+DSBQ4cOWex/9NFHh2Us9bSJz7QqHEnLtGkuIu7llBx/E1Jt05h226upgvkUTkTSdhKXy8J0hlNqglnRaWrgMu2fbC62U05Fp3FrYXFI7koX0NzcQ+DK0OIUyey7xGoT5T2ts++eUnxZlo5Fonvf1YTgPYX2k8vvmQATYAJMgAkMgMCVK1fwxhtvSC1ERUUhIUE6THuIixYatTmaQhYPiiPrsbjShB1qOhvd/GQomlFdbRehYdWqputW15nmTlcbyD3oar8hnckiDxWWCClLe/5EAK/Luj0Ox7M3wzKQQ6rwWCUNsLkCyppNmG6ljjtqlFCZx95bonvPGub51lgUep4pt8gEmAATYAIBTODFF1+0jD6bAjuGp+jopCNzz6HRlBy/f1bo9a2WaOOmMgpaKXOnHRJ0PQauuGqDFKCVOvWepC7BSM3OQCR5CVvpvyplPbYnmwJRhHhVKc3L624munc1em+4zqLQG2aBbWACTIAJMAG/IfCb3/zGMpZFixYN27g6ulQVZZ535QnszbgOSyPitMYkxDld97VqhfoKVVDCd2dZ43vrzIvvB6ctQUa0EmXNlL6mqhQ1HbQPW3gxO1SoMLsJZcl9THTvheNlUeiFk8ImMQEmwASYgG8SeO6553D58mXJ+IceegixsbHDNBCrU4j0dOxiP60IDjXvE6Tn41YcwAknZwj3s2kfe2w68jJjUbaLchbqKZWNqoOC84LRpio3Lx1TonvyJvammb190Bxo4u0zxPYxASbABJiAzxDYtm2bxdbCwsJhtFsORVdQsb4ZGkrx158izjfvkoXN/W2kPx174TPxImehZJceqvIqSvPfRl/NS8ehqcjzg6SmLAq98IPHJjEBJsAEmIDvEfjd734Hnc50Flo6HY32/9s7F+goq6vv//n6lUmVEDQp0aiQiAQqJFggaCBUiMFPJAWRS6FBEbA1BJcJtJDIzQJWuSwh2BJiLWCRCIWIgkBQKIgk2EJATVBhWG1CLGAiVEJQM9S1+p39JDNMJjPJzGTu8z/vmjeZ5zmXfX4n78vffc7eR662816JRFxs58bhy7DfxvVxUuHsLpXE2sYZwBCVrsoYo1J1uNj7aWLMLmGvv2ywfkWn3dCvb6rX1toAYN6XugkqrTFmqHZ/AXap5NeF+xvahaUERqJ7ikK7/3hYkQRIgARIgARsEzD3Es6bN892RQ+9iVe5NhtkoQH7V66Etet4q/dlYnT67sYce1YMi0xVVz02iqfylcje5KTL0UrXTj1S29lagLIqBnUFnTri53QJCTPeAa+EcZneDoHZFWPTkhrOZxoOIy99JRo0oaT8SfGJQGmnYTQ25JnCthJke68QUCngTeO+8sorTa6T8opBHNQuAt9//z2qVP6xEPVf+5Kqg8U/CJw6dco/DPWilRJx/O9//1uzICUlBX369PGiNQ1DhwyehRlxBXhO7mg7k4eJo3XIXz1L5QIMQf3ZMuxamYPn1suVcToltNQFc1adZZEYq66Fyzu8CuUq1cz+6Q9jtH4ZFs1IQXzk9Xwx9dVlOKw8juvk3vC0t3BghpsiTUJUeh21La7lTlTX6i3cNAn5Y9UDNX6xmmdC6gD7z/XFJyBOV4BiNe+aN1/AurTXMC0+DLUqz+LhWpXYe3DXZmsYOUblaswuVmLQgPJjjZcld1bC2XYCx2Z9+PIDikJfXh3aZpPAD37wA9O7iooKyIfFvwj861//8i+Daa1GwPz/9ojkOoFly5aZvjz77LM+giYWGfmLsCv5OW17uKZ4FR7tt8rCts5IyX0NqbtGI6txK9TS+JD4xdicW4WHs95ElaEK+1dNVB8ddI1eO4MSi+bbr90bdtAtu3HR91ikTkvACznHVKqcGuye3g93TG/sunMa9jgiCsULOvg5FEv0cG0xch64CzmNXelGbECqEoXN0iSqnIXTUnKwf/f10J0uY1tK6u2iaXuoG24fewg0h3EtgaSkJISGhrq2U/ZGAiTQIgE5I+e9aNoWTfPqy7y8PBj/I+f+++/HgAEDPGOPTkUYN44UorOedCYkfhb2HFiLtDjj+cLrpnWOG4Pcvx3HW0pkRTY216k+rZWuk15DieonI6VLY+CJAQaVw7CmpsYkCMO6JGBExlq8NsvSS6gEZEjjAK2kx7l+ZPD63CztiZ2xAa9NjTMFwBjfh8UnocmFfaa5mEViN+ksEpPyN2NmUmeLlD06xA6IbS4ItbadkDJjjNk4cq2dh9bbEoQbvreTm5jd0C+7JAG3E6itrUVZmbV7KN0+NAdwkkBBQQHkI0XOX8XH+/F9UE4y8Ndm7dq1g/zHGEtTAiIGRQQaGrM1FxUVeZRT/WWVbsagtn9Vdmrrcu66vZfPHkW5Xok4Jac6d43DgFjzjNb1uFytvF8q2rjVPNf11er/9+pRXVOrvHVq7M5d0SU2tpX70utVEE6t8i6q/ls0tMEOg6pntjtt/c/u8lmU6au0bW9dZCziYyMtGDT2pVM3pbQ8KOqr9aqv6ob5xMY32RpvNvjlQkzoPQWas7D7TBw/sRiWMrhZGz95QFHoJwtFM0kgEAi88MILePHFF7Wp7NmzB4PVrQAsz3poTAAAIABJREFUJODPBMQzeOLECW0KI0aMwJYtW/x5OrTdDgLV64ajV1bDvdJxi47jSDPPqB2d+GgVbh/76MLQLBIgARIgAd8mINvGRkEYERGBtWvX+rbBtM4FBPQoWNcgCFVYC9LGBIqPsAENRaEL/kTYBQmQAAmQQHARkG3jhQsXmib96quv4qabbgouCME427J1WNcYdKxLSsPY5gHKfk2FotCvl4/GkwAJkAAJeIPAlClTTOcIH3/8cS0NDUugE6jH4bzCxtyIgXGtneWKURRaEuF3EiABEiABEmiBgPm28e23347ly5e3UJuvAoeAHseO1mjT0XVJQ0aqv9903HxlmKewORM+IQESIAESIAGrBI4cOYLs7GzTuw0bNuDGG2+0WpcPA41APGadqMOsQJuW2XzoKQzgxeXUSIAESIAEXEdAr9djzJgxpg4zMjJw3333uW4A9kQCXiZAUejlBeDwJEACJEACvk9AkjRLypmrV69qxiYmJsL8FhPfnwEtJIHWCVAUts6INUiABEiABIKYwDfffIPU1FR8+eWXGgW51eXNN98MYiKceqASoCgM1JXlvEiABEiABFxCYPz48fj888+1vm655Rbs3r2b12y6hCw78TUCFIW+tiK0hwRIgARIwGcISLqZDz74QLMnLCxME4S33nqrz9hHQ0jAlQQoCl1Jk32RAAmQAAkEDIHFixfjrbfeMs3n7bffRqy645eFBAKVAFPSBOrKcl4kQAIkQAJOE5g3bx5efvllU3u507h///5O98eGJOAPBCgK/WGVaCMJkAAJkIBHCHz77bd47LHH8N5775nG++Mf/6hFHrOQQKAT4PZxoK8w50cCHiZQX1/fphEl0pOFBLxB4KuvvsKQIUOaCELxFk6ePNkb5nBMEvA4AYpCjyPngCQQ2ARCQkIwduxYyHZbbW2t3ZOVf5BXr16N3/3ud3a3YUUScBWBzz77DAMHDjRFGcstJRJUInccs5BAsBDg9nGwrDTnSQIeJDBu3Dg8+eST2oh9+vTBoEGDMHjwYFRVVZmsKC8vx9mzZ1FSUoJ//OMfOHPmjPautLTUg5ZyKBIA3n33XW3L+LvvvtNwyH3GEmDSs2dP4iGBoCLQ7n+qBNWMOVkSIAGPELj77rvxxRdfODTW6NGjsXHjRofasDIJtIXA8uXLsWTJElMXffv21RJTR0REtKVbtiUBvyTA7WO/XDYaTQK+T2DOnDkOG/nss8863IYNSMAZAuK1TklJaSIIR44cqXkNKQidIco2gUCAojAQVpFzIAEfJPDEE08gMjLSbsuGDx+On/zkJ3bXZ0UScJbA5s2bcd9992nHFoxl1qxZKCgogJyJZSGBYCVAURisK895k4AHCMyePdvuUebOnWt3XVYkAWcIXLp0CXJl3a9//WvU1dVpXXTu3BnvvPMOFi1a5EyXbEMCAUWAZwoDajk5GRLwPQLR0dGQf4xbKkOHDsXOnTtbqsJ3JNAmAuvXr4fcUGL+tyjbxWvWrEGnTp3a1Dcbk0CgEKCnMFBWkvMgAR8lkJWV1aplzpw/bLVTViABReCjjz5CYmIiMjMzTYKwY8eO+NOf/qRtF1MQ8s+EBK4ToKeQfw0kQAJuJXD16lUttYetnIX33nsv9u/f71Yb2HnwEbh48SIWLFiATZs2NZm85NBcunSpQ+ddg48eZxysBOgpDNaV57xJwEMEOnTogKefftrmaIw4tomGL5wgcPnyZS2iOC4urokg7N69O4qKirBhwwYKQie4sklwEKCnMDjWmbMkAa8SkH+oxVtoeYWdJLYuLi72qm0cPDAIiEf6D3/4g3ZG0NwrLTeTyH94yPYxCwmQQMsE6ClsmQ/fkgAJuICAnNtKT09v1hO9hM2Q8IGDBL799lu89NJL6NWrF1544YUmgvCpp57CyZMnKQgdZMrqwUuAnsLgXXvOnAQ8SkDOeEkewvr6em1c8RweO3bMozZwsMAhUF1djby8PG07+Ouvv24yMbmybv78+YiKigqcCXMmJOABAvQUegAyhyABEoB2S8TUqVNNKOgl5F+FMwTKysogidHvuusurFy5sokglGsSP/74Y00sUhA6Q5dtgp0APYXB/hfA+ZOABwl8+eWXkAP/3bp10/7xZiEBewkUFhZqYs/Su6zT6TBhwgQ888wziI2Ntbc71iMBErBC4P9aecZHJEACJOASAhJgsnv3boh3p7y8XOtTopENBgMefvhh7fvgwYO1SNHU1FSXjMlOAofAqVOnsHHjRvz1r39FTU1Nk4mJ5/nJJ5/UzqqGh4cHzqQ5ExLwIgF6Cr0In0OTQKASkKTAEgVqFIL2zlOEYUZGhiYUWYKTwH/+8x9NBMrf0CeffNIMgngDZ8yY0eQoQrNKfEACJOAUAYpCp7CxEQmQgDUC8g+5RIBWVVVZe233MxGFa9euRdeuXe1uw4r+S0CE4I4dO/D222/jwIEDzSYSEhKCUaNG4fHHH8fPfvazZu/5gARIwDUEKApdw5G9kEBQE5Bt4l/+8pc4fPhwEw533HGHti0sIi8+Pr6ZyJNtZflIu127duHKlStN2i9btkzzHLIEHgGJRhcRKJ9Dhw5ZnWDfvn0hkcS/+MUvEBoaarUOH5IACbiOAEWh61iyJxIISgIi6iZOnNjEO5iUlIS5c+c6vA0sV5JlZ2c3EYdpaWnIz88PSraBNmlJI2MUgraSlsv5QBGBU6ZM0dIWsZAACXiOAEWh51hzJBIIOAIiCCVgxHiDRMeOHbFlyxaHxaA5GPE6yha0bB8bi3ga9+zZE3D8An1CkpOypKQEBw8e1LaFbZ0x7dy5s+ZRHjlyJB544IFAx8L5kYDPEqAo9NmloWEk4NsELAVh7969NUHoqnOA4jWcPn26CQI9hr7992C07vjx45oIlM8HH3xg0+guXbpoIvDnP/85Bg4caLMeX5AACXiOAEWh51hzJBIIGALizRs0aJBpy1i2izdv3gy5zs6VRYSnjGMsS5cu1SJPWXyHwIkTJ7T7q+Vz5MiRJtfMWVopQSLDhg3Dgw8+iLvvvtvyNb+TAAl4mQBFoZcXgMOTgD8SkC1jY1CJeAiLiorsFoQi9MLCwuz2KFp6DGU7UoJWWDxPQLaDP/roI038yTrIR+4etlUk0EhEoHyGDh2KG2+80VZVPicBEvABAhSFPrAInjPhPIo3fQjdyDFI6OihUSvew7qyaIwbFQtPDemhmQXtMCIGjYmn5QyhCAR7t4zfeecdLUpZysmTJ+1uN2fOHNMZQ54v9NyfXmlpqSYCxRson88++6zFwSWHYGJiorYdLD9jYmJarM+XJEACvkWAN5r41nq4zxpDOdZlzEdRj4VY70l1FhOL8OWZSCubiQ0LhiDCfTNkzx4iINHBxiIpY+wVhNLGPNBAchna21YimSVlzRdffKF5KOXjbILrf/7zn3j55ZexevVqDxHz/WHk+kG5PUQ+skYiAEW0t1YkZYwcHbj33nu1nzfffHNrTfieBEjAhwlQFPrA4hiuXMLFOkObLNGFhiO0ow46a70YKvF6RiZeD81B4Zx+HvbYReOR3OdxaVompiwJRcECT49vDQifOUtAPH1GYSfbxpMmTXK2K4fayVlFEYbGwBO5LcVRUajX6/H73/8e27dvR2ZmpkPjB0rlc+fOmcSfCMDTp0/j008/bZYf0tp8xev305/+VPv069cPCQkJkKTSLCRAAoFDgKLQS2t55dT72LZtO4oOnsDpS9dcY0Vod4xb/DIWJpvfA1qHY8tnI/diKtbnPegdT50uDtNyc/DJ+NmY03MT8sdFuWa+7MXjBMRbZywi0jxZRIBKqhrxFsp9yhLsYk9gi4gfEYOSH89YJCFyIBY531dZWdnkc/bsWVRUVGhBQd98841d077zzjtNAtAoBJk82i50rEQCfk2AotDjy3cJB5Y8g/mFZ1BnMXb79u3Vk2u4Zq4R1TN52rxY1JMKdWewd68e2cmJJo/huR3zkLkTGL9uBvpYdSM279ktTyIexMI5+zB2QSZW9diEmfHeNMYtMwyKTkWMSZGzhJJKxNNFvINvvPGGNqxsIbdkg4jBJUuWYOdO9X8AZqVHjx6Qj78VuQFEtnkvXLig/TT+Lgmhz58/rwlBqeNo6dWrF/r06aN9JIBHflIAOkqR9UkgMAhQFHp4HSvWP4PZhZfQM205lmcm4jadhTi6uAtTHliMUrEr+nFs2/E0bOf0N+BixYdYlz0fm05fQ/voEVig+jT1eOV95K7+O9o/tBZZPiDCIh6aiWmbxiN3+RY8smkyeATdw398bRxORJgxSbWjW7dtHNrUXBIcG0WheC2tiULZDhUxaBSwlmP72t254vkUYSciT37W1NSYvsvvIvhk27et5dZbb0W3bt20VDBG8XfPPfe0tVu2JwESCCACFIUeXszQoc9hZ1qsEoPWBzac/hCnG1+FxvdrRTjpEBEzBNMyZyLGEIeHks0jfA34ZPUq7K3rjQXp/ayfNbRughufRmF8Zio2PfkXLN4xDBtGeX4b2XBOj4uhir8ng23cSNSTXcs2pLF4KyWMuRiV7VDz8sknn+D5559X3vK9LWJxlxCS7Wz5fP3116af5r8b38mzr776ShN+8rurilwPJ4E78pHzf5IcWn5KWpju3bu7ahj2QwIkEMAEKAo9vLgRKhq3pXKq5GTjtnJ7xPfvbZeYixg0BuMtO63YgiU7LyD8oRyMus3ypfe+6xImYHyP7cqDuQYHhv4eyR4VZ3rkZUzCjv7r8P6COO9B8NORzUVYXJxtfrbutJVpm/chASvt2rWzSkO2p60JT2tnCEUMLl68GO+9957Vviwfivdtx44d+O9//6t9rqnzGvLz+++/N/1ufFdXV6fl4ZOzeMaf8rv5d3neUq4+y/Ed/S4RvZGRkRBPn/yMiorSft5yyy2mn/KMQR+OkmV9EiABSwIUhZZEvPr9PMpKLzRaEIvkQaFOWmNA8frNOH3tZowb2dcuYenkQE40i8aocX2R9/whrNt5HsmTPOktNED9G6/+4b/qhN1sYk7AmjiT93I+zdKDZ4uceWoba3Ukn+Err7xi7ZX27OOPP8YjjzyCv/3tbzbrWHuxaNEia489+uyGG27QBJ3xI3f/mn83/n7bbT70X3QeJcTBSIAEvEGAotAb1G2NeeUESioaX/YYhkHOJvW7cghbD/4HiJqAUQk29qlt2eCB5xFDhyF++QmUqujrU5NaOjPpAWM4hEsJyBapq4rx/KJlf7ItKsLz6tWrDgtCy76c/S6BGCLs5IYO+SnfO3TooEVD33TTTdpHfjd+xNtn/F3e/ehHP3J2aLYjARIgAbcRoCh0G1rHOzaUf4iyxsjj6IdUEIrjXWgtrhzZhRLlEQsfen8LQSpOdu6KZhGJSFbBn6Xlh1B0SolC25E0rhiNfXiQwJYtW0zX31kbVoJVjNvL4glsKXl1WlqatS5MnkjJk/jnP/8ZkkD7rbfeslrX2sNf/epXuOuuu/DDH/6wxY9kAzCKPuNPJme2RpTPSIAEAoUARaHPrKQBZQeON54n7IrkgdFOWmZA6d6TKrGNOpM4KNbHto6NU4pCQv+u6nqLsyg5UomZPZ2dq5OI2MwpAnJfsbHI/cXWIpDlmbXnxnaSZ9AoCiXvYEt1WzNSPG+yXb1x40acOXMGL730EgoKClprhiFDhmDkyJGt1mMFEiABEgg2AhSFPrPiehQdUVu+UqISMdxp79lJHCiTM3PdkRDX2pnEOhRljcKSUpULsb26DUV2miVXopYo0QCDQXIhxiLr7XyMN9vKPrZkFDLfNVxvoxndWD9qMgrsSDcTEx+tZOtZnC45jotTo72TVFuz29XFe0xdPRPL/swDP8wjkS3rufO7iFFrRaJr8/PzMW/ePKxcuVLzINoqcn0bRaEtOnxOAiQQzAQoCn1l9cv2oeR8gzFR6syd05qw4iROXVL9hEYjxo4zibrwWxEeLiLwGi5VXmiaUDv0ZkRHqevzLBgp/QhD3X+a1G2v6kaoc1WhoR3sIqrr0Vul2zmkrtk6gVOGMUjyvaOPds3DWiVvMbVmiyufyVk+Y2kpwtiVY1r2JdvPxiI5Cy2LpF9ZtWoVcnJykJubi3Xr1uG7775rUu3o0aOWzfidBEiABEhAEfg/pOAbBE4d+BANmvBmDBpqO91Ha9YaKvTQYlWibOdCvN5HKJIXFOCdHdvUZysKZvdtKgANUXhk8UIMtxCXfebsQOHs3g03rYTfhzmv/w3Hi/fi3aIdeGftmFZyKzaOHtEVUXJVS10lTjt+CUNrGLz43otM3TxrOf8n5/ikSDoZb3gLza/Za2nrWaJ3X3zxRch9xxLlbL71LdHKV65ccTMtdk8CJEAC/keAotAn1kxtHR80Jga+hmP5zyB9eiufJe/B2h0HF89fUD4/tQssHkCH5qZDzKQVWD5WnfUzlmsnkZuxGAcsRNuVYyvxzGp1brF9b2TlrsBj8Za+RDsG1oXjNk1sXkBFpR31/bKKh5l6gJH5DSL2nN9zpUkiQo0eSvEIWstjaDmenDucP38+Pv/8c+2WE0nwLMWRwBTLPvmdBEiABAKVAEWhL6xs2S7srTQachWVpX9XARgtffRAvPIEWrH9YkWDv1EXEQrH80KHImnBCiwYaLYFfGkfZmf8EZ8YGgYznNqMzJlbUImuGJe7GtOcvj4vHGpnWpWr6r5Wy1ugrUzMbx95kqn7IZlHBOfl5Wk3d3iqSJCKsUiQiiNFUsZkZWVB7kNevnw5tm7d6khz1iUBEiCBoCDAM4VeX2YDju14r3HruCsemj0ZtnJWX1NX4B24eCtGTZ2sAlGse+fq6hoSM9t7tq/59KMxftkKVDyWiU2VDflxrp3eiMzsaLW93B7LMlahtO5mDF28GgttGdq8UytPdAhtPEcot0aoQ5BW6gTKI08xdT8v2UKWVDJy/7DkEZSt2ZYSTLvKIvESGu88lttOMjIynOpabv2YPn269qmvr+ctIE5RZCMSIIFAJUBR6O2VNZzADkk0rUr7/pORPSm1hUjc1ObX2TWxX93Y0ZjnUKKJnS4d+yE7L0cJw8UokaAVVS4dXIyRJRKY3AHxs1/GijbfW9weOol0VpvdBi3a2bliOFeOssqrKvbZjmLQ47waynD+OA6oucjorZb2HRATH2fzrupW2xsreISp3da0qeLcuXMhZ/vkXJ4INTnbZ6/nTuquWbNGO+PX0lV55gaKN3LixImmRzNmzNASQbe18Fq4thJkexIggUAjQFHo5RW9UrIdBzTh1QHJafe3IAjtMVQUj1Fg2SV5bHd6WyqW5+oxddoWdV1eQzXRbqEDc7B6kivyHyr7GnWr4YqzovA8ts6chuWnbU/D6ptS5flUH3tLj8ytKFRpc9pc3M60zRba1YF4C0WYSSCHFIn0lfN99pzxE1F47py107C2h5b+JbBFipwldNZLaHsEviEBEiABEhACFIVe/Tu4hL0FHzakdolKxaTktm6hKu+brsH7Jh64tpaO8bOweskFpGUfQqPDEHWlr2LdsURkJ7TVVhGwDRbqOjorYKPw2J/fRtJ5Oz2F18qRl7UMJT1mYn1mP/sSe+s6IComqq0oTe3dy9RlZrbakXgLjVu6so388MMPY8+ePXYJw1Y7N6sg29PGgBbZNpYbU1zhJXTEBtYlARIggWAhQFHozZWu2IzXS0W8qdtH0iagT5tt0Wm5p6Vcu2bXhmorI15CRYneJAgbOj6LTTPnocdfX8Yj1iJdWunR/LVx27hhG9mBhuZVO0Yhxu6IGoPmidVFRKNPz1gnB2xrM/cybat1jrSX6+UkmbQkgzYKw7Vr18I8QtmR/izrpqenN7mhRMazxxtp2Q+/kwAJkAAJ2EeA0cf2cXJDLRVgsmmXiuJVJTQRk0a6xhtlDDCpu9gQcOK84QZ8kjsbmTsvoH2PERg38ObrXdX9HUsyluJYm1K91UGLL1GCOFRFSgdHcTdTz1IUj11RUZEpd6EIQwlCEe9eW6KSJUH1wIEDmwhCEZv2nlv0LAWORgIkQAKBQ4Ci0FtreXEf1u1sCDCJGvk4htvt7WrZ4IjGrc66uktoi2Y7t0MJwg0qF2HUMKzIew4LVT7CqT2ub/Neq9yuIpLfbEiU3bJJNt5ewnlNFKrt2ajgEIXuZ2oDtRsfG4WhiEFjkVQ1kuRazhw6kuBavI7iHZStaOMZQtkypiB04wKyaxIgARIwI8DtYy/9OVTs3IwSbee4L6ZPjXOZFRExt6rkLidRd75S2/Z1RmteLFmEqQv/jkuhfbEgbyGStSTTcZipfj8/fj72Nh4wrDuyDM8siULBgkTHxzFcwjktKbba/nUsy7bLWHmyI48w9eSEzMYSYShpaSQAxRh8Il5DySsoH4kyli1luSZP6ly9ehUdOjTkwhQhKB/xDlZVVTWZgQSVyBlCbhl7aWE5LAmQQNARoCh025LX4dTe7dha0h6j5k9EH/MMMYYPsW7TGW3kqJG/anaNXFtM0kXLncL7UHb+LCrUscIYBzPTGMr+iPSs3Tjfvjum5q3AePMOIh7E83kXcP6xNShrjGOpLJyHzJj1yJ8UbV/ghnFyyj5JD4PwrujZxrOJbeHlibYeY+qJybQwhgSfSHJrEYLGnIJSXbx+Rs+fefP//e9/aNeuXbMexTso0c3SHwsJkAAJkIDnCHD72B2srxzHqkmjMC57DbbtXIMla8ub5NE7t+1VFIm3LfR+ZE23MwrWXjtvi0UP8bxdq0Tj5Sb2tsSVsr8gPWOjSkHTAUOXvIyZVq6v0/WcjNXLhin/nrFcRemKTCwpcexWEkPFSW3ruX2Pfuhht4X+V9GTTH2BjngCxWsowSeSINp4V7I12ywFYVJSEpYuXYpPP/2UgtAaMD4jARIgATcToKfQHYArD2FvuTHQ4xpOb8jE1LocLJh+P3q2P4Hc9eqsHm7FQ4tzXOolbJhKbyT174Bt715AcVkdpsW0dF5Pj9enL8WOSyro46LyAF5qdP+pq+cOPj8V6XWrkT8uuimhc7uwbH0lDMbMN9rbC9iRNQ6nekRp3sLQ5BzkT205uvdUqV5LmhM/qK/jW89NLfKxb95j6ksgRBzKdXJS5FyhbBGLt1C2iY33F8s5RKknuQtli5mpZnxpBWkLCZBAMBKgKHTHqserm0lGHtIidxvKVZQVzse4QuUZU2Lqmq43Jqmt2exB7jhMp0P/5H4IffcQTpechGFUYgvbunWoOK1HRV1DfsPwcHXOS2Wolv8xyJm/c0aRaAaprhKnTp9RuRVV1HComoyakBZ+otqdr9SrdtcQGtOa17ASpaXCpjuSB133ObpjKTzfp7eYen6m9o4owk8+cq5QtpaNolCiiUUQspAACZAACfgGAYpCt6xDOJKXrMfamKXKq3YIlUaNFNoVCSMnY3p6Kvo4EwFip60dB45A/9BDOFj6Ho4ZEpFk81xhPyw8UIyFdvarVev5NN459rQjLZrXPafucJZbSOJS8VBM89f+/cRLTP0bGq0nARIgARLwAQIUhW5bhHAkTV2hPnW4WHEBde3DEXVbeAteOxca0jERj/2/m3GwUG1jlxqQNMimKnThoPZ3de7gLpQp/+KgcQ8iwGNM7IfCmn5A4CgWDpyI9XoDOqdtwJHVwxDiB1bTRBIgARKwlwADTewl5XS9UETExCLGU4JQs1OHhKmTEd/+Kg5sO9SmfIVOT9tmQz22blOR1+pav+kPuWP73ObA6kWo2vJW/zu0IR1KSzX5jgSaE6iBXl+DWkMtzuirUd+8Ap+QAAmQgF8ToCj06+VrwfjbHsWccV1Rd/Av2CZhvj5Srhx4Fdsqb8ZDs2c0TdPjEfuiMfP1ImzLdF1eSI+YzUFIgARIgARIwAMEKAo9ANk7Q+jQJ30WRkWdwetr3/cRb6Ee2/IPwdD/V8hKbikq2o3EOoajo2/tprtxsuyaBEiABEiABOwnQFFoPyv/q6nOFmbNHga8uwrLHcwj6I7JVmxairyK3siaP4ZnCd0BmH2SAAmQAAmQQBsIUBS2AZ4/NI1IXojVUzqg6PmVKG7LZchtnWzFZszPP4/kZSvwWMBFHLcVDtuTAAmQAAmQgPcJUBR6fw3cbIHaRs5aged7fIg5MzdrV995vFz5EPOzXoVhrLIj2dPBJR6fLQckARIgARIgAb8kQFHol8vmqNFRGL4sH3NCt+CZBe/joqPN21LfoMe6jKWoGLoC67PiPJOSpy32sq2fEKjH0TVPYPTw0XhiXZkWCVx/dhcWThiIbrffjtu79cXw7H2otjaby2UoXJmJCcl90avb7Yi4vRt69U3GhMw12KW/bK2Ffc/0m5A+ejiGj34Ca462EJus7Mye0FBv5eE2jGefVaxFAiRAAnYTYJ5Cu1H5eUVdNB7J3YSYHR/iovIWRngq2OLiJYRPX4v16uYSTw3p5ytF8+0iUIvyXW9if7GqHDIGyxIO44mHc1Bca2xci+LCXdCre7ojzfqr3peNiVPycMxUr+FlVW0Nqs4cw+71K5G0aDM2zxqATnbZYVZJvwuFyiDNGZ+6DDMG2MhiWKXq7S5GjapWm7IIswY7PJKjlrE+CZAACdhFgKLQLkyBUikUfUY96NnJ3JaIR5ih2rPMg2206l1In7i7QRB2jkPK4Hjoqo+iuDoEYWYsLh/OxPBH10NlyQTC4jBm1iykJcUiMqQWVccKseaFAhTX1KD4uUcxpXMJ3prUNdhIcr4kQAJBToCiMMj/ADh9EvB7AuW7sV9NIixpKfZsnoF4a463+qN4Md0oCJOQe+AtTIu97smLjx+M1NRUpCc/ioKqWuzPeQ77xr6GYTacfX7PjBMgARIgASsEeKbQChQ+IgES8DMC3adisy1BqKZyef9KJfZkTjq1PZzfRBCaZho5DIueTWo45lC7HwX7WzgX6Gd4aC4JkAAJ2EOAotAeSqxDAiTgwwS6IyN/GWzxHnBuAAAFnUlEQVQfzavHscJiaMcIdQlIS7W9LRw5YBhitZnWouyo3ofnTNNIgARIwPUEuH3seqbskQSCnkBlZSWio6Od4vDpp5+iV69e9rfVxWOwraAOrRc9Dpc1RpaEhKC2bB/22eq99qzpTU2N1dhlWy35nARIgAT8ngBFod8vISdAAr5H4Le//S0iIyMxYsQIDBo0CGFh5iEfze2trq7GwYMHUVhYiHvuuccxUdi8O4sntVDxIw1FbQvnPConEFsv9dw9bh0Sa5AACQQUAYrCgFpOToYEfIPAnDlz8MADD2Djxo2aQXFxcRg8eDDKyspMBm7atAnyKSkpwdmzDR66G264Aa+++qqLJ1Gv5THUiq4L4hK6NIlKbj5YCEJCIpEyLaH5Kz4hARIggQAmQFEYwIvLqZGAtwgMGDAAQ4YMwfvvv6+ZUF5ern3MyxtvvNHMvIyMDNx0003NnrftQRjCjEkyw1KwrGg1Bretw1Zbe+PioFaNYgUSIAESaIUAA01aAcTXJEACzhGYP3++Qw1D1Hm/zMxMh9rYVzkSscbYktoqnHXXJSK6TjBmsKmvtciObZ+hrEUCJEACXiVAUehV/BycBAKXwL333oukpCS7J5ieno5OnawlGbS7CxsVuyI+rnPDO8MxdQuKmw4LqnOTxpOTVWVnr29ZN7HqMg4XHtVuM2EhARIgAV8jQFHoaytCe0gggAjk5OTYNRv3eQkbhh+cNhZdtF9r8ebC59DS1cR2GWytUmwcYhu3qWv3F2B/M4+kEoQLH8XE9dqdKiwkQAIk4HMEKAp9bkloEAkEDoH7778fcr6wtTJt2jRERES0Vs359wNmYdGIRm/hmTw8mpyOdYctvXmXcfaouu4ufbiKfk7HPkcdip1SkTa40VdY+ybSH12IXfpq1Ksw5uqyXVg4ehBGrzqGWl0Y7wF3fiXZkgRIwI0EKArdCJddkwAJAM8++2yrGLKyslqt07YKkRib/xpmJjSIttryAmQ93Bs/jrgd3bp1U5/bERF6B3o/MAU5BcWoqr4Mx4NFOmHssmfROARqj63CxH534cc//jHuGjQRq/ZXwRCWhEV7liLFGPjStkmxNQmQAAm4lABFoUtxsjMSIAFLAikpKYiPj7d8bPr+1FNP4ZZbbrH53voLHXQhjcpKBajYpbE6DcbiPSXYvmgM4jo3tjBIDsMa9altEIG6zohLScPSzcuQ2uzeY50pkESnxrRaYmdg+/ZcjOlumZdRhy4pM7H9eBFmDYg09WOjF6td8yEJkAAJuJtAu/+p4u5B2D8JkEBwEygqKsL48eOtQjhz5owTolC6Utuy1bXQhUWik8Pqqh6X9XqUn61Gba3yCaogkS5dYxEbe12wWTW2/jKqVf0wlZi75SGVbfoylOlrYFDbxV3iExAfadZC60cNG3k9YtnqeHxIAiRAAh4kQFHoQdgcigSCmUBCQgJOnTrVBIGcJczNzQ1mLJw7CZAACfgMAW4f+8xS0BASCGwCCxYsaDbB3/zmN82e8QEJkAAJkIB3CFAUeoc7RyWBoCMwcuRI9OzZ0zTvyZMn44477gg6DpwwCZAACfgqAYpCX10Z2kUCAUhg7ty5plllZ2cH4Aw5JRIgARLwXwK8+9h/146Wk4DfERg9ejTuvPNOJCYm0kvod6tHg0mABAKdAANNAn2FOT8S8DECW7duRb9+/bT8gCwkQAIkQAK+Q4Ci0HfWgpaQAAmQAAmQAAmQgNcI8Eyh19BzYBIgARIgARIgARLwHQIUhb6zFrSEBEiABEiABEiABLxGgKLQa+g5MAmQAAmQAAmQAAn4DgGKQt9ZC1pCAiRAAiRAAiRAAl4jQFHoNfQcmARIgARIgARIgAR8hwBFoe+sBS0hARIgARIgARIgAa8RoCj0GnoOTAIkQAIkQAIkQAK+Q+D/A4OLNYRrQm3fAAAAAElFTkSuQmCC)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercice\n",
    "\n",
    "- Implémentez cette connexion sous une forme de couche dans la classe `ResidualLayer` ci-dessous. Vous pouvez la tester avec le réseau `ShallowResnet` fourni ci-bas. Dans notre cas, chaque *weight layer* consiste en une couche à convolution à 100 filtres 3x3 avec padding de 1 suivi d'une batch normalization. Prenez pour acquis que le nombre de *canaux* en entrée de votre couche sera de 100."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ResidualLayer(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        pass\n",
    "    \n",
    "    def forward(self, x):\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Testez votre implémentation de votre couche résiduelle avec le réseau peu profond ci-dessous."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ShallowResnet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        \n",
    "        self.conv1 = nn.Conv2d(3, 100, 3, padding=1)\n",
    "        self.bn1 = nn.BatchNorm2d(100)\n",
    "        \n",
    "        self.res1 = ResidualLayer()\n",
    "        self.res2 = ResidualLayer()\n",
    "        self.res3 = ResidualLayer()\n",
    "\n",
    "        self.fc1 = nn.Linear(100 * 8 * 8, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.bn1(self.conv1(x)))\n",
    "        x = self.res1(x)\n",
    "        x = F.max_pool2d(x, 2)\n",
    "        x = self.res2(x)\n",
    "        x = F.max_pool2d(x, 2)\n",
    "        x = self.res3(x)\n",
    "        x = x.flatten(1)\n",
    "        x = self.fc1(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Il n'est pas nécessaire d'effectuer l'entraînement au complet, juste de confirmer que votre implémentation a des \"bonnes\" performances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = ShallowResnet()\n",
    "model.cuda()\n",
    "\n",
    "optimizer = optim.Adam(model.parameters(), lr=lr)\n",
    "history = train(model, optimizer, cifar_train, n_epoch, batch_size)\n",
    "history.display()\n",
    "print('Précision en test: {:.2f}'.format(test(model, cifar_test, batch_size)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercice\n",
    "\n",
    "- Codez le réseau à connexion résiduelle ci-dessous qui prend en paramètre le nombre de couches résiduelles désiré. Encore une fois, vous allez devoir utiliser la classe [`nn.ModuleList`](https://pytorch.org/docs/stable/generated/torch.nn.ModuleList.html#torch.nn.ModuleList) ou [`nn.Sequential`](https://pytorch.org/docs/stable/generated/torch.nn.Sequential.html#torch.nn.Sequential) pour pouvoir avoir un nombre variable de couches. Notez qu'ici `num_layers` représente le nombre de couches résiduelles et non le nombre de couches à convolution. Le réseau devrait consister d'une couche `Conv2d -> BatchNorm2d -> ReLU` suivi de `num_layers` couches résiduelles et terminer par la couche de classification. Faites un maxpool après les couches résiduelles 1 et 2. \n",
    "\n",
    "    Entrainez le réseau avec 25 couches résiduelles (donc au total 51 couches à convolution si compte la première couche non résiduelle)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Resnet(nn.Module):\n",
    "    def __init__(self, num_layers):\n",
    "        super(Resnet, self).__init__()\n",
    "        pass\n",
    "\n",
    "    def forward(self, x):\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Resnet(25)\n",
    "model.cuda()\n",
    "\n",
    "optimizer = optim.Adam(model.parameters(), lr=lr)\n",
    "history = train(model, optimizer, cifar_train, n_epoch, batch_size)\n",
    "history.display()\n",
    "print('Précision en test: {:.2f}'.format(test(model, cifar_test, batch_size)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Question\n",
    "\n",
    "- Comparez les résultats et la vitesse avec laquelle le réseau a entraîné par rapport au réseau à convolution de 51 couches entrainé plus haut."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercice facultatif\n",
    "\n",
    "- Comme le défi lancé précédemment, tentez d'implémenter le réseau à connexion résiduelle en n'utilisant que l'interface `nn.Sequential`. Vous pouvez utiliser la couche `ResidualLayer` définie précédemment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_sequential_resnet(num_layers):\n",
    "    assert num_layers > 2\n",
    "    layers = []\n",
    "    \n",
    "    # TODO Ajoutez les couches à la liste \"layers\" ici.\n",
    "\n",
    "    return nn.Sequential(*layers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Il n'est pas nécessaire d'effectuer l'entraînement au complet, juste de confirmer que votre implémentation a des performances similaires à votre implémentation précédente."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = create_sequential_resnet(25)\n",
    "model.cuda()\n",
    "\n",
    "optimizer = optim.Adam(model.parameters(), lr=lr)\n",
    "history = train(model, optimizer, cifar_train, n_epoch, batch_size)\n",
    "history.display()\n",
    "print('Précision en test: {:.2f}'.format(test(model, cifar_test, batch_size)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}