2-neural_network.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Artificial neural network\n",
    "\n",
    "Inspired by biological neural networks, perceptron was created in the late 1960s. Since then, a generalized form of neural network has gain popularity, and new structures were created for different proposes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The core idea is to use an artificial neural network to approximate a function rather than creating a function through experiments and/or deriving it.\n",
    "\n",
    "Say we want to model kinetic energy. We might carry out a series of physical experiments and figure out the relationship between mass, speed, kinetic energy, and arrive at a function like:\n",
    "\n",
    "$$ \\text{kinetic energy} = \\frac{1}{2} \\text{mass} \\cdot \\text{speed}^2 $$\n",
    "\n",
    "Say we want to model aerodynamics of an airplane, we might derive a function with known physical laws.\n",
    "\n",
    "However, there might be somethings we want to model that is too complex to derive this way.\n",
    "\n",
    "One approach to model such problems is artificial neural networks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A function takes in an input and produce an output.\n",
    "\n",
    "$f(x) = y$, where $f(\\cdot)$ is the function, $x$ is the input, $y$ is the output.\n",
    "\n",
    "A neural network is no different, it takes in an input and produces an output as well. However, rather than a formula like in the kinetic energy example, neural networks calculate the output with a network of neurons."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is inspired by biological neural network which is a network formed by neurons. With each neuron taking in impulses from some neurons and decide whether to fire off an impulse to other neurons.\n",
    "\n",
    "![bio_neuron.png](img/bio_neuron.png)\n",
    "\n",
    "Retrieved from: https://simplebiologyy.blogspot.com/2014/08/conduction-of-nerve-impulse.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is an example artificial neural network with input of size 2 and output of size 2. Meaning there are two input scalars and two output scalars. A scalar is just a number, e.g., $1.3$, or $\\text{-}3.2$.\n",
    "\n",
    "Usually, the input and output of a neural network is an array of scalars (vector).\n",
    "\n",
    "![small_neural_network.png](img/3_layer_neural_network.png)\n",
    "\n",
    "This is type of neural network is a fully connected feedforward network.\n",
    "\n",
    "The grey cell represents the input values.\n",
    "\n",
    "Each of the white cells (\"hidden 1\", \"output 1\", etc.) are a neuron.\n",
    "\n",
    "These neurons are organized into layers. By convention, the layer between input layer and output layer is called a hidden layer. There can be multiple hidden layers in a network. \n",
    "\n",
    "The output of each cell is fed to the connected cell as their input."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To understand how the network as a whole, we need to know how the individual neurons work."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neuron\n",
    "\n",
    "Let us start with a simplified neuron. This neuron takes the inputs and compute a weighted sum as the output.\n",
    "\n",
    "![simple_neuron.png](img/simple_neuron.png)\n",
    "\n",
    "In this case, there are three input connections. \n",
    "\n",
    "The three input values are: $0.5$, $3.1$, $\\text{-}1.3$.\n",
    "\n",
    "The weights for each of the inputs are: $\\text{-}1.4$, $0.2$, $2.3$.\n",
    "\n",
    "The output of the neuron is $\\text{-}3.07$.\n",
    "\n",
    "The output is the sum of each input multiplied by its weight.\n",
    "\n",
    "$0.5 \\times \\text{-}1.4 + 3.1 \\times 0.2 + \\text{-}1.3 \\times 2.3 = \\text{-}3.07$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Complex systems tend to be [nonlinear](https://en.wikipedia.org/wiki/Nonlinear_system) in nature. For the neural network to model nonlinear system, some form of nonlinearity needs to be included.\n",
    "\n",
    "For this reason, neuron includes an activation function. The weighted sum of the neuron is inputted into the activation function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is possible to have no activation function (or identity function as activation) for some layers. It is just some nonlinearity needs to be present in the network.\n",
    "\n",
    "Identity function: $f(x) = x$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a variety of [activation functions](https://towardsdatascience.com/activation-functions-neural-networks-1cbd9f8d91d6). A highly popular and useful one is the ReLU.\n",
    "\n",
    "However, we will be using [sigmoid](https://en.wikipedia.org/wiki/Sigmoid_function) for now.\n",
    "\n",
    "$$\\sigma(x) = \\frac{1}{1+e^{\\text{-}x}}$$\n",
    " \n",
    "where $\\sigma(\\cdot)$ is the sigmoid function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot of sigmoid:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de3zcdZ3v8ddnJrfe0lvSe0pbWqDl2hKgisvFQmmrW8ALFnVV9IjuLqse1z0H132wPtA9D8XH8Rw9i6uoKF6goC7ahVKoXASVQlt7T1saekvSNklvSdNcZ+Zz/pgpDGHSTtJJfjOT9/PxmMf8Lt+Z+eQ3v7zzy/d3M3dHRERyXyjoAkREJDMU6CIieUKBLiKSJxToIiJ5QoEuIpInCoL64LKyMp82bVpQHy8ikpPWr19/2N3LU80LLNCnTZvGunXrgvp4EZGcZGb7epqnLhcRkTyhQBcRyRMKdBGRPKFAFxHJEwp0EZE8ccZAN7MHzazBzLb2MN/M7LtmVm1mm81sXubLFBGRM0lnC/2nwKLTzF8MzEo87gT+4+zLEhGR3jrjceju/qKZTTtNk5uBn3n8OrxrzGyUmU1094MZqlFE8pC7E4k5HZEYnZEYHZEoXRGnMxqlM+JEYjG6ok4kGiMac7piTjQWIxrjzWd3YjEn5k405rhDzJ1Y4tnfMhx/jn92YlpiGCA+9ub4qRrfnP/2tt3bv+Xne+sP+5Z5C2aP59KKUX1bcKeRiROLJgM1SeO1iWlvC3Qzu5P4VjxTp07NwEeLSFAi0RhHTnZyuKWDoyc7OXqyk2MnO2lqi9DU1sWJ9i5aOiKcaI/Q0hGhrTNKa1f8ua0zSnskHtSDhdmbw+NKS7I20C3FtJTfkrs/ADwAUFlZOXi+SZEcFI05B463sfvwSfY0tlBzrI26Y23UHW/jUHM7R1o66CmPhxWFGTmkkOElBQwvLmBESQHjS4sZVlRASVGYIYXxR3FBiOLCEMUFYQrDIYoK4o/CkFEYDlEQTjyHjIKwEQ6FCJsRDp16QMiMUGJaKGQYEA4ZZmDEpxvxQDU7NT3+ulNtrFuKnZr+5vCp6ZY0nNw+VQwOvEwEei1QkTQ+BTiQgfcVkQHSEYmyta6ZTTXH2X6wmR2HTvBa/Qk6IrE32pQUhpg8agiTRw9lzsRSxpcWU15aQvnwIsYMK2bMsCJGDy2kdEghhWEdQBeETAT6CuAuM1sOXAU0qf9cJLt1RKKs33eMP+46zMu7j7CtrpnOaDy8y4YXM3viCP5m/jnMHDec6WXDmF4+jPLhxVmzJSqpnTHQzewR4DqgzMxqgX8FCgHc/fvASmAJUA20Anf0V7Ei0nfN7V08t72BlVsO8uKuRtq7YoRDxmUVo7jj6mnMnTqaeVNHMa60JOhSpY/SOcrl9jPMd+DvM1aRiGRMLOa8VH2YR9fu5/dVDXRGY0woLeG2ygqumVXOVTPGMKKkMOgyJUMCu3yuiPSflo4Iv1yzj5+9vI+6422MHlrIR+efw3sumcjcilGEQuo6yUcKdJE8cry1kwf/tJeH/ryXprYu5s8Yw92LL2DhheMpLggHXZ70MwW6SB7oisb4xZp9/N/f76KprYuFc8bzd9fP5LJ+ONZZspcCXSTHvbSrkX9dsY3djSd518wyvvKe2cyeWBp0WRIABbpIjmrtjPC/Vm7nF2v2M71sGD/+eCXvvmCcDi0cxBToIjlo/b5jfPGxjew/2sp/e9d0vnTT+ZQUqo98sFOgi+SYR17dzz2/28r40hIe+fR85s8YG3RJkiUU6CI5oisa4+tPVPHQy/u45rxy/t/tcxk5RMeQy5sU6CI5oLUzwmd+vp6Xdh3m0381nbsXzyasY8mlGwW6SJZr6YjwyZ+sZd2+o9z3/ku47YqKM79IBiUFukgWa2rr4hM/eZXNtU189/a5vPeSSUGXJFlMgS6SpU52RPjYj1+h6mAz3/vIPG66cELQJUmWU6CLZKFINMY/PLKBLXVN/OBvKrlxzvigS5IcoEAXyTLuzr+u2MZzOxr4t1svUphL2nRbEZEs84MXd/PLV/bz2WvP5SNXnRN0OZJDFOgiWeSlXY18c9UO3nvJRP7HTecHXY7kGAW6SJZoaG7nvz+6kZnlw/nWBy7VNcul19SHLpIFojHn88s30tIR4eFPz2dIka7LIr2nQBfJAv/+XDUv7z7Cfe+/hPPGjwi6HMlR6nIRCdjGmuN859nXuOWySXywckrQ5UgOU6CLBKgrGuPu32ymfEQx995yka5lLmdFXS4iAXrgxd3sOHSCB/7mckpLdOVEOTvaQhcJyO7GFr7z7C6WXDyBhTqtXzJAgS4SAHfny/+5hZKCEF9demHQ5UieUKCLBGDFpgO8suco/7xkNuNGlARdjuQJBbrIAGvvinLfqp3MmVjKbZW6trlkjgJdZID99M97qTvexr+8Z7bOBpWMUqCLDKAjLR3c/1w1Cy4YxztnlgVdjuQZBbrIAPrus7to7Yry5SUXBF2K5CEFusgA2XP4JL98ZT/Lrqhg5jid3i+Zp0AXGSD3P19NOGR8/oZZQZcieSqtQDezRWa208yqzezuFPOnmtnzZrbBzDab2ZLMlyqSu2qOtvL4hjo+fNVUHaYo/eaMgW5mYeB+YDEwB7jdzOZ0a/YvwGPuPhdYBnwv04WK5LLvvVBN2IzPXHNu0KVIHktnC/1KoNrdd7t7J7AcuLlbGwdKE8MjgQOZK1Ekt9Udb+PX62u57YopTBiprXPpP+kE+mSgJmm8NjEt2VeBj5pZLbAS+IdUb2Rmd5rZOjNb19jY2IdyRXLPD/7wOu7w2Wu1dS79K51AT3Xmg3cbvx34qbtPAZYAPzezt723uz/g7pXuXlleXt77akVyTENzO8vX1vCBy6cwZfTQoMuRPJdOoNcCyecnT+HtXSqfAh4DcPeXgRJAZ03IoPfQy3vpisb42+u0dS79L51AXwvMMrPpZlZEfKfnim5t9gMLAMxsNvFAV5+KDGrtXVEefmU/N84ezzljhwVdjgwCZwx0d48AdwFPA9uJH82yzczuNbOliWb/CHzazDYBjwCfcPfu3TIig8pvN9RxrLWLO66eHnQpMkikdccid19JfGdn8rR7koargKszW5pI7nJ3HvzTHmZPLGX+jDFBlyODhM4UFekHf379CK/Vt3DH1dN0n1AZMAp0kX7w4B/3MHZYEUsvnRR0KTKIKNBFMmzv4ZM8t7OBj1w1lZLCcNDlyCCiQBfJsIdf3U/YjI/OPyfoUmSQUaCLZFBnJMZv1teyYPY4xpXqNH8ZWAp0kQxaXVXPkZOdLLtyatClyCCkQBfJoOVr9zN51BCumaVLW8jAU6CLZEjN0VZe2nWYD1ZOIaybP0sAFOgiGfLYuhrM4LbKijM3FukHCnSRDIhEYzy2roZrzytn0qghQZcjg5QCXSQD/vBaI/XNHSy7QjtDJTgKdJEM+M1fahk7rIgFs8cFXYoMYgp0kbPU1NbF77c38NeXTqIwrF8pCY7WPpGz9NSWg3RGYtw6t/udGUUGlgJd5Cw9vqGOGWXDuGTKyKBLkUFOgS5yFmqPtfLKnqPcOneyLpMrgVOgi5yF322M3173FnW3SBZQoIv0kbvz+IY6rpg2mooxQ4MuR0SBLtJX2w40U93Qoq1zyRoKdJE++u2GOgrDxnsunhh0KSKAAl2kT2Ix58ktB7lmVjmjhhYFXY4IoEAX6ZMNNcc42NTOey/V1rlkDwW6SB88sfkgRQUhbpg9PuhSRN6gQBfppVjMWbnlINeeV86IksKgyxF5gwJdpJfW7TtGfXMH771E3S2SXRToIr305OYDFBeEWKDuFskyCnSRXojGnJVbD/HuC8YxvLgg6HJE3kKBLtILr+45SuOJDt6j7hbJQgp0kV54cssBSgpDvPsC3chCso8CXSRNsZjz9LZ6rj9/HEOL1N0i2SetQDezRWa208yqzezuHtrcZmZVZrbNzB7ObJkiwdtQc4zGEx0sumhC0KWIpHTGzQwzCwP3AzcCtcBaM1vh7lVJbWYBXwaudvdjZqb/RyXvrNp6iKKwulske6WzhX4lUO3uu929E1gO3NytzaeB+939GIC7N2S2TJFguTurth3i6pljdTKRZK10An0yUJM0XpuYluw84Dwz+5OZrTGzRaneyMzuNLN1ZrausbGxbxWLBKDqYDM1R9vU3SJZLZ1AT3VfLe82XgDMAq4Dbgd+ZGaj3vYi9wfcvdLdK8vLy3tbq0hgVm09RMjQtVskq6UT6LVARdL4FOBAija/c/cud98D7CQe8CJ5YdXWQ1w1fSxjhxcHXYpIj9IJ9LXALDObbmZFwDJgRbc2vwWuBzCzMuJdMLszWahIUKobWtjV0KLuFsl6Zwx0d48AdwFPA9uBx9x9m5nda2ZLE82eBo6YWRXwPPBP7n6kv4oWGUhPbzsEwMIL1d0i2S2tsyPcfSWwstu0e5KGHfhi4iGSV57ZdohLK0YxceSQoEsROS2dKSpyGoea2tlU28RN2jqXHKBAFzmN1dvrAVg4R4Eu2U+BLnIaz2w7xIyyYZxbPjzoUkTOSIEu0oPm9i7W7D7CjXPGY5bqdAyR7KJAF+nBCzsb6Yq6jm6RnKFAF+nBM9sOUTa8mMsqRgddikhaFOgiKXREoryws5EbZo8jHFJ3i+QGBbpICmt2H6WlI6LuFskpCnSRFJ7ZdoihRWHeeW5Z0KWIpE2BLtJNLOasrqrn2vPKKSkMB12OSNoU6CLdbKlrouFEBzfqZCLJMQp0kW5WV9UTDpluNSc5R4Eu0s3qqnqumDaaUUOLgi5FpFcU6CJJ9h9pZWf9CW6co2ufS+5RoIskeaYqce1z9Z9LDlKgiyRZXVXPBRNGUDFmaNCliPSaAl0k4djJTtbuPaqjWyRnKdBFEp7b0UDMUaBLzlKgiySsrqpnQmkJF08eGXQpIn2iQBcB2ruivLirkRvmjNO1zyVnKdBFgD+/fpjWzigLdbii5DAFugjwzLZ6RhQXMH/G2KBLEekzBboMetGY8/vt9Vx3wTiKCvQrIblLa68Mehv2H+NwS6dOJpKcp0CXQW91VT2FYeO688uDLkXkrCjQZVBzd57edoh3nFvGiJLCoMsROSsKdBnUqhta2HukVd0tkhcU6DKoPVNVD+jsUMkPCnQZ1J6pqufSilGMLy0JuhSRs6ZAl0HrwPE2NtUcV3eL5A0Fugxaz2yLX/t88UU6O1TyQ1qBbmaLzGynmVWb2d2nafcBM3Mzq8xciSL9Y9W2Q5w3fjgzyocHXYpIRpwx0M0sDNwPLAbmALeb2ZwU7UYAnwNeyXSRIpl2pKWDV/ccZdGF2jqX/JHOFvqVQLW773b3TmA5cHOKdl8D7gPaM1ifSL/4/fZ6Yg43qbtF8kg6gT4ZqEkar01Me4OZzQUq3P2J072Rmd1pZuvMbF1jY2OvixXJlFVbDzF1zFDmTCwNuhSRjEkn0FNdHNrfmGkWAv4P8I9neiN3f8DdK929srxcp1lLMJrbu/hj9WEWXTRB1z6XvJJOoNcCFUnjU4ADSeMjgIuAF8xsLzAfWKEdo5Ktnt/RQFfUuUn955Jn0gn0tcAsM5tuZkXAMmDFqZnu3uTuZe4+zd2nAWuApe6+rl8qFjlLq7YeYtyIYuZWjAq6FJGMOmOgu3sEuAt4GtgOPObu28zsXjNb2t8FimRSa2eEF3Y2ctOFEwiF1N0i+aUgnUbuvhJY2W3aPT20ve7syxLpH8/vaKStK8qSiycGXYpIxulMURlUnth8gPIRxVw5fUzQpYhknAJdBo2THRGe29HAkosmEFZ3i+QhBboMGs/uaKAjEuM9l0wKuhSRfqFAl0HjiU0HGF9aTOU5o4MuRaRfKNBlUDjR3sULrzWy5OKJOrpF8pYCXQaFZ7c30BmJ8d5LdHSL5C8FugwKT2w+wKSRJcytUHeL5C8FuuS9ptYuXnztMIvV3SJ5ToEueW/l1oN0RmPcctnkMzcWyWEKdMl7j/+ljnPLh3HRZF0qV/KbAl3yWs3RVl7de5T3zZuiS+VK3lOgS1773cY6AJZeqpOJJP8p0CVvuTuPb6jjymljqBgzNOhyRPqdAl3y1pa6Jl5vPMmt87QzVAYHBbrkrcc31FEUDrHkIp1MJIODAl3yUiQa4782HWDB7HGMHFoYdDkiA0KBLnnpuR0NHG7p5Na56m6RwUOBLnnp0bU1lI8o5voLxgVdisiAUaBL3jnY1MbzOxv44OVTKAxrFZfBQ2u75J1frasl5vChKyqCLkVkQCnQJa/EYs6ja2u4euZYzhk7LOhyRAaUAl3yykvVh6k73sayK6YGXYrIgFOgS155dO1+Rg8tZOGF44MuRWTAKdAlbzSe6GB1VT3vmzeF4oJw0OWIDDgFuuSNh1/ZT1fU+fBV6m6RwUmBLnmhMxLjF6/s47rzyzm3fHjQ5YgEQoEueeHJLQdoPNHBHVdPD7oUkcAo0CXnuTsP/nEvM8cN55pZZUGXIxIYBbrkvPX7jrGlrolPvHOa7kokg5oCXXLeT/60l5FDCnmfrnsug1xagW5mi8xsp5lVm9ndKeZ/0cyqzGyzmT1rZudkvlSRt6s73saqbYdYdmUFQ4sKgi5HJFBnDHQzCwP3A4uBOcDtZjanW7MNQKW7XwL8Grgv04WKpPKDP7xOyODj75gWdCkigUtnC/1KoNrdd7t7J7AcuDm5gbs/7+6tidE1wJTMlinydvXN7SxfW8MHLp/CpFFDgi5HJHDpBPpkoCZpvDYxrSefAp5KNcPM7jSzdWa2rrGxMf0qRVL4wR92E405f3vtzKBLEckK6QR6qsMGPGVDs48ClcC3Us139wfcvdLdK8vLy9OvUqSbwy0dPPzqPm65bDJTxw4NuhyRrJDOXqRaIPnC0lOAA90bmdkNwFeAa929IzPliaT2w5d20xmJ8ffXnxt0KSJZI50t9LXALDObbmZFwDJgRXIDM5sL/ABY6u4NmS9T5E3HTnby85f38deXTmKGTvMXecMZA93dI8BdwNPAduAxd99mZvea2dJEs28Bw4FfmdlGM1vRw9uJnLX7n6+mrSvKXder71wkWVoH7rr7SmBlt2n3JA3fkOG6RFLad+QkD728l9sur2DW+BFBlyOSVXSmqOSU+1btpCAU4osLzwu6FJGso0CXnLF+31Ge3HKQz1w7g/GlJUGXI5J1FOiSE9ydrz+5nXEjirnzmhlBlyOSlRTokhNWbDrAhv3H+dLC83XNFpEeKNAl6x1v7eRrT1RxyZSRvP9yXVVCpCfa1JGs929PbudYaxc/++RVhEO63rlIT7SFLlntj7sO86v1tXzmmhnMmVQadDkiWU2BLlmrrTPKPz++hellw/jcgllBlyOS9dTlIlnrG09tZ//RVpbfOZ+SwnDQ5YhkPW2hS1ZatfUgD728j09ePZ35M8YGXY5ITlCgS9apOdrKP/16M5dOGcndiy8IuhyRnKFAl6zSGYlx1yMbAPj3D8+jqECrqEi61IcuWcPd+doTVWyqOc5/fGQeFWN04wqR3tDmj2SNH/9xDz9fs487r5nB4osnBl2OSM5RoEtWWLnlIF9/cjtLLp7A3YvUby7SFwp0Cdy6vUf5wqMbufyc0Xz7tssI6WxQkT5RoEug1u49yid+spbJo4bww49V6nhzkbOgQJfA/Pn1w3zsx68yrrSYRz49nzHDioIuSSSnKdAlEC/sbOCOn6xlyughLL9zPhNG6oYVImdLhy3KgHJ3fvKnvXz9ySrOn1DKLz51JWOHFwddlkheUKDLgOmIRPmXx7fyq/W1LJwznm9/6DKGF2sVFMkU/TbJgHi9sYUvPrqRTbVNfO7dM/nCDefpaBaRDFOgS7+KxZyHXt7LN57awZCiMN//6DwWXaSThkT6gwJd+k3VgWa++l/beHXPUa4/v5xvvv8SxpVq56dIf1GgS8Y1nujg26t3snxtDSOHFPKN913Mh66owExdLCL9SYEuGXOoqZ0fvbSbh1/dT2ckxh3vnM7nF8xi5NDCoEsTGRQU6HJW3J0tdU38cs1+Ht9QR9SdpZdO4q53z+Tc8uFBlycyqCjQpU8aTrTz1JZDPLq2hqqDzZQUhvhg5RQ+e+25uuytSEAU6JIWd+f1xhb+8NphVm09yLp9x3CHCyeV8rVbLmLppZMYOURdKyJBUqBLSrGYs6uhhb/sP8a6vcf4U/VhDjW3A3DBhBF8fsEsFl80kfMnjAi4UhE5RYE+yLk7jS0d7Gk8yeuNJ9lxqJntB5vZfvAELR0RAEYPLeSd55Zx9cwy/mpWmbpURLJUWoFuZouA7wBh4Efu/o1u84uBnwGXA0eAD7n73syWKr0VjTnHWjs5erKTwy0dNDR3UN/czsGmduqOt1F7rI3ao62cSAQ3wPDiAi6YMIJb507msopRzDtnNNPGDtUhhyI54IyBbmZh4H7gRqAWWGtmK9y9KqnZp4Bj7j7TzJYB3wQ+1B8F5yp3JxpzoqeeE49IzIlEna5oLDEcoyMSoysaozMSozPx3BGJ0d4Vpb0rRltXlLbOCK2dUVo7o7R0RGhpj9DSEaG5vYvjrV00tXXR3N6F+9trGVYUZsrooUwePYQrpo1metkwZpQPZ0bZMKaMHqLwFslR6WyhXwlUu/tuADNbDtwMJAf6zcBXE8O/Bv7dzMw9VZycncfW1vDAS7vfGO/pI7yHkVOD7p40DKfG3HlLCKZqF3ujTXw45o53e465E4vFh6OJ6ZlWEDKGFIUZUVzA8JIChhcXMGZYEdPLhjFySCGjhhYxdlgRY4YVMXZ4EeNLSxhfWqILYonkqXR+sycDNUnjtcBVPbVx94iZNQFjgcPJjczsTuBOgKlTp/ap4NHDijh/fLcdcT1sUCZPTt7qtDemJQ/bm+0NTo2danPq5YYRCiWGDMJmb7QJhYxQ4n3CIcPMCFl8OGRGOJT0MKMgbBSEjHAoREHYKAwbBaEQRQUhisIhCsMhigtDFBfEpw0pDFNSGKakIMyQojBFBbqcvYi8KZ1ATxWX3bc302mDuz8APABQWVnZp23WG+eM58Y54/vyUhGRvJbOJl4tUJE0PgU40FMbMysARgJHM1GgiIikJ51AXwvMMrPpZlYELANWdGuzAvh4YvgDwHP90X8uIiI9O2OXS6JP/C7gaeKHLT7o7tvM7F5gnbuvAH4M/NzMqolvmS/rz6JFROTt0jrcwd1XAiu7Tbsnabgd+GBmSxMRkd7QYRIiInlCgS4ikicU6CIieUKBLiKSJyyoowvNrBHY18eXl9HtLNQsobp6R3X1XrbWprp652zqOsfdy1PNCCzQz4aZrXP3yqDr6E519Y7q6r1srU119U5/1aUuFxGRPKFAFxHJE7ka6A8EXUAPVFfvqK7ey9baVFfv9EtdOdmHLiIib5erW+giItKNAl1EJE9kbaCb2QfNbJuZxcysstu8L5tZtZntNLObenj9dDN7xcx2mdmjiUv/ZrrGR81sY+Kx18w29tBur5ltSbRbl+k6UnzeV82sLqm2JT20W5RYhtVmdvcA1PUtM9thZpvN7HEzG9VDuwFZXmf6+c2sOPEdVyfWpWn9VUvSZ1aY2fNmtj2x/n8+RZvrzKwp6fu9J9V79UNtp/1eLO67ieW12czmDUBN5ycth41m1mxmX+jWZsCWl5k9aGYNZrY1adoYM1udyKLVZja6h9d+PNFml5l9PFWbM3L3rHwAs4HzgReAyqTpc4BNQDEwHXgdCKd4/WPAssTw94G/7ed6/zdwTw/z9gJlA7jsvgp86QxtwollNwMoSizTOf1c10KgIDH8TeCbQS2vdH5+4O+A7yeGlwGPDsB3NxGYlxgeAbyWoq7rgCcGan1K93sBlgBPEb+D2XzglQGuLwwcIn7iTSDLC7gGmAdsTZp2H3B3YvjuVOs9MAbYnXgenRge3dvPz9otdHff7u47U8y6GVju7h3uvgeoJn4j6zdY/Aai7yZ+w2qAh4Bb+qvWxOfdBjzSX5/RD964+be7dwKnbv7db9z9GXePJEbXEL/7VVDS+flvJr7uQHxdWmDJN6ftB+5+0N3/khg+AWwnfs/eXHAz8DOPWwOMMrOJA/j5C4DX3b2vZ6CfNXd/kbffrS15Peopi24CVrv7UXc/BqwGFvX287M20E8j1U2ru6/wY4HjSeGRqk0m/RVQ7+67epjvwDNmtj5xo+yBcFfi394He/gXL53l2J8+SXxrLpWBWF7p/Pxvufk5cOrm5wMi0cUzF3glxex3mNkmM3vKzC4coJLO9L0EvU4to+eNqiCW1ynj3f0gxP9gA+NStMnIskvrBhf9xcx+D0xIMesr7v67nl6WYlqfblqdjjRrvJ3Tb51f7e4HzGwcsNrMdiT+kvfZ6eoC/gP4GvGf+WvEu4M+2f0tUrz2rI9hTWd5mdlXgAjwyx7eJuPLK1WpKab123rUW2Y2HPgN8AV3b+42+y/EuxVaEvtHfgvMGoCyzvS9BLm8ioClwJdTzA5qefVGRpZdoIHu7jf04WXp3LT6MPF/9woSW1ap2mSkRovfFPt9wOWneY8DiecGM3uc+L/7ZxVQ6S47M/sh8ESKWeksx4zXldjZ815ggSc6D1O8R8aXVwq9ufl5rQ3gzc/NrJB4mP/S3f+z+/zkgHf3lWb2PTMrc/d+vQhVGt9Lv6xTaVoM/MXd67vPCGp5Jak3s4nufjDRBdWQok0t8b7+U6YQ33/YK7nY5bICWJY4AmE68b+0ryY3SATF88RvWA3xG1j3tMV/tm4Adrh7baqZZjbMzEacGia+Y3BrqraZ0q3f8tYePi+dm39nuq5FwP8Elrp7aw9tBmp5ZeXNzxN99D8Gtrv7t3toM+FUX76ZXUn89/hIP9eVzveyAvhY4miX+UDTqa6GAdDjf8lBLK9uktejnrLoaWChmY1OdJEuTEzrnYHY89vHvcW3Ev+r1QHUA08nzfsK8SMUdgKLk6avBCYlhmcQD/pq4FdAcT/V+VPgs92mTQJWJtWxKfHYRrzrob+X3c+BLcDmxMo0sXtdifElxI+ieH2A6qom3k+4MfH4fve6BnJ5pfr5gXuJ/8EBKEmsO9WJdWnGACyjdxH/V3tz0nJaAnz21HoG3JVYNpuI71x+5zO9PxwAAAB3SURBVADUlfJ76VaXAfcnlucWko5O6+fahhIP6JFJ0wJZXsT/qBwEuhL59Sni+12eBXYlnsck2lYCP0p67ScT61o1cEdfPl+n/ouI5Ilc7HIREZEUFOgiInlCgS4ikicU6CIieUKBLiKSJxToIiJ5QoEuIpIn/j81rjmSgc3EnwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-10, 10, 100)\n",
    "y = sigmoid(x)\n",
    "\n",
    "plt.plot(x, y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the version of neuron with sigmoid activation function:\n",
    "\n",
    "![full_neuron.png](img/full_neuron.png)\n",
    "\n",
    "The inputs and weights are the same as the simplified neuron.\n",
    "\n",
    "The only different is there is a sigmoid activation function. So the output is now 0.04.\n",
    "\n",
    "$\\sigma(\\text{-}3.07) = 0.04$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.04436182758915638"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sigmoid(-3.07)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For three inputs, a neuron would look like:\n",
    "\n",
    "![neuron.png](img/neuron.png)\n",
    "\n",
    "Where x1, x2, x3 are the inputs, w1, w2 ,w3 are the weights, h(·) is a activation function, and y is the output.\n",
    "\n",
    "The neuron takes the weighted sum of the inputs and pass it into an activation function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The general formula for any input size is:\n",
    "\n",
    "$n(\\cdot)$ is the neuron function and $h(\\cdot)$ is an activation function.\n",
    "\n",
    "$n(x_1, x_2, \\dots , x_n) = h(w_1 \\times x_1 + w_2 \\times x_2 + ... + w_n \\times x_n)$\n",
    "\n",
    "where $x_i$ are the inputs, and $w_i$ are the weights.\n",
    "\n",
    "or\n",
    "\n",
    "$n(\\vec x) = h(\\vec w \\cdot \\vec x)$\n",
    "\n",
    "where $\\vec x$ is the vector of $x_i$, and $\\vec w$ is the vector of $w_i$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fully connected feedforward neural network\n",
    "\n",
    "In a fully connected feedforward network, all possible connections between any two layers are present. \n",
    "\n",
    "There will be one input layer and one output layer with any amount of hidden layers.\n",
    "\n",
    "Here is a sample two layer network:\n",
    "\n",
    "![2_layer_neural_network](img/2_layer_neural_network.png)\n",
    "\n",
    "There are three inputs and two outputs.\n",
    "\n",
    "Here is the three layer network shown at the start:\n",
    "\n",
    "![3_layer_neural_network](img/3_layer_neural_network.png)\n",
    "\n",
    "There are two inputs and two outputs with a hidden layer of size 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By the way, fully connect feedforward are often just referred as fully connected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recall the formula for a neuron is: $n(\\vec x) = h(\\vec x \\cdot \\vec w)$\n",
    "\n",
    "From that the formula for fully connected layer can be derived."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is an fully connected layer with 2 inputs and 2 outputs.\n",
    "\n",
    "![feed_forward.png](img/feed_forward.png)\n",
    "\n",
    "The two inputs are $\\text{x1}$ and $\\text{x2}$.\n",
    "\n",
    "The two outputs are $\\text{y1}$ and $\\text{y2}$.\n",
    "\n",
    "The $h(\\cdot)$ is the activation function (not shown in image).\n",
    "\n",
    "Using the formula for neuron, the outputs should be:\n",
    "\n",
    "$\\text{y1} = h(\\text{w11} \\times \\text{x1} + \\text{w21} \\times \\text{x2})$\n",
    "\n",
    "$\\text{y2} = h(\\text{w12} \\times \\text{x1} + \\text{w22} \\times \\text{x2})$\n",
    "\n",
    "Alternatively, we could use vector and matrix representations:\n",
    "\n",
    "The inputs: $\\vec x = (\\text{x1}, \\text{x2})$\n",
    "\n",
    "the outputs: $\\vec y = (\\text{y1}, \\text{y2})$\n",
    "\n",
    "The weight matrix: $\\mathbf W = \\begin{bmatrix}\\text{w11} & \\text{w12}\\\\ \\text{w21} & \\text{w22} \\end{bmatrix}$\n",
    "\n",
    "Then it becomes:\n",
    "\n",
    "$\\vec y = h(\\vec x \\cdot \\mathbf W)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### General form\n",
    "Say $\\text{ff}(\\cdot)$ is the function of a fully connected layer, then:\n",
    "\n",
    "$\\text{ff}(\\vec x) = h(\\vec x \\cdot \\mathbf W)$\n",
    "\n",
    "Where $\\vec x$ is the input of the layer, and $\\mathbf W$ is the weight matrix of all the input connections of the layer.\n",
    "\n",
    "Note in this fully connected layer each neuron uses the same activation function $h(\\cdot)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some versions adds a bias for each neuron. Which results in:\n",
    "\n",
    "$\\text{ff}(\\vec x) = h(\\vec x \\cdot \\mathbf W + \\vec b)$\n",
    "\n",
    "where $\\vec b$ is a vector of bias.\n",
    "\n",
    "Bias is just a scalar to shift the function along the x-axis (input axis).\n",
    "\n",
    "For example, $f(x)=x-1$ is shift to the left by one compared to $f(x)=x$. The scalar $1$ is the bias in the example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A network of fully connected layers would have network function that is consist of nested $\\text{ff}(\\cdot)$.\n",
    "\n",
    "For example, a two layer fully connected network function $N(\\cdot)$ of:\n",
    "\n",
    "$N(\\vec x) = h_2(h_1(\\vec x \\cdot \\mathbf W_1 + \\vec b_1) \\cdot \\mathbf W_2 + \\vec b_2)$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example of fully connected network\n",
    "\n",
    "Now we know how neurons work and one way to build a network of them, we can use a simple neural network to model a system.\n",
    "\n",
    "Let us create a model to classify iris flowers with the petal width and petal length of a flower.\n",
    "\n",
    "That means given a set of width and length of petal, our model can guess what type of iris flower it is.\n",
    "\n",
    "There are three types of iris flowers:\n",
    "\n",
    "##### Iris setosa:\n",
    "\n",
    "![Iris_setosa.jpg](img/Iris_setosa.jpg)\n",
    "\n",
    "Retrieved and downscaled from: https://en.wikipedia.org/wiki/Iris_setosa#/media/File:Kosaciec_szczecinkowaty_Iris_setosa.jpg\n",
    "\n",
    "##### Iris versicolor:\n",
    "\n",
    "![Iris_versicolor.jpg](img/Iris_versicolor.jpg)\n",
    "\n",
    "Retrieved and downscaled from: https://en.wikipedia.org/wiki/Iris_versicolor#/media/File:Blue_Flag,_Ottawa.jpg\n",
    "\n",
    "##### Iris virginica:\n",
    "\n",
    "![Iris_virginica.jpg](img/Iris_virginica.jpg)\n",
    "\n",
    "Retrieved and downscaled from: https://en.wikipedia.org/wiki/Iris_virginica#/media/File:Iris_virginica_2.jpg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the dataset\n",
    "\n",
    "We will be using the [iris data set](https://archive.ics.uci.edu/ml/datasets/iris)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "\n",
    "data_path = os.path.join('data', 'iris.data')\n",
    "dataset = pd.read_csv(data_path, names=['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)', 'class'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is what the data look like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)  \\\n",
       "0                5.1               3.5                1.4               0.2   \n",
       "1                4.9               3.0                1.4               0.2   \n",
       "2                4.7               3.2                1.3               0.2   \n",
       "\n",
       "         class  \n",
       "0  Iris-setosa  \n",
       "1  Iris-setosa  \n",
       "2  Iris-setosa  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The \"class\" column is the type of flower.\n",
    "\n",
    "The other four columns are the length and width of sepal and petal, these are the four features given in the dataset.\n",
    "\n",
    "Out of the 4 features given, we are only going to use the petal length and petal width."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize the dataset\n",
    "\n",
    "To visualize the dataset, we will be using seaborn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcAAAAFgCAYAAAAsOamdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdd5iU1fXA8e/Z2d3Z2V5pSwdFAWliBRWNxhJ7T4yxRE00JpbElvwSTdEkRo0aNZZoIonG3kIssYMgShFBEFRA2gLb+/S5vz/u7OzOdtid3R3mfJ5nHnnv3LnvnVU5+77vveeIMQallFIq0ST19wSUUkqp/qABUCmlVELSAKiUUiohaQBUSimVkDQAKqWUSkjJ/T2BXXXccceZ119/vb+noZRSLUl/T0Dturi7AiwvL+/vKSillNoDxF0AVEoppXqDBkCllFIJSQOgUkqphKQBUCmlVELSAKiUUiohaQBUSimVkDQAKqWUSkgaAJVSSiUkDYBKKaUSkgZApZRSCSlmAVBERojIuyLyuYisFpGr2ukzR0RqRGRF+PWrWM1HKdVz3oCXssYyyhrLcAfc3f5ctaea0sZSKtwVAPiCvsg4jf7GWE1XqU7FMhl2APipMWa5iGQBy0TkTWPMmlb9FhhjTozhPJRSvaDGU8MLX73AQysfIhAK8O19vs3Fky8mLy2v08+V1Jdw/fzr+bTsU8Zkj+G+b9zHwpKF/GX5X/AEPZy595lcPvXyLsdRqrfF7ArQGLPdGLM8/Oc64HOgOFbnU0rF1td1X3PXsrto8DfgDXr5x+p/sGTHkk4/U+2p5ob5N/Bp2acA1Ppq2Va/jds+uo06fx3+kJ9/r/037299H2NMX3wNpSL65BmgiIwGpgMftfP2ISLyqYi8JiKT+mI+SqldN3/r/DZtb256E3/I3+Fn/CE/K8pWRI73ytuL5TuXt+n31qa3dumWqlK9IeYBUEQygeeBq40xta3eXg6MMsZMBf4CvNTBGJeJyFIRWVpWVhbbCSul2jVj0Iw2bQcOOZCUpJQOP+MQB+Nzx0eOt9RtYUL+hDb9Zg6eidPh7J2JKtVNMQ2AIpKCDX5PGGNeaP2+MabWGFMf/vOrQIqIFLbT72FjzExjzMyioqJYTlkp1YF9C/bllHGnIOHar4cVH8Y3Rn6j08/ku/K5/fDbKXLZ/29rvDWMzRnLORPOIUnsXz8HDjmQk8adhCPJEdsvoFQrEqv77iIiwONApTHm6g76DAF2GmOMiBwIPIe9IuxwUjNnzjRLly6NyZyVUp2r9dbSGGjEGIMr2UVuWm6XnwmGglR5qnAH3DiTneQ4c/AFfTT4GwiZEK5k156wAEYrwsehWK4CnQWcD6wSkaaHAD8HRgIYYx4EzgQuF5EA4AbO7Sz4KaX6V7Yzm2xn9i59xpHkoDA9+saO0+EkKzWrN6em1C6LWQA0xnxAF78VGWPuA+6L1RyUUkqpjmgmGKWUUglJA6BSSqmEpAFQKaVUQorlIhilVIyFTIgKdwUf7fiIJJI4YMgBFLoKsYuwe6a8sZxSdykf7/iYaUXTKM4spihdtyGpPYcGQKXiWFljGWfPO5tKTyUAha5Cnj7xaQalD+rRuA2+BuZtmMedy+6MtF0w8QIu2e+Sbm19UCoe6C1QpeLYi1+9GAl+AOXucl7d8GqPx6311fLXT/8a1fbE50/QGNDKDWrPoQFQqThW7alu01bjremVsb1Bb9RxwAQImVCvjK3UQKABUKk4ds4+5+CQ5hRiyZLMaXud1uNx05LTOH708VFthww7hNSk1B6PrdRAEbNUaLGiqdCUauYOuNlSu4VHVj2CiHDZfpcxPGs4aclpPR67rLGMVze+yqKSRUwfNJ3T9zq9x88W92CaCi0OaQBUag/gDrgRpFcCX0vBUJBaXy2ZKZmkODqu+qA0AMYjXQWq1B7AleyKybiOJMeekKhaqXbpM0CllFIJSQOgUkqphKQBUCmlVELSZ4BKxblAKEC1txoM5KXl4UhyUOutxRv04nQ4yXZm2z6eahDIdeaSnJTcpk93BENBqrxVYCA3zY7TYR8gx5lDSlLbxTMhE6LKU4XBkJ2aTaqjf7dXGGOo9FQSMiGyndk4Hc5+nY/qGxoAlYpjNd4a3vj6DR5e+TAiwnUzr2NiwUR+t/h3fFL6CdMHTeeXh/ySz8o+445ld2CM4dqZ1zKtaBq/W/w7lpcuZ0rhFG4+9GaKM4s7PVetr5YFWxdw7/J78Yf8XDz5Yk4cdyK5zubUaHW+OhZuW8g9y+/BE/RwwcQLOHWvU6P61PvqWbx9MXcuuxO33835E8/njL3O6LcUa+6Am1Vlq7j1o1up9FRyxl5ncMGkC3TxTwLQW6BKxbEvq77kt4t/y87Gnexo2IEn6OHqd69mYclCGgONLCxZyDXvXoMn6GFHww52Nu7EE/Bw7XvXsmDbAhr8DXy4/UN+/PaPqXBXdHqukvoSblxwIyUNJZS5y/jjkj+yunx1VJ8dDTu4bv51bK3fSrm7nDuX3cknOz+J6lPaWMo1713D1rqtVHgquHv53Xy046Ne/9l0V5WnisvevIwNNRuo9lbz6GePMm/DPIKhYL/NSfUNDYBKxbF5G+ZFHRdnFrOual1U2+eVnzM8a3jkeFT2KFZXRAeuL6u/bJP6rLW3Nr3Vpu2V9a/gD/kjx+9uebdNn/9s+A/eQPPYC7YtaPd7uP3uTs8fK6vKVxE00cHu9Y2vU+er65f5qL6jAVCpODa5cHLUsSfgibrdCJDnzMMdaA4ujf5GCtIKovpkp2a3+6yupX0L9m3TNqlgEsnS/CRln/x92vSZmD8x6lnh3nl7tx07f99+ew44KntUm7bxeeN7PamAGng0ACoVx44ccSRTi6ZGjpfsWMLvZv0ukrPT6XBy6+xb+XjHx5E+i0sWc+vsWyMLPVKTUrl19q1tAmdr04umc+iwQyPHE/Mn8q2x34qqPTi5cDKHFx8eOZ6QN4FT9zoVR1JzvtIJ+RM4csSRkeO9cvfi7AlnR/XpS0PSh3DmXmdGjodnDufyqZdrAEwAmgpNqThX5amizleHIGSmZuJKdlHrq6XGW0OOM4fs1GzcATf1vnoMhqzULNKT06nx1UT16c5f+FWeKur99YRMiKzULPLT8tv0qfZUU++vJ2iCZKVkke/qpE8oSGZqJgWugjZ9+lKNt4YGfwPeoJes1CwKXYW7OoSmQotDGgCVUqrnNADGIb0FqpRSKiFpAFRKKZWQNAAqpZRKSBoAlVJKJSRNhaaUale9r55qbzXrqtYxPnc8ec68NjlD63311HhrWFu1lnE548hPy+92XtG+UumuZGv9Vmq8NUzIn0B+Wn67OUxV4tH/CpRSbfhDft7b+h43Lbgp0vazmT/j7L3PxpVii+8GggEWlizkuvevw2BXk1814yq+s893SE9J75d5t1bpruSa965heelywG74f+rEpxiRNaKfZ6YGAr0FqpRqo9pTze8/+n1U273L76XO35werMpbxW0f3RYJfgD3r7ifen99n82zKxtrNkaCH9iE3g+seCAqM45KXBoAlVJtGEybXJi+kI9AKBDVp9pbHdUnEApE5Qbtb6Xu0jZtOxt34g8OnDmq/qMBUCnVRpojjTkj5kS1TS2aiivZFTl2Jbs4ZuQxUX0m5k+M6tPfZgyaQZojOsPNuRPOHXDPKVX/0EwwSql2VbgrmLt6Lou2L2Ja0TQum3IZRelFbfo88fkTLNi2gKlFU7lsymUMSh/UTzNuyx/0s6l2E39e9meqvFWct+95zC6eTY4zp7dPpZlg4pAGQKVUh3xBH/X+etKT0zvMFdqdPv2tzldHIBQg15kblby7F2kAjEO6ClQp1aFURyr5jrbJrHe1T3/LSs3q7ymoAUifASqllEpIGgCVUkolJA2ASimlEpIGQKUSlDvgptxdTqO/MdJW5ami0lMZOfYEPFS4K9rt09kCuu70iRVf0Ee5u5x638DZkK8GJl0Eo1QCKmss4/4V97NkxxKmD5rOldOvZFvdNu5YdgfBUJAfT/sxEwom8OCnD/LR9o+YUjSFq2ZcxY6GHdy+5Hb8IT+X7HcJhww9JGpPXaO/kdUVq7l72d14g14unnwxs4tn99m+u0p3JXPXzOWtzW8xLmcc1x1wHcWZxbFa+aninAZApRJMtaeaGxbcwJIdSwDYXLeZjTUbOWPvM/is/DMANtZu5Im1T7CwZGGkz/rq9Xx34ndZVb4KgJ+9/zMeP+5xZgyeERm7tLGUS/53CSETAuCGBTfw6Dcf5cChB8b8e7kDbu5fcT/PfPEMAJtqN7G6YjX/PvHfFLmKuvi0SkR6C1SpBOMNeiPBr8nK8pWMyh4VOZ5UOCkS/Jp8Xvk5xZnFUW3PfvFsVHq0tze/HQl+TZ5Z9wy+oK+3pt+hel89r258NaptZ+NOvRWqOqQBUKkEkyRJZKdG35JMT04nGApGjhv9jeQ586L6pDnS2jzTG5MzBoc4Isctg2iT0Tmjo/rESpIkMSRjSFSbIAMqNZsaWDQAKpVgctNy+eXBvyRJ7P/+gnDjgTeyePviSJ/3trzHLw+J7nPdAdfx8Y6PI32GZw3ntPGnRT1fmz5oOpMLJkeOh2UM4+wJZ+NIin0ALHAVcPMhN5OalBppu2S/S8hIyYj5uVV80lRoSiWgBn8Dtd5aNtdtZkTWCHKcOXgCHnY07iAUCjEscxiuZBc1vho2125meNZwclJz8Aa97GzcSSAUYFjmMApdhW3GrnBXUNpYii/kozizuN0+seINeKn2VrOpdhNDMoaQ48yJRd7P9ugqmzikAVAppXpOA2Ac0lugSimlEpIGQKWUUglJA6BSSqmEpAFQKaVUQopZJhgRGQHMBYYAIeBhY8w9rfoIcA9wAtAIXGiMWR6rOSkVbyrcFVR4KkiSJPLT8klzpFHrq6W0sZTB6YPJdmbj9rup8FQgIuSn5eNyuKj11bKjcQdD0oeQlZpFekp6f3+VHvMFfVR7q9nZsJMCVwFZqVndqvNX7ammxltDY6CRovSiPl2Vqga2WKZCCwA/NcYsF5EsYJmIvGmMWdOiz/HAXuHXQcBfw/9UKuGVu8u59H+X8lX1VwBMKZzCrbNv5dSXTyVogiRLMncdeReLSxbz5NonAbhw4oXsV7Qf18+/PtLnT0f8iSOGH0GKI6U/v06Pra1cy/ff+D6eoAdBuHbmtZy191md7vOr9FTymw9/w9ub3wZgcPpg/nXCv9psmFeJKWa3QI0x25uu5owxdcDnQHGrbqcAc421GMgVkaGxmpNS8eTVDa9Ggh/YdGWLShYxudBuNA+YADcvvJnZxbMjfQ4pPoRff/hrgibY3GfRzVR7q/t28r2swl3Brxb9Ck/QA4DBcPeyu7tMc7a1bmsk+IFNjfbwpw/jDXhjOl8VH/rkGaCIjAamAx+1eqsY2NLieCttgyQicpmILBWRpWVlZbGaplIDRjAUZG3l2jbtW+u3RiV2rvJWRd3edDqc1Ppqoz5T66slYALEM2MMW2q3RLUFTTASEDuyuXZzm7aNtRvxBjUAqj4IgCKSCTwPXG2MqW39djsfabMz3xjzsDFmpjFmZlGRZnVXez5HkoMz9j6jTfuc4XNYUbYicrz/4P35surLyPGm2k2RK8QmUwqnkOZIi91k+4ArxcUxo4+JahucPrjLNGczBs8gWaKf9Jw87uRuPTtUe76YBkARScEGvyeMMS+002UrMKLF8XCgJJZzUipejM8dz62zb2V09mjG547n7iPvZlT2KKYVTaPIVcTxY47nD4f9gUGuQYzJGcPYnLEUuYq464i7OHbUsRS5ijh21LHcNecu8tLyuj7hAJaRksF1M6/jrL3PYlD6IGYNm8Vjxz5GQVpBp5/LS8vj0WMfZWLBRIozi7l2/2s5csSRWh9QATFMhRZe4fk4UGmMubqDPt8CrsSuAj0IuNcY02nhME2FphJJMBSMPL/LT8tHRKj31eMOuElPSScjJYOQCVHlqWq/T3I6Gal7TjJod8BNva8ep8O5S0V2qzxVBE2QXGcuyUkxWfunETUOxTIAzgYWAKuw2yAAfg6MBDDGPBgOkvcBx2G3QVxkjOk0umkAVEoNQBoA41DMtkEYYz6gi/8ojI2+P4rVHJRSSqmOaCYYpZRSCUkDoFJKqYSkAVCpOOMP+an2VBMINe/tq3RXUuWu6vRz9b76LjeO96UGfwN1vrr+noZKYLFMhaaU6mXl7nKe/PxJluxYwqziWZyx1xmU1Jfw4MoHEYQfTPkBo7NHk5PWXAXd7XezvmY9D6x4AIPhiqlXMC53XL/lB/UGvGyp28J9K+6j0d/IxZMvZlLhJN2bp/qcBkCl4kS1p5qb5t/E4h2LAVhRtoK1lWuZUjiFD7Z9AMDCkoU8f/LzUQFwe+N2znv1PELGLsZeVLKIF05+gXG54/r+SwAVngrOmXcOvpAPgA+3f8jc4+YyffD0fpmPSlx6C1SpOOEOuiPBr8k7m99h6qCpkeOQCfHCl9E5J1748oVI8Gvq8+y6Z2M72U68v+X9SPBrMnfNXDyBztOaKdXbNAAqFScc4iA1KTWqLT0lHX/QH9XWMlcowCDXoDZjFaX3X0rBAlfb7C2D0gfhSHL0w2xUItMAqFScyErN4oppV0S1XT3jat7f+n7keGjGUI4fc3xUn+PHHB9V/mdw+mBOGndSbCfbiRmDZzA2Z2zkOMeZwwWTLiAlKb7LNan4E7NMMLGimWBUIqvx1lDuLmdNxRomF06mwFWA2+9mZdlKkiSJyYWTGZwxuM3nmj5jjGFS4aR+Lwpb4a7gi6ovaPA3MKVoCgVpBfF+BaiZYOKQBkCllOo5DYBxSG+BKqWUSkgaAJVSSiUkDYBKKaUSkgZApZRSCUkzwSg1QNR7aqgLNNAYcJORnE5uag61/no8QQ8igtPhZFB62z19VZ4qGv2NIJCRnEFuWu5unb+0sRRv0IsxhrTkNPKceVR7q2nwN5Cekk5Oag7OZGfUZ/xBf1Sf7NRs0pLTovoEggGqvFWd9mlPjbeGRn8jARMgPTm93f2DSvWEBkClBoBGbx0f71zKjR/chDvgJisli/uPvp/XNrzGv9f9G4DDiw/n5kNvjgqCle5KblhwA4u32wwxs4tnc+usW8l35e/S+UsbS7ll0S0s2LYAgJmDZ/KHw/7A6S+fTq2/FleyizuPuJODhx5MisPu1zPGsLZyLT9864fU+mpJc6Rx+xG3M2vYLFIdzRv211Wv47L/XUatrxanw8nvZ/+ew4Yf1mkQrPJUcdfSu3hp/UsATMyfyP1H39/v2zfUnkVvgSo1ANT46/i/hb/EHXADUOev48b5N3Lg0AMjfeZvm8/yncujPjd/2/xI8AP4YNsHfLzj410+/7KdyyLBD2DpzqW8tfktTtvrNADcATc3LriRam91pE+Fp4LrF1xPra8WAE/Qw00LbqLGW9Pcx13BTfNvivTxBr38YuEvIscd+br260jwA1hTuYan1z4dVQFDqZ7SAKjUAOAL+qjzR5cGKmkoIS8tL6ptVfmqqOPWARHgk9JPdvn8rccFWF2+mvF54yPHtb5afMHmHJ4hE2Jr3daozzT4G/AGvVF9NtZujOrjDri7zPu5tmJtu3P0Brzt9FZq92gAVGoAcDqcDM8cHtW2X+F+bKyJDh7fGPmNqOMTxpzQZqxjRx+7y+dvPS7AMaOOYf7W+ZHj4VnDo54BpiSlMH1QdAWHoRlDo25tpjpSOWDIAVF9BqUP6rIU00HDDmrTduzoY/uthJPaM2kAVGoAGOQaxP3fuJ/9CvfDIQ4OHHIgtx9+O76gj6yULPKcedxwwA2MzBoZ9bl9Cvbh2v2vJTs1mxxnDjcccAPjcna9zNGorFFcP/N6cp25ZKdm86NpP2JiwUSqvdU4xMGUwik8ePSDFKQ1L0TJS8vj9sNv5+ChB+MQB5MLJ/PQMQ9F9clx5vD72b/n0GGH4hAHEwsm8sgxj5Cf1vkzyiJXEbcffjuD0gfhSnZx0eSLmDNiDiKacEX1Hk2FptQAsrN+O0YgmSQKMwbT6G+MPC/Lc+a1WYUJdiVmjbcGxAac3U0q7Ql4Is/4slOzSU9Jp8Zbgy/oIzkpuc3t2Cbd6VPrrcUb9OJIcnQZ/JoEQ0GqvLbKfVZKVrvffQDRyByHNAAqpVTPaQCMQ3oLVCmlVELSAKiUUiohaQBUSimVkDQAKhUjvqAvNhu3jQG/2/5TKbXbNBWaUr3M7XezpX4L//jsH2SkZHDBpAsYkj6EZEcv/O/WUAZrXob178Lex8GE4yFD04MptTs0ACrVyzbXbebseWcTMiEAXl7/Mq+c+gpDMob0bGB3Ncz7KXz+sj1eOw+mfReO/wM4s3o4a6USj94CVaoX+YI+Hl/9eCT4gU399e7md3th8AZY+0p028p/g7e+52MrlYA0ACrVi5Ikqd10Xa4UV88HFwFxRLc5UnUHmlK7SQOgUr0oOSmZCyddiCu5OeANSh/ErGGzej64MwsOujy6bdY14Mzp+dhKJSDNBKNUL/MH/VR4Knh789ukJ6czu3g2RelFvTN4YwWUfQGbFsKYI6BgHKTvWu0/FRN6HR6HNAAqpVTPaQCMQ3oLVCmlVELSAKiUUiohaQBUSimVkLq1EV5E8oBhgBv42pgWm5yUUkqpONRhABSRHOBHwLeBVKAMSAMGi8hi4AFjTC/s7lVqD+SuAl+j3bvnzGo/U4u72m5uF4HUTEjL3s1ztRwnA9L6bltEyISo9FTiDXhJdaSSl5ZHcpImmFLxobP/Up8D5gKHGWOqW74hIvsD54vIWGPMo7GcoFJxp6EM/nO1TVWWlAwH/RAOuxbSC1r0KYd519q0ZkkOmHkJHHEDZBR0PG675yqHN34Bq54GxKZGO/rmPssPuqFmAz9660eUNJSQn5bPXXPuYkrhFFIcu1eVXqm+1OEzQGPMMcaYf7YOfuH3lhljrtbgp1QrxsDql2zwAwgF4MP77N69ltb+tzmnZygIHz8Epat3/XwbF8DKp+x5TQg+mQtbl/TsO3RThbuCn73/M0oaSgCo9FRy1btXUe1t81eGUgNStxbBiMgUETlZRE5vesV6YkrFJb8HNrzXtn3TouY/B3ywoZ2nB19/sOvn++qttm3r++bJRCAUYH31+qi2Gm8NnqCnT86vVE91GQBF5DHgMeAM4KTw68QYz0up+JSSZssUtTZuTvOfk1NtGaM2fY7a9fPt8622be2dPwZSHCnsm79vVFtBWgEuRy/kPVWqD3TnCvBgY8xMY8wFxpiLwq+LYz4zpeKRiA1uM75nn/+lpMNRv4S8MdH9xn0D9r843McFc26Cwr13/XwjD4KDf2STYic7YdbVMGxq73yXLuSn5XPHEXewd56d9/DM4dx/9P3kpeX1yfmV6qkuU6GJyKPAncaYNX0zpc5pKjQVFzx14Ku3CbLScm2Qa81bD94628eZA6ltq0h0i68BPLU9H2c3VXoq8QV9JCclU5BWgEhCZgVLyC8d77qzXvlx4EMR2QF4sf+ijTFmSkxnplQ8S8uyr844M+2rp1Iz7Kuf5KdpMm4Vn7oTAB8DzgdWAboBXiml1B6hOwFwszHmla67KaWUUvGjOwFwrYg8CfwHewsUAGPMCzGblVJKKRVj3QmALmzg+2aLNgNoAFRKKRW3ugyAxpiLdmfg8P7BE4FSY8zkdt6fA7wMbAw3vWCM+c3unEupmGkog43zYdNimHQqDNq36wrs9RXgrYblj9vFKVO/Y/NzbnwPNiyAfU+EwfuBrw6W/9OuEJ16LmQNsWnRmvgaoX4nLJ8LjhSYfj5VznQ+LVvFwpKFzBkxh4kFE3EH3Lz01UskSRKnjj+VQleh5uNUqhu6sw3iceCqppRo4coQd3a1F1BEDgfqgbmdBMCfGWN2aVO9boNQfaaxEl76IXzxRnPbsbfBAZfazewdKV0LDx0GQZ89Ts+HHyyA+w4Cf709/t4r8LejIRDOmuLKg8sXQfaw5nEq1sODs8DvBqD+m7/hTqp47svnAchIyWDu8XP57qvfxR1wR9pePPlFhmYO7bUfg+oW3QYRh7qzEX5Ky3ygxpgqYHpXHzLGzAcqezA3pfqXrz46+AHM/5Ot9NARTx18+Jfm4Ac2kK79L5z6gD3e+3hY8rfm4Ad2zDX/aT4OBeHjhyPBD6Bh9KG88NWLkePZw2bz0lcvRYIfQIO/gVfW65o1pbqjOwEwKXzVB4CI5NPNOoLdcIiIfCoir4nIpI46ichlIrJURJaWlZX10qmV2g0m1PXv+qF2dgsZA0nh/91E7HHbD7Y6DHZ+HoH27uCEtFynUt3SnQB4J7BIRH4rIr8BFgG398K5lwOjjDFTgb8AL3XU0RjzcDgd28yioqJeOLVS3ZCaYVOWtTT7WpvZpSNpWTDrx/aZXRNXnn3u9/KV9njda3DA9236ssjncmDiKc3HSQ5bRinZGWlK37SYk8aeFDleuG0hp40/jTRHWvOpkl2cMr7FOEqpDnX5DBBARCYCR2F/9327u2nRRGQ0MK+9Z4Dt9P0amGmMKe+snz4DVH2qoQy+fBM2fwiTz4AhU7qxCKYMPFWw5DEbRPe/AFJzYP2btlLExFNg2HSbBm3Jo3YRzP4XQOZQcLRYBOOtt4tgljwKjmSY+X0q0zJZunMZH2z7gKNGHMXUQVNp9DfyzLpnSEpK4qy9z2KQaxDJDl0E08f0GWAc6jAAikimMaa+0w930aezACgiQ4CdxhgjIgdiC/COMl1EZA2ASqkBSANgHOrs18SXRWQFdqvCMmNMA4CIjAWOBM4GHsEGrjZE5N/AHKBQRLYCNwMpAMaYB4EzgctFJAC4gXO7Cn5KKaVUb+n0FqiInACcB8wC8gE/sA74L/CoMWZHX0yyJb0CVEoNQHoFGIc6fVBgjHkVeLWP5qKUUkr1me6sAlVKKaX2OLpUTCUOT60tHisCrvzOs7nsilAI6krsHsGk5OhsLk2MsStKQwG7/SGjEBqrbGYYBJzZkJYN9aXRffpZpacSf9Bvi926Cvp7Okr1Kg2AKjE0lMH//g9WPQupWXDMb2xuz7Scno3r98LOVfDCpVC5AYr3hzP+Bvljm/sEA1D2OUEmMfMAACAASURBVDx7gU1vNmw6nPEYrHoaFtxl9wwe+wcong7PXQTlX9rtFmc/Hj1OH9tcu5mfvv9T1lauZUzOGO444g7G544nSfTGkdozdOu/ZBFxiMgwERnZ9Ir1xJTqNcEgfPqUfYWC4KmG//wE6nb2fOzGcnjybBv8ALYtgxd/CHXbW/SpgH+dboMfQMkn8ML3IWuYTZnma4CsQXac8i9tnx0r4ZnvQUOn22JjptJdyTXvXcPayrUAbKzZyBVvXUGlR7Mbqj1HlwFQRH4M7ATexK7+/C8wL8bzUqr3+Orgi9fbtm/5qBfGbrABrvW4LdOh+Rvtrc2Wti2HgnHNx6mZUFsS3WfHquicon3IH/LzRdUXUW07G3fiDXg7+IRS8ac7V4BXAROMMZOMMfuFX1NiPTGlek1qBow8pG370Km9MHY6OLOi2wZPjl4Un5LW9lZr0QSo3dZ8HPS1zTCTP9Y+U+wHjiQHI7JGRLXlOfNIdfTSc1PVZ0TkFhH5WX/PYyDqTgDcAtTEeiJKxYwjBQ68DEbNssdJyXDYTyFneM/HTsuFMx5tzg+aMxxOfwiyi5v7uArg7H/ZnKBgF8mc8Shs+8QeSxLsXANnzW3ukzUEzp4LGf2T+7YgrYC75txFocsuxMl15vLnI/9MnjOvi08qFT86S4V2bfiPk4AJ2Fufkfsfxpi7Yj67duhGeLXbGirs7cgkh71qa33ltru8DTb3Z8ADyWmQOcTm7mwp4AN3pe2T4oL0Qvt8L+AGBFLSwZVrb6cGPJDssqtAWxbI7WPBUJBKTyXeoBenw0meM09zjHZswGyEF5HvAT8DDLASWA/UG2PuEJFLgcuAVOAr4HxjTKOInIXN1hUEaowxh4cr9Pw93DcJOMMY82Xff6PY6ey/5qa/HTaHX6nhF9gfrFLxJaMAiMFSfmeGfXUmOdVe1bWUNbhtv9Z9+pEjyUFRulZfiSfhoPULYJYxpjxcvu4nLbq8YIx5JNz3d8D3sdV4fgUca4zZJiJN5U5+CNxjjHlCRFKB/vttLEY6DIDGmF8DiMhZxphnW74X/m1BKaXUwHIU8FxTVR1jTKVI1MXp5HDgywUygaaKzwuBf4jIM8AL4bYPgV+IyHBs4Nyjrv6ge88Ab+pmm1JKqf4ldH6H7h/AlcaY/YBfA2kAxpgfAv8HjABWiEiBMeZJ4GRssYI3ROSoWE68P3R4BSgixwMnAMUicm+Lt7KBQKwnppRSape9DbwoIn82xlSEb4G2lAVsF5EUbKGDbQAiMs4Y8xHwkYicBIwQkRxggzHm3nAVoCnAO333VWKvs2eAJcAy7G8Ay1q01wHXxHJSSimldp0xZrWI3Aq8LyJB4BPg6xZdfgl8BGwCVtG81uNPIrIX4aLnwKfAjcB3RcQP7AB+0ydfog91WRFeRFKMMf4+mk+XdBWooqEMdnwGpZ/DXsfYVZdpu7Gis7bEjrH9U9jrm5AxyK7AXPdfux1hzOF2y8TO1TZ7y/ijIWuoXbm57jW7inTMHLuqs+xzu7l97By7zSHghi/eBGcmjD7Mrgot/9JmihlzOOSMsKs+1Z5iwKwCVd3X2TaIVXRyL7m/NsNrAExwjRXw0uXwRfjZvQic+xTsfaz9c3fVlsDrN8Gal+xx5mA4/0X429F2qwTYjehnPQ4PHWaP0wvggnnw6NE2AwxA3mg490n466HNY594N2xdAiuesMfH3gaVG2HJI819TrgDZnwPkp27/CNQA5IGwDjU2SKYE4GTgNfDr/PCr1fpoAq8UjHnqW0OfmCrLLz5f/aqcFcEfc3BD2xi7A/+3Bz8wOb3LFluM7sATDwFPryvOfgBVH0NmxbBsBnNbW//2o7XZPhMWPpo9Pnf+S24q3ZtzkqpXtXZNohNACIyyxgzq8VbN4rIQvbA+8EqDrSXG9PXYAPhLo3T6q5+sis6sEXGrrepzMBucm+d97Pp/E19APxu27eJMbZUUkt+967NVynV67qzDSJDRGY3HYjIoUAXu36VihFXPhTuHd120OVt82h2JSU9OhfouldturSWnFn2+V1JOGXZF6/DAZdG32pNzYBxR9lbnk1mXmSfETYp/xJGHx499vTzey8TjVJqt3RnEcz+wGNAUzbfauBiY8zyGM+tXfoMUFG3A5b8zVZLmPYdG6R2NQCCfQ649DE7zr4nw7gjoXozLH7A5vacdbUNcEsfhe0rYMKJdtFNzRb48H67wGX2NTbR9fJ/wpbFsO8ptk/dDvjgLlt78LCfQqoLVjwFmxfCPifChBMGRMFb1Wv0GWAc6jIARjqKZIf792tibA2ACrBV0wNeG6B6IuADb104TVqYp9au/kxND/fx28rtrhaJoD11Nk9nU59gEIIee2XZdIXod9tE100LXULBcC7Q9F1bsKPigf4LjUOdbYT/rjHmXy2SYje1A/2XDFspIBygeiExc3IqJLfKD5qW3apPCiS3qoLQetuFwwGOVsE4xRV9nOToecBWqgsiUm+MyezgvUXGmEPbe68XzvtzY8xtsRg7Vjr7G6Tp/1R9UKGUUjEw+sb/fge4DRiJLTrw86//8K0ne/s8IuIwxgRjFfzCfo79LnGjO88A04wxnj6aT5f0FqhSagDa5Vug4eD3CJDeorkRuLQnQbDpClBE5mBLHG0HphljJrZ4byjwNDa1ZTJwuTFmQatx2i2HJCLfxVaYSMVmlbkCuBW4DptdZrUx5rzw3cOLw8P9zRhzt4hkAM8Aw7HVJX5rjHlaRH6F3XbnAhYBPzDdfT7XA925h/SZiOwEFgDzgYX9/RxQKaX2ALcRHfwIH98G9NZV4IHAZGPMxlbt3wHeMMbcKiKOduYB7ZRDEpF9gXOw5Zb8IvIAcJ4x5kYRudIYMw0iiycvAg7C/nLwkYi8D4wFSowx3wr3a1pceZ8x5jfhtn9i96H/p5d+Bh3qMgAaY8aLyEjgsPCkHhCR6qYvqlTc8HvAU2MXoKQXQlKSLUobCtrFLB1tS2isgGDAPtNr/XywSUMFhPzhPjnt92kt4GveDJ9eaJ8jqkQychfbd8fH7QQ/gCXAY+Gk2C8ZY1a006dNOSQR+QawP7AkvB7EBZS289nZwIvGmAYAEXkBG0NeB+4QkT8C81pcdR4pItdjA3E+sJo+CIBd7gMMf/lZ2MlPx07s6RjPS6ne1VgB7/8RHjgIHv2m3bKwczU8cRbcOw1evtJuXWgpFISyteE+U20KtjZ9Qnaf37/PseO8+EO7vaLL+VTCR3+FBw+FR+bAqmfAXd1rX1fFhc272L472snuAMaY+cDh2GoQ/xSR74nIaSKyIvya2UE5JAEeN8ZMC78mGGNuaecU7d4SNsZ8gQ2gq4Dfi8ivRCQNeAA4M1ym6RHCZZpirTu3QDdjf1u4LVwzSqn4EgrBmlfsvjywV12hADx+UnNmlzUv2Wwtp9zffJXXUA6Pnwz1O+3x2nn2c6c9DK7wVV5DGcw9BWq32eN1r9qtDmf+vfNk11s+hjd/1Xz80g/hBws0QXZi+TntPwP8eaxPLCKjgG3GmEfCz+VmGGOuBl5s0Wcsbcsh/Q94OVxuqTRcbikrnDnM36J4wnxsgd0/YIPhacD5IjIMqAzvMKgHLqQ52JWLSCZwJn2UbrM7mWCmA3OB74jIhyIyV0S+H+N5KdV7vLXw2fPRbUmOtmnNvnwjOkWZr645+EX6/M8GuCb+hubg12TDu9F9Wgv6YGU7N1HW/rfjz6g9Tnihy6XY0kQm/M8eLYDZBXOwhW8/Ac4A7mmnzznYNSArgH2AucaYNdjCuf8TkZXAm8DQcP+HgZUi8kQ4Uco/gI+xC2X+Zoz5BNgP+Dg85i+A3xljqrG/CKwCXsJecPWJbm2ED0fl2djboN8FjDFmdGyn1j5dBap2WcALb//GJrJucslb8Nhx9oquydCp8N3nIaPIHteWwD1To/OPDtoXvvcfyGzqsx3+Mj06cBaMh4teg8xB7c/HGJtt5o1Wv+h/+ymYcPzuf0/Vn3QjfBzqzjPApdiHoacBa4HD+yv4KbVbkp1w6JVQNKG5rewLOOFOu6EebJaXUx5oDn5gF7OcdA84UsLHuXDaQ83BD+yt0JP+Ao7U5s+c/nDHwQ/sIpz9zoLiFhUk9j7OVo1QSvWZ7uwDLDLG7GKtmdjRK0C12+pLbdozR6rN45mUCt4a8Nbb537phbZwbUu+BrtyNNKnoDkg7kqf9jSU2/k0ZYhJL+j6M2qg0ivAONSdbRADJvgp1SOZg9pemTm7SE2WmtF1+rLu9GlPRqEmxFaqH3VnEYxSSim1x9EAqJRSKiF1Vg3i9M4+aIx5ofeno5RSSvWNzp4BntTJewbQAKiUUgNMf5VD6i4ReRX4Tnj/36587hag3hhzR2/NpcMAaIy5qLdOohJAQzlUb4HaLTBsf7uiMsXZf/NprLT7+Cq+stsNXPl2Y3vJCrvlIX9c9HYGpfrDLTltyiFxS028lkNqeb5kY0ygvfeMMSf09xyadKuiqIh8C5hEi/xsTZm7laKxAl75sU0DBnbf3ffftBvL+4O7Gt7+LSx7zB4nOeC85+HD++GrN23bsBnwnac736+nVCzZ4NcyFdoo4BFuyaE3gmDrckjArpZD+gi42BizOnz8HvBT7H7wv2CzuiQDtxhjXhaRC4FvYeNEhoic1945RORrYKYxplxEvgf8DHtXcaUx5vxwmrbHgCKgDLjIGBOVH1VEpgEPhn9268PzrArPcRE2f/UrwJ2d/Yy6sxH+QWxKnB9j97qchf0XpZRVX9Yc/MBmXnnj59BY1T/z8dY2Bz+wSa1fux4mt3isXbIcStf0/dyUatZZOaTeciDwC2PMxFbtTeWQpgFTgfaqQTwFnA0QDpjDjDHLsCnM3jHGHAAcCfwpnE8U4BDgAmPMUV2dI1xv8BfAUcaYqcBV4bfuw6ZdmwI8AdzbztzmAjeE+6zCBvomucaYI4wxnQY/6N4q0EONMd8Dqowxvw5/wRHd+JxKFJ52buXXl9ryQP2hvTycDWVtE03X7WzbT6m+09/lkC4KP1fbzxhT106fZ7AXPGAD4bPhP38TuDGcz/M97BVf05zfNMZUdvMcRwHPGWPKAVp87hCa6yH+E5uGMyJcQzDXGPN+uOlxbGWLJt2uVtSdANiU5LAxnMnbD4zp7glUAsgb0zaLycyLwdVPmU3SciFvdHTbtPPgq7ebj1NcMHo2SvWjgV4OaRtQISJTsHcBnwp/XLDV4ZtKIo00xnze+nztnaPVNAR767Mru1oZvt3v3J7uBMB5IpIL/AlYDnxN8w9CKZs/89J3YPJZNp/lSffClHP6r8Br5iC48L8w43tQvD9881Y49Ccw9kgYeQhM+BZc+l503k+l+t7PseWPWurLckilxphHgEex5ZBebBHUmvJNPgVcD+QYY1aF294AfizhirgiMr2752jV5W3gbBEpCPfPD7cvAs4N//k84IOWHzLG1ABVInJYuOl84H12Q3cWwdxujPECz4vIPOzlbie1XlTCSUqyV1wn32Of/6Xl2rb+lDMcjvsjBNzgzLXBeN8TYdQsm++zo+rvSvWVW2qe5JYc6INVoO2YA1wnIn6gHmh9ddbkOWyppN+2aPstcDe29JFgL4pO3NVzGGNWi8itwPsiEgQ+wdYH/Am2Wv11hBfBtDP2BcCDIpIObOigT5e6kwx7uTFmRldtfUWTYSulBiBNhh2HOssEMwQoBlzhS9ymf8HZtF25pJRSSsWVzm6BHou9HB0O3NWivZY+uEetlFJKxVJnmWAeBx4XkTOMMc/34ZyUUkqpmOvOSoWFIvKoiLwGICITReT7MZ6XShQBr02hVrejiz6bO+8TDNi9h40Vu3Z+T43dD+gP7/YJBe04Dbs4jlIq7nQnAP4du+x1WPj4C+Dqrj4kIo+JSKmIfNbB+yIi94rIVyKyUkT6ZVGN6kd1O2HRvTD3ZHjuIti2HLwNbft8eD/881R49gLYusRWXm+psRKWPAJ/Px7+/W3Yugz8XSxUNgaqvoYXLoNHj4G3fwN122H5XPjHt+DJs2Dzh+BrvUpdKbWn6E4ALDTGPAOEAMLJRYPd+Nw/gOM6ef94YK/w6zLgr90YU+0pAn5Y+Qy88zuo3ACbFtkA5q5s7hMMwpqX4O1fQ8V62LzYBqeWfYyBda/B6zfaxNdbPoK/HwuN5Z2fv74U/n4CfPE6VG+CbUth/Xsw72oo/wK2LbPnaiiNyddXSvW/7gTAhvBGRQMgIgcDNV19KJwFoLKTLqdg870ZY8xiIDecb04lgoZS+Oy56LaAB7a3SBfYsBNWPdOqjxe2fNx87K6GFf+K7hP022DZGV8d1G5rPh59eNv5hIKw/t3Ox1FqgBGR+k7eW9QL458sIjfuxue6PLeI/E1EWuctjZnubIS/FptVe5yILMRm6D6zF85dDGxpcbw13La9dUcRuQx7lcjIkb2ZJk/1m5Q0yBkRHfDAbmCP9HFB7kjY2mrfZ26LXOzJTpuKbdOijvu0e/50kCQwIXvcUGrP1Vr+2M7HUaoH9nt8vzblkFZdsGpAl0MyxryCjQmtz9Fp+aHunNsYc0kPp7dLurwCNMYsB44ADgV+AEwyxqzshXO3t3G03V35xpiHjTEzjTEzi4o0fdUeIb0Ajr4Z0vOb2/Y9GbKGNB+78uDI/4OMwua2Cd+KDpKp6TDnxuiyRnsfB/ldpKtNzYQ5LXbzrP2vTZeWXdzcNuYIGNxnv4yqBBMOfo9gq+tI+J+PhNt7TETmiMi7IvIktmJC5OpQRIaKyPxw3s/PWqQVa/n5j8IVG5qO3xOR/UXkQhG5L9z2DxG5S0TeBf4oIkUi8qaILBeRh0Rkk4gUtjr3nPBYz4nIWhF5okVatfdEZGb4z8eFx/lURN4Otx0oIotE5JPwPyf05GfU5RWgiKQBV2AzchtggYg8aIzpaTq0rURXlRgOlPRwTBVP8sbADz6wi1HS8221hqxWd8FzR8Fl8+1zOleuDYqt++SMsOPUbLEpztILIaOLRNxp2XDgpTDlLKjdblO5ufLh0nehZqsNrBlF0cFXqd7VWTmk3roKPBCY3E5FiKZSRbeKiKOdeUBzOaSbW5ZDEpH9WvXbGzjaGBMMB8Z3jDG/F5HjCN+5a8d0bI3ZEmAhtn5fJOeniBRhfzk43BizsUWe0LXhtoCIHI39WZ3RnR9Ee7pzC3QuUIctgAjwbWyJirM6/ET3vAJcKSJPAQcBNcaYNrc/1R7MkQw5xfbVkz4ikDXYvnaFK9e+WlaOSNmNcZTaPf1dDukxEUkBXjLGtFcP8BngTWytvZblkFp71hjTtDByNnAagDHmdRHpqCjox8aYrQDhskqjiU56fTAwv2nuLUol5WD3p++FvSBL6WD8bulOAJwQLlbY5F0R+bSrD4nIv7HJUAtFZCv2h5gCYIx5EHgVOAH4CpsBfbeSmSqlVJzaTPvFxfukHJKIHI6t4P5PEfkT9kKnqbDsJcaYpSLSshzSD7pxju7mRPW2+HOQtrGoo1JJvwXeNcacJiKjsfUId1t3AuAnInJweKUmInIQ9pK1U8aYb3fxvgF+1K1ZKqXUnufn2Nt8LW8/9mU5pG3GmEfEVnOfYYy5GnixVdf2yiF15gPs1eIfReSbQN5uTvFD4H4RGdN0CzR8FZiDrS8INlVnj3RnG8RBwCIR+VpEvg5P7AgRWSUivbEYRimlEk54teelwCbs1c4m4NJYrAJtxxxghYh8gn2Gdk8H/Z7D1uZ7poP3W/s18E0RWY7d670de2W5S4wxZdjnhy+E7zg2VXm/Hfh9eEdCjwuOdqccUqfryY0xm3o6iV2h5ZCUUgOQlkMCRMQJBMOLVA4B/mqMmdbf8+pIl7dA+zrAKaWUilsjgWdEJAnwYa9wB6zuPANUSimlumSM+RK7xSEudOcZoFJKKbXH0QColFIqIWkAVEoplZA0ACqllEpIGgCVUkolJA2ASimlEpIGQKWUUglJA6BSSqmEpAFQKaVUQtIAqJRSKiFpAFRKKZWQNAAqpZRKSBoAlVJKJSQNgEoppRKSBkCllFIJSQOgUkqphKQBUCmlVELSAKiUUiohaQBUSimVkDQAKqWUSkgaAJVSSiUkDYBKKaUSkgZApZRSCUkDoFJKqYSkAVAppVRC0gColFIqIWkAVEoplZA0ACqllEpIGgCVUkolJA2ASimlEpIGQKWUUglJA6BSSqmEpAGwl/kDITz+YH9PQymlVBeS+3sCewp/MMT2Gg8Pvr+eWrefyw4fy9jCTDLT9EeslFIDkf7t3EvK670cf/d8Gnz26u+/q7bz0o9mMXV4bj/PTCmlVHv0FmgveX9dWST4ARgDD72/HrfeDlVKqQFJA2AvyXC2vZjOcqbgkH6YjFJKqS5pAOwlB48tYHieK3LsSnFwxZHjSE129OOslFJKdUSfAfaSoiwnL15xKIvWV1DnCXDUPoMoykrt72kppZTqgAbAXlSUlcYp04r7expKKaW6QW+BKqWUSkgaAJVSSiUkDYBKKaUSkgZApZRSCSmmAVBEjhORdSLylYjc2M77F4pImYisCL8uieV8+poxhrI6D5srG9lZ68EX0E3xSik1UMRsFaiIOID7gWOArcASEXnFGLOmVdenjTFXxmoe/WljeQPfe+xjtla5yU5L5p5zp3PIuALSUnRvoFJK9bdYXgEeCHxljNlgjPEBTwGnxPB8A0plg5drnl7B1io3ALWeAFc8sZwat7+fZ6aUUgpiGwCLgS0tjreG21o7Q0RWishzIjKivYFE5DIRWSoiS8vKymIx114XDBlWbauJanP7gzR4A/00I6WUUi3FMgC2lwXTtDr+DzDaGDMFeAt4vL2BjDEPG2NmGmNmFhUV9fI0YyPFkcRBYwqi2rJdyWS2kzNUKaVU34tlANwKtLyiGw6UtOxgjKkwxnjDh48A+8dwPn0qNz2VO86ayvQRthzS8DwXj190IHkZKf08M6WUUhDbVGhLgL1EZAywDTgX+E7LDiIy1BizPXx4MvB5DOfT54rzXDx64QH4AkEcSUJBhpOkJC0PoZRSA0HMAqAxJiAiVwJvAA7gMWPMahH5DbDUGPMK8BMRORkIAJXAhbGaT3/Jz9CE2EopNRCJMa0fyw1sM2fONEuXLu3vaSilVEt6aycOaSYYpZRSCUkDoFJKqYSkAVAppVRC0k1p7ahq8LFqWw2vfbadQ8cVcui4AgoynVF9dtR4+Hx7LW+s3sGMUXkcsXcRGHh22RaqG/2cd/Aosl3JrNtRx7xPtzNzdB6H711EYatxqht9rNtRx8srSpgxKpc5Ewa16aOUUqr36SKYVtz+IH99bz33vv1lpO34yUP4/en7kZtuV3TWNvp4bNHX3P1Wc59v7DuIb04czA3PrwIgOUl4/vJD+eG/lrG9xgPA4XsXcfc50yIrQ/3BEE8s3sQt/2lOj3rI2HzuP29/XT2qVHzRRTBxSG+BtlLn9vPI/A1Rba99toNGX3Mlh3pfsE2ftz8vZa9BWZHjQMjw6AcbOW7ykEjb/C/KaPQ1p0KravBx37tfRY3z4YZKTZemlFJ9QAPgbjAGRNr+wtdOE3F2ga2UUglDA2Ar2a4UfnDE2Ki2E/cbSnpqcwmjLKeDyw6P7nPMxMGs21kXOU5OEi45bAyvf7Yj0jZnQhHpqc2PXfMyUvnJUXtFjTNrXAEZTi2XpJRSsabPANtR1eBjzfbayCKYg8bkt7MIxs26nXX8b/VOZozKY/b4QjDwwifbqHH7OPeAkWS7kvlyZz3zVpZwwOh8Zo0vbDNOdaOPL0vreWXFNmaMtAtlWvdRSg14+gwwDmkAVEqpntMAGIf0FqhSSqmEpAFQKaVUQtIAqJRSKiFpAFRKKZWQNBVaB7bXuAmGDEkiDMl2UucJUB/eDO90CIVZaVTUe/EFQyR3Uuy2ssGLN9B5H6WUUn1PA2A7NpTVc+WTn7Bmey0j89N5/KID+O+q7dz/7nr8wRCnzyjmmmP25tK5S/lsWy3D81zc953pTBqWQ4qj+aJ6W1Ujlz+xnJVbayjOdXHvt6cxZXgOKQ7d56eUUv1Nb4G2sr3GzVVPrWDN9loAfIEQX1c0csf/vsDtDxIIGZ5ZupXXVu0gOy0FgK1Vbi78+xKqGnyRcaobffz02U9ZubUGgG3Vbi76+xKqGvx9/6WUUkq1oQGwlZCBVdtqIsd7D85kydeVbfot3lDBvkOzI8fVjf6ofKH+YIiPN0Z/rtYToF7zfCql1ICgAbCVJGzQa7KhvIEpw3Pb9Js2Mpf1ZfWR40xnMq4W6dIcSdLmc+mpDjKcetdZKaUGAg2ArQzNdXHPudMpznUB0OANMH5QBhceOhpHeAHLN/YZxGnTitkQDoC56Sk8+N0Z5KWnRMbJz3Dy53OmMTI/HYBsVzIPnDeD3BZ9lFJK9R9NhdaOQCBEWb0XTyCEMzmJ/IxU6j1+vAFDyBhSHMLg7DTK6314/EGcyUnkZaS0WdxijKG83ovHb8fJTU8hNVkXwCi1B9Ll3XFI78e1Izk5iaHhK8AmaSltA1dRVudJq0WEoqy0Xp2bUkqp3qG3QJVSSiUkDYBKKaUSkgZApZRSCUkDoFJKqYS0Ry+CMcZQWuflvXWl+IOGo/cdTFFmKo4W6cpCoRA767y883kpnkCQ4yYNwSFJ7Kh1894XZew/Ko9JQ7PxBkO8s7aUBm+Q4ycPIS0lie01Ht5ZW8r0EblMGJKNwfDeujLqPH6OnzyUTGcSO2p9vLeulCnDc9lnSFabau+1bn94nJ1MHpbDxGHZWhFeKaX6wB69DWJnrYcT7/2AsnovANlpybx21eEU5zWv8Nxe7eaU+xdSWmf7PPn9g/hsew23vboWgOQkeP/6Iznzrx+yvcYD2A3tL10xi7Mf/pDqRpva7Mz9hzNteA7/9/JqAFwpDub9eDZ/eP1z3lxTCsAp04bx65MnkZueCkAwGOI/K7dz9dMrIvM5nZ3MtwAADFNJREFUYfIQbjt9v0gfpVRc0G0QcWiPvgX62qrtkeAHNhXZPxdvIhRqDvpvrNkRCX4AIwvT+cvbX0WOz5o5kvfWlkWCH0CjL8hjCzdy3KQhkbbnlm1lUnFO5NjtD/LA+19xw3H7RNpeXlFCQ4t0aZWNPv7w2tqoOb/62Q4avEGUUkrF1h4dANvLu1nvjU5G3TrYCOANhCLHGU4HDb6247j9QVKTo398oVYX025fMJI9JtKnRSdj7DitBUKhNm1KKaV61x4dAE+dVoyzRZByJAkXHjo6qibfyVOH4Wqxyb2iwcc5B4yIHD+9ZAvHTR5Ceos8n0kC5x88irc/L420HTQmn00VDZFjEbj0sLE8+dGmSNv0kblR4+Skp3DJ7DFRc96vOIdMzReqlFIxt0c/A/QFgpRUe3ho/gZ8gSA/OGIcI/JduFKaA0yj18+OWi8Pz9+A1x/iB4eNIS/LybtrS3lnbSkzRuVx1v7F1HqCPDJ/Aw2+AN+fPYbB2WnM/6KMtz7fyZThuZy5/3C8gSAPvbeBOm+Ai2ePZki2iyVfVzJvZQnT/7+9O4+xq6zDOP59Zu8MXaCtWimkClgECwUqLghhUSOKWywEgoBLonGF8AcgRnGJBqKiRFFTSxUVlFqsIhHEBdQq1pZaWrCgpaItoG3p3k5n/fnHfQfuTGfm3mnn9tw75/kkk7n33Pe85+mkk9+c95zzvkceytyTpzNlwOwxW3d38uC6Z/n5yqc4cfokzp9zRMkZZsys6vgaYA0a0wWwT1dPLxHsM2RZbG9nN70ErU3PT1a9dXcHhzQ10JjOEHfv7aInggnjmoZss6ejm+7e3n5t2jsLw6UDh0OLldPGzKqWf3FrUC7G2opXaR9KS9O+P4pD2/qfibW17LuSw8A2rYMMXxYvkzSUctqYmdnoGdPXAM3MzIbiAmhmZrnkAmhmZrmUi2uAA23b00lXTy8NdXUc2lbejCvdPb1s3dNFEExsaaS5sZ7/bt9LbwQR9JtdxszMql/uCuDT29q5atHDLP3XFmYdPpGvXDCbGZNbkYa+iWt7exe/XfM/rr/nMfZ29XDdW4/n1S+dzCcXr+ZPT2zm2BdN4EtzT2DG5NZBb6YxM7Pqk6sh0C27O/jo7StYsvZZunqCFf/ZxqULlrJ5V+ew+z2zrZ0rFz7Mxp0d7NjbzYlHTOTqO1fxwD820dUTrH5qO5cu+Ctb9nQN24+ZmVWPXBXAzu5C0Su2fks7eweZjqzY/Y9v7Pd+XFMDS9Zu7rdt484O2js9h6eZWa3IVQGsr4PpA67VTWhpGPYBeYBZh0/q976rp5ejph7Sb1tLY52f5TMzqyG5KoCT25q56cLZjE8Pq7c01nHjBbOZNG7fB9yLvXzaeM47Ydpz7x9cu5kvn38CE9N+zQ11fPGds2huzNWP08yspuViKrRind09bNvTxa6ObtqaGpjY2khLY+kzt217Otnd0UNPby9tzQ001dexo6Ob3amf5gYxZXzLfucys5rmqdBqUO5uWWxqqOcFE+p5wQj3m9TaxKTW/tvGlzhzNDOz6uUxOzMzyyUXQDMzyyUXQDMzyyUXQDMzy6WKFkBJb5L0uKS1kq4Z5PNmSXekz5dKmlHJPGZmZn0qVgAl1QM3A+cCxwEXSTpuQLP3A1sj4mjgq8ANlcpjZmZWrJJngKcCayNiXUR0Aj8G3j6gzduBW9PrRcA5Gm5WajMzs1FSyQJ4OLC+6P2GtG3QNhHRDWwHJg/sSNIHJC2XtHzTpk0VimtmZnlSyQI42JncwGlnymlDRMyLiDkRMWfq1KmjEs7MzPKtkgVwA3BE0fvpwNNDtZHUAEwEtlQwk5mZGVDZArgMOEbSSyQ1ARcCdw1ocxdwWXo9F/hd1NrkpGZmVpMqOhm2pDcDXwPqgQUR8QVJnwOWR8RdklqAHwAnUTjzuzAi1pXocxPw7/2IMwXYXLJVdXHmg8OZD55azF1O5s0R8aaDEcZGT82tBrG/JC2PiDlZ5xgJZz44nPngqcXctZjZyuOZYMzMLJdcAM3MLJfyVADnZR1gPzjzweHMB08t5q7FzFaG3FwDNDMzK5anM0AzM7PnuACamVkujfkCKGmBpI2SHsk6S7kkHSHpfklrJD0q6fKsM5UiqUXSXyU9nDJ/NutM5ZJUL+lvku7OOks5JD0pabWklZKWZ52nHJImSVok6bH0//o1WWcajqSZ6efb97VD0hVZ57LRNeavAUo6A9gFfD8iXpF1nnJImgZMi4gVksYDDwHviIi/ZxxtSGkVj7aI2CWpEVgCXB4Rf8k4WkmSrgTmABMi4rys85Qi6UlgTkTUzAPlkm4F/hgR89PMUK0RsS3rXOVIS7s9BbwqIvZnEg6rUmP+DDAi/kCNzS8aEc9ExIr0eiewhn1X0qgqUbArvW1MX1X/15Wk6cBbgPlZZxmrJE0AzgBuAYiIzlopfsk5wBMufmPPmC+AtU7SDApTxS3NNklpaShxJbAR+HVEVH1mClP1XQX0Zh1kBAK4T9JDkj6QdZgyvBTYBHw3DTXPl9SWdagRuBD4UdYhbPS5AFYxSYcAdwJXRMSOrPOUEhE9ETGbwsofp0qq6iFnSecBGyPioayzjNBpEXEycC7wkTTMX80agJOBb0XEScBu4JpsI5UnDde+DfhJ1lls9LkAVql0He1O4LaI+GnWeUYiDW89AFT75MCnAW9L19R+DJwt6YfZRiotIp5O3zcCi4FTs01U0gZgQ9GIwCIKBbEWnAusiIj/ZR3ERp8LYBVKN5TcAqyJiBuzzlMOSVMlTUqvxwGvBx7LNtXwIuITETE9ImZQGOb6XUS8O+NYw5LUlm6MIg0jvhGo6jucI+K/wHpJM9Omc4CqvaFrgIvw8OeY1ZB1gEqT9CPgTGCKpA3AdRFxS7apSjoNuARYna6pAVwbEb/MMFMp04Bb0x1zdcDCiKiJxwpqzAuBxYW/kWgAbo+Ie7ONVJaPAbelIcV1wHszzlOSpFbgDcAHs85ilTHmH4MwMzMbjIdAzcwsl1wAzcwsl1wAzcwsl1wAzcwsl1wAzcwsl1wALXOS3iPpxWW0+56kueVuH4Vc1xa9nlHuiiKSrpB06Sgc/6OSqv5xAbNa5QJo1eA9QMkCmIFrSzfpT1ID8D7g9lE4/gLg46PQj5kNwgXQRlU6U3pM0q2SVqU14FrTZ6dI+n2axPlXkqalM7c5FB6SXilpnKRPS1om6RFJ89LMOOUef59jpO0PSLohrVn4D0mnp+2tkhamrHdIWippjqTrgXEp022p+3pJ30nrHd6XZrwZ6GwKU2d1p/6PlvQbFdZJXCHpKElnpowLU5brJV2csq2WdBRAROwBnpRU7VOdmdUkF0CrhJnAvIg4AdgBfDjNbfp1YG5EnELh7OYLEbEIWA5cHBGzI6Id+EZEvDKt3zgOKGuNvqGOUdSkISJOBa4ArkvbPgxsTVk/D5wCEBHXAO0p08Wp7THAzRFxPLANeNcgMU6jsH5jn9vSPicCrwWeSdtPBC4HZlGY9edlKdt8CrOm9FkOnF7Ov9/MRmbMT4VmmVgfEX9Kr39IYRjvXuAVwK/TCV09zxeDgc6SdBXQChwGPAr8oozjzixxjL5JxR8CZqTXrwNuAoiIRyStGqb/f0VE39R0xX0Um0Zh/UbSnJ2HR8Ti1P/etB1gWUQ8k94/AdyX9l8NnFXU30bg2GEymdl+cgG0Shg4v14AAh6NiNcMt6OkFuCbFFY8Xy/pM0BLmcctdYyO9L2H5//vlz28WrR/Xx+DDYG283ze4fou7qu36H0v/X8vW1KfZjbKPARqlXCkpL4idBGwBHgcmNq3XVKjpONTm53A+PS6r3hsTushjuTuzuGOMZQlwAWp/XEUhiT7dKVh1ZFYAxwNkNZw3CDpHan/5r7roSPwMqp8tQezWuUCaJWwBrgsDSceRmEh1E4KxewGSQ8DKylcEwP4HvDttPJFB/AdCkOBPwOWlXvQEscYyjcpFM1VwNXAKmB7+mwesKroJphy3AMUL1B7CfDx1P+fgReNoC8oXFP8zQj3MbMyeDUIG1WSZgB3pxtYql5avqkxIvamuy9/S+GGlM4D6HMxcFVE/PMAs50EXBkRlxxIP2Y2OF8DtLxrBe5PQ50CPnQgxS+5hsLNMAdUAIEpwKcOsA8zG4LPAM3MLJd8DdDMzHLJBdDMzHLJBdDMzHLJBdDMzHLJBdDMzHLp/5zDIsobDlNYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 463.5x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.relplot(x='petal length (cm)', y='petal width (cm)', hue='class', data=dataset)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal of our network is to determine which class a data point belongs to.\n",
    "\n",
    "Given set of petal length and petal width, the network would output the class of the flower."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One way to see it is for the network to decide on some boundary for classification.\n",
    "\n",
    "One possible decision boundary may look like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcAAAAFgCAYAAAAsOamdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOzdd3jUVdbA8e/NlGSSEJKQANKLCFKkBZAiglhYcEUExYa9vqKi666KrisWbGuXXQs2QEVFBAXFRRRsSO8IKNJrQnqZlJn7/nEnfZIMkMnMkPN5njxkfvMrd9Bwcts5SmuNEEIIUd+EBboBQgghRCBIABRCCFEvSQAUQghRL0kAFEIIUS9JABRCCFEvWQPdgGM1fPhwvXDhwkA3QwghylKBboA4diHXA0xJSQl0E4QQQpwEQi4ACiGEELVBAqAQQoh6SQKgEEKIekkCoBBCiHpJAqAQQoh6SQKgEEKIekkCoBBCiHpJAqAQQoh6SQKgEEKIekkCoBBCiHrJb7lAlVItgelAU8ANvKm1frnCOUOAecBOz6E5WuvH/NUmIYR/ZRVkkVeUh0IRY48h3Bpe5blaa446j1LkLsIWZiM2PJY0Zxou7cJusRMXEVeHLRf1kT+TYRcBf9Nar1FKNQBWK6UWaa23VDjvR631hX5shxCiDqQ6U3l6xdN8s+sbIiwR3NnzTi5qfxEx4TGVznW5XfyR/gf3LLmHvVl7uabzNQxoNoCHf36YlLwUuid2599n/5umUU0D8ElEfeG3IVCt9UGt9RrP91nAb0Bzfz1P1LL774fvvw90K0SIcLldfLnjS77e+TVu7Sa3KJdnVj7DodxDXs9PzU/ljsV3sDdrLwDntz6fu7+/m5Q8k+x+ffJ6piyfQlZBVp19BlH/1MkcoFKqDdATWO7l7f5KqfVKqa+VUl3qoj2iBl9+Cc8+C+edBy++CFoHukUiyOUW5fLT/p8qHV93ZJ3X8wuKCjicexgAe5idnKIc8l355c5ZfXg1ziJn7TdWCA+/B0ClVDTwGTBRa51Z4e01QGutdXfgVWBuFfe4RSm1Sim1Kjk52b8NFjBiBDz4ILhccO+9cPXVkJsb6FaJIOawOujTtE+l490Sunk9326x0yiiEQAF7gKibFFYw8rPyHRN6Eq4peo5RCFOlF8DoFLKhgl+H2it51R8X2udqbXO9nz/FWBTSiV4Oe9NrXWS1jopMTHRn00WABYLTJkCs2dDVBR8+CEMHAg7d9Z8raiXrGFWxnYYy6DmgwCwhdm4rfttnBJ1itfz4yPieeWcV0hwmB/3RbsW8cxZz9DA1gCA9rHteaT/I17nD4WoLUr7aXhLKaWA94FUrfXEKs5pChzWWmulVF9gNqZHWGWjkpKS9KpVq/zSZuHF5s0wejT8/jvEx8OsWWZoVAgv0p3p5LnysCgL0bZoIm2RVZ7rcrtIdaZS4Cog3BJOtD2azIJMCt2FRFgiaORoVIctP2FSET4E+TMADgJ+BDZitkEATAJaAWitX1dKTQBux6wYzQPu1Vr/Ut19JQAGQHq6GQZdsADCwuDpp+G++0DJz7wQHvLDEIL8FgD9RQJggLjdMHkyPObZpnnZZfDOO2aIVAghATAESSYY4ZuwMBMA586FBg3gk0+gf3/YsSPQLRNCiOMiAVAcm1GjYMUK6NgRNm6EpCRYuDDQrRJCiGMmAVAcu06dTBAcNcrMD44YYVaNhthwen2VW5hLcm4yac60E75Xviufg9kHOZh9sNI+PiGCnQRAcXxiYmDOnNI5wYcegrFjIUsydwSzo3lHmbJ8CqPmjuL2b29nW+o2ilxFx3WvlNwUPv/9c65ZeA3jvx7PnO1zSMlNqeUWC+E/EgDF8QsLg3/+02SOadjQBMQzz4Tt2wPdMuFFXmEeL695mXk75pFVmMXmo5u5fuH1pOUfX09wV+Yunlz+JIdyDnE49zBTVkzhz4w/a7nVQviPBEBx4kaOhJUroXNn2LIF+vSB+fMD3SpRQXZhNt/vLZ/fNaswi1Rn6nHdb+HOynO/X+/8+rjuJUQgSAAUtaNDB/j1VxgzBjIz4a9/NcOjbnfN14o6YQuz0SamTbljCkXD8IbHdb+OjTpWOtapUafjupcQgSABUNSeBg3g00/Nghil4F//MllkMjIC3TIBxEbE8uiAR4kNjwUgTIUxsddEom3Rx3W/s1ucTc/GPUted0/sztCWQ2ulrULUBdkIL/xj4UK44gqzSrRjR7N/sJP0DgKtyF1EmjONjIIMom3RRNmiaGBvcNz3O5J7hJzCHAAirZE0iWpSW00NNbIRPgRJABT+s2OH6QFu3Gh6h9Onw8UXB7pVQviDBMAQJEOgwn/at4dly2DcOLM9YvRoeOQRmRcUQgQFCYDCv6Ki4KOP4LnnzLaJxx83C2TS0wPdMiFEPScBUPifUqZ6xDffmJJKX31ltkps3hzolgkh6jEJgKLunHsurFoFPXrAH39Av36m6K6oU8ULYXILcwEocBWQ6kzFWeQEwFnkJDUvlUJ3IQA5hTmkOlNxuV0+3T/NmUZWgfeMQIXuQlKdqeQV5Xl9P7cwl1RnKkXu48tOcyKO9XOK0Gd59NFHA92GY/Lmm28+essttwS6GeJ4xcXBNdfA7t2werXZNpGfD0OHmiFS4VepzlQ+2fYJL65+kd/Tfuf0+NN5Z+M7/Gfdf9iZsZPT4k7j3U3v8p/1/8GmbETZo3hmxTN8+NuHFLoLadmgJQ6rw+u9M/IzWLp3KU+teIql+5bSIa4DDWwNsIZZS579/qb3eXXtq2xP207n+M5E20u3YBzIPsCzq55l+pbp5BTm0LZh2yqfVZtcbhf7svYxZfkUZm2dhRs3zaObH+uzJ/urfcJ/rIFugKiHIiNhxgxTSeK++0yB3bVr4cMPzRCp8Iv8onymb57O25veBmB4m+E8sfwJluxdAsC2tG1sObqFsaeN5Z3N79A2ti3jvxpfkiptS+oWnC4n13a5tiSolbXuyDru++G+ktcrD61k/uj5nBJ9CjmFObyw6gXm7ZhX8qz1yet5+/y3aeRoREpeCtcuvJZDOYcA2Jq6lYyCDO7ocQfhlnB//rWQ6kzligVXkFmQaT7n8i0Uugu5stOVWMIsfn22CCz5lVsEhlIwcSIsWgQJCWZ+MCkJNmwIdMtOWpkFmcz9Y27J6+6Nu7N079Jy52xI2UDrmNbEhseSkZ9RKU/oZ79/Rnp+5QVMOQU5zNo2q9yxAncByw4uAyCvKI8FOxeUe39H+g5yi8wwbEZ+RknwKzbvj3lk5mce46c8dnuy9pQEv2Kzt8/2+jnFyUUCoAisoUPNUGjv3rBzpymyO2tWzdeJY2YJs5DgSCh5nVeUR0x4TLlz7GF2NBpnkZMYe0zFW5DoSMSqKvf+bBYbTSObVjreNKr0WKOIRuXboyzYw+wAXocbExwJWJT/e2DFmXEqPtsWZvP7s0VgSQAUgdeqFfz4I1x7LeTmmgwyf/87FNX9QoiTWXxEPA+d+VDJ8OXcP+Zyb+97y51ze/fb+d+u/+F0OdmTuYcL2lxQ8l64JZz7+95PbETlgGG32Ln5jJvLBZNuCd3oGGfyhcaFx/HPM/9JmCr9J+embjcRZYsCINoWzWWnXVbynjXMyqR+k4h3+H9IvFFEI4a1GlbyOsISwT/6/KPSLwfi5COZYETw0Br+8x8zNFpUBMOGmd5gQkLN1wqfOIucpOenszV1K82jmxMXHke+K59tadtoH9ueaGs0mYWZ7MrYRaf4TtgsNo7mHeVQziE6xXciLiIOu8Xu9d5u7eZo3lG2p22ngb0BLaJblAtguYW5pOensy11G60btqZRRKNyibjTnekk5yWzP3u/eVZ4HOFW/87/FUtzpnE49zBHco/QMa4j8RHx2CzH1AOUTDAhSAKgCD4//miK6x45Aq1bw+efQ8+eNV8nROBIAAxBMgQqgs9ZZ5l5wb59zXaJAQNg5sxAt0oIcZKRACiCU4sW8MMPcOON4HTC+PFwzz1QWBjolgkhThISAEXwCg+Ht96C118Hmw1eegnOP98MjQohxAmSACiCm1Jw662wZAk0bWr+TEoyKdVEUCh0F5akTfMm35UfFOnFtDbbO0Jt3YPwH8kEI0LDgAFmXnDsWFNiadAg0zO87rpAt6zeyi/K52DOQd7d/C4A13W5jmZRzUpWbmbkZ7Dl6BZmb59Nu4btuLTjpTSObByQtqY6U1m6dylL9y1lQLMBnNv6XOIjJOtQfSerQEVoKSiAu+82wQ/gjjvghRfA7n1pvvCfvVl7GTV3VEnvzxZmY96oebSMaYlbu1nw5wIm/TSp5PwW0S2YMWJGuc34dSG7IJtnVjzD3B2lWXDOb30+/+r/r9rc6yerQEOQDIGK0GK3w3//a+YG7XaYOtXsFzx0qOZrRa36bPtn5YY+C92FfLL9E8Dsq3tn0zvlzt+XvY/DOYfrtI0AuUW5fPnnl+WOLdq9qCQNm6i/JACK0HTTTWaVaPPm8NNPJpXa8uWBblW9EmmLrHSsOLNLmArzmsQ6UOnFKibvtigLSjpt9Z4EQBG6+vUz84JnnQUHDsDgwTBtWqBbVW+Maj+qXCaXGHsMF596MQBxEXHc0/ueckGme0L3Oh/+BGhgb8CNXW8sd+zqzleXK8Uk6ieZAxShr7AQ/vY3ePVV8/rWW+Hll802CuE3LreLo86jLNm7BK01Q1oOMQmsPSWEcgpzSM5NZvGexbSJaUOPxj1o5GhUw139I92Zzu7M3Sw/uJykpkm0bdiWuIi42nyEdCdDkARAcfJ4/30T/PLzTVWJ2bOhWbNAt0rUDxIAQ5AMgYqTx7XXmvnAli3NVoneveHnnwPdKiFEkJIAKE4uSUlmXnDIELMydOhQs2UixEY6hBD+JwFQnHwSE02l+eLcobffDjffbHKKCiGEhwRAcXKyWs0G+ZkzweGAt9+Gs8+GffsC3TIhRJCQVGji5HbVVdC5M4weDStWmHnBTz81WyZEjY7mHWX14dX8mf4n57U5jxh7DD/t/4lUZyoj241Eofh2z7cADGs1DKUV83fOJy4ijsHNB5MQWbrtISM/g71Ze1m6dyndE7vTJaFLba/ELJFVkMWB7AMs3rOY0+NPp3ti9zqpLi9Ci6wCFfVDSgpcfjksXlzaO5wwwSTbFl6lOlO5c/GdbEjZAIBC8czgZ/ho60f8dvQ33h3+LrcuupXMgkzA7AN847w3uH7h9ThdTppFNeODkR+Q4Eig0F3I7O2zmbJ8Ssn9R7QZwaQzJ5XbS1gbXG4XC3ct5IEfHyg5NqjZIKacNcVvARdZBRqSZAhU1A8JCbBwIfz971BUBHfdZRJp5+UFumVBKyUvpST4AWg07256l5HtRtK/WX++3f1tSfADyCzIZNHuRQxsPhCAAzkHWHdkHWB6f6+tfa3c/b/a9RW5hbWfjiwtP41X1rxS7thPB34ipzCn1p8lQpsEQFF/WK3w7LMwaxZERsL06aaqxJ49gW5ZUCpyF1U6VuAuwKqsWMOsXksgFbgKyqUdy3flm280Xs934669BldoZ6Vnaf88S4QuCYCi/hk3zuwTbNcO1qwx84Lffx/oVgWdJpFNaB3TutyxyztezuI9i1l2YBnntz4fe1hpFQ57mJ0L2lzAL/t/AaBheEP6Nu0LQLQ9mvGdx5e7V9+mfYm0Vs4neqJiw2O5vsv15Y51ju9MtE1Sn4nyZA5Q1F+pqXDllfDNN2CxwHPPwcSJMi9YRnJuMnP/mMvv6b8ztsNYWse05oOtH3A07yg3dbsJhWLmbzPRWpcEuLc2vkWjiEZcefqVJDoSS1KjpTnTWHN4DV/v+ppejXtxQZsL/JYaLd2ZzsaUjXyx4wu6JnRlZLuR/s5DKv/ThCAJgKJ+c7ngn/+Ep54yr6+80pRaiqz9nkmocms3LrcLm8VUcnC73bhxlwx1Fg+Vln2tUCWBr6ICVwG2MBuqDn7RqMNnSQAMQTIEKuo3iwWmTDF5Q6Oi4MMPYeBA2Lkz0C0LGmEqrCT4AYSFhZWb57OGWSu9rir4Adgt9joJfnX9LBF6JAAKATBmjKkn2KEDrFtnUqotWhToVgkh/EgCoBDFunQxm+VHjjTzg8OHm3nBEJsmEEL4RgKgEGXFxsIXX8Ajj4DbDf/4h9lAnyN7yIQ42fhtEYxSqiUwHWgKuIE3tdYvVzhHAS8DI4Bc4Dqt9Zrq7iuLYESdmTcPxo+HrCzo1g0+/xzatw90q6qltSYlL4Vtaduwhdlo37A9Sik2JG8g3hFP86jmFLmL2HR0E82jm5PgSMBZ5GRr6lbax7UnISKBmPCYQH+MGhV/zq2pW4mwRtCuYbtqV5RmFWSR5kzjt9Tf6BjXkfiI+Nr+nDLRGIL8mQu0CPib1nqNUqoBsFoptUhrvaXMOX8BOni++gH/9fwpROCNGmWGRC++GDZuNPOCH31khkaD1JG8I1wx/wqS85IBaBPThscHPs7d39+NRtO7cW+u6XIN9yy5h7+0/QtJTZJ4/NfHS66f2GsiV3S6gkhbcK+CPZx7mMvnX85R51EA2jVsx9sXvO11q0N+UT7/2/U/Hl32aMmxf/T5B2NPG4vD6qirJosg5LchUK31weLenNY6C/gNaF7htFHAdG38CsQqpU7xV5uEOGadOpkgOGoUpKfDiBFm1WgQzgu6tZtPtn1SEvwAdmXuYmPKRno07gHA6iOr0WgaRzZmVPtRvLym3KAMU9dNJbswu07bfayK3EV8+NuHJcEP4M+MP1l5aKXX8zMLMnlu1XPljr2y5hWyCrL82k4R/OpkDlAp1QboCSyv8FZzYG+Z1/uoHCRRSt2ilFqllFqVnJxc8W0h/CsmBubMgcceM68fegjGjjVDo0FEa82h7EOVjh/NO0qMvXS4L82ZRrQtmghrRKUgUOgu9JoCLZi4tZuDOQcrHT+cc7jK8yvmAXW6nLi0yy/tE6HD7wFQKRUNfAZM1FpnVnzbyyWVfrXWWr+ptU7SWiclJib6o5lCVC8szGyY/+KL0oB45pmwfXugW1bCEmbhitOvKH9MWRjcYjBrDpup9UhrJB3iOrArcxdrj6xlaMuh5c7vntg96IcF7RY7V51+VbljVmVlWOthXs93WB2c3eLscsf6NOmDwxLcn1P4n1/rASqlbJjg94HWeo6XU/YBLcu8bgEc8GebhDghF14IK1ea+oJbtkCfPvDBB+Z4EGgT04a3zn+LNze8iS3MxoQeE8h35dMuth2JjkTu7HknO9J30LlRZw7nHGZSv0l0iOvAT/t/omfjntzQ9QZ/lgyqNe1j2/P6ua8zbeM0wi3h3NXrLhId3n85jgmPYfKAycz4bQa/HviVpCZJXNflOmIjYuu41SLY+HMVqALeB1K11hOrOGckMAGzCrQf8IrWum9195VVoCIoZGXB9dfDZ5+Z15Mnw8MPm55iEMjMz0QpRQN7AwDS89OxhdmIskXhcrvILMgkwhqBw+qg0F1IdkE2kdZIwq3hAW75scnIz8CiLETba050XegqJLvQb59TVoGGIH8GwEHAj8BGKKl5MgloBaC1ft0TJF8DhmO2QVyvta42ukkAFEFDa3j6aTMnqDVcdJEpsdSwdgu8ipAgATAESTJsIU7UwoVwxRVmlWjHjjB3rlk9KuoTCYAhKDjGa4QIZcOHw6pVZrP8tm3Qt68JgkKIoCYBUIja0L69KbI7bpyZHxw9ujSdmhAiKPl1FagQ9UpUlMkUk5QE998Pjz8Oq1ebVaKxgVtxmJKXQpozjUhrJA6bgwJXAUfzjhIbEUuEJaJcCrGcghyyC7NJdaaS4EigYXhD7BZ7NXevPVkFWeQU5pCen06jiEbERcSVK7MkRG2TOUAh/OHbb01vMDUVTj3VDIl26VLnzTiYfZBrFl7DoRyzQf6idhfRv1l/HvzpQRSK27rfxriO42jkaEReYR5f7/qax5Y9hku7cFgdvHnem3RP7O73mnrZBdl8vO1jXl7zMhpNjD2Gd4e/y2lxp/n1ubVI5gBDkAyBCuEP555r5gV79IA//oB+/UzR3TqUV5TH1HVTS4IfwBd/fkFseCxx4XFoNK+vfx2nywlAVmEWT/76ZEmGlLyiPCb9NIlUZ6rf25pdmF0S/MCkL5v8y2TSnGl+f7aovyQACuEvbdvCzz/DVVeZckqXXgoPPgiuuknBlV+Uz58Zf1Y6vi97X8mwp0aT7kwvOb/AXVD+3Kx9uLX/5zGzCrJKgl+xXZm7gj4tmwhtEgCF8KfISJgxA158ESwWs2+wuOCunzWwN2BE2xHljlmVldPjT2d35m4AYuwxJcHQYXNwSlT5XPRnNT+LcIv/N8fHhcfRMLz8/slzW51LtK3mDe5CHC8JgEL4m1IwcSIsWgQJCfDNN2ahzIYNfn2sJczCyHYjueWMW0hwJHBa3Gm8ef6bbE/bTrQtmu6J3Xn7grdLUp81imjEtPOncWbTM4kNj+Uvbf/CowMerZP6gHERcbw3/D16JPYgLjyOMR3GcFevu3DYJF+n8B9ZBCNEXdqzBy65xKwOjYyEt982Fef9qMBVQGZ+JmEqjHhHPJn5meQU5mBRFhpHNa50fkZ+BgWuAiJtkUTZovzatorSnekUuguJtkcHfVLuCmQRTAiSAChEXcvLg9tuM2nTAO67D556Cqyy5D+ESQAMQTIEKkRdczjgvffg1VdN0Pv3v002mZSUQLdMiHpFAqAQgaAUTJgA330HjRvD4sVmXnDt2kC3TIh6QwKgEIF01llmPrBvX9i9GwYMgJkzA90qIeoFmXQQItBatIAffoA77jCLYsaPN0Hx2WfBZqvx8tTcFJzuArR2Yw+zEaasON1OFIpwS3i5VGcAR/OO4ixyYrPYaGhveEy18dKcaTiLnLi0C3uYnUhrJNlF2YCpNl9xxWhqXip5rjxsYTZi7DFEWCNK3ityFZGWn0a+K58IawRx4XFYwixVPju7IJucwhw0GofVUWnbhBDHSgKgEMEgPBzeestUmL/zTnjpJVi3Dj7+2AyRVuFIzmHe3PAWs3+fjUu7GNxiMHf0uIMrFlwBwMXtL+aOnnfQONLcY3/2fm7/9nZ2Zuwk0hrJ5AGTObvF2T5tN0jOSWbm1plM3zydIl1En6Z9eHLgk4yeN5rcolxGtB3B/X3vL9lWcSjnEBMWT2Bb2jYiLBE80PcBLmhzAdH2aIrcRWw8upGJ308k1ZlK48jGTB02lY5xHb2mXUtzpjF13VQ+3f4pbu3mnJbn8K8B/yI+Iv54/raFAGQIVIjgoRTceissWQJNm5o/k5JMSrUq7MzcycfbPy5JX/bDvh/4cd+PDGw2ELd2M+ePOWxN3QqYKvGPLXuMnRk7AcgtyuXBHx8ksyDTp+YdyjvEO5veoUib7CwrD63kk+2fMKHnBDSaBTsXsPLQSgByCnN4ftXzbEvbBoDT5WTyssklz0pzppUEP4AjuUe4Z8k9HHUe9frs7Wnb+XjbxyVZab7b+x2Ldy8m1Faxi+AiAVCIYDNggBkC7d8f9u6FQYPMqlEv1h6uvGhmy9EttGnYpuT16sOrAch35bP56OZy5xZpMwzpi00pmyodW5+8vtyzlh1YBpg8ohtTNpY7V6M5kH0AMHsTK+YY3Ze1r8rUZ8WfoazlB5dT6C70qe1CeCMBUIhg1KyZ6QHedhvk58P115uh0cLy/+Cf2ax/pUt7N+nNttRtJa8HNR8EgMPqoG+TvuXODbeE+zyM2LtJ70rHBjQbwG9Hfyt5fU6rcwCIskXR75R+5c61KAstG7QseW7TqKbl3u8Q2wF7mPfSSwOaDah07JxW59RZqSZxcpIAKESwstvhv/81c4N2O7z2GgwbBodKqzu0iDqFO3veicPqwBpmZWyHsSQ1TWLtkbU4rA5uO+M22sS0ASDaHs39fe8nqUkSAE0imzB12FQa2n1bTBIfHs/9fe8n2haNRVkY0XYEI9uO5N1N72IPs3ND1xvomtAVMMF2Qo8JDGw2EDBp1l4a+hIN7A3MvRzxTB02taRtp8WdxotDXiTe4T0Yt2nYhrt63lXyOS/veDn9vQR/IY6FZIIRIhQsXw5jxsD+/aZ3OGeOKbEE5DgzySrKQQMOSzgqzEJuUS5okxA72l4+oXS6M50CdwEKRXxEfLUrLyvKLcwlMz8TjSbCGkGENYLsgmw0mgb2BpXSl2XkZ5DvyieMMGIjYisVuD2ad5QidxG2MFuVwa+Ys8hZUjUiyhZV52naaiCZYEKQBEAhQsWhQ6ak0k8/mR7h1Klw002BbpUwJACGIBkCFSJUNG1qMsZMmAAFBXDzzaVzhEKIYyYBUIhQYrebHKLvvWf2Dr7xBgwdCgcOBLplQoQcCYBChKJrrzVDoS1bwrJl0Lu3qT4vhPCZBEAhglBqXirJucmkOavZo5eUZPYLDhli5geHDoXXX4eK8/r52ZB1GLIPg8v7Pjsh6iMJgEIEEa01uzJ2cdu3t3HOp+dw+7e3sydzT9UXJCaaSvP33GP2CN5+u5kbdDrN+znJ8NV98GJneH0QbFsA+Vl182GECHISAIUIIqnOVCZ8N4HfUs3m8s1HN3PX93dxNM97ijDA1BR84QVTRcLhMAm1zz4bdu+CVe/C+o/AXQTZR+DTayG3mnsJUY9IABQiiOS78tmdubvcsR3pOyhwFdR88VVXmXnA1q1hxQro0xfmf1T+HK3hgNQcFAIkAAoRVGxhNhpFlC9f1CSySaUN5FXq2dMkzx42DJKT4cUNsKKg/LxgYqdabLEQoUsCoBBBJC4ijueHPE+M3dTVaxjekOfPfv7Yyv4kJMDChfD3v4PLDV87YZ4TXGEw+O8Q3bTmewhRD0gmGCGCTKGrkLR8U3jWYXUQFx6H1XKcpTs//hhuuAFyc6F7N/jkIzitS+02WIBkgglJ0gMUIsjYLDYaRzamVUwrEiMTjz/4AYwbZ/YJtmsH6zfCwCHw/fe11lYhQpkEQCFOdmecAStXwgUXQEoKnHcevPhi5f2CQtQzEgCFqA/i42HBAnjwQXC54N574eqrzdCoEPWUBEAh6guLBaZMgU8/hago+PBDGDgQdu4MdMuECAgJgEIEo/xsyDpk/vQmN9WkNivyYX9gRaMuhO8WwKntYd06k1Jt0ZqmVWsAACAASURBVKITa28VClwFpOSmkJGf4Zf7C3EiJAAKEWwyD8AXE+CtofDl3eZ1saJ8OLAOPhoH086Dn1+CnGPI7JKTDN89Bj//H9zTAc4bAqmpMHw4PPdcrc4LpjnTeH3961z11VXc/f3d/J72O0WSi1QEEdkGIUQwyUmBDy6FA2tKj7XsB5d/BFGNTDB8pScUOUvfP/9J6He7GeKsTkEufPsvWPFm6bGIeEi/BJ55wby+7DJ45x0zRHoCCl2FvLHhDd7Y8EbJsUhrJPNHzycxMvGE7h2kZBtECPKpB6iUilNKdVFKtVNKSa9RCH8pyisf/AD2Li8NeIc3lw9+ABtmgTO15nvnZ8LmOeWPOVNh4vUwdy40aACffAL9+8OOHcf/GYCM/Ay+2vlVuWO5Rbnszdp7QvcVojZVGcyUUg2VUpOUUhuBX4E3gE+A3UqpT5VSQ+uqkULUG2E2CI8pf8wRB2Ge3l1M88rXxLUFq8O3e8e2rnw8oiGMGmXyh3bsCBs3mnnBhQuPvf0edoudFtEtKh1v5Gjk5WwhAqO63txsYC9wlta6o9Z6kNY6SWvdEngaGKWUurFOWilEfeGIg7++VBrwwixw0avg8KRCi24CZ1xeen5kIzhvMoRH13zvqEZw4YtgL3PuWX8rDbidOpkgOGoUpKfDiBFm1ehxTJPEhMfwQL8HSlK6AYzrOI7Y8NhjvpcQ/iJzgEIEm/xscGZA5n7T43PEgr3MnFxuKuSlQl4GNGwOUY0hzMeZCVehKYeUvgeiG5venyOu/DluNzz5JPzrXyb4XXIJvPeeGSI9Bi63i1RnKgdzDhIXHkdMeAwNwxse0z1CiMwBhiCfAqBS6gygDVCSk0lrPafKC/xIAqAQdWT+fFNiKTMTOnc284QdOgS6VcFKAmAIqvHXRqXUO8A7wBjgr56vC/3cLiFEoF14oUmh1rkzbNkCffqYoCjESaLGHqBSaovWunMdtadG0gMUoo5lZcH118Nnn5nXkyfDww/7PuxaP0gPMAT58n/wMqVU0ARAIUQda9DApE+bMgWUMnODo0dDhmR3EaHNlwD4PiYIblNKbVBKbVRKbfB3w4Q46TgzoSDHx3OzIH1v6flF+ZCXZhaogNnU7qwmAOWkmMUyHlkFWeQV5QGgtSYzP5MC1zGkUVPKJNL+6iuIjYUvvoB+/WDrVt/vIUSQ8aXQ2DvAeGAj4Pb1xp65wwuBI1rrrl7eHwLMA4oz8c7RWj/m6/2FCBnODDi43qQts8fA0Achrg1Yw72fn3kQVk6DPb9A60GQdANsmQdbv4S+t0LTbvDDs5CxH5JuhLaDIdKzkjP3KBzZCj+/CJZwMoZPYU3mDj7c+hGNIxtzZ887+Xn/z8z/cz6nxp7KTd1uoklUE98/y/DhsGqV6QFu3Ah9+8L06XDxxSf81yREXfNlDvA7rfU5x3xjpQYD2cD0agLgfVrrY1pQI3OAIuTs/gXe/Uvpa2sE3LkaGlbeKE7GfvjyTvhjcemxjiNg+NPw8hlwwzfw0eWmN1jskjeh22Wml7Z/NUwbZrYvtOrPogE3cO/yxwEY1X4UiY5Epm2aVnJpi+gWzBgxgwRHwrF9ppwcuPFGU3Ee4J//hEcfrc/zgjIHGIJ8+b91q1LqQ6XUFUqpS4q/arpIa/0D4EN+JiFOYgW5sGxq+WNFTvi9iuoL7qLywQ9g+9cmuMW2hvTd5YMfwK//NT2/onxYMa1k43pGx78wc+eCktMGtxjMvB3zyl26L3sf6fnpx/65oqLgo49MAu2wMHj8cfjrX80GeiFChC8B0AHkA+dT+9sg+iul1iulvlZKdanqJKXULUqpVUqpVcnJybX0aCHqQJgVorwkf46qoselwkwPsSyrA1BQmGM2rlfkiAeLDZTFZHspvqwgi4b20s3rOYU5XjeiR1giKh3ziVJw333wzTem4O5XX5mtEps3H9/9hKhjNQZArfX1Xr5uqIVnrwFaa627A68Cc6tpw5ueNGxJiYknZSZ5cbKy2mHQPeUDV2JHU+HBG5vDpCcra8gDZgFNTorJ5NKqf5n7h8N5j5n7W6zQ95aS4Bq1/mPuPv3akgA394+5TOgxgbAy+eyHtxlOtN2HNGrVOfdcMy/Yowf88YdZHDN79ondU4g64Msc4PvA3VrrdM/rOOB5X4KgUqoNMN/bHKCXc3cBSVrrlOrOkzlAEXJcLsg5AnuWQUQMND3DpCGrSuZ+U+Nv/0po0c+kKss+DMlbofVAE/SO/mFKI7UeAJGJYPMsqHEVmXP3/ALWCApb9iVVwerDq0lwJNCuYTvyXfmsPryaNg3b0LJBS+Ii4qpuy7HIzYVbboEPPjCvH3gAnnii5jJNJweZAwxBvgTAtVrrnjUdq+LaNlQRAJVSTYHDWmutlOqLSb7dWtfQIAmAQgQxreHll83QqMsFF1wAH35ohkhPbhIAQ5Avc4Bhnl4fAEqpeHzYPqGU+ghYBnRUSu1TSt2olLpNKXWb55SxwCal1HrgFeDymoKfECLIKQUTJ8KiRZCQYOYH+/SBDbJ1WAQfX3qA1wAPYnpoGrgMeFJrPcP/zatMeoBChIg9e0wlidWrITIS3n4bLr+85utCk/QAQ5Avi2CmYxJhHwaSgUsCFfyEECGkVSv48Ue45hozP3jFFfD3v0NRUaBbJgRQTQ9QKRWttc6u9mIfzqlt0gMUQUVrM+xXG9zu6jeSV3xWxfNrsy21SWuYOhXuuccEv2HDYNYsM0R68gjCv3hRk+oC4GJgHSZd2WqtdY7neDtgKGYo9C2tdZ2ud5YAKIJC9hHYPBcObYCe483WBsdxVjvPOmTus3kuNO8FnUZCg1PKPCsZts6Hfaug+zhodCqs+8DkCu13m6kGv3o65GdBv5uhQXOwO2rnc1aj0FXIkbwjzNo6CzAV35tENsFmsXm/4McfYexYOHIEWreGzz+HnjWupQsVEgBDULVzgEqpEcBVwEAgHigEtgELgLe11ofqopFlHVcAXLfO7FESojZkJ8OMi+HwptJjo98w6ciONRVYQS4s/y8sLpMGt81ZJr1ZTDOz92/WVbD319L3h/3LPHvLPJMabeYYcHoysIRZ4fZfTED2swPZB7h43sUlSbYjLBHMvXguzaObV33Rvn0wZgysWAEREfDWW3D11X5vax2QABiCqv1p1Vp/pbW+SmvdRmsdo7VupLUeoLV+MhDB77h89JH5LfO++2TuQdSO3OTywQ/gh+cgt9otrN7lpcIvr5Y/tutHKHSa7/Ozygc/MAGzy2izmX7Xj6XBD0wqtZ9egqJjqPRwnD7b/llJ8ANwupzM3l7DgFCLFrB0qckj6nTC+PFmaLSw0M+tFaKykz9zbVYWWK3w/PMwYgSkSnpScaK8/NiEWTjuToC3ebviY17fs5h5Ne02qdO8taUO+iNhXnq7FuXDpvfint/rr4PNBi+9BOefb4ZGhahDJ38AvOUW+PZbSEw0e5P69DFlXIQ4XlGNoHnv8seGTKo6v2d1HPEwqELqs1PPLc0Hao+GtmeXf3/AnbDpM9i3wpMJpjT/J9ZwGDgRLPZjb8sxuqTDJUTZokpeR1ojuaRDjXnyDaXg1lthyRJo2tT8mZRkUqoJUUdq3AcYbI57EcyePaZm2dq1JpP9+++buQghjkf2EfjjWzMU2m0cxLfxnqjaF1mH4ejvsOULswim3dmVF8HsXAr7VkPXS6Bhc/jtS0jbDUnXgy0SNs8x+UJ7XgXRTcF2nAmuj0GRu4iUvBS+3PElbu3movYXkehIxGrxpcxoGQcOmMUxy5ZBeLjpGV53nV/a7EcyBxiCfAqASikL0IQyGWC01nv82K4qndAq0NxcuPlmk5oJ4OGHYfLk+lzDTIjgUFAAd99tgh/AhAnwwgtmiDQ0SAAMQTX+y6+UuhOzCX4RZvXnAmC+n9vlH5GRMHMm/PvfJug98QSMGgUZGYFumRD1m90O//2vmRu02+G118x+wUOhsdZOhCZfUqH9AfTTWh+tmyZVr9b2AS5aBOPGQVoadOwI8+aZP4UQgbV8uZme2L8fmjWDOXNMiaXgFrQ9QKXUo0C21vrfgW5LsPFl7G8vcPJ1kc47D1auhK5dYds26NsX5odmx1aIk0q/fmYxzKBBZn5w8GCTR1SIWlZlAFRK3auUuhf4E1iilHqw+JjneOhr395MvI8ZA5mZcNFF8OSTZom5ECJwmjaFxYvNXGBBAdx0E9x2G+TnB7plQU8pdY1SaoNSar1SakaF925WSq30vPeZUirSc/xSpdQmz/EfPMe6KKVWKKXWee7XIRCfx5+qS4X2r2qu01rrx6p532/8kgpNa5gyBf75T/P9mDHw3nsQfYKVskXoyM+BnMNmZWdCR2jSGaISfb8+8wDsXW5Wh3b6qxkQ2/EduF1mW4MKg20LzErR1oOgKA+2LYS4NtCsh8kIs2OxKZYb3xayD8HuX6HNQFPwNnWHWXHafqhZ5RkR46+/ieDz/vtmy0R+PvTvb6rNN2sW6FZVFBRDoEqpLsAcYKDWOsVTvu4uPEOgSqlGxdNZSqknMDVZX1VKbQSGa633K6VitdbpSqlXgV+11h8opeyARWudV9WzQ1GV65W11pPB/Gagtf607HtKqUv93bA6pRQ89BCccYZJy/TZZ2ZYdO5c00sUJ799K2HmaLO5HEzQGv2m2fNXk8z9Jh3Zkd9MKrIWSTDrSpPjEyAyHq76DP73MMS3h4hY+Ohyk7UFTEaXAXfC1/8wr88YB6ecAd88BCOeg10/w5a5pc+7bIbJFxpWLyqtw7XXQpcuprTSsmXQu7cJggMHBrplwegcYLbWOgVAa52qyidT6OoJfLFANPCN5/jPwHtKqU8wARRMPdeHlFItgDla69/r4gPUJV/mAB/08Vjo++tfzQR8x46waZPZNL9oUaBbJfwtOxn+N6k0+IHpCeb5mDXo0EYT/MBsWv/j29LgB5CbCps/hw4XQPfL4cfnS4MfmJ6jzQEOT93pDR+boAjQrGf54AcmkOYGxZq0ulO8SX7IELMydOhQs2VCpisqUpi6rVV5D5igte4GTAYiALTWtwEPAy2BdZ6e4ofARUAe8I1S6hx/NjwQqpsD/IunC9xcKfVKma/3gJM3qWanTiYIjhxpVogOH27SqMkP2knMDfleqnoV+Tjf5Mws/d7m8H6v/Czzns0BBV7eL8gxWVxKmuQyIxNuLz9qBVn18//Hxo3NL6TFuUNvv93s63U6A92yYLIYuEwp1QjAMwRaVgPgoFLKhil0gOe89lrr5VrrR4AUoKWn8s+fWutXgC+AM+rkE9Sh6nqAB4DVgNPzZ/HXF8AF/m9aADVsCF98YYZF3W6TSHv8eMg7qYa/RTFHIzjz9vLH4tpCdGPfrm89wAxrAuz8AU6/EMqWBAqzQLcxpme47SvoeU356xs0NenMinuNp3Q3w6pam95p487lz+9zy/GXXgp1VqvZID9zJjgcZnXo2WebKhMCrfVm4ElgqVJqPfBChVP+CSzH7OveWub4c0qpjUqpTcAPwHpgHLBJKbUO6ARM93f765ov+wBtWuugSdVe5/UAZ882aZlycqBXL1PDrFWrunu+qBu5qaaywtoZ0LiLqbMXc0rN1wEU5kPWfvjxRVMRYvD9YLWboU7tgkH3mATWS56C8AYwdJIZNl3zPjRsDQPvhMObYeU0aJ4Eva+F7f+DrV9AlzFm4cuqd+DAWlNyqcO55fN/1ldr18Lo0bB7t+kdfvqp2TIRGEGxCEYcm+pWgW6kmrFkrXVAusMBKYi7caPJI/rnnyap9uzZgfxBE/6Un22GIqsq6lodZxa4CkoXzhQPjRav2MzLML3BcM/q4tw08yx7pOntFWSD1QEWqxkCLcw1eT7DLOAqMitHwxuc+Gc8maSkwOWXmy0TViu8+CLccYf3Khr+JQEwBFUXAFt7vr3D82fxfpKrgNyTahuEL1JTzQ/aokXmB+2ll+D//i8QP2hCiLKKimDSJHjuOfP6mmvMAhmHoy5bIf8QhCBfhkB/1loPrOlYXQlYAATzg/bAA2ZRDJiinlOnmgz2QojA+vhjuOEGk/S+7qcrJACGIF+2QUQppQYVv1BKDQCiqjn/5GW1mkTaM2eaop5vv22WZR84EOiWCSHGjTP7BNu1gzVrzH7B778PdKtEEPMlAN4ITFVK7VJK7QL+A9zg11YFu6uugp9/hpYt4ddfzR6lX38NdKuEEGecYXL8XnCBmR887zwzL1gft42IGvlcEFcpFeM5P6CJsQM6BFrRkSNw6aXwww+mhMt//mOGRcXJLzfVrPjMOgiNTjMZYLIPgzMDGrWHqMbl60xmHYbUP82Cl5hm5dOsFeSBMxWSf4fYViZzTHXbHHJSIOcI5ByFhA4mVZqlnmSF8ZXLZVIbPvWUeX3llabUUmSkv54oQ6AhqLpFMFdrrWdWlfhaa11xf0mdCKoACGZD7j33mLlAMCvQXnwxlAp5imOVmwbfPQ6rPBUKLv8QVkyDP78zryMbwc3fQ5xnHVnGfpg2zARLgBZ9zTXRiaZnsusnk4bN5dltdPaD0P//vOf7zDkK8yfCb1+Y1xEN4abvIOFU/33eUFZ2G1OPHqa0Utu2/nhS0ARApVS21tprImOl1C9a6wF+eu4krfUUf9zbX6obAi2e52tQxZcAE+heew2mTTO9wKlT4dxzTe9QnJzyM0qDX8MWZstCcfADk6Zs6dMmwXVRISybWhr8APatgIPrzPc5yfDlXaXBD+CHZ0y2F2+yDpYGPzA9zkWPmC0YorKxY01mpw4dYN06M10RROkN2zyw4Mo2DyzY1eaBBW7Pn1f64zlKKQuAv4KfxyQ/3tsvqgyAWus3PN8+o7WeXPGrjtoXOm68EZYsgVNOMUOiSUlmIl6cfPLSS793xEOWl0VQabvNnkB3gankUOn9neZPrU3Wl7K0GwqrSO+V7aVCesZecEmZoCp16QIrVpj0hqmpphcYBDzB7i2gNaYH2Rp4q7aCoFJqiFLqe6XUh8BGz7Fsz5+nKKV+8JQ62qSUOsvL9V7LISmlri5z/A2llEUp9TTg8Bz7wHPevZ57b1JKTfQci1JKLfCUXdqklBrnOf6Ip0zTJqXUm0rVzf4yXxbBbFJK/ayUelopNUIp1dDvrQpV/fubhL1nngl795ps9R98EOhWidpWnLoMIPk3aN4bLPby5/S+zszj2aMgqcK8cJgFTj3ffG+Pgq5jy78f16Z0s3xFjbuYa8rqOb40kbbwLjbWpDd8/XWzhzc4TAEqTkpGeo7Xlr7AQ1rrCvn0uBL4RmvdA+gOrPNy7W3Ay55zkoB9SqnTMSnSBnqOu4CrtNYPAHla6x5a66uUUr2B64F+wJnAzUqpnsBw4IDWurvWuiuw0POs17TWfTzHHMCFtfdXULUaA6DW+lTgCsxvEBcC6z254YQ3zZqZnuANN5gkvVdfbXKJFp28+cPrnahEuHERnPYXkzM0bQ/c+D9oc5bJ23nhS6acUrEWfWD0G9C0G7TqD9cvLM0zGh4N506GgROh0anQeTRc+yVEN6ni2Qlw47fQbigkdoILnjJ5RutLaaQTERZm6goGz77dqjYp1ubmxRVa651ejq8ErldKPQp001p7G0NfBkxSSt0PtPbUAhwG9AZWeuLAMKCdl2sHAZ9rrXO01tmYEktnYeLIuUqpZ5RSZ5VZVDlUKbXck4HsHKDLcX/iY1BlPcBinlpQAzGN7w5sBn7yc7tCW3i4mRPs3RvuvttsnN+wAWbNgviKydlFyAmzmJWel7wJRU7T+7LYYNwMk7IsMr58QIqMMzk82w8zxyMr/D8QnWjyg/a/w1SLqC7dmcVmivVe+p6ZN3TEywrQ0LUHM+zp7XhtyfF2UGv9g1JqMDASmKGUeg7IAooLod+ktf5QKbXcc843SqmbMEO172utayqJ53UIU2u93dM7HAE8pZT6H/AsZntdktZ6rycoRxzTpzxOvgyB7gEmAl9rrftrrUdqrZ/yc7tCn1ImVdq335r8oYsWmfqCGzcGumWitkTEmJ5ccd5QR5wJZt56Y2Fh5r2Kwa+YNdzcy9dcn45Ycz8JfqFsEpBb4VgudbCYxJPq8ojW+i3gbaCX1vpzzxBmD631qirKIS0GxiqlGnvuE18mbWahp8wSmIoSFyulIpVSUcBo4EelVDNMKs2ZwL+BXpQGuxSlVDRQYU7Af3wJgD0xZTCuVEotU0pNV0rJZjdfnX22mRfs1csk0+7f31ScF0LUa7ueHvkhcDOwG1N4YDdws+e4vw3BFL5dC4wBXvZyTqVySFrrLZjCuf9TSm3AlFUqLpvyJrBBKfWB1noNpvjuCkz5pWla67VAN2CF554PAU9ordMxi4E2AnMxw7N1wqeN8J6oPAgzDHo1oLXWbfzbNO+Cbh+gr3Jz4ZZbShfFPPQQPPZY+c3SQohQFTT7AIXvavzXVym1CjMZOhpTQHFwoIJfSIuMhBkzTC7RsDB48kkYNQoyAppYRwgh6i1fqkEkaq2T66g9NQrZHmBZixaZxL1padCxI8ybZ/4U1SvKh7xUU7MvvAFExIGtjlb0FeSBM93U7AtvYBarFOSYmn32Bmb+Tkpj1WfyHz8E+bINImiC30njvPPMvGDXrrBtG/TtC/PnB7pVwc3lgn0r4dUkeC0JXu0Fe5aVz6DiL0VO2LEYXulhnj21n6no/un18GpveGuIyfMphAgpMgEVKO3amdItY8ZAZiZcdBE88YRkra9KbgrMvt70wMD0vj67waQd87e8NPj8VhMIwfQEv7gT+t1qXmcegAV/K58hRggR9CQABlJ0NHz6qQl8YLLXX3opZGcHtl3ByF0I2RXyq+YeNenG/K3QWRp4i6X+WbqZHUxGmKIq0pcJIYJSlRvhlVKXVHeh1jo4EuqFOqXMitDu3U2dwc8+M8Oic+dC+/aBbl3wsIZDk65weFPpscROYK2D/bL2SJP0OmNf6bHWA+DIltLXHYZDuJfqDUKIoFVdD/Cv1XzVSZ62euXCC03W+o4dYdMms2k+iLLWB1xUoikh1HqAqb3X6ky4Ylb5Xpi/RCbC+LnQvJd5druhMOo/sHW+yQHadQyc85AJlEIEWHHC6yre+6Uu21JFG75SSlVT8LLK6x5VSt1Xq23xtSBusDgpVoFWJyPD5A+dP99sl3j2Wbj3XllhWCw31Sx8sdiqzqriLzkppvSRNdxkYslONpUbbJEQIRXC6rnj+wF9tOGVmOTXrTBZtybxaMYJbYT3Vg9QKWXRWrtO5L7H2Aar1rpWEyB7UqRla63/XVtt8GkOUCk1Uin1D0/JikeUUo/4cp04Dg0bmm0RDz8MbrdJpD1+POTlBbplwSEyHho0qfvgByYRdYMmpdXaoxPNawl+4niY4FepHJLn+AmrhXJIy5VSXcq8XqKU6u0pafSOp3zRWqXUKM/71ymlPlVKfYnJFOP1GUqpXUqpBM/313hKLa1XSs3wHGutlFrsOb5YKVUpObhSqodS6lfPOZ8rpeLKtHGKUmopcHdNf0e+bIR/HZMS507Mf6RL8Z7AVdSWsDB4/HFTzToqymSPGTQI9tRmjlwhRIAFezmkWcBlYAIm0ExrvRqTwuw7rXUfYCjwnCffJ0B/4Fqt9Tk1PcMTXB8CztFad6c0YL2GSbt2BvAB8IqXtk0H7vecs5HSJN4AsVrrs7XWz1f3FwO+9QAHaK2vAdI8hXD7Ay19uE6cqDFjzFaJdu1Mcd2kJFNsVwhxMgj2ckifYDo8YALhp57vzwce8OTzXIJJZl3c5kVa61Qfn3EOMFtrnQJQ5rr+QPEw8AxMGs4Snpq0sVrrpZ5D7wODy5zysZfP4pUvAbB47C3Xk8m7EGjr6wPECerWDVauNJvnk5Nh2DCYOlX2CwoR+qoa0qmTckiYoLEfUw7pGqXUaM9w5TqlVJLWej9wVCl1BmYUcJbncgWMKVM5opXW+reKz/P2jArNUJgk4DU51n/svH5mb3wJgPM9K3aeA9YAuyj9i6iSZ4z4iFJqUxXvK6XUK0qpPzzjuL18bXS9Ex8PX31VWlh3wgS46SbIzw90y4JbXgak7zVfFfcQAmQdhvQ9ZnuDM7Py+9lHIGOvOc9dw/qBwjzIOmg2xeelmXRtmQfNl7d7CxHk5ZA8p84C/gE01FoX13L7BrhTKbMyT5lK7z49o8Ipi4HLlFKNPOcXT+z/Alzu+f4qKtSf9RTRTSszbzkeWMpx8CUAPqu1Ttdaf4aZ++sEPOHDde8Bw6t5/y9AB8/XLcB/fbhn/WW1wnPPwcyZEBEB77wDQ4bAgQOBbllwyjoMv/7HpC57uRt8dR9kHSp9P/MgfDEBXuoGU/vCmunlg2TKH/DeSHixK7w1FA6sNcVuvclLh9Xvwyu94IXT4cA6WDLFPPelLrDoEbOCVIiyzGrPSuWQTnQVqI+GUHM5JIDZmGD0SZljjwM2TOmjTZ7Xx/wMrfVm4ElgqVJqPfCC5627MEOnGzDBzdtilmsxc48bgB7AY1V+0mr4kgx7jda6V03Hqri2DTBfa93Vy3tvAEu01h95Xm8DhmitD1Z3z5N+G4Qv1qyBiy+GvXvhlFNgzhw488xAtyq47FsF04aVP3beE3Dm/5msMkufhZ8qzJHfvsxUW89JhhmjTb7PYpGN4PZfoEHTys9K3maCKJjN+Wf9DebcXP6cS9+HLhef+OcSwUr2KYWgKnuASqmmypSudyileiqlenm+hlB55dLxaA7sLfN6n+eYqEmvXiaZ9uDBcPCgKbr79tuBblVw2fWjl2NLIT/DDFHu/qny+/s8v1i5isoHPzBp1wqr2Iqyf23p941Ph30rKp+z/WuzrUUIETSqGwK9AFOyvgWma/q85+seameM2ttvTF67o0qpW5RSq5RSq5KTpTgFAI0bw7ffwh13QEGBmROcMAEK66A6QihoPbDysTaDPGWUYqFV/8rvt+ht/gyzmrRrZUXGLqbnhAAAIABJREFUmxJI3jTrUfp98lZo0afyOR0ukOLHQgSZKn8itdbva62HAtdprYeW+RpVS3lA91F+O0ULwOuEltb6Ta11ktY6KTExsRYefZKw2eC112DaNLDbzerQc8+FI14WfNQ3sa3MUKQ13GTR6XQhnHG5ySBjd0C/26C9Z4jU5oBzHzUpz8BscB/7LsS3M68bNIUrPjbDoN40aALnP2nuc+Q3iGoMfW81gVSFQc+roe1g79cKIQLGlznAppiJymZa678opToD/bXWNY651TAHOBKYAIwA+gGvaK371nRPmQOswq+/wiWXmCHRli3h88+hd+9Atyqw8tKhIMtsGbFGVM4bmnXYVHBQYSaRtaNh6Xtam7nAonyT7zMyASyWqp9VkAvOjNLUaBYr5Hu2PdmjIUISZZ/kZA4wBPkSAL8G3sVkE+iulLICa7XW3Wq47iPMKqAE4DBmp74NQGv9umcJ7WuYlaK5wPVllt5WSQJgNQ4cMJvnf/3VrBSdNs1UmBBC+JsEwBDkSwBcqbXuo5Raq7Xu6Tm2zpPeps5JAKxBfr6ZFyxeFPO3v8HTT5ttFEIIf5EAGIJ8mZXP8WxU1ABKqTOBDL+2Shy/8HB46y0zH2i1wvPPw4gRkJpa87VCiJCn/FwOSSl1kVLqgeO4rsZnK6WmeabZ6oQvPcBewKtAV2ATkAiM1Vpv8H/zKpMe4DH44QcYO9akUGvXzhTZ7VbtyLUQ4vgcVw+w2/vdKpVD2njtxpAsh6T8UALJ32rsAWqt1wBnAwOAW4EugQp+4hgNHmz2C/bqBX/+Cf37m4rzQoiA8wS/SuWQPMdPmPJfOaTrlFKveY69p5R6QSn1PfCMUipRKbVIKbVGKfWGUmp3mdJH2WXatUQpNVsptVUp9UGZtGpLlFJJnu+He+6zXim12HOsr1LqF2XKMP2ilOp4In9HvpRDisCkpnkcmAzc4TkmQkGrVvDTT2YxTE6O6REW1xoUQgRSqJZDqug04Fyt9d8wix2/82QK+5yqK1v0BCYCnYF2QLmNu0qpRMwvB2M8pZKKq1JsBQZ71qM8wgn+XfkyBzgd6IIZBn3N0+AZJ/JQUcccDpgxA/79b7MZ+8knYdQoU31eCBEooVoOqaJPywyvDsJTLEFrvRBIq6Zd+7TWbkzwbVPh/TOBH4rbXqZUUkPgU08O0hcxsem4+RIAO2qtb9Raf+/5ugUT8UUoUcqsCF24EOLiYP586NcPtm0LdMuEqK9CtRxSdc/wdS70/9u78/go62vx458zk0kySSBsAcMmgoqtIAIBUVv0VuwLi4Jlue6/qvXaq1K7eG2rVtsiauu12mql/aHQikKt4orSiiBFQQUCKhAQZVGWgAlZyTYzmXzvH98ne4AAGZ5Zzvv14pWZJ888cyAhJ893OadpK5sw0HKZ+qFaJd0PLHf2ll+G7UV4zNqTAD9yVn7aqETOAVYdz5sqF118sZ0XHDLEJr/Ro20yVEqdaLHaDulwVtI4bPptoOsxhvgBcIGInOJcq75VUiY2aQNcf4zXbtCeBHgO8L6IfCEiXzQJbKPTikLFmoEDbaf5KVOgvBwmToSZM7XJrlInkLPas1U7pONdBdpOF3Ls7ZAO5zfAt0VkPbbl3T6greHVwzLGFGLb5L3stEqq7/L+MPCQiKwCDlOaqX3asw3i5CME+uXxBnE0dBtEBzIGHnwQ7r3XPp4yBf72N8jIOOJLlVLN6EZ4QERSgLAxplZEzgX+7FbRlPY4YnmQE53g1AkkAvfcA8OG2VWiL71kh0VffRUGDXI7OqVU7OkPvCAiHiCIvcONWtqfRcGll8KaNTB4MGzaBKNGwdtvux2VUirGGGM+N8YMN8YMM8aMMsasdTumw9EEqKzBg2H1apsMS0pg/Hi7bULnBZVScUoToGqUmQmvvda4Uf7OO+Haa6Gq5UI1pZSKfZoAVXMeD9x/PyxcCOnpsGABfOMbsKsjtyYppZT7NAGqttX3FRw4ED76CHJybHFtpZSKE5oA1aENGQJr19rN84WFcNFFts2SzgsqpeKAJkB1eN26weLF8D//A7W1MH063HSTbbyrlFIxTBOgOrKkJPjf/4XnnoPUVJg7Fy68EPLz3Y5MKaWOmSZA1X7XXAOrVkG/fnZ+MCfHflRKqRikCVAdnREjbDHtsWNh3z644AKYM8ftqJRS6qhpAlRHr2dPWLrUzgcGg3ZO8LbbIBRyOzKllGo3TYDq2Ph88MQT9u4vORlmzYJx46CgwO3IlFKqXTQBquNz442wYgVkZ9t9gjk5sG6d21EppdQRaQJUx2/MGJv0xoyB3btt5Zj5892OSimlDksToOoY2dnw73/D978PNTW2hmj93kGllIpCmgBVx0lJgaeestVikpLg97+H73wHiovdjkwppVrRBKg6lgjceissWwZZWbav4KhRsHGj25EppVQzmgBVZIwda+cFR4yAHTvg3HNtx3mllIoSmgBV5PTrBytX2goylZUwdWpjr0GllHKZJkAVWX4/PPusnQ/0eOCBB2DiRCgrczsypVSC0wSoIk8EfvpTeOst6NoV3nwTzjkHPv3U7ciUUglME6A6ccaNs3VEhw6FrVttEnzjDbejUkolKE2A6sQaOBDef9/OB5aX2+HQmTN1XlApdcJpAlQnXkYGvPCCnQ8EuPdemDYNKircjUsplVA0ASp3iMDdd8OiRdC5M7z8st0qsX2725EppRKEJkDlrgkTYM0aGDwYNm2ym+bfftvtqJRSCUATYBwqqgjw9ub9PLR4C+u+LKG0Kuh2SIc3eDCsXg2XXQYlJTB+PDzyCBjjdmRKqTimCTDOlFYF+eWrm/iveev4/+/uYMqf3+eVj/YSrI3yRSaZmfDqq3Y+sK4O7rzTFtSuqnI7MqVUnNIEGGcqA2H+uWl/s2OPLf2MsuoovwsEu1F+xgxbMi09HRYssK2Vdu1yOzKlVBzSBBhnDK2HDcPhGBtKnDwZPvzQbpn46CPbZHfFCrejUkrFGU2AcSYtOYlvnNq92bGbLxhI51SfSxEdoyFDYO1auPhiKCy0m+iffFLnBZVSHUZMjP1AycnJMbm5uW6HEdWKKgIs2fwVa3YWc9lZ2Qzv35Wu6cluh3VsamvhrrvsohiAG2+EWbNs70Glooe4HYA6epoA41i4rg6vJ05u8hcsaOw2f845dt9g795uR6VUPU2AMShOfjqqtsRN8gO4+mpYtQr697dbJnJy7DyhUkodozj6Cani3ogRdl7wggtg3z77cc4ct6NSSsUoTYAqtvTsaSvFTJ8OwSDcdBPcdhuEQm5HppSKMZoAVezx+eCJJ+zdX3KyXRQzbhwUFLgdmVIqhkQ0AYrIeBHZKiLbROQXbXz+ehEpFJGPnT83RTKeRFVUEWD9rhIWfZJPfmk1NaGw2yF1jBtvtPsDs7Ph3XftvOC6dW5HpZSKEUmRurCIeIEngYuBPcBaEXndGLO5xan/MMZMj1Qcia64MsCdL37CO1sLAUj2enjxv89lWL8uLkfWQcaMsUlvyhT44ANbOebpp+Gaa9yOTCkV5SJ5Bzga2GaM2WGMCQLPA5Mi+H6qDQcqgg3JDyAYruP+NzZTEu0Fso9GdjYsX27nA2tqbA3RO+6wewiVUuoQIpkA+wC7mzzf4xxraYqIbBCRhSLSr60LicjNIpIrIrmFhYVtnaIOoaKmdRIoqQoSrout/Z9HlJICs2fb+cCkJHj0UbjkEigqcjsypVSUimQCbGtjaMufuouAAcaYs4ClwDNtXcgYM9sYk2OMycnKyurgMONb365+emQ0rwJz3ZiT6eqPsdJo7SECt9wCy5ZBVhYsXWr7C27c6HZkSqkoFMkEuAdoekfXF8hveoIxpsgYE3CePgWMjGA8CSmrUwqv3no+U0f2ZdSArjwybRiTzu6D1xvHC4DHjrXzgiNGwM6dttP8woVuR6WUijIRK4UmIknAZ8BFwF5gLXC1MSavyTnZxph9zuPvAj83xow53HW1FNqxqQ6GCdSGyfT7EEmQqk3V1XDzzfDcc/b53XfD/ffbtktKdawE+U8VXyL2k8AYUwtMB94CtgAvGGPyRGSGiEx0TrtdRPJE5BPgduD6SMWT6PzJXrqkJSdO8gPw+2HePPj9723Se/BBmDgRysrcjkwpFQW0GLZKDEuXwhVXQHExDB5su8+fcYbbUan4kUC/WcYPHQtSiWHcOFtHdOhQ2LrVdpR44w23o1JKuUgToEocAwfC++/D1KlQXm6HQ2fOhLo6tyNTSrlAh0BjgDGGAxUBCg4GSE9JItPvo2ta49aGksoAVaE69pVWk52Zis/robbOUHgwwEmZqSR7PVQEaimtDtGrUwrdM1LwehpHbIoqAhRX2o3x3dKT6Z4R581mjYGHHoJf/tI+njwZnnkGMjLcjkzFLh0CjUERK4WmOs6ekmom//l9Cg/aHSOXn92b+y47k27pyVQHa1n7RQm3LVhPKGzweoTfTh7Kqm1FvPrxXlJ9HmZfl8Pjyz4n98sSuqT5eOmW8xiUZX/YFx4McP1f15CXXw7A0D6Z/PWGUfSI5yQoYleEDhtm+wy+/DJ89pmdFxw0yO3olFIniA6BRrmqQC2PLNnakPwAXv04n/1l1QAUVwa5+5WNhML2Tj5cZ/jNos3856i+ANSE6rjvtU1877wBAJRWhZixKI/yats+aEne/obkB7BxbxnLtiRIV4UJE2DNGrsYZtMmu2n+7bfdjkopdYJoAoxyNbV17DxQ2er4nhKbAOuMrffZVEWgttkQ567iKrI6Nd7RfVFURaDWzntt3ldOS20di1uDB9vO8pddBiUlMH48PPKIHRpVSsU1TYBRLtPv4/Kzm5dQTfZ6GNonE4AkjzD6lG7NPn/GSZ3YV1rT8Pzir/di7RfFDc8nDutNF6cU2tSRfVu95+ThbZVsjWOZmXb489577YKYO++0BbWrqtyOTCkVQZoAo5zXI1w+vDc/ufg0ememclbfTJ7/wRi6pttFMNld/Dx2xdlMGHoSWRkpXPy1nsy+biTvbz9AVqcUpuX05b5Lv866L0vo1TmFH4wdyPXnDcCXZL/0A7PS+eOVZzOwRzqDstJ5/KrhnNIj3c2/sjs8HpgxA156CdLTYcEC21pp1y63I1NKRYiuAo0Rwdo6SquDJHmEbumtF6gUHqyhJlRHSpKHnp1TKasOEQiFyUhJIi0liZLKIKFwHZl+Hyk+b7PX1tUZiiqDiEC3tGQ8ngRf0LZpE0yaBDt22KLaL74IF1zgdlQquiX4f5rYpAlQqbYUF8NVV8GSJba90h/+ALfealeQKtWafmPEIB0CVaot3brBm2/a+cDaWpg+3TbcDQSO/FqlVEzQBKjUoSQlwcMPw/z5kJoKc+faodD8/CO/VikV9TQBKnUkV18Nq1ZB//6wejWMHAkffOB2VEqp46SVYGLEvrJqAqE6fF4hLdlLVTBMoNYueklL8RIKG6qDYfw+L13Tffi8jQtdqoK1lFfXEgrX4fd56dEpjqu8RMqIEZCbC9OmwYoVcOGFMGsWfP/7bkemlDpGmgBjQH5pNTc/m8umveWc0j2NJ68ZyX8/t45dxVX4fV5mfncIW/eXM/vdnWT6fcy+biQjTu6Kz+vhYE2Il9fv5cHFWwjU1nFqzwzm3Tia3l38bv+1Yk9Wlq0Uc8cd8MQTdk5w/Xp47DFITj7y65VSUUWHQKNcRU2IJ975nE17bXWWq8eczP1vbGZXsd2kXR0K8/OFG/jOkGwAyqpD3DJ/PSVVtjpMaVWIX72e11D5ZVtBBQ+8uYWKQMiFv00c8Png8cftfGBysr0LHDcOChKkfJxScUQTYJQ7GKhl497GDuYDuqeTl9+8o3ltnaG8ppZkr/1yFlcGCYRswssvrW51zQ17S6kOhiMYdQK44QZ4913o3Rveew9ycmDdOrejUkodBU2AUa6L38f5g3o0PN+cX8aYgd2bneP3eUlPSSIYtkmvb1c/qT77pe3fLY2kFhvbx56WRUaKL8KRJ4BzzrHzgueeC7t328ox8+e7HZVSqp00AUY5f3ISN37jFC4dmo3XIyz6JJ+7vnMG5w2ySbBvVz9/vWEUizfapfmn98pg7vWj6O5Ui8lM8/H093Lo1TkFERj3tZ78aNxp+JO9h3xPdRSys2H5cjsfWFNja4jecYfdO6iUimpaCSZGFFUEGubxuqcnU1IVJGwAY+iZkcLBYG1DP8CWvfzC4TqKq4LUGXu32Nmvd38dzhj4y1/g9ttt8hs3Dp5/Hrp3P/JrVTzQSjAxSBOgUh3p3Xdh6lQoLIRTToHXXoOhQ92OSkWeJsAYpEOgSnWksWPtYpiRI2HnTjs/uHCh21EppdqgCVCpjtavn10Zeu21UFlpN8/fc4/tNaiUihqaAJWKBL8f5s2DRx+1vQYffBAmToSysiO/Vil1QugcoEtC4TpKKoMYwJ/spXNq84UpZVVBKp29epmpSVQEwtQaA8bQze+jpKYWYwwG6JmeQmFlfZcCoVcnH/sPBhFAROjVKYUi571SfR4y/c2rllQHw5TXhBCgk9+H36crRDvU0qVwxRW2xdLpp9t5wTPOcDsq1bF0DjAGaSk0F1QEalmxtYD7XsujpCrI+CHZzJh0ZsPqzYLyGv60fBvPr9lNr04pvHjLecxYlMe/8vYz4uSu/OGKs7lz4QY+2F5Eny5+Hpl2Fks3f8WcVV9wWs8M/njlcB5f9hn/yvuKEf278ocrhvH31V/yl/d28q3BPfntlKFkdUoF7Kb5uSt38PTKnQjCf40dyA3nDWjoOK86wLhxsHYtXH45bNwIo0fb/YKXXeZ2ZEolNB0CdUFpVZDpf/+Iokq7NWHxxn3MXbmTkLPN4b3PDzDvgy8Jhut4YPJQ/rpqJ4s37afOwO3fOpUZizbzwfYiAPaWVnPj33L5zlm9Afi8oIKbn81l8oi+AKzfVcLPXtrA1WNOxhhY9mkBjy39vKESzMY9pfxp+XZqQnVUh8I8vuxz8vLLXfhXiXMDB9oOEtOmwcGDtuP8zJk6L6iUizQBumBzfjktR57/vbWQ8poQVcEQyz79quF4dmYqK7cdaHjeu4ufD3cUNXttdShMZaCxFNqekmoym+z1+3BHMV5P45f6vc8LG2qB/nPT/lbxvZXX+pjqAOnp8I9/2PlAgHvvbUyISqkTThOgC07tmdHq2PD+XUhLSSI1ycvIk7s2HC+tCjGsb2bD8+LKIGf2yWz22iSP0Cm1sRRa9/RkakKNtT7P7N2ZcJM7jbP6dGmoBHP+qT1o6dxBunk7YkTgrrtg0SLo3Bleftluldi+3e3IlEo4mgBd0D0jmZ+PH4zPa+fNh/TpzO0XnYbf58Xj8XDpWb258PQsAH7+8idM/9ZpDHWS3mNLP2Pm5UM4uXsaAGnJXh6eehaLN+4DoEuaj8evGs6C1bsAewf5+2nD+Ncm+/nTe2Vwz4SvNdQCPW9Qdy4dmt0Q28RhvTnnlG4n4F8hwU2YAGvW2MUweXkwapRttaSUOmF0FahLKgO1HKyxTWrTkr10b1G+rKC8hkBtHSLQKdVLRaCOUG0dPq/HeR6mJhQmJcmD3+eluraOQChMis/jNMytf+6lS6qPgwGnIW6yt1WptNKqIFXBMAKkJSeRmaal0k6YsjK47jp7R+jxwO9+Z2uJii4qjDH6BYtBmgCVcltdHfz613D//fb51VfDU09BWpqrYamjogkwBukQqFJu83hgxgx46SW7UGbBAtta6csv3Y5MqbimCVCpaDF5Mnz4od0y8dFHtsnuihVuR6VU3NIEqFQ0GTLEbpr/9rfhwAG7if5Pf6LVvhml1HHTBBgljDEcqAhQeLCGcPjIm6Org2EKymsorQoCUFhew+7iKr4qq4l0qCrSunWDxYvhzjttb8Ef/tA23A0EjvxapVS7aQKMAgdrQizbUsCVsz9k8p/f5+9rdjUktrYcqAgw883NTHh8JbNXbGdvSRX3vZ7Hd2et4q5XNrKrqOoERq8iwuuFhx+284F+P8ydCxdcAPn5bkemVNzQBBgF9pfVcNO8XLYVVLC7uJpfvpbHx7tL2zy3KlDLI29tZf7qXRRWBJia04+fvPAJ/9y0nwMVQd75tICbn81lb4kmwbhw1VWwahX07w+rV9s+gx984HZUSsUFTYBRYMnmr1odeyF3N4HacKvjFYHaZuXLfF4Pa3YWNzvn0/0HCYV1zihuDB8Oublw4YWwf7/9OGeO21EpFfM0AUaB03t1anXsayd1JsnT+svj9QgDujfuDzNAj4zmnRsyUpLwefVLG1eysmDJEjsfGAzaOcHbbrOPlVLHRH9KRoHh/btw7sDG8mODsjK4YlQ/vJ7We2u7Z6Tw0OSzyEixnazW7izioclDGwphez3CbyadSXqK9vSLOz4fPP64nQ9MToZZs+wq0YICtyNTKiZpJZgoUVQZoKwqRChs6J6R3KpcWVP1zXS/OhigW1oyyUkeguE69pdVc1JnP8leIatz6gmMXp1wq1fbfYP5+dCvH7zyip0fVG7RSjAxSBOgUrFq3z6YMsUuiklNhaefhmuucTuqRKUJMAbpEKhSsSo7G5Yvt/OBNTVw7bW2kHZtrduRKRUTNAEqFctSUmD2bDsfmJQEjz4Kl1wCRUVHfq1SCU4ToFKxTgRuuQXeeQd69oSlS21/wQ0b3I5MqaimCVCpePHNb9r9giNHws6dttP8woVuR6VU1IpoAhSR8SKyVUS2icgv2vh8ioj8w/n8ahEZEMl4lIp7/frBe+/Z+cCqKpg2De65B8KtiyoolegilgBFxAs8CVwCfB24SkS+3uK07wMlxphTgceA30UqHqUSht8P8+bZ+UCPBx58ECZNst3nlVINInkHOBrYZozZYYwJAs8Dk1qcMwl4xnm8ELhIRHQ5sVLHSwR+8hN46y3bXeLNN2H0aDh40O3IlIoakUyAfYDdTZ7vcY61eY4xphYoA7q3vJCI3CwiuSKSW1hYGKFwlYpD48bZ/oJDh8KECdCpddk9pRJVUgSv3dadXMtd9+05B2PMbGA22I3wxx+aUglk4EC7WT7l0NWFlEpEkbwD3AP0a/K8L9CymVnDOSKSBGQCxSilOlZ6ut0nqJRqEMkEuBY4TUROEZFk4Erg9RbnvA58z3k8FXjHxFptNqWUUjEpYr8SGmNqRWQ68BbgBeYaY/JEZAaQa4x5HZgDPCsi27B3fldGKh6llFKqKS2GrZRSx09Xr8cgrQSjlFIqIWkCVEoplZA0ASqllEpImgCVUkolJE2ASimlEpImQKWUUglJE6BSSqmEpAlQKaVUQoq5jfAiUgh8eQwv7QEc6OBwOkK0xgUa27GI1rggemOL1rig/bEdMMaMj3QwqmPFXAI8ViKSa4zJcTuOlqI1LtDYjkW0xgXRG1u0xgXRHZs6fjoEqpRSKiFpAlRKKZWQEikBznY7gEOI1rhAYzsW0RoXRG9s0RoXRHds6jglzBygUkop1VQi3QEqpZRSDTQBKqWUSkhxnwBFZLyIbBWRbSLyC7fjqScic0WkQEQ2uR1LUyLST0SWi8gWEckTkR+5HVM9EUkVkTUi8okT22/cjqkpEfGKyEci8obbsTQlIl+IyEYR+VhEoqqbtIh0EZGFIvKp8z13rtsxAYjIYOffq/5PuYj82O24VMeK6zlAEfECnwEXA3uAtcBVxpjNrgYGiMhYoAKYZ4wZ4nY89UQkG8g2xqwXkU7AOuDyKPk3EyDdGFMhIj5gJfAjY8yHLocGgIj8FMgBOhtjLnU7nnoi8gWQY4yJus3mIvIM8J4x5mkRSQbSjDGlbsfVlPNzZC9wjjHmWIpwqCgV73eAo4Ftxpgdxpgg8DwwyeWYADDGvAsUux1HS8aYfcaY9c7jg8AWoI+7UVnGqnCe+pw/UfEbnIj0BSYAT7sdS6wQkc7AWGAOgDEmGG3Jz3ERsF2TX/yJ9wTYB9jd5PkeouSHeSwQkQHAcGC1u5E0coYZPwYKgLeNMdES2x+AnwF1bgfSBgMsEZF1InKz28E0MRAoBP7qDB0/LSLpbgfVhiuBv7sdhOp48Z4ApY1jUXHHEO1EJAN4CfixMabc7XjqGWPCxpizgb7AaBFxffhYRC4FCowx69yO5RDON8aMAC4BbnOG36NBEjAC+LMxZjhQCUTNPD2AMyw7EXjR7VhUx4v3BLgH6NfkeV8g36VYYoYzv/YSMN8Y87Lb8bTFGSr7NxANBYjPByY6c23PA98SkefcDamRMSbf+VgAvIKdGogGe4A9Te7iF2ITYjS5BFhvjPnK7UBUx4v3BLgWOE1ETnF+k7sSeN3lmKKas9BkDrDFGPOo2/E0JSJZItLFeewHxgGfuhsVGGPuMsb0NcYMwH6PvWOMudblsAAQkXRnMRPO8OK3gahYeWyM2Q/sFpHBzqGLANcXW7VwFTr8GbeS3A4gkowxtSIyHXgL8AJzjTF5LocFgIj8HbgQ6CEie4BfGWPmuBsVYO9mrgM2OnNtAHcbYxa7GFO9bOAZZ1WeB3jBGBNVWw6iUC/gFft7DUnAAmPMv9wNqZkfAvOdX1B3ADe4HE8DEUnDriD/gduxqMiI620QSiml1KHE+xCoUkop1SZNgEoppRKSJkCllFIJSROgUkqphKQJUCmlVELSBKhcJyLXi0jvdpz3NxGZ2t7jHRDX3U0eD2hv5w4R+bGI/L8OeP/pIhI12wKUijeaAFU0uB44YgJ0wd1HPqU5EUkCbgQWdMD7zwVu74DrKKXaoAlQdSjnTulTEXlGRDY4vd7SnM+NFJEVTlHmt0Qk27lzy8Fuhv5YRPwicp+IrBWRTSIy26lO0973b/UezvF/i8jvnH6Cn4nIN53jaSLyghPrP0RktYjkiMhvAb8T03zn8l4RecrpRbjEqUbT0rewpbNqneufKiJLxfYwXC8ig0TkQifGF5xYfisi1zixbRSRQQDGmCrgCxGJltJlSsUVTYBRjg6uAAAClklEQVQqEgYDs40xZwHlwK1OfdEngKnGmJHYu5sHjDELgVzgGmPM2caYauBPxphRTp9EP9Cu3nqHeo8mpyQZY0YDPwZ+5Ry7FShxYr0fGAlgjPkFUO3EdI1z7mnAk8aYM4FSYEobYZyP7aFYb77zmmHAecA+5/gw4EfAUGzlndOd2J7GVkeplwt8sz1/f6XU0YnrUmjKNbuNMaucx89hh/H+BQwB3nZu6Lw0JoOW/kNEfgakAd2APGBRO9538BHeo76w9zpggPP4G8AfAYwxm0Rkw2Guv9MYU18eruk1msrG9lDEqcHZxxjzinP9Guc4wFpjzD7n+XZgifP6jcB/NLleAXDGYWJSSh0jTYAqElrW1zPY1lR5xphzD/dCEUkFZmE7mO8WkV8Dqe183yO9R8D5GKbxe7/dw6tNXl9/jbaGQKtpjPdw1256rbomz+to/v8y1bmmUqqD6RCoioT+IlKfhK4CVgJbgaz64yLiE5EznXMOAp2cx/XJ44DTk/BoVnce7j0OZSXwn875X8cOSdYLOcOqR2MLcCqA00dxj4hc7lw/pX4+9CicTpR0b1Aq3mgCVJGwBfieM5zYDdvwNIhNZr8TkU+Aj7FzYgB/A/7idJ8IAE9hhwJfxba0apcjvMehzMImzQ3Az4ENQJnzudnAhiaLYNrjn0DThrPXAbc7138fOOkorgV2TnHpUb5GKdUO2g1CdSgRGQC84SxgiXpOayWfMabGWX25DLsgJXgc13wF+Jkx5vPjjG048FNjzHXHcx2lVNt0DlAlujRguTPUKcAtx5P8HL/ALoY5rgQI9ADuPc5rKKUOQe8AlVJKJSSdA1RKKZWQNAEqpZRKSJoAlVJKJSRNgEoppRKSJkCllFIJ6f8AQRI5t7O+9SsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 463.5x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.relplot(x='petal length (cm)', y='petal width (cm)', hue='class', data=dataset)\n",
    "plt.plot([5, 0], [0, 1.4], 'r', linewidth=2)\n",
    "plt.plot([6.7, 2], [1.2, 2.5], 'r', linewidth=2)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Structure of the fully connected network\n",
    "\n",
    "A neural network takes scalars as inputs and output scalars.\n",
    "\n",
    "It is easy to see how input petal length and petal width as they are both scalars.\n",
    "\n",
    "But how to make the network output one of the three classes of flower?\n",
    "\n",
    "One way is for the network to output three scalars, each representing a class. The class with the largest value will be the chosen class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since there are two inputs, the input layer will consist of two units. There are three outputs, so three neurons in output layer.\n",
    "\n",
    "Now we decide on how many hidden layers and how many neurons in each layer. There are no hard and fast rules for choosing these. Often these parameters are chosen with trial and error.\n",
    "\n",
    "We will be using a single hidden layer with 3 neurons.\n",
    "\n",
    "With that our network would look like:\n",
    "\n",
    "![small_iris_network.png](img/small_iris_network.png)\n",
    "\n",
    "The first output would represent iris setosa. The second would be iris versicolor. And the third being iris virginica."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Implementing fully connected network\n",
    "\n",
    "Let us start by implementing fully connected layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class FullyConnected:\n",
    "    \n",
    "    def __init__(self, input_size, output_size, activation):\n",
    "        # Init weight matrix to random values\n",
    "        self.weight = np.random.rand(input_size, output_size)  \n",
    "        self.activation = activation\n",
    "        \n",
    "    def run(self, x):\n",
    "        x = x @ self.weight  # @ is the dot product operator in Python\n",
    "        # Apply the activation function to each weighted sum\n",
    "        x = self.activation(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is an implementation of the general formula for a fully connected layer (without bias).\n",
    "\n",
    "Note the weights are randomly chosen. Since we have not learned how to assign these weights.\n",
    "\n",
    "Now, we can make the network with fully connected layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "class IrisNetwork:\n",
    "    \n",
    "    def __init__(self):\n",
    "        self.hidden = FullyConnected(2, 3, sigmoid)\n",
    "        self.output = FullyConnected(3, 3, sigmoid)\n",
    "        \n",
    "    def run(self, x):\n",
    "        x = self.hidden.run(x)\n",
    "        x = self.output.run(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "network = IrisNetwork()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With a network defined, we can test it with the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "to_class = {0: \"Iris-setosa\", 1: \"Iris-versicolor\", 2: \"Iris-virginica\"}\n",
    "i = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data is the petal length and petal width "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1.4 0.2]\n"
     ]
    }
   ],
   "source": [
    "data = dataset.loc[i, ['petal length (cm)', 'petal width (cm)']].to_numpy(dtype=np.float_)\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The target is correct output for the network. Which is the type of flower."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iris-setosa\n"
     ]
    }
   ],
   "source": [
    "target = dataset.loc[i, ['class']].tolist()[0]\n",
    "print(target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now feed the network the input and the class represented with the largest output is chosen."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The network output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.77957496, 0.75517361, 0.78954698])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = network.run(data)\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The index of max in output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c = np.argmax(y)\n",
    "c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The class of max in output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Iris-virginica'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "to_class[c]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the weights in the feedforward were randomly chosen, the output is random as well.\n",
    "\n",
    "Now we learn how a fully connected network works, we can learn how to assign proper weights to model a given dataset.\n",
    "\n",
    "In the next lesson, we will learn about backpropagation which is the most popular method for training a neural network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Practice:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise 1:\n",
    "\n",
    "Implement a fully connected with a input layer of size 4, 2 hidden layer of size 5, and a output layer of size 2.\n",
    "Sigmoid is the activation function all hidden and output layers.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise 2:\n",
    "\n",
    "Input the vector $(0.4, -1.3, -32.3, 2.1)$ to the network.\n",
    "\n",
    "Note the output will be random since the weights are random."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solutions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution to exercise 1\n",
    "class Network:\n",
    "    \n",
    "    def __init__(self):\n",
    "        self.hidden1 = FullyConnected(4, 5, sigmoid)\n",
    "        self.hidden2 = FullyConnected(5, 5, sigmoid)\n",
    "        self.output = FullyConnected(5, 2, sigmoid)\n",
    "        \n",
    "    def run(self, x):\n",
    "        x = self.hidden1.run(x)\n",
    "        x = self.hidden2.run(x)\n",
    "        x = self.output.run(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.79984323, 0.73817321])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Solution to exercise 2\n",
    "x = np.array([0.4, -1.3, -32.3, 2.1], dtype=np.float_)\n",
    "network = Network()\n",
    "\n",
    "network.run(x)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}