Navigation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Navigation\n",
    "\n",
    "---\n",
    "\n",
    "In this notebook, you will learn how to use the Unity ML-Agents environment for the first project of the [Deep Reinforcement Learning Nanodegree](https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893).\n",
    "\n",
    "### 1. Start the Environment\n",
    "\n",
    "We begin by importing some necessary packages.  If the code cell below returns an error, please revisit the project instructions to double-check that you have installed [Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Installation.md) and [NumPy](http://www.numpy.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from unityagents import UnityEnvironment\n",
    "import numpy as np\n",
    "import gym\n",
    "import random\n",
    "import torch\n",
    "import numpy as np\n",
    "from collections import deque\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "from dqn_agent import Agent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will start the environment!  **_Before running the code cell below_**, change the `file_name` parameter to match the location of the Unity environment that you downloaded.\n",
    "\n",
    "- **Mac**: `\"path/to/Banana.app\"`\n",
    "- **Windows** (x86): `\"path/to/Banana_Windows_x86/Banana.exe\"`\n",
    "- **Windows** (x86_64): `\"path/to/Banana_Windows_x86_64/Banana.exe\"`\n",
    "- **Linux** (x86): `\"path/to/Banana_Linux/Banana.x86\"`\n",
    "- **Linux** (x86_64): `\"path/to/Banana_Linux/Banana.x86_64\"`\n",
    "- **Linux** (x86, headless): `\"path/to/Banana_Linux_NoVis/Banana.x86\"`\n",
    "- **Linux** (x86_64, headless): `\"path/to/Banana_Linux_NoVis/Banana.x86_64\"`\n",
    "\n",
    "For instance, if you are using a Mac, then you downloaded `Banana.app`.  If this file is in the same folder as the notebook, then the line below should appear as follows:\n",
    "```\n",
    "env = UnityEnvironment(file_name=\"Banana.app\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\t\n",
      "Unity brain name: BananaBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 37\n",
      "        Number of stacked Vector Observation: 1\n",
      "        Vector Action space type: discrete\n",
      "        Vector Action space size (per agent): 4\n",
      "        Vector Action descriptions: , , , \n"
     ]
    }
   ],
   "source": [
    "env = UnityEnvironment(file_name=\"Banana.app\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Environments contain **_brains_** which are responsible for deciding the actions of their associated agents. Here we check for the first brain available, and set it as the default brain we will be controlling from Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the default brain\n",
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Examine the State and Action Spaces\n",
    "\n",
    "The simulation contains a single agent that navigates a large environment.  At each time step, it has four actions at its disposal:\n",
    "- `0` - walk forward \n",
    "- `1` - walk backward\n",
    "- `2` - turn left\n",
    "- `3` - turn right\n",
    "\n",
    "The state space has `37` dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction.  A reward of `+1` is provided for collecting a yellow banana, and a reward of `-1` is provided for collecting a blue banana. \n",
    "\n",
    "Run the code cell below to print some information about the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agents: 1\n",
      "Number of actions: 4\n",
      "States look like: [1.         0.         0.         0.         0.84408134 0.\n",
      " 0.         1.         0.         0.0748472  0.         1.\n",
      " 0.         0.         0.25755    1.         0.         0.\n",
      " 0.         0.74177343 0.         1.         0.         0.\n",
      " 0.25854847 0.         0.         1.         0.         0.09355672\n",
      " 0.         1.         0.         0.         0.31969345 0.\n",
      " 0.        ]\n",
      "States have length: 37\n"
     ]
    }
   ],
   "source": [
    "# reset the environment\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "\n",
    "# number of agents in the environment\n",
    "print('Number of agents:', len(env_info.agents))\n",
    "\n",
    "# number of actions\n",
    "action_size = brain.vector_action_space_size\n",
    "print('Number of actions:', action_size)\n",
    "\n",
    "# examine the state space \n",
    "state = env_info.vector_observations[0]\n",
    "print('States look like:', state)\n",
    "state_size = len(state)\n",
    "print('States have length:', state_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Take Random Actions in the Environment\n",
    "\n",
    "In the next code cell, you will learn how to use the Python API to control the agent and receive feedback from the environment.\n",
    "\n",
    "Once this cell is executed, you will watch the agent's performance, if it selects an action (uniformly) at random with each time step.  A window should pop up that allows you to observe the agent, as it moves through the environment.  \n",
    "\n",
    "Of course, as part of the project, you'll have to change the code so that the agent is able to use its experience to gradually choose better actions when interacting with the environment!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score: 1.0\n"
     ]
    }
   ],
   "source": [
    "agent = Agent(state_size=37, action_size=4, seed=9)\n",
    "env_info = env.reset(train_mode=False)[brain_name] # reset the environment\n",
    "state = env_info.vector_observations[0]            # get the current state\n",
    "score = 0                                          # initialize the score\n",
    "while True:\n",
    "    action = np.random.randint(action_size)        # select an action\n",
    "    env_info = env.step(action)[brain_name]        # send the action to the environment\n",
    "    next_state = env_info.vector_observations[0]   # get the next state\n",
    "    reward = env_info.rewards[0]                   # get the reward\n",
    "    done = env_info.local_done[0]                  # see if episode has finished\n",
    "    score += reward                                # update the score\n",
    "    state = next_state                             # roll over the state to next time step\n",
    "    if done:                                       # exit loop if episode finished\n",
    "        break\n",
    "    \n",
    "print(\"Score: {}\".format(score))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When finished, you can close the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. It's Your Turn!\n",
    "\n",
    "Now it's your turn to train your own agent to solve the environment!  When training the environment, set `train_mode=True`, so that the line for resetting the environment looks like the following:\n",
    "```python\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DQN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "agent = Agent(state_size=37, action_size=4, seed=9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 100\tAverage Score: 12.19\n",
      "Episode 108\tAverage Score: 13.02\n",
      "Environment solved in 8 episodes!\tAverage Score: 13.02\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABcRklEQVR4nO29eZQk113n+72x5VZVXXuppVZL3VpbsmxZliXhRbbcsjEGgzFwDMbgmQdPhzcwBh4wmO1hzwCHnQEeA7axH8aAMd6FzRhbsmxJ2LIt2dq7tXS1WuqWupbuqs6qXGO574+IX8SNyIjMyKrIysrM+zmnT1dlVmbcyIj8xS++v41xziGRSCSS0UHp9wIkEolEsrNIwy+RSCQjhjT8EolEMmJIwy+RSCQjhjT8EolEMmJo/V5AGmZnZ/nFF1/c72VIJBLJQPHAAw+scs7noo8PhOG/+OKLcf/99/d7GRKJRDJQMMZOxD0upR6JRCIZMaThl0gkkhFDGn6JRCIZMaThl0gkkhFDGn6JRCIZMaThl0gkkhFDGn6JRCIZMaThl0gkkl3I6XN1/MkXn8Diymbm7y0Nv0QikexCnlur4i+//DROrdcyf29p+CUSiWQXstmwAABFI/sGC9LwSyQSyS6k4hn+sZw0/BKJRDISkOEv5dTM31safolEItmFbDZsANLjl0gkkpEh8Pil4ZdIJJKRoNKwYGgKdDV7My0Nv0QikexCNhtWT2QeQBp+iUQi2ZVUGlZPAruANPwSiUSyK6k0bZR6kMMPSMMvkUgku5KKlHokEolktKg0LBSl4ZdIJJLRwQ3uSo1fIpFIRoZKQ2r8EolEMlK4WT3S8EskEslIwDlHpSmDuxKJRDIy1EwbDu9NuwZAGn6JRCLZdWz6LZllcFcikWyD9WoTv/O5x9G0nL5s/y/vfApPLW30Zdu95H1fPYZHT53L9D0rXmdO6fFLJJJtcc9Tq/jbe4/jyAvlHd92tWnhT770JD774PM7vu1ewjnHH3zhKD5+/3OZvm8vO3MC0vBLJCNDuW4CAKpNe8e3TdtcqzZ3fNu9xHI4HA4sbzQyfV/f8Mt0TolEsh026q4xqZnWjm+75hn+9aq549vuJQ1PNlsq1zN930qzd9O3AGn4JZKRoVyTHn/WNH3Dn63H38vpW4A0/BLJyEAef38Mv7vts5XhMvwNy/0sVzYa4Jxn9r5S45dIJJmw4Wn8tT4Y/qGVekzX42/aTqb7NrCGnzF2IWPsLsbY44yxxxhjP+89Ps0Y+xJj7Cnv/6lerUEikQSU++rxB1JPlp5xv2kIqbFLG9np/Jt+cHfwNH4LwC9xzq8CcBOAn2WMXQXg3QDu5JxfBuBO73eJRNJjAo9/54O7VdM1/A3LQc3c+QtPrxBrIrLU+SsNC3ldgdaDebtADw0/5/wFzvm3vZ83ABwBcAGAHwDwYe/PPgzgLb1ag2Q4WN1s4NyQSQT9oFzro8ffCC42a0N0LEnjB7LN7Nls2D0L7AI7pPEzxi4G8FIA3wCwwDl/wXvqNICFhNfcxhi7nzF2/8rKyk4sU7JLue3v78f/c/uj/V7GwEMef6WPUg8ArA1RgFeUepYzNPy97MwJ7IDhZ4yNAfgkgF/gnIdKBrkr9sUKfpzz93POr+ecXz83N9frZUp2KY7D8fgLZZzZHB5j0S9I4++H1CPKO8OU0il6/FkWcVWbVs+Kt4AeG37GmA7X6P8j5/xT3sNLjLG93vN7ASz3cg2SweaFch110+lbf5lhwXa4HzDsZzonMFxSD52XCsta6rF6VrwF9DarhwH4IIAjnPM/FZ66HcA7vZ/fCeCzvVqDZPBZXNkEEPasJN2zWQ8Mbz+Cq+LFZn2oPH7X8O/dU8g4uGsPrNTzSgA/AeB1jLEHvX9vAvD7AF7PGHsKwK3e7xJJLMdXKwDCWqqke6hPD9Afj7/WtDFTMgAMVxEX5fFfOF0YKI2/Z+/MOb8XAEt4+nCvtisZLhZXpOHPAjL8jPUvj388r8HMuNCp3zRsz/BPFfHAiTU4DoeiJJm99Gw2LIwNqsYvkWyXY57UIzX+7UHtGmZKuf7k8TdtFAwNUyVjuIK7nmy2f7oI0+aZ7dvAZ/VIJNshkHqkxr8dqEHbeXty/ZF6TAtFQ8Vk0RguqcdzSPbPFAFkU8TlOByVpt2z6VuANPySXUzdtHFqvQYg0FIlW4M8/oXxfF969VSbNoqGiumiPlxSj2f4L5gsAMimbQNVOUuPXzKSPHOmAs7dL9WwaPyfe/h5vOf2x3ry3n/070fxLwmToKh4a2FPHlXT3vF+ObWmjYKuYqo4XFJP03JgaAoWJvIAuiviet9Xj+Fvvnqs5fFeN2gDpOGX7GKOe4HdQ3vH0bSdoWju9ZUnVvDp75zqyXvf/tDz+MoT8WUxZcHjtx2Opr2zF1Ly+CeLxpBV7trIaQrmJ3IAgOUupJ7PP/IC/uVbrRfqYNC6NPySEWTR0/evOG8cwHBk9tRNG5WG1ZOLmFvoFv++G3UTBV3FRME1Jjst91Bwd7qko9K0hyZY37Ac5DQVOU3FVFHvSurZqFt49mwVZuQiXPWGsBR71JkTkIZfsos5trKJ8ybymCq6+d/DYPgblgPL4T3Zl4Zpw3Li37dcszBR0HxjstMB3lozCO4Cw1PE1TAd5DTXjM6P57sK7pZrJiyH47mz1dDj0uOXjDTHVys4OFdCTneN1TB4iXUvcFdpZJ9SWbecFu+R2GiYGM/rKHi54Ttp+DnnqJqu1EMX8bNDYvibtmD4J3KpNX7OuR9wp1oVQmr8kpGFc47FlQoOzJaQ83qSD0NKJ2UnVRrZ7ovjcDQtB6YdL/WUaxYm8hqK3kV0J6WehuWAc6BgqJgq6QCAtcpwZPY0TBuGZ/gXJvKpG7U1LMePs1DKMhEMWpeGXzJinK00ca5m4uDcGHI6Gf7B9/jp4rWZscdPRiTR46+7Hn8g9excERfdXRT1wOMfGqnHcvw70oWJHJY3GnCczvEbsYXG4upm6Dkp9UhGFvKCDs6V/FvpYcjlr5PHn7HhJQnJSvL46xYmCjoKZPh3sFEbXWSKhjZ0Uk/Dsv070nkvY+pMiqwlGooDAMcSpR4Z3JWMGKR7HpwtIad5Gv8OpyD2AvL4s9b46W6ovcevoWjsfFYPefwFQ8Vk0ZV6hqWIq2k5/h3pgpfSmaY9M9VVzI3nWqSeTU8GHNh+/BLJVjm2uglDVbBvquhrqI0+z2o9vlrBwyfXt/Ue9R5p/OTxJxl+V+PXO2b1HD1dxtPLm7HPbRVf6jFU5HUVRUMdmlx+N52TgrtuEddKCp2fArsv2TeJlY2GfyEAXKegaKiZNHtLQhp+ya7k+EoFF80UoSoskHr6rPH/6icexn/7xMPbeo96jzx+uqDEBXfrpo2m7WA8rwVST4LU9JuffhTv/ddsK4tpW7TtqaIxRFKP49+Rzo15RVwpcvlJ47/2wj0Awpk9vW7QBkjDL9mlLK66GT0A/C9WPw3/WqWJ+0+c9ZudbRWKU2Qd3CUJyYrx+MnITBQ6e/znamYqj7UbSFYi6WKqNDz9ehpWkNUz7c0bSDNhzPf4L5wEEA7wbjasngZ2AWn4JbsQy3Zw4kwFB+fGAMDXUPuZx3/XE8tw+PYGlXPOe+7xN2M8fjIyE3kNea294a827cyNsij1ABiqfj1NQeopGioMVUm1b+RAXHPBHigsaE8CBO0teok0/JJdx6n1Gkyb4+Cc6/EbuyCP/44jSwC2FxQ1bQ7q1LCZcVaP7/HHVO6SkZnI61AUhoKuJvbkrzYtnK02M20pUROCu4Bn+IdQ42eMYaqkp9q3jboFVWHYU9Bx4XQRx4QA76aUeiSjiJjRA6DvefwNy8bdT66CMTezKE5OSUNduHD1TOOP+YzI4x/Pu8akaKhtPf6m5WQ6l1dM5wSAqaI+NAPXG2aQxw/Q3UznfSt7WVaMMRycLbVo/FLqkYwcNHXLl3pI4+9TVs83Fs9is2HhpgMzALaeA183RcPfo6yemOIhX+opuKmUBUONvXOxhR5CWRpm+rxIvpgsGijXzS1fQHcTDcv270gBYLKopypO26i7WVYAcGB2DM+sVvzCLxnclYwki6sVTBZ1P1hGt9L9yuO/88gS8rqCw4fmAWxd7hEL0LIP7gZ5/FGZhoK7nTx+0cvPUoqpNW0wFhzH6ZIBzt1A8iBj2Q4cHuwX4O5bmglj5ZrpH4+DcyXUTBunvfz/zUZvp28B0vBLdiHHV4KMHgBCHv/OG37OOe44soxXXTrnX4i22uBMjFFk3TKBPH7OXc9dhHLEycMsGFrsXYu4piyDr9WmjaKugjE3L52KuAZd7qGLLUmRgHs3kyY4Lnr8FMsiuafSsHpavAVIwy/ZhSyubuLg7Jj/u6YwKKw/Gv/R0xs4tV7DrYfmfY16q0a7HvL4s5V6xM/Gihj+cs0NJJLUUkwI7op3MplKPV4vfmJY+vXQZy5KPdNFA+s1s2O/HtL4AeAST9J8cmkDddNGzbSl1CNpzx984Sh+8WMP9nsZXbG62cANv3sHHj11ruW5zYaFpXLD94IAN1sip6khqafSsPDK3/8y7ls809O1fvXJFQDA666c9w1nktRzrmript+7E9965mzs8+Txj+e0HgR3gzVFJbGNuomxnOZ73ElSj/hYt1LP3/3HcXzPn98Ta/CoFz9Bd079Grr+zg99E++LjDxc3qjj5b97B544vZH6feh4isHdyaIO2wlaLgPAlx5fwmv+6K7QMdrweicBwPx4DmM5Df/9c4/jyt/6AoAgHtMrentZkfScR0+dS9UbZDfx7NkqljcaePz5Ml50wZ7Qc8+shjN6CENTQsHdpXIdp9ZreOL0Bm46ONOzta5uNFDQVcxP5PGsNzAjSeo5trqJ0+U6Hnx2HS+/eLrlefL4Z8aMnvXqAVobtbkN2oKvelJwN2T4u/TGv3bsDI68UMZDJ9fx0v1TLe8rGv5+9+u5/5mzmCqGDetzZ6tY2Wjg6OmyP/GtE1RXImr8dDezVm1ij7eNbz+7hhNnqjh9ro6LvfNa1PgZY/jLt78Ujz9fBgDoKsMPvvSCbexhZ6ThH3Dqpp3Yg323QkYnzrhEM3qInKaEjBsZqV4PFKEBIgCEdgfx26R5q0kl++QhTpcMPJVxPxzxohjt17NRNzGeCwxdYnBXeKxbo0xjMu88stxi+Gum7X92QNg47jQNy0alabekq9aa7mdWrqe/IPsav9Z6N7NWbeJiuEaezoulsmv4HYdjs2lhPB8ck1uumMctV8xvYY+2hpR6BpyaOXjzS8noxPVrWVypgDHgopli6PGcHjb89MVNKkTKilozMFp+Z0szfptk8JPG7wUefy7zubv1Noafxi4SRUOLjVOIj3Ujw1ClNRAUuoXfN+zxFw0Vhqb0pV8PXdBqkUQBOp/EZmmdoGQDQwuncwLhixqdFzSkZaNhgXO3krpfSMM/4NRNZ+DaFZOBWY+ZwnR8tYJ9UwXk9XA6W05TQxe4HfP4BX26U58bktySpDfy+GdKBhweDvZuF/GiGL0DLHtDWIiCocYWaNFj43mtK2+cKq0vXxjD0dMbOLkWniFbbdoo6IGRY4xhqqjHHv9eQ/tVj3r83u9in/xONG1P44+TeoR9i54X0SyrfiAN/4BTaw6ex99O6llc3cSB2bGWxw1VCaVDkqff64EiYkZKoUNwd8mXejp5/K5xyDKXXzRk0cIoMXUQcLN6TJu33BnQBe2CyUJXUg+lIf70qw8CcOUekWhwF+hfvx4yyFHDX9+Gxx8y/KVWGSt6XgQFddLjl2wRV+MfLMNfTTD8nHMcX6m0BHaBVqmH3qPXA0VqXg46AP//LXv8Jnn8bvveLAO84mcTvQMUUweB5FiFaPi7kXooLnP4ynkcnCu1yD2VmKZjfTP83jaj5w0Z/i1p/MLd6UReg6qw0J0FFarReUG9k8alxy/ZKvUB1PjptjqaK75UbqDStHHJXIzh15RQAVcg9fRW4682bX8EnqYqMFQl0fBTO+Nq04715uvWTnn8gdTjOBybDSuUHpg0hYvuos6fLHSVY398tYI9BbfS+tZDC7hv8UzIcxbjJMRUqT/9enyDbEX3fQsev/ceYh4/YwyThWDfxBbXgdRD3VKl4ZdsAc45aqYNy+GpBjzvFnyNP2JcqCd5nNST01Q0BE+2tkMav5uREk2FjDfYS+W671nHef1knEkHznLtYrxAvAPcbLYGEulCFr1oVps2VIVhfjyHStNO3Q110au0Zozh1kMLMG2Oe55aBeCeo9UYqcetcO2H1EMef9hZos+vm3kLcZW7gCv30L7ReTCe13ypJ9pCox9Iwz/ANL1eIQBgxrTjzQrOOd5/9zEsZ1QvEEg9Ziizxe/KGePxR/P4u5V67jyyhF/5+EP+vwdOrKVcq+VLPEByKmTDsrFWNXGNV5cQZ/gblgNDVfwvfLZSjw3NG9UnBnfjvMtCgmRFrRVIp06r8x9frfjH7Lr9k5gs6rjj8SVvXe45Woy0IJj2ulhmmdmUBvLEk4K7G1tK54wY/qLuS2Wk719zwR4/rTPaNK8fSMM/wIheXi/lnuWNBn7v347iC4+dzuT9yFjbDg9pqifXajBUBed5s0tFcpoSyerxgrspDf/ffPUYPvvQ8/iPp1fxqe+cwj9949lUr6tGZIqCocYGlOlLfc2+PaHfReqmjZym+C13s5V6HIx5FxQzUuEMAMWcePGi1hOtckfBULvKs680LJwu1/22A5qq4LsOzuDbz6757+luM+rx6y3HfydIyuoJNP4tePxaa/yCLprkAFyzbw82GxY2G5ag8UuPX7IFwrnbvfOcSFvPqkmaaHDE2/3lch1z47nYIdM5TY0P7qbM6inXLLz28jl87dcO48BsKTEXP0otJgc97i6DbuPJ448r4mpYbu926sOStcdPhkQcxtKM6SeTNHeXitWmSl4ueop0y+Ne4ZbYVO/S+TE8t1ZD03JaWjIT/erXQ1KP5YSzmupb8fi91xgtHn8QuF7aqMNQFVw+71YDL5fr2GhYKOgqdLV/5lca/gFGNEC99PhJ681qApZo+MXskaWNOhYmcrGvScrqSWs8N4Rc9naDSESalgPL4WHDr8f32SEZ7MBsCUVDjS3iqps28rriG/7MPX6vOrdpBU6A30hMME5JPYeqDQsFQ+vK46eKXVGeOzBbgu1wPHu26sdDClGpp0/9esSAsug00M/VZvosOcqeiko9kyUdaxVXxlopNzA3nsN5e9y72KVyI9SuoV9Iwz/AiJkJvUzpJOORVXfMatPy9WhRR14qN7AQI/MAMXn8nseeVuPfEPrVFHQV1RTdMYORgZHgbsxdBt3SnzeRx8JEPl7jN90xfSXP8GY5jKVh2RjPtfH4Ywx/rMbfpdSzuLIJxoCLZwLDT+02jq9Wgnm7eqvUA+x8vx7xDkO8YxbPo82UXn9cHj/gxi+atoNq0/adGXJoljfqoQZt/UIa/gFGPFl72bI48Pizk3r2TroGPlzoUk80/Ekef9W0OwYIbYdjo2GFPf4UUk/FHxnYObi7tNGArjJMFQ3Mj+diNf6GZSOvq9BUBXld8d8/CxoJGn+cV+pLPZELGEk93RjlxZUKLpgMV1qT7LO4stkyaJ2gi8tOe/xnK03/AlkXMnvqwrmVVudvWA4MTfG7nhLihZOcmXnvvF4uN1rqKvqBNPwDTM3cWY8/Kzmp1rRx/p4CgOCLX2va2KhbmBtPkHq8lg1k5KtCgLhTywqSVCb8CVRaKqknzmgldbZcKtcxN+bGJxYm8rEaf90MBnOP5bRspR7L9oPGYrynGROADPL4w9unCtu8rqJoqKmM8vHV8NAcANhT0DE7ZmBxpdIyaJ2Iq3DtNZbtoFy3fKdD/P7UhWOaVudvWDZyMTq936+nYmKpXMf8eA7jOQ15XcFSuY5y3epr8RbQQ8PPGPsQY2yZMfao8Nh7GGOnGGMPev/e1KvtjwKNHcrqCaSejDR+08LCRB4KC7xKMpSJHn9k/KJofDvJPZRFEUygijfeUXyjJXizpYQGZysbDd+rmx/PYancaLkTIY8fAEoZ9uS3HQ7T5vEef4zU0zad0x+I3rmylnOOxZVNP6NH5ODsWFjqiWj8VOG6k1IPVdDu9ZyOekTjpwt82lz+puW05PADQfzi+XM1bNQtzE/kwZjrECxtNLBRM/vaoA3orcf/dwDeGPP4n3HOr/X+/VsPtz/07JjHn3FWT61pYyyvYVLMfvCkkcTgLo1f9AyZaHw7ee/R3ihpg7tVX+oRO1smSD3lIDC9MJFHzbSxETHsddPxDX/R0DLT+MUBL0C4cpcaiYlZParCkNOUmMrdIHV1qqR3NMrLG26ldVzdxcG5EhZXN4XPMOzxU4XrTnbopHPt/DiP37Qx791tpk0xbVhOSyon4BanAcCT3lAXcmYWxvPD7/Fzzu8GED+KSJIJO5XVQ152I6OLCxUKTRZ1wfCn8/jp4lNr2r4X28mIB5WS3Xn8pIFH8/gbltMy11YMTM9TIC8S4KU8fgAYy6mZefxUzzGe0uMH4i9gdFwA1+OPk3o453h6eRMPn1zHXUfdZmxRqYceW91s+sc1KvUA4QrXdmzUzdStORyH4/Hny3j45DoePrkektwoo4c8/mhWz/y4e/y60fijgV0g8PiP+obfPR/mJ3JuOmfd7GuDNqA/g1h+jjH2kwDuB/BLnPN0JZSSFsSsnl62ZqZ85Sw8fmozUTRUt3qzQlKP5/GPJxl+13DQflZNG7MlA8+fq3c04uTxk2Es6hqatgPLdqC1yaWOKz7yUyHNQFOnRly+ZzcRpO5d6uVvA66hEKWerAKb5PHTepqpDH84zuE4PCR3TBYNPHc23F4ZAB57vozv+8t7/d8ZAy4T9pGgzJ7HvKlSUY8fcCtc09QK/PSH78dEQccHfvL6jn/7mQdP4f/+l4f838+byOO+Xz8MIMjh3+ulVjYiHv+cZ6BTa/ym3fK5Am6MgzHg6Gl338Xz4ouPL6FpOX3t0wPsfHD3rwFcAuBaAC8A+JOkP2SM3cYYu58xdv/KysoOLW+wEA1eTwu4MtT466YDzt0USVHqWS7XYWhKoidk+B5/kG89M5bzfm7/RY1q/MWErJYo8cHd1oHr1IiLAtMkGUQDvKLHX8owuEse/5i3f6LUE5fH7+6HGipiIyei4Gv88U3UqNf+e7//anzwndfj0//llX6OugjJP2T48zGSSNo4wqOnzuErTyynaqBGkuHfvOM6/PiN+3G6XPcvsIHU0+rx100Hc975lFrjt+M9flVhmMjrfnHbvHBe0IV4mDX+FjjnS5xzm3PuAPgAgBva/O37OefXc86vn5ub27lFDhChAds7Etzd/jZEzXcqIvUsTORaUuMIUeO3HY6m5fhdLjsZ8I1IU6xOffWJoPhILOBqfW1UppoXPH4R0eMfM7IL7tIFuWioYCw+ndNQ20s90YvcVNHAuZrZ0tv/rOehv/6qBRw+tIBrL5yMXdOFU0WoCsOzZ6so6GpsNXYaw08dW8XGb+2oNS0wBnz31efh1kMLANy0UkCUejyN30vnpLvQUk7FWE7rwuOP1/gBV+5xuHvB3ePl7Isy5tBq/HEwxvYKv/4ggEeT/lbSmaSOjFnTzDCdkwxMwVAxXQoadS2VG4kyDxB0QGxYjn/xmPU8tI5ZPb7U437Zgu6UaT3+cHA3+tpoYHosp2Esp7UUcdVN298PN6snm+AunQd5XYGuKLHpnFHDX9DDhj+adknDyM9FvF8y1JSrnoShKdg/7Y7PjJN5AK/CtUOjNurYCsSPdYziTvtSwRgL6gk8z3ut0oShKb4GT46TaXPYDkdBVzGR17rQ+O3YrB4gSOkUnZl5IXGh3xp/L9M5Pwrg6wCuYIydZIz9FIA/ZIw9whh7GMAtAH6xV9sfBWo75vFnV8BFa3YLhQw0LQc108byRnLxFiBo/JbjG6lZX+rp7PHndcWXO2gMYCeJyL9I6eHgbnSbvscvXLjcQF7g8XPOQ1kgYzkVlWY2c3fJgOU1FbrKWoK7uspaPO5oz6EWj9/Psw8bwfVqE3ldiQ3WRqGBOkl/O+0d/3bHjzq23nBgGncdXW4JqkepCnGKfVMF6Crz32Ot2sRUUQ/u+LzPjWSuvK5iPK+n7slP3VbjoAujeE6MhMfPOf8xzvlezrnOOd/HOf8g5/wnOOfXcM5fzDn/fs75C73a/igQMvw70rJh+x6qaGDIqzxbaWK53Ah5RFF8jd+y/feY9aSeTgPXy7XI6MG0Uo+nyauC0YwbYkKNuMjLA9wvvKjx02eYFzx+zrPpyS/2hddUJSTPNBOMU3TgejTtMqltw9mK2dHbJ8jjTvL407SGWFypoKCreMdNF2GtavpdP5MQU1I1VcFFMyUcXw2knqmiAUNVwFhwwaTiLdfwa6nn7ibl8Yv7Jp7T80Jx4qgFdyUZUjeD/iw99fgzzOMnA1PQNd+rfH7d7Vg4307qEdI5yVj6Gn8nj78RLpHvNDRdXGvUaAWvFYK7XiMuMT4xP5ELafxBX5cgqwfIpkMnGbCc5nZ8bIby+J3YzJNoSmtQrBYUcAFBJgyxXm2mNvyU2RNt0EakaQ1xfHUTB2ZLeO0Vc9AU1lHuqTYtlITtHZgtBR5/xV07YwwFXfU/N3KgCrqKiYKOjUY36ZxJFzV338Rzeiyn+eePbNkg2TJ10/abPfVU4/eKgLK4q6iFPH7XgDzhp70le/z0BWtYjp+NknaSVbQpVtLM2ShiJWv0teLdVlxXUWrURlJOw5cTgpYNQDYdOoO7CVfqafH4Ywx/MTJXICr1+G0HIt74WrXpt23uBGX2RBu0EWk6dC56Q14m8jpuPDjdMsg9SnR+wsG5Ek6cqcJ2eGjtBT1otucbfqM7j79h2bFZPUAglYnyDlXvAv0dwgJIwz/Q1JpBD/ad8fizlXqmvS/h0UiFYxxBywbbD4qOef1POvXkd9vgxsyc7dCoLdqLn9Yt7gcQ31V0fjyHhuX4RsQPwGpq4vtslcDjV6CrSsgJaCQY/kI0qyfSN386QeMnuSQNpPGXcvGGf7KD1NOwbDx3tuq/z+ErF/D08iae8YK1cUSP2SWzY2jaDk6t1bAurD2vq35Wjxgcn+hC40+6qAKCxh9xCObHc1AY/A6t/UIa/gGmbjoYz2stKXycc/zWZx7F/c9kUzidZTqnmD0y6Xv84QrHOIwYqadgqC1adRwbdSuUN51e6okx/H5gOBzcjRp+v4jL0/kpgJjrscevqQymE87qidX4dc2dN+D3PgqnrhYNFYaqtEg9a11IPXPjOYzltESpZyoi9dy3eAa//PGH/PnRz52twuGBZETpme3kHjerR5B6vLuOYyubobXndcU/JrWoxl9PF3RPqtwV9y3uvBjLaYlpyzuFNPwDTM20vdv7Vl33I/edwJce75z+lgYxq2e7WShi/5tJ73aXDP98Co9flHqKhtaSlhhHuR72+NPn8dstGSnBa4N5AHFdRSn+sLrp6vyNiMefpcZPd2I5XYGhKjCtqMcf1y6BpBzX6EZTVxljmB0zsLoZGH7b4ThXM32j1gnGGH75DZfjR162L/Z5qnAlqedD9x7HJx44iUdOnQMAHIvMYN4/U8TlC2Nt5R6x+hgI7joefG4dDg8krIKh+kHdekTjt70q5k600/i/65IZ/PiN+/HS/ZOhx3/0hgvxrsOXdXzvXiMN/wBTN92c5ZwankdLHmBWLQHE995uhbAoKWiqgom8ho2Ghbyu+IHqOHI6afx2SC5KGoUoUhaGsACB5twpj75qWi0av6Ep0BTmryGpq+h0ZGB51OPPcgpXOJ1TgeV0Du5S0JFSUeOqlOci7aXP1UxwHujXafhPrzyAmy+PL8B0j7+O9WoTddP2C7TIo6egrNgL6NZDC/jmM2dxLiEgXGmEA/LTJQN7CrqfDUTHJa8FGn89ovED6KjzW7ZbSJgk9UwWDfzuD17Tcv684pJZ/PSrD7Z9751AGv4Bpk4ev6bEzg+NK7nfCo3QRWV7mnStaYOxwIMXg2Dtbn99jV/I4y94hr+dx1837ZbeKJrqesadhrFEA4WEqI8ndRWNpir6Hj9V7voefzbpnAoDdJW5Uk8ouBvfM57SDKndRPS4AMDCeC5UhJa2eKsbqDXE14+dQc3LUrvD8+gXVzYxN54L3a0dPrQA2+H4ypPxXn/0Lo0Kub7z7Hpo7QWjNasnr6n+edJJ508auzgopF41Y6zAGLuil4uRdEfN8/h1lYU9fs/IZDXIuhFzN7FVqAMkGfnJmEKXODSFgTGq3PW8U131jHCyAfdbMudbs3PSSD1xGSliT/6krqLBMA73GIheORAEPLNK58xp7mcaDe4mBSCDRnLu+iteGqR48XUHygQpqXQ+TaaUetIwVXLbNnzpyBJKhorbbj6IIy+UcXKtiuOrFV+qIa69cBIzJSNW7uGchwq4iINzJf/Oitae11XUqNNrnMffwfAnjV0cFFKtmjH2ZgAPAviC9/u1jLHbe7guSQrIuzG0aCaHeyJn1es8TkbaKq4XHRjhacp3bhPYBVzPLacpvuE3VLdYqdM0rWhLZiJNT/644G70tWQ45yMaf05TUTJU/66rReoxsg3uUpqoW7nbWeqhhmR0xxIXz1iYyGG9avoXLerTM92F1NMJav9855El3Hz5HN70Yrery5ePLvupnCKqwvC6K+dx1xPLLSnMTU9+icor4qAYX+oR8vjrwt0YpVl26skfFM31Nztnq6S9XL0HbkO1dQDgnD8I4EBPViRJTd2rHNRVJdQrv+57/FlJPdm1hogWRQVpb+09fiAYv1hrWr6R6uS5R1syE2k9/riMFPG1KxuNUCMuEbH7aDS4qygMRSObnvzk8QNwNf4UlbuGpmCmZPhZR3EXOQq2kxzUC6lnsqjj6OkNLJUbOHxoAZfMjeHAbAmffOAkzlaaODjbOt3r8KEFbNQtfCuStRY3MQ0IxwjoDrOgK4LhD2osJnyNv4PH730nklo27HbSrtrknJ+LPNa7PsCSjlCHyoKuxmRyuCflerXpp8ZtBzFtbbsaf9TA0Bcx6jHHYWiKH9yl9yh2yOrxWzIX4jz+ZKNr2Q6atpPK40/qKjpV0gOpJ+LxA16jtgwGrruTvdz31ZRIhlebXPP5ibw/LIaam4WeH6e7AvdveiH1TBcN2A4HY8AtV7hB4FsPzeOhk665iZvu9erLZmGoSovckzTYnd7DbZfsdWgVC7iaNhTmGvFA429/XPxZxgktG3Y7aVf9GGPs7QBUxthljLG/BPC1Hq5L0gExBc3QlFBVLXn8Dk8/TagdDdPxpZLttm2ISgpUxJXO41fcPH4zeI9Srn0ef5LHX9TbS0TRgiaRgqH5z7frKuq2HXY//6jHD7hFPNkEdwOP39DSVe4CrmEnDb9mtranWIi0lz5bMaGrzA9MZwEF91+2f8qfr3DYy9cH4qd7lXIaXnHpDO44shRKLxbrO0QunimBMTeQTBdot4Ar8Pipo+d4SsPvSz0J6Zy7nbSG/78CuBpAA8A/ATgH4Bd6tKaR5OjpMs5shvu3c87xtWOrsbnzwe2p5/HHaPxANpk9TdvxPSXxAlNtWvjsg6fw8fufa/n31NJG7HtFpZ7JmGZWSZDGL1ZnugNFwj1nHjgRSAB04Ys2xYq+rmWdjXgjArh3GZTHv9Smq6jYbz7R408p9dRNG99YPJPwXNjjj/bjTzL8CxO5UDpnVBun/aKUzvVqE5Ner5usoLsH0dhff9EU9hR0aArDhV5r5yiHDy3gxJkqjq0EbZuDdiDh/cjrKs7fU/DPNXqsYTn+5DHKtsrrCnSVdQ7u0vEc0OBux0s3Y0wF8HnO+S0AfqP3SxpN3vmhb+JN1+zFb7/5av+xR06dw9s/8A380/95I15xyWzo78XGUnokj1/s079WbeIAWr2mbmiYNsa9L6Do8d/+4PN496ceiX3NS/btwWd/7lUtj1ebNqZLgZG/fGEchqaEAnBJ5DTV78dPFbRFXYVpc5i2A11V8PEHnsNv3/4YvvFrhzE/kW8ZwkKUcipOrbcx/AlDwukxP4+/3MDNl8VftNzRgpTV05oFMjOWw8m1Wsf9BoB/feh5/MonHsbn3/UqXH3+ntBzbl/4QOM3IxO4knTohYk8VjYabsFS0/bbXIvr11Xme/zU1jhLLp0bg6EpeOOLzvMf01QFb37JXjz2fBl6wtpvvsz9PnzrmTV/vGW7Y3b9xVOh7whd0BteW3Ay/OT1d0rnpO9B0kV1t9PR8HPObcaYwxjbE6PzSzLibKXZUnB1xvtdrJ4kfI/fcPP4q7XAiIU8/gyKuBpW4PGL701DOr74izeH9OE/+vcn8LVj8d5ptLLyhgPTeOQ9b0h1y0waf61pB0E6of3CnoKCk2s1cA48vbzpGX7L640SCe7qWtvgbtCLPzm4W2lY2GxYyR5/yUC5bsGyHb+hl+gtv/rSWfzuvx3BybUq9k3Fe7YEXSC+9PhSi+Gn1h0AYvvxJ3ml8xN5OBw4s9mIDe4yxjA/HsQB1rpoyZyWGw/O4OHffoNveIn3vPnqtkFEqpQWB8VUzeS7tD/5kZeEPvu895nUTBsN0wm9ZiJFo7bGiOTxbwJ4hDH2QcbYX9C/Xi5slGhY7mi56G0//R7nfQRNv1rL9MMe//akHtvhsBzuGxYxnZOM4yVzY7hwuuj/u3i2hDOVRsvYPnpN1MCk1UnFdE4/uBvpj09GiqYulWsmxnJa7CCSdrGBWhuNnzx+f0B8gkxFRnK9Znpj+sJft8OH5gG4qYudoG3F5a/XBY9VzOPnnLeXesaDlM6k1NWFiZyf+dNNn55uiBp9wPX6k7x9wL3TVRUW+m7UEoK79H7iXAUy9HXT9jz+YFvdePzDrvF/CsBvAbgbwAPCP0kGUIAvmtNNhj/O+xCLTgyNRYK7gSe73SKuYDi0Hvqd1hAdVAK4xoLz+DuVuHzxtOR01Tf8YjMxILjNJ1mCyv2jLZmJTnn8SRkigBvcrZk2XjjneuFJHn/Qb76JhmW3GLiDc2M4OFtK1VOJLmiPnDqH0+fCIx1Fr15TmT9s3XI4OE9OOZwXNPxa04q9u3E9fpJ6zNQtmXsNY6zFMw8K+zoHn+lY1Ez3DlK8Y6VGbe1oxMRsBolUq+acfxjARxEY/H/yHpNkABn4aIbHpvd7nPch5iy3BneDn7fbr4dOcDKeotQTN6gEaO0DQ3DOE1+Thpym+KMaxeCuuxYvy2aDPH436Bdt0EYUDNUf3B5H3KB1grb97JkqgGSPX2xt7AZgW9/r1qsWcN/imY4e5tJGHfumCgCAO4+GLxSix2+oQYaXP2+3TXAXAE6X67EVr/Q3NFegmyEsO0HUM293zKL4hr9pox65KKdpzUzfsaHO42eMvRbAUwD+CsD/AvAkY+zm3i1rtKBc7mhOt+/xx0o9QVZPa3DXfY6GmW8HOsGpgVpU6olmUACBQYka/oblwOGtWRdpIY3fbcTlBXcjg1HIOz1OUk+kJTMRfV2UuEHr0dceP+NuY65NOifgXnyThnYcvnIeps39BmVJLJUbeMUlM9g/XWyRe8TKXU1lfpO2ToZ/diwHxoBnz1bBebzBnJ/Io1y3sLLZgOXwXWb4w555u7u0KOThU8woH/X4O2j8o5LH/ycA3sA5fw3n/GYA3w3gz3q3rNEi8PiTNP5kqSevx7VscAdsT5eMbUs9pGX6Gr8QP0iSbYI0wHB6ajsNNg05TUGtaaPhFa6J71Vt2tj0gq1FQ8VzZ6toWHbLEBaCKnKTdP62Uo+37ROr1VC1ZxRR6kny+F/mpS626zFv2Q7ObDZw3kQehw/N4z+eXg2tO1q5azscjsN9zz/J8OuqgplSDidWq4n7SseSWmdnWby1XaKeeVW4C+5E0F7b8fP4/fctpPf4h13j1znnT9AvnPMnAeyeM2DAIUknKvXQHUBc+bjYSlZXlZAnXjdt5DUV014flO1AYxdJ6gnn8cfLAzMlAwoLdGn/79sETNOQ01Q/i4OanJEuXWta/vauv3gaDnelmI1IS2aiKNzqxyF2AG15rXfReOZMpW1X0WC0oOkZ59avm6Yqbu+Zo8uJstOZShMOd73vWw8toGE5uFe4Q6iHevW4/5uOE3j8beSIhYkcnvHuXJKkHiAw/Fn26dkuUc+cgrTRQH4cBUHjr5tOi8ZfadqxyQnEoOfxp131/Yyxv2WMvdb79wEA9/dyYaOE7/E3w5N/Kr7G3+qVUuZOQVeRa2nL7CCnq5gs6tvu10PbKeXcSV/i+MW40YSAa8xmx8LDxt2/Jw12a1JPTlOE6syw1CNm2dx0cBqAm9lTrpstxVvR18UhdgBNeu0zZyptu4pSVbUb3I33+AE3u2etavo946OIHUBvODCN8bzmyz3UFz7w+F2jZ9o80KHbGKf58cDwxx0Xitcc9T3+3WP4o565KAF2gi6UcVk9dL60a6Dn5/EPs8YP4P8C8DiAd3n/Hvcek2QAnWCchw3RZhuNP5B6lBaNn/RksXI0Lb/2qUfwx//u39wJt7Ru2mhI448ZVELMC2mA/t+3MaZpEL2rYozUQwbypoMzANxc/s1GvMbfaeB61bT8DqBJr62bDubaVBwzxrx+800vAJtQjHT5HDSF4a6EtE66gM6P56CrCl5z+Ry+/MQyOOeo+2MXg8pdwOs1lEKOWJjI+xf3uOMS9fizLuDaDlGNP8kRiSOU1WPayBthjx9ozab7668cw7s++h0AXkW0mu7uYjeS1vBrAP6cc/5WzvlbAfwFgMEUt3YhorYf93Osxk9zQr2pSw6HLxU0vBJ+6nWedlziRt3EJx54Dt88HrQ8EI0H5dETSYNKALe//nLE4+8m+BZHyPBHsnpqTdvf3qXzY5gdy+HRU+fAeWtLZvf17TX+WtNGMWFIuLj+TnME3LbDZtsxfRN5HedPFnBqPb6KNzrl66X7p7Cy0cCZSjMYu0gevz+U3kk1LEQcdxl3XPYUdBiagieXdp/UM5HXsdmw/PM+SXqMg6SdasPymx0SdG7UIw0Jv/3sGm5/6Hksb9Rj6zIGibQrvxNAQfi9AOCO7JczmojGfjPG8Mdq/J5XryjMv5UnuYeadk0VdbcwrEP7YeLuJ1dh2jy0BjFfmfLoiaRBJYDX+THi8bfTzdMg9j4P8viD4edL5bo/wvHgXAkPPbcOoLVdg/u6zlJP0r6JdzntBsQDruGn0YLtMkCm2mRgLZUbYAyY9eb4UrfJ46uVFo/f8KQey+Yds3qi6y/GNF9jjGFhIudP+YqTzfoFHVc6X93mfWmlHvfYrnvfLVGGo88r2pCQzv27ji57bTKG3/DnOed+NyTv5/Y15pLUbApBXTHAu+lr/3ZL4K/eFKs13S87nZjUtMsf/5cywEuZJaIX3Cr1iHn8yR7WwkQOq5vNUOyhXYpkGsIev/seqnfhq5oWljYafrD1krkSnvcKneIKuDoNXG9XaBby+Dt0FZ0qkdST7PEDroSSlIG1XK5jdizny06XeD3qF1c2Q2m9QCD1mILU09bwj7f3+MW/2VPQd5W04Q9N8Yx3rWmllhHp86LvhujxJ7Ugp7urO44sJ845GBTSrrzCGLuOfmGMXQ8gXXcpSUeSPX7hIhCRe2pCClouxuPP66rf8jaNzm/ZDu56Ytlbg9j3JzD8OV1p8fiTPKyFyBAPoH0TrTQYMVIP/Vxt2Fgu130jJbbz3ZrHnxy/ELfdqasotWZ2j0kbj79NBtZSuR6aV3DBVAGGqmBxpdIyAlD3zwWealiIuP6kNEj6m26GrO8EFLshKbQbqYccBrrLijf88R7/PU+t4FzNHNjpW0B6w/8LAD7OGLuHMXYPgH8G8HM9W9WIIRZuid52pWElzgCtC42lKIWvKXj8bnDXm/uaIrPngRNrWK+a2DdVCF2Iwhq/6v/eblAJEAzxEHP5xTYTW0H0+MX3KHnjF5c3Gr6REic3xWb16J3z+JPWKT4+n0LjX/eDu+08fiMxA2vZu5MhVIXhopkiFlcrQrtn71xQKKsnpcffQeMHgn3cTcVbQHBc6bvRbTuQvKb4TpEo29BnGZ0217QcjOc01E0HXzt2Zng1fsbYyxlj53HOvwXgSgAfA2DCnb17fAfWNxJUGpYv1wQZPhyVpoW9e9wvXdTw14S88KjGT0aGUu/SFHHdeXQZusrwxqvPQ80MpCUxX1kM7nbKyY8O8wayCO6GvXzC7a1veZ6xu11xclOcx99R6kloYQC4HjT1J+qk8U8WdTjc9cDbGYqpohuojBttuVRutGzn4FwJiyubrR6/Slk9nQu4gKDmAkiW4OhY7qaMHgAtQ1O68fgB9xwgwy96/HSH1CL1WDZuPDiDkqFis2ENr+EH8D4AZDW+C8Cvw23bsAbg/T1c10hRadi+wSJ5p2bacDhw3h43ph5NLasLU6iiHj+NSvR7xaTQ+O84soSbDs74X3K6CxG7EBqa4uucnQK15Hkvxxj+/BarHUNSj9CIq2ioWPY6TJKBvHC6CM2zaHEav6G5AzeqZpvgbsK+McZQ1FUUDbXjNCoxC6adxz9Zir9Im7aDM5VGy53FgdkxPHu26t+d+Rq/50A0bSdVPxmquWAMiVIUfaa7zuMvhOfjtpPn4ijoKta8AfLieUzef5zUM5HXcPPl7ojIQe3FD3Q2/CrnnHL73gbg/ZzzT3LOfwvApb1d2uiw2bB8Qxlt2Ha+5/FHS8jFMnM6AZsRj39PQQdjwNkOUs/iyiYWVyq49dACSp4ho3WIRUAhj7+D9z5TykFVWKiIy+0AqW45QJgk9RR0Fc/4DdPcz0tXFez3hsfEefz0urbB3TZdHguG2rZqlxCNZTsPcboYNHQTWd1sgPPWIPLBuRJMm+NpbwIVXUwN3+MX8/jbf80XJvL+6MGk54Hdp/EHHr8n9ZhdSj266l9o83Eav9kq9Riaglu9aWGD2q4BSGH4GWN09h8G8GXhuewGb+4yNuomPnLfidT579ul0rAw500/8jN5vP/P86We1uCu2JERCHv8ea9f+Z5CcrYIQVWghw/N+60QaPvNkOEPNH7SxpOMo6owzI3lQimdlS5vxaPQF00TUlgB9+Kz6o2tFIOVB+dKnkSVnJZJ+2HZDj5473F/vzt1ES0aqj8MpB1TKT1+klGiAV6xeEvkEk/KOvJCGUDgpVLmj5kynZPeu92+0rZ3m8cfxL8smLYD0+ZdFQfmddVPdQ4Hd92fG3arx5/TFNxy5TwUNrjtGoDOxvujAL7KGFuFm8VzDwAwxi6FO3d3KPnfj57Gb33mUbzikplUIwG3ixvE1VHQVd/w0AXgfE/qiXr8Yg9xXfiyAwj1hREHfifx4Ml1XDRTxL6pol+hSZk9DcuGrjKoCvOyetzH0+j18xO5iMe/9V78QGDcou8RzqsPPONbDy20zAoIvy7oyf/NZ87if3zucagM+E+vPNBRL37VZbM4f7KQ+Dwh6uLtsnqS4jHLQrsGkQNe8JoMfzS113ScVBo/4FYOx8lhxIXTRbxk3x5ct3+y7fvsNLqqoKCr2KibiYPW2yEa+/g8/laNP6ermC4ZeOt1+0JxpEGjreHnnP8uY+xOAHsBfJEHLrACdwD7UEJeVzSFsldsNiyM5VR3+HYz7PEvkMffovEHvV98qcdywLnbo4UyEyaFua9JLJfrfhA5TuohDyinppd6ADcb5ORa1f99O734geDOJvoe4SybwDP+0Rv240dv2J/4fjRCEQCOeYNb7jy6jJ/4rovdDqBt1vo7b7km1ZonQ1JP8vuJvftFlhKmfE2XDEwWdX/d0eCuaaVr0gYA73zFxXhnm+fzuho7P3k3MFFwG7UlDVpvh3ghFn+OS+f0v1fec3/8Iy/Z1rr7Tcd7Fc75fZzzT3POK8JjT3LOv93bpfUPivRH2yT3Ajd7x0Ypp2EspwadOr0LwJ6CjqKhxmr8QUfGIIWvEdF1p1P063GzRlzDT8HKTd/w2/6FRczjTzP0YmEiF0rnrLbJ+08DefzRL3cwhrFzsDX6OrqAHfcM6H2LZ/zag+1cpIiJvObfdbT3+Cn1ttXjV5g7mD3KwdmSn32Vj9z9WY4r9SgMsf2GhoXxvI6NhrmlGpFonIjIaWHpFOg8zWzQGI69yBjykNt158sKmgJVymko5TRUfanHNUhjORUTeT02nTMuuEsBqbzv8SfnhwPuhWepXPcNf9TjF8f6hTX+zh7WwkQeZytN/zXtWjykgTzmaKERfYHTBFvDr9P8rJ7F1U2vyynHFx59wX9+u1CjNnH9ceR11csyiWr8btVunGR1QKhVoOHhmuAEtJu3OyxM5DVs1K0tST2ivCO+jjHW0pDQd6gGuE2DyHDsRcbQ7XZ0IlYvIAM75hn+aHC3aGgY905ugnMeSucUg7ukwZN3OVXU2/bkL9ctNCzHl0j84G6TNH7R8Hen8ZM8seIFXrvNs45C64i+B6V2RgOgnSjqqn/nsrhSweFD85gs6vjXh1/wn88CknvaefxA/MS0aPGWCGnMqsJ8r96IBHeHxUNNYjyvo1wzt1QjIhr+aIqxeK4DaGmGN+gM91mxRSjAttmIT/XLEkrbdKWeVo2/lNMwUQh7/E3bHWEY1fhN2/Fb7NIJOlUyvGET8ftCwcP5iNTja/xCjxnDS+fknKdquBadvdttul0U2s/W4K4a2oe0kNTTsGycXKvi0vlx3HLFPB44sRZ63+1CqZqdjMak18JZJK54i6DMnrzg1WsR2c8YEkOVhNuT39qa1CN8f6IpxtH2JFEJddDp2V4wxj7EGFtmjD0qPDbNGPsSY+wp7/+pXm1/O5CHvBMa/6bv8XvBXe9CQI+XDLXF4683w3KO3tbjp2yR5M6PALDgecsFXYXCBKlHkAtymgLuVaGm6a0fLeLabnA3yeP3pZ4uPX4K7j57pgqHu5o55WiL77tdSL9P5/G3avxJF7SDXsaZ2DMmyPByQjLdsOL25DcDR6RN7UUUMvxxPYpEWRPoPL940OjlXvwdgDdGHns3gDs555fBbfX87h5uf8uQkdwJw08efhDcDTz+vO4OAnFniwqG3wrnHotf9haPPyE/nFiKpAsyxlAytFBwV9T4Afdi0G5QCRG0bRClnm0EdzUK4sYHdzt1yoxCHj9lxhycK+Hmy2f9YPl21ioyldrjD8djmpaDM5VmYs///dNFt+JWMEa6Ikg9I6Dx0zCWrUk93h1kjOGnu1ti0GfsRunZWcE5vxvA2cjDPwDgw97PHwbwll5tf6s4Dvd7dEdn4G4VzjlOrlVx4kwFJ85UYrtxFg0NRUMT8vhtX3ZxZ4sGBsEfwkI92IX0s3rU409oBUDQlKz5UF92NSz1RLdj2qly8qeLBjSF4enlTZw4U9l2Hr+uMjDWTurp1uPXUDNtHPOqXw/MljCe13HjgZnQ+24XOgadAoPReExcUZpIXlexb6oQ9vg1sUmbPfQa/0ReR9Ny/O/sVjT+uDuxnNCeBBj8GbtRdrr6doFz/oL382kAC+3+uB9s1IOJPll5/J944CR+5RMP+79fvjCGL/7ia0LboOButWnDcTiqTcvPsCEdk3MOxljQ5TJSuWvaPNRbB3CbcAHAC+fCQ1GI5XID4zkt5N2KklPDcvziHjG/OU2gVlEYzp8s4CP3ncBH7jsBAJhsUyjUCcbcSuTpSAXpdMk1jPumuhsRQet//IUy5sZzfguAN1y9gHufXvUlmu1y3kQOmuLeSbVjqmigXDdh2Q40VfGPWbtGcFcsTPgXCKB19OKwe/zUmpnuXLsq4DLI8MdJPWGPf9iknr61XeCcc8ZYYk8ExthtAG4DgP37k4twskbUWDczyup55kwFqsLwRz/8Ynzh0dO48+gyHIdDUZgQxFUx5mfUWKg0LN9QjOc1v+lWXleD4RuU1SPkHQeDOdzHDs6NYbpk4J6nVvBDL9vXsrbljXqLRzkmZBeJmSHksTYtJ/V8079+x3V+NbCqMLzuyvlUn1kS/3zbTdg7Ea6YffnFU/jYbTd1XVlK63/s1DkcFPr3v/2G/bjyvImuLyRJvO3l+3HdRVP+hTyJqaIOzoFzNRMzYzk8s+pKUBfNJFeI/s5bXhQK3Ot+k7bRkHrIKSHD322TNiD+YhHV+IctuLvThn+JMbaXc/4CY2wvgPjp0gA45++H1wH0+uuv35mmOQDOCoY/K49/rWpiqqjjrdftw2bDwhcfX8KZShNz4zkhX18Tcuhtr5rX8/iFvuN5XQ0GrXtevaowKCxcwEVejKow3HLFPO44suR7kiJi8RZREiQnccSc38PEclJ3Qrz6/D24+vw93XxcbbnyvImWxxhjuNEbsN4N9MV/5kwV33XJrP+4piq44cD01hcZ3Y6h4sX7Jjv+3ZRQvTszlsPxVddhoGZzcVAvJ4IxBl1lgcc/5FLPuODx5zSlbYuOKL7UE6PbG5oSmUQXnnsw6Oz0WXE74FeHvxPAZ3d4+x0hLbxoqNkZ/krTz+WOpjiKaZt+KmXTQqVh+zn1fjMqr20DeXiip2JoCpp24PGLnsnrr5rHuZqJ+700RRGxeIsQ6wnEPH6xT3m7QSWDgnjhEj3+fjEV6dezuLqJ/dNFP3ifFk1R/Kyeoff48+TxN7qOybT3+CPBXXO4PP5epnN+FMDXAVzBGDvJGPspAL8P4PWMsacA3Or9vqug/tz7pgqZ5fGvVZu+Lu2nOHpB1UrT8nrDK760U2m4Uk8x4vFT2wbK3ClE0vjcdM6wxw8Ar75sDoaq4I7Hl0Lr4pxjudxoKXway6lBP36xV4/Qp7zdoJJBoZgL1r8bGm6R4acA7+JKZUsXJF1l3ujF4Tf8FJc5fa7edRZWvl06p66E0zlTNrwbFHqZ1fNjnPO9nHOdc76Pc/5BzvkZzvlhzvllnPNbhV7/uwbS+PdNFTPz+Nerph8oJO962UtxrAiSTknok7PZsDBmUHA3aD8LtGb1APDaDcR7/KWchpsumcEdR5ZCrabP1Uw0baclT1wM7opeo5/OmTK4u9sRaxAO7kAX1k7QObJeNeE4HMdXK1u6IOmqMjotG7zvxlaKA+n7E5dtldNU6fGPEmvVJlSFYWEin5nhP1tp+t4c9d1f8g1/IOmMCRp/pRFk9cQNnADiPf4gnTP8JXj9oXk8c6bq56yLa4hmjYzlIhq/UMBFj3UaVDIIkIeoKQz7pjq3WO41QYfOJp4/V0PDckL9eNKiq4o7etFykBt6jT/IvOpa6jGSPX63V09cOudgOzvEcJ8VW4ACseN5LZMmbZxzrFdNP3BnaApmSoafP78pZO+Q9LDZMFFp2n6Wjx/cjWj8eSNs+E2hSVvUM3mdV5F655FA7okWbxGlnIaGVwXszooNZw81TGfbVbi7Afri75/pXkfvBUVDhaEqOFttYlEoKusWTWUjo/GXDNWfGRxnwNvRtnI3qWWDbNI2nFAgtmS4xs+KTOHplmrTRtN2QgM55sZzfhsD0bMnj5/aApeEAi5A1PhbZ9f6wV2vjXK0S+UFkwUc2jvhT9sCBMM/3mr4gaCCOcjq6S6Pf7dD6z+4Ba+6FzDGMFnUsV4xsegVlW3F8BuqAtMZjXROxpjv9Xd7PgYFXPHB3bh0zmHJkhqOveiSD9y9iLufXIl9bq3axFRRb+lSmYam5eA3Pv0ITq3X/McoUCeOrVuYyAtST2D46X96jn4vGu4YxbIg9agK83O2AZJ63AKufMKX/fWH5nH/ibP+mqhXfjSPv+R9gc5sun8X5PGr/vY7DSoZBMhQXLILArsE9es5vlrBWE7zpcFu0FQG03Lv/obFULWDdP4tB3djzuPklg3D8XkOx150yV995Wm/kjTKetXEVNFo6VKZhkdOncM/fuNZfFmQU8hrFmevugNKAqmHJB0KNpInTmtwvZqgUdujp8pen5bA8JPHT+Ph4jh8aAEOB77yhOv1L5frmMhrLR4PXXDoAhH1+GmfBt3jn8jrePuN+/Hml5zf76X4UIfORS+w2818AUJXFVjOaAR3AWA853r83ToiE3kN77hpP15z+VzLczlNhe1w/46/kXAnPagM/1kRgXOOjbqF46uV2OcpEBsdSJIGek9xzixlCYlSz8JEHisbDdgOd4O7nqeiKAwlQ/U9cbHSkxq1bTYsfP3YGRyOVMAanpfnjmSMP6zXXLAHc+M5X+6JK94CggsOFbNFNX7KM89iUEk/URSG3/vBa/CiC7IrMNsuNCN5q6mcgFuAVjfdAT+jYPgDj787w88Yw++8Jf74R8cvuu3Jh+ezHJ49SUm1acN2OE6cqbTo9xSInSzpLSMI00C6LHnzQGD4xdmr8xN5OBw4s9kIST2Aa+wDjT84kalR271PraBpOzh8KNzmSPT44yoRAdfQ3XpoHl99cgVNy8HSRmvxFq0BAM56PWCiWT2+xz8kVYy7iamSgaVzdZxar20powdwnQCqwxiWLJR2kMafpfQYHb/YtKXhH2hILjFtjpNrtdBzFIidDnn86TV+ysQIefyeXDItSD1UMLVUbqDStEJzYsdyGk6fC0s9QODxf+nxZUzkNVx/cXiUAWX11IVumnEcvnIBmw0L3zh+JrZ4CwguOCT1kNdIevGaUN0syZapoo4Nz9nYalGZpiioeuftSHj8FNzNML3YENqTAOGBRMPA8J8VEcRJVlG5RwzEklHrxuMPpB7R4zfBmDs0nSAv+8TZChyOFo+f8vTFYNV4XsN6rYm7nljGLVfOt6QfGn7lbrLHDwCvvHQWOc2t4nUbtCVLPWdI4/eMB2MMhqb4Hv+gB3d3I2ISwFYNv64p/nk7Coafst6ydETEmhX6X3r8A8yGYPipDzshBmK7De7aDsfxM67hJ40ecL3jibweah5FBVN0hzAmSDqivBPy+As6nlrexNlKs0XmAdwvezOFx18wVLz6slnc/tDzMG0e2/KXLkRrEY3f/VnBeo08/sHW+HcjouE/sEWNX1eY32Bs2Au4gKBDZ6ZSjx7W+IetJmJ49iQlVAQFAIsRj18MxJLxq6Zszfz8eg1Ny8H5e/I4W2n62uBa1QzJPAAwO5YDY0FMINQLP9QXP6zxc+5WmcZlIRhUwNXB4wfc7B4a6t0uuEvpnLlQawjVf62UerJnquQasb178lu+sOqq4kuUw2SskpjoiccftCcBws0Kh4Hh2ZOUkNSzp6Dj+Eq84Z8U0jnTNmqji8hNXnvgFS8wul5ttgz00FUFM6Wc/5qo1OP/bIQ1fgC44cB0SDYiSOpxs3o6GH4hIyjO46f2tvR5iLngOU0Rsnqk4c8a8vi30zROU9nQNRVrx8QWC7jaYcRKPcNzvg//WRGBGp295MJJLK6GpR4xEJvXldDQccCtqP33x077/8TsHfLeb7rENfyk84t9ekTmx3P+hWcsxvAXDRWKIA+RjnlrjMwDuCP3TJujbnbWIucn8njJvj3eOlo9fnfururHPFqbwXF/jZJs8Q3/NqqJxQv1KBRw0Xcjy/RiX+M3BY9/SNo1AH2cwNUvSOO/dt8e3P3kSmjgiRiIZYyF+tIDwHv/9TF87uEX/N9fc/kcPvx/3ADA1evHcxqu2usOCqGWDOtVM3Z4yMJEDo+/UAYQ1fVV77HwoblwughDVfD6q+INv6G6E4NUhaUaFvF9Lz4fi6sVzMVk9dD2KVaRi7SGILLMopC4zI3nUDJUvHjf1msLNDVc2DfsXOgNnb9gMn4o/VbwDb8daPy50vB8liP3zS3XLBiqgkOegT6+UsE13pcsGogVu1QCwHNrNVy3fxL/4y0vwke+fgKf/PZJbNRNjOd1v4UuTUSilE5qARFF1NbjPP6xiOF//aEF3Pfrh1viBYSuubf3zExXVv5TrzqAH3rZvkRZqJTTYHtppeL7iRcVKfVkTymn4d5ffV2snJcWMeNrFAz/iy7Yg2//5utD1fHbxZ82J3j8w/RZDs+epMQ11Jrff12Ue6KB2FJO8wthANeLPzA7hqvP34O3XrcPps1xz1Or7vusbLrzbYsGNIVhqVxH3XQnVcWdkGIaZSmSx+8+FjaqisISjT7gZm80vf4snTT+NO8nrkk84ekioClsqL4Iu4mpkhGS+bpl1Aw/gEyNPiA1/qGjXLcwUdBx0UzRy6wJArzRQGzJUP3gruNwrGw0/GDodfsnMVnUcceRJVSbFp4/V8eB2RIUhbndNzcaQXpojMYvBlXjgrulLvVK+rI37eSWDd0gpphG0zkBqe/vZsTmfaOg8fcC2bJhyCCPP6+r2DdVCKV0RgOxJUHqOVttwnK4L9FoqoJbrpjHXUeXW3qnz0/ksVSuCwVhrbftYlC1ZIh5/PFSTyf0kFe+faMsXnjiPH6Zw7970dXW4yXpDgrkypYNQ0K5ZvrpXwdmx3BckHqoMychGv5gaEngqd/q5cN/4oGTAIJMjIXxHJbLDT/tcbKNx5/TFGjCF5UuAsUuDb/o2WXj8bvb11UWKj6ji4r0+Hcv2ghKPVmTU1tbNgzTZzk8e5KSjbrlp38dnC3h+ErFn0MbDcSOCRo/zcidEzz1my+fha4yfOxbzwEALp4tAnD72y9t1P1CJyrKEaE7h6hnH3j83RnWzD1+bx1RqYC8HhnY3b0YI5bV0wuCyl0bnHOp8Q865Xrg8V8yV0KlaWOp3IgNxJZyql8BGefxj+d13HhgBjXTxvlCpeXCeB7rVROnvddMx3j8MyUDCmtN2xzbosafy9jjp3VFU0MNqfHverQRy+PvBfS5NUwHlsPh8OGSzYZnT1IievzU9nZxZTM2ECvm8VNOezTv/dZDbhUsZQkBgTf/5OkNAPFSj6YqmB3LtRj+6DSutOha4OWlyerpBN1xRE/2wOOXGv9uZRSzerJGUZhbDe/NLgaG67Mcnj1JgWm7s2KpfzcFY3/mHx7A9/7FPQDCgdgxQ0PTcmDaDpbKdUyXjJbbPWqYJjbUolGGR5c2UDLUxBNmYSKP8QSPny5OaTHU1syb7eB7/FHDr4enhUl2H7qUejLB0BQ0TGfoxi4CI1bAtem1a6CJPXv35PHLb7jcn5Gb11W84tJZ/+/FKVxLCb3rL5wu4nfe8iLcdHDaf0z0+GfGkvOLf+W7rwgFTgFgdszAb7/5KnzPi/Z2tW/ilz0Lj5+kpqjhoFtgKfXsXnQp9WRCTlPQsGw/lz9NRfygMFKGnxq0kcfPGMPPve6yxL+nIqrNhpXYux4A3nHTRaHf6QJRM+3YHH7i5pgum4wx/OdXHmizF/HEpVxuh8DjD5/sMri7+9E8Z8JQh2dGbD/IaW5RZHMIPf7h2ZMU0PStiZQyijiFa7ncwEJCX5soU0XD98CjnTl7RTidM4usngSNX5ce/27Hn5g2RIaqH+R0FQ0rkHqG6fMcnj1JQdTj7wQZ/o26iZXN+MHkcSgK8wu02rVFyBIxnTPLPP5oR0K6A5DB3d2LpkjDnwWG6kk9Jnn8w+PsjNSZQUNYSOPvBBm/E2eqsJ34aVVJUIC3ndSTJeGe+b3L45fpnLsfutuU+v72yOmK5/F7Gv8QXUiHZ09SQC2ZJ9J6/J5XS7N0kzT+OEjn3ympJ1Smn6XHn6DxS8O/e6FzQXr820Nq/ENC2df40xljMn6+4U+p8QNBZs9OST3h4G52Hn+i1DNEGQ7DhjT82ZDTpMY/FJDHP5Y6uOsaNxrKnlbjF/82rnirF2TdqycxuCubtO16NCn1ZIIRTeeUGv9gUq6507aiufNJkNf7zBnX40+aVhUH3R3EdebsBeSNMJbNFz6nqdDV1p77UuPf/UiPPxty0QKuIRq9ODx7kgJqyZwWGjpeNx3MjhkhHb0TL7lwEgsTOVw2P76VpXYNBfRyWna529dfNI2r9oZHAB6YLWFhIodL5rY+E1bSW/zgrjT82yKnuS0bZOXugCM2aEsDDR0v163YoeTtuHxhHN/49Vu7XeKWoS95Fjn8xEdvu6nlsQunizu6X5LuIQdlmAxVP8hpasjjH6YL6fDsSQrEBm1poQDvfBepnP1AftklhC/1SI1/W/gavyk1/oGm3KXUAwQ6/0KXHv9OQ1/yLD1+yWAipZ5scHv1DKfUMzx7koINb95uN/iGf5d7/IrCoClsqE5OydaQwd1syOkyj38oKNe69/gDqWd3e/yA+4WXHr9EpnNmg6GqsByOatMauoZ3I3NmcM5dj7+L4C4Q5LN3U7zVLwxNQX6IdEjJ1iCDP0zph/2APr/NhjVU3j7Qp6wextgzADYA2AAszvn1vd5mzbRhOTx1gzYikHoGw+OXX3aJ5gd3pROwHcjYl+vW0H2v+pnOeQvnfLWXG3AcjhfKdVwwWQhaMqds0EZQv55BMPw5TRmqzAPJ1pDB3Wyg71K5Zg7d92qoz4xf/eTDeOv/+g9wzlGuddeSmZgq6jBUBbNtJmntFko5tesYhmT40BUFhqr4c5MlW8MQPP5hu4j2y0pwAF9kjHEA7+Ocvz/6B4yx2wDcBgD79+/f0kZuPDiDjz9wEo+eKqNpu5H5tENYiHe+4mK86rI5//Z5N/Nnb7u26xiGZPhQFIZ/+Okbcdm8rK7eDiT1bNRNqfFnxKs456cYY/MAvsQYO8o5v1v8A+9i8H4AuP766/lWNnLLFXNQGHDHkSVcu38SQPce/8xYDjNjuz+wCwBXn7+n8x9JRoIbDkx3/iNJWwLDb+H8Pbtf6u2GvlzGOOenvP+XAXwawA292M7MWA7X7Z/CnUeXfI1/T5cav0QiGU1ouHq5Zg6d1LPje8MYKzHGxulnAG8A8Givtnf40AIePVXGk6c3AHTv8UskktGE0mIbliODuxmwAOBexthDAL4J4POc8y/0amOvv2oeAPDZh04BSD+ERSKRjDZiCqfU+LcJ53wRwEt2anuXzI3hopkiTpypQlNYJkNKJBLJ8CMa+2HL4x+uvYmBMYbDVy4AAMbz2lCVXUskkt4hyjvD1v5iuPYmgVsPuXJPtw3aJBLJ6BLy+KXGP3i8/MA0xvOaLG6SSCSpGWapZyQsoa4q+NlbLu33MiQSyQAhevkyuDug/MxrLun3EiQSyQAh5u7LPH6JRCIZAQyp8UskEslooSrM73Q6bFLPcO2NRCKRZAh5+tLwSyQSyYhAco8hpR6JRCIZDcjTlx6/RCKRjAi+4R+yPP7h2huJRCLJkEDjl1KPRCKRjASBxj9cpnK49kYikUgyRGr8EolEMmKQti8Nv0QikYwIUuOXSCSSEYP68EuNXyKRSEYEKfVIJBLJiCHz+CUSiWTEkBq/RCKRjBiGTOeUSCSS0YIM/rANWx+ZCVwSiUTSLT9w7QWYKhpQFNbvpWSKNPwSiUSSwBXnjeOK88b7vYzMGa77F4lEIpF0RBp+iUQiGTGk4ZdIJJIRQxp+iUQiGTGk4ZdIJJIRQxp+iUQiGTGk4ZdIJJIRQxp+iUQiGTEY57zfa+gIY2wFwIktvnwWwGqGy9mNyH0cDuQ+Dge7aR8v4pzPRR8cCMO/HRhj93POr+/3OnqJ3MfhQO7jcDAI+yilHolEIhkxpOGXSCSSEWMUDP/7+72AHUDu43Ag93E42PX7OPQav0QikUjCjILHL5FIJBIBafglEolkxBhqw88YeyNj7AnG2NOMsXf3ez3bhTF2IWPsLsbY44yxxxhjP+89Ps0Y+xJj7Cnv/6l+r3W7MMZUxth3GGOf834/wBj7hncsP8YYM/q9xu3AGJtkjH2CMXaUMXaEMfZdw3YcGWO/6J2njzLGPsoYyw/6cWSMfYgxtswYe1R4LPa4MZe/8Pb1YcbYdf1beZihNfyMMRXAXwH4HgBXAfgxxthV/V3VtrEA/BLn/CoANwH4WW+f3g3gTs75ZQDu9H4fdH4ewBHh9z8A8Gec80sBrAH4qb6sKjv+HMAXOOdXAngJ3H0dmuPIGLsAwLsAXM85fxEAFcCPYvCP498BeGPksaTj9j0ALvP+3Qbgr3dojR0ZWsMP4AYAT3POFznnTQD/DOAH+rymbcE5f4Fz/m3v5w24xuICuPv1Ye/PPgzgLX1ZYEYwxvYB+F4Af+v9zgC8DsAnvD8Z6H1kjO0BcDOADwIA57zJOV/HkB1HuKNdC4wxDUARwAsY8OPIOb8bwNnIw0nH7QcA/D13uQ/AJGNs744stAPDbPgvAPCc8PtJ77GhgDF2MYCXAvgGgAXO+QveU6cBLPRrXRnxPwH8NwCO9/sMgHXOueX9PujH8gCAFQD/nydn/S1jrIQhOo6c81MA/hjAs3AN/jkAD2C4jiORdNx2rQ0aZsM/tDDGxgB8EsAvcM7L4nPczc8d2Bxdxtj3AVjmnD/Q77X0EA3AdQD+mnP+UgAVRGSdITiOU3A93gMAzgdQQqtEMnQMynEbZsN/CsCFwu/7vMcGGsaYDtfo/yPn/FPew0t0C+n9v9yv9WXAKwF8P2PsGbjy3Ovg6uGTnmQADP6xPAngJOf8G97vn4B7IRim43grgOOc8xXOuQngU3CP7TAdRyLpuO1aGzTMhv9bAC7zsggMuIGl2/u8pm3had0fBHCEc/6nwlO3A3in9/M7AXx2p9eWFZzzX+Oc7+OcXwz3mH2Zc/7jAO4C8MPenw36Pp4G8Bxj7ArvocMAHscQHUe4Es9NjLGid97SPg7NcRRIOm63A/hJL7vnJgDnBEmov3DOh/YfgDcBeBLAMQC/0e/1ZLA/r4J7G/kwgAe9f2+Cq4HfCeApAHcAmO73WjPa39cC+Jz380EA3wTwNICPA8j1e33b3LdrAdzvHcvPAJgatuMI4L0AjgJ4FMBHAOQG/TgC+CjcmIUJ987tp5KOGwAGN7PwGIBH4GY49X0fOOeyZYNEIpGMGsMs9UgkEokkBmn4JRKJZMSQhl8ikUhGDGn4JRKJZMSQhl8ikUhGDGn4JUMNY8xmjD0o/Gvb+Iwx9jOMsZ/MYLvPMMZmt/C672aMvdfr+Pi/t7sOiSQOrfOfSCQDTY1zfm3aP+ac/00P15KGV8Mtcno1gHv7vBbJkCI9fslI4nnkf8gYe4Qx9k3G2KXe4+9hjP2y9/O7vNkHDzPG/tl7bJox9hnvsfsYYy/2Hp9hjH3R6z//t3CLd2hb7/C28SBj7H1ey/Doet7GGHsQbivj/wngAwD+M2NsoKvNJbsTafglw04hIvW8TXjuHOf8GgD/L1xjG+XdAF7KOX8xgJ/xHnsvgO94j/06gL/3Hv9tAPdyzq8G8GkA+wGAMXYIwNsAvNK787AB/Hh0Q5zzj8Httvqot6ZHvG1//9Z3XSKJR0o9kmGnndTzUeH/P4t5/mEA/8gY+wzctgqA2zbjhwCAc/5lz9OfgNtf/63e459njK15f38YwMsAfMttWYMCkpuvXQ5g0fu5xN2ZCxJJ5kjDLxlleMLPxPfCNehvBvAbjLFrtrANBuDDnPNfa/tHjN0PYBaAxhh7HMBeT/r5r5zze7awXYkkESn1SEaZtwn/f118gjGmALiQc34XgF8FsAfAGIB74Ek1jLHXAljl7kyEuwG83Xv8e+A2XQPc5l0/zBib956bZoxdFF0I5/x6AJ+H28P+D+E2FbxWGn1JL5Aev2TYKXieM/EFzjmldE4xxh4G0ADwY5HXqQD+wRuTyAD8Bed8nTH2HgAf8l5XRdCO970APsoYewzA1+C2JQbn/HHG2G8C+KJ3MTEB/CyAEzFrvQ5ucPe/APjTmOclkkyQ3TklI4k36OV6zvlqv9cikew0UuqRSCSSEUN6/BKJRDJiSI9fIpFIRgxp+CUSiWTEkIZfIpFIRgxp+CUSiWTEkIZfIpFIRoz/H8scLk+Chr6gAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def dqn(n_episodes=1000, max_t=500, eps_start=.5, eps_end=0.01, eps_decay=0.97):\n",
    "    \"\"\"Deep Q-Learning.\n",
    "    \n",
    "    Params\n",
    "    ======\n",
    "        n_episodes (int): maximum number of training episodes\n",
    "        max_t (int): maximum number of timesteps per episode\n",
    "        eps_start (float): starting value of epsilon, for epsilon-greedy action selection\n",
    "        eps_end (float): minimum value of epsilon\n",
    "        eps_decay (float): multiplicative factor (per episode) for decreasing epsilon\n",
    "    \"\"\"\n",
    "    scores = []                        # list containing scores from each episode\n",
    "    scores_window = deque(maxlen=100)  # last 100 scores\n",
    "    eps = eps_start                    # initialize epsilon\n",
    "    for i_episode in range(1, n_episodes+1):\n",
    "        env_info = env.reset(train_mode=True)[brain_name]\n",
    "        state = env_info.vector_observations[0]\n",
    "        score = 0\n",
    "        for t in range(max_t):\n",
    "            action = agent.act(state, eps)\n",
    "            env_info = env.step(action)[brain_name]        # send the action to the environment\n",
    "            next_state = env_info.vector_observations[0]   # get the next state\n",
    "            reward = env_info.rewards[0]                   # get the reward\n",
    "            done = env_info.local_done[0]        \n",
    "            \n",
    "            agent.step(state, action, reward, next_state, done)\n",
    "            state = next_state\n",
    "            score += reward\n",
    "            if done:\n",
    "                break \n",
    "        scores_window.append(score)       # save most recent score\n",
    "        scores.append(score)              # save most recent score\n",
    "        eps = max(eps_end, eps_decay*eps) # decrease epsilon\n",
    "        print('\\rEpisode {}\\tAverage Score: {:.2f}'.format(i_episode, np.mean(scores_window)), end=\"\")\n",
    "        if i_episode % 100 == 0:\n",
    "            print('\\rEpisode {}\\tAverage Score: {:.2f}'.format(i_episode, np.mean(scores_window)))\n",
    "        if np.mean(scores_window)>=13.0:\n",
    "            print('\\nEnvironment solved in {:d} episodes!\\tAverage Score: {:.2f}'.format(i_episode-100, np.mean(scores_window)))\n",
    "            torch.save(agent.qnetwork_local.state_dict(), 'checkpoint.pth')\n",
    "            break\n",
    "    return scores\n",
    "scores = dqn()\n",
    "\n",
    "# plot the scores\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "plt.plot(np.arange(len(scores)), scores)\n",
    "plt.ylabel('Score')\n",
    "plt.xlabel('Episode #')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.save(agent.qnetwork_local.state_dict(), 'checkpoint.pth')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "drlnd",
   "language": "python",
   "name": "drlnd"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}