Unsupervised.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Unsupervised methods\n",
    "\n",
    "Most of the methods we've looked at so far have applied *supervised* machine learning, where the learning methods require examples of text **input** (documents, words, etc.) and the **correct outputs** (classes, tags, etc.)\n",
    "\n",
    "By contrast, *unsupervised* methods do not expect outputs, but instead aim to identify commonalities or structure in their input."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example: language models\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Statistical language models estimate the \"probability\" of text, i.e. $P(w_1, w_2, \\ldots, w_n)$\n",
    "\n",
    "Basic N-gram approach:\n",
    "\n",
    "* Assume (incorrectly) that $P(w_i)$ only depends on the N-1 previous words\n",
    "    * e.g. for bigrams $P(w_i|w_{i-1},w_{i-2}, \\ldots) = P(w_i|w_{i-1})$\n",
    "* Count occurrences of N-grams in a large amount of (unlabelled) text\n",
    "    * e.g. [Google Web 5-grams](https://catalog.ldc.upenn.edu/LDC2006T13): 1,024,908,267,229 words (~1 trillion)\n",
    "* Estimate probabilities from counts and apply smoothing\n",
    "\n",
    "---\n",
    "\n",
    "A model that can estimate $P(w_i|w_{i-1},w_{i-2}, \\ldots)$ can be used to generate text: given a \"prompt\" of words, pick the most likely next word, and repeat.\n",
    "\n",
    "Recent example from [Open AI GPT-2](https://openai.com/blog/better-language-models/) neural language model:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"margin:1em\">\n",
    "<table style=\"font-size:100%; text-align:left\">\n",
    "  <tr>\n",
    "    <td><b>Prompt (human input)</b>:</td>\n",
    "    <td><i>In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.</i></td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td><b>System output</b>:</td>\n",
    "    <td><p>The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science.</p>\n",
    "\n",
    "<p>Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved.</p>\n",
    "\n",
    "<p>Dr. Jorge Pérez, an evolutionary biologist from the University of La Paz, and several companions, were exploring the Andes Mountains when they found a small valley, with no other animals or humans. Pérez noticed that the valley had what appeared to be a natural fountain, surrounded by two peaks of rock and silver snow.</p>\n",
    "\n",
    "<p>Pérez and the others then ventured further into the valley. “By the time we reached the top of one peak, the water looked blue, with some crystals on top,” said Pérez.</p>\n",
    "\n",
    "<p>Pérez and his friends were astonished to see the unicorn herd. These creatures could be seen from the air without having to move too much to see them – they were so close they could touch their horns.</p>\n",
    "\n",
    "<p>While examining these bizarre creatures the scientists discovered that the creatures also spoke some fairly regular English. Pérez stated, “We can see, for example, that they have a common ‘language,’ something like a dialect or dialectic.”</p>\n",
    "\n",
    "<p>[...]</p>\n",
    "</td>\n",
    "</tr>\n",
    "</table>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Distributional semantics\n",
    "\n",
    "Methods that aim to derive (representations of) meaning from text statistics. Based on a comparatively old idea (mandatory quotes):\n",
    "\n",
    "<div style=\"text-align:center; font-size:120%; margin-top:1em\"><i>If A and B have almost identical environments we say that they are synonyms</i> (Harris 1954)</div>\n",
    "\n",
    "<div style=\"text-align:center; font-size:120%; margin-top:1em\"><i>You shall know a word by the company it keeps</i> (Firth 1957)</div>\n",
    "\n",
    "<div style=\"text-align:center; font-size:120%; margin-top:1em\"><i>Words which are similar in meaning occur in similar contexts</i> (Rubenstein and Goodenough, 1965)</div>\n",
    "\n",
    "---\n",
    "\n",
    "Example:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"margin:1em\">\n",
    "<table style=\"font-size:100%; text-align:left\">\n",
    "    <tr style=\"background-color:lightgray\"><td>A large</td><td><b>___</b></td><td>runs in the yard</td></tr>\n",
    "    <tr style=\"background-color:lightgreen\"><td>A large</td><td><b>dog</b></td><td>runs in the yard</td></tr>\n",
    "    <tr style=\"background-color:lightgreen\"><td>A large</td><td><b>cat</b></td><td>runs in the yard</td></tr>\n",
    "    <tr style=\"background-color:yellow\"><td>A large</td><td><b>hat</b></td><td>runs in the yard</td></tr>\n",
    "    <tr style=\"background-color:orangered\"><td>A large</td><td><b>the</b></td><td>runs in the yard</td></tr>\n",
    "</table>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Basic approach:\n",
    "\n",
    "* Create a **word-context matrix** that records occurrences of words in context (for various definitions of context)\n",
    "    * Most typically **word-word matrix**\n",
    "* Process a large corpus of text, summing up (word, context) counts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"figs/word_context_example.png\" width=\"60%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Apply smoothing and/or statistics measuring association strength (e.g. PMI, TF-IDF)\n",
    "* Perform dimensionality reduction\n",
    "\n",
    "The process creates **word vectors** that aim to reflect word relationships: e.g. `similarity(cat,dog)` > `similarity(cat,hat)`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hypothetical example with *very* small corpus:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "words = ['a', 'cat', 'dog', 'eats', 'runs', 'the', 'yard']\n",
    "\n",
    "word_word_matrix = [\n",
    "    [  0, 43, 57,  0,  0,  0, 12 ],\n",
    "    [  0,  0,  0, 13,  9,  0,  0 ],\n",
    "    [  0,  0,  0,  9, 11,  0,  0 ],\n",
    "    [ 21,  0,  0,  0,  0, 16,  0 ],\n",
    "    [  5,  0,  0,  0,  0,  2,  0 ],\n",
    "    [  0, 33, 38,  0,  0,  0,  7 ],\n",
    "    [  0,  0,  0,  0,  0,  0,  0 ],    \n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>cat</th>\n",
       "      <th>dog</th>\n",
       "      <th>eats</th>\n",
       "      <th>runs</th>\n",
       "      <th>the</th>\n",
       "      <th>yard</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>0</td>\n",
       "      <td>43</td>\n",
       "      <td>57</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cat</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dog</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>eats</th>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>runs</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>the</th>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>yard</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       a  cat  dog  eats  runs  the  yard\n",
       "a      0   43   57     0     0    0    12\n",
       "cat    0    0    0    13     9    0     0\n",
       "dog    0    0    0     9    11    0     0\n",
       "eats  21    0    0     0     0   16     0\n",
       "runs   5    0    0     0     0    2     0\n",
       "the    0   33   38     0     0    0     7\n",
       "yard   0    0    0     0     0    0     0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from pandas import DataFrame\n",
    "\n",
    "\n",
    "DataFrame(word_word_matrix, index=words, columns=words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, similar words should have similar vectors (rows):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dog \t [ 0  0  0  9 11  0  0]\n",
      "cat \t [ 0  0  0 13  9  0  0]\n",
      "eats \t [21  0  0  0  0 16  0]\n",
      "runs \t [5 0 0 0 0 2 0]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "vec = dict((words[i], np.array(word_word_matrix[i])) for i in range(len(words)))\n",
    "\n",
    "for w in ['dog', 'cat', 'eats', 'runs']:\n",
    "    print(w, '\\t', vec[w])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Vector similarity is typically measured using *cosine similarity*, the dot product of normalized (length 1) vectors:\n",
    "\n",
    "$$cos(a,b) = \\frac{a \\cdot b}{\\|a\\|\\|b\\|} = \\frac{\\sum_i{a_i b_i}}{\\sqrt{\\sum_i{a_i^2}}\\sqrt{\\sum_i{b_i^2}}}$$\n",
    "\n",
    "(i.e. for normalized vectors, this is just the sum of the elementwise products)\n",
    "\n",
    "As the name suggests, this corresponds to the cosine of the angle between the vectors:\n",
    "\n",
    "* if $a$ and $b$ pointing in the same direction, $cos(a,b) = 1$\n",
    "* if $a$ and $b$ have a near 90 degree angle, $cos(a,b) = 0$\n",
    "* if $a$ and $b$ point in opposite directions, $cos(a,b) = -1$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sim(dog ,cat ) = 0.96\n",
      "sim(dog ,eats) = 0.00\n",
      "sim(eats,runs) = 0.96\n"
     ]
    }
   ],
   "source": [
    "from math import sqrt\n",
    "\n",
    "\n",
    "def cos(a, b):\n",
    "    return np.dot(a,b)/(sqrt(np.dot(a,a))*sqrt(np.dot(b,b)))\n",
    "\n",
    "\n",
    "for a, b in [['dog', 'cat'], ['dog', 'eats'], ['eats', 'runs']]:\n",
    "    print('sim({:4s},{:4s}) = {:.2f}'.format(a, b, cos(vec[a], vec[b])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's not very surprising that an artificially constructed example can be made to work. Can we do this with real data?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "# Recap\n",
    "\n",
    "Yesterday, we talked about annotation and started looking into unsupervised methods, briefly covering\n",
    "\n",
    "* Language models: estimate probability of text $P(w_1, w_2, \\ldots, w_n)$\n",
    "* Distributional semantics: representations of meaning from text statistics\n",
    "* Word-word matrix and count-based word vectors\n",
    "* Cosine similarity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Neural word representations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Count-based word vectors are easy and effective, but recent *prediction-based* methods offer better performance at many tasks (see e.g. [Baroni et al. 2014](http://www.aclweb.org/anthology/P/P14/P14-1023.pdf)).\n",
    "\n",
    "In prediction-based models, the task is to predict a word given its context (or vice versa):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"figs/cbow_and_skipgram.png\" width=\"60%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"text-align:center; font-size:80%\">(figure adapted from <a href=\"https://arxiv.org/pdf/1301.3781.pdf\">Mikolov et al. 2013</a>)</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The basic idea:\n",
    "\n",
    "1. Create a random vector for each unique input word and context element (above, also words)\n",
    "2. Map inputs and contexts to their vectors, try to predict one from the other (black box)\n",
    "3. Update word and context vectors (and other model weights) to minimize errors (prediction differing from word/context example)\n",
    "4. (Repeat 2-3)\n",
    "\n",
    "**Note**: The word vectors are learned as a \"side effect\" of learning the prediction task."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "In the following, we'll be working using the [gensim](https://radimrehurek.com/gensim/) library to work with vectors created using the [word2vec](https://github.com/tmikolov/word2vec) implementation of the skip-gram model.\n",
    "\n",
    "The English word vectors were created by Mikolov et al. on 100 billion words of Google News texts. These are available [here](https://code.google.com/archive/p/word2vec/)\n",
    "\n",
    "Let's load some vectors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gensim.models import KeyedVectors\n",
    "\n",
    "# This is where the word vectors are found on the server\n",
    "EN_WV_PATH = \"/course_data/textmine/wordvecs/GoogleNews-vectors-negative300.bin\"\n",
    "\n",
    "\n",
    "en_wv = KeyedVectors.load_word2vec_format(EN_WV_PATH, binary=True, limit=100000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's have a look at the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(300,)\n"
     ]
    }
   ],
   "source": [
    "print(en_wv['dog'].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are 300-dimensional vectors. Can we make sense of one?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>290</th>\n",
       "      <th>291</th>\n",
       "      <th>292</th>\n",
       "      <th>293</th>\n",
       "      <th>294</th>\n",
       "      <th>295</th>\n",
       "      <th>296</th>\n",
       "      <th>297</th>\n",
       "      <th>298</th>\n",
       "      <th>299</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>wv[\"dog\"]</th>\n",
       "      <td>0.05127</td>\n",
       "      <td>-0.022339</td>\n",
       "      <td>-0.172852</td>\n",
       "      <td>0.161133</td>\n",
       "      <td>-0.084473</td>\n",
       "      <td>0.057373</td>\n",
       "      <td>0.058594</td>\n",
       "      <td>-0.08252</td>\n",
       "      <td>-0.015381</td>\n",
       "      <td>-0.063477</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.180664</td>\n",
       "      <td>0.103027</td>\n",
       "      <td>-0.275391</td>\n",
       "      <td>0.261719</td>\n",
       "      <td>0.246094</td>\n",
       "      <td>-0.047119</td>\n",
       "      <td>0.0625</td>\n",
       "      <td>0.416016</td>\n",
       "      <td>-0.355469</td>\n",
       "      <td>0.222656</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 300 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "               0         1         2         3         4         5    \\\n",
       "wv[\"dog\"]  0.05127 -0.022339 -0.172852  0.161133 -0.084473  0.057373   \n",
       "\n",
       "                6        7         8         9    ...       290       291  \\\n",
       "wv[\"dog\"]  0.058594 -0.08252 -0.015381 -0.063477  ... -0.180664  0.103027   \n",
       "\n",
       "                292       293       294       295     296       297       298  \\\n",
       "wv[\"dog\"] -0.275391  0.261719  0.246094 -0.047119  0.0625  0.416016 -0.355469   \n",
       "\n",
       "                299  \n",
       "wv[\"dog\"]  0.222656  \n",
       "\n",
       "[1 rows x 300 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([en_wv['dog']], index=['wv[\"dog\"]'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Doesn't look much like a dog. A couple of points:\n",
    "\n",
    "* The vectors don't mean anything *in isolation*; they can only be interpreted with reference to other vectors (of the same collection)\n",
    "    * `wv[\"dog\"]` is a good representation of \"dog\" if its relations to other vectors (e.g. `wv[\"cat\"]`) reflect the relations of the words (e.g. \"cat\")\n",
    "* The specific values are (largely) arbitrary: re-running the method would not re-create these vectors\n",
    "    * (It is not possible to look at this vector alone and determine that it represents \"dog\")\n",
    "    \n",
    "So, can we see the relations?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col0 {\n",
       "            background-color:  #f5c6c8;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col1 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col2 {\n",
       "            background-color:  #c1f2cc;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col3 {\n",
       "            background-color:  #f6c8ca;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col4 {\n",
       "            background-color:  #b6f0c2;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col5 {\n",
       "            background-color:  #f5c5c6;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col6 {\n",
       "            background-color:  #f5c8c9;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col7 {\n",
       "            background-color:  #c0f2cb;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col8 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col9 {\n",
       "            background-color:  #a8edb7;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col10 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col11 {\n",
       "            background-color:  #bdf1c8;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col12 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col13 {\n",
       "            background-color:  #cdf5d6;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col14 {\n",
       "            background-color:  #f3babc;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col15 {\n",
       "            background-color:  #f9ddde;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col16 {\n",
       "            background-color:  #74e28b;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col17 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col18 {\n",
       "            background-color:  #e1f9e6;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col19 {\n",
       "            background-color:  #d5f6dc;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col0 {\n",
       "            background-color:  #f9dadb;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col1 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col2 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col3 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col4 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col5 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col6 {\n",
       "            background-color:  #fbe9ea;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col7 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col8 {\n",
       "            background-color:  #f4c2c4;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col9 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col10 {\n",
       "            background-color:  #fae3e4;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col11 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col12 {\n",
       "            background-color:  #f2f2f2;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col13 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col14 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col15 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col16 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col17 {\n",
       "            background-color:  #f3bbbd;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col18 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col19 {\n",
       "            background-color:  #f5c4c5;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col0 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col1 {\n",
       "            background-color:  #66df7f;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col2 {\n",
       "            background-color:  #a7edb6;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col3 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col4 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col5 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col6 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col7 {\n",
       "            background-color:  #71e189;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col8 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col9 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col10 {\n",
       "            background-color:  #cbf4d4;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col11 {\n",
       "            background-color:  #f4bdbf;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col12 {\n",
       "            background-color:  #f7d4d5;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col13 {\n",
       "            background-color:  #b8f0c4;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col14 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col15 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col16 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col17 {\n",
       "            background-color:  #e0f9e6;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col18 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col19 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col0 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col1 {\n",
       "            background-color:  #fbe5e6;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col2 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col3 {\n",
       "            background-color:  #fbe9ea;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col4 {\n",
       "            background-color:  #c7f3d0;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col5 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col6 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col7 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col8 {\n",
       "            background-color:  #a2ecb2;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col9 {\n",
       "            background-color:  #fae0e1;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col10 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col11 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col12 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col13 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col14 {\n",
       "            background-color:  #f5c6c8;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col15 {\n",
       "            background-color:  #76e28d;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col16 {\n",
       "            background-color:  #f6c9ca;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col17 {\n",
       "            background-color:  #64de7e;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col18 {\n",
       "            background-color:  #f6cccd;\n",
       "            color:  #000000;\n",
       "        }    #T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col19 {\n",
       "            background-color:  #f3b8ba;\n",
       "            color:  #000000;\n",
       "        }</style><table id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdb\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >0</th>        <th class=\"col_heading level0 col1\" >1</th>        <th class=\"col_heading level0 col2\" >2</th>        <th class=\"col_heading level0 col3\" >3</th>        <th class=\"col_heading level0 col4\" >4</th>        <th class=\"col_heading level0 col5\" >5</th>        <th class=\"col_heading level0 col6\" >6</th>        <th class=\"col_heading level0 col7\" >7</th>        <th class=\"col_heading level0 col8\" >8</th>        <th class=\"col_heading level0 col9\" >9</th>        <th class=\"col_heading level0 col10\" >10</th>        <th class=\"col_heading level0 col11\" >11</th>        <th class=\"col_heading level0 col12\" >12</th>        <th class=\"col_heading level0 col13\" >13</th>        <th class=\"col_heading level0 col14\" >14</th>        <th class=\"col_heading level0 col15\" >15</th>        <th class=\"col_heading level0 col16\" >16</th>        <th class=\"col_heading level0 col17\" >17</th>        <th class=\"col_heading level0 col18\" >18</th>        <th class=\"col_heading level0 col19\" >19</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                        <th id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdblevel0_row0\" class=\"row_heading level0 row0\" >wv[\"dog\"]</th>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col0\" class=\"data row0 col0\" >0.0512695</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col1\" class=\"data row0 col1\" >-0.0223389</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col2\" class=\"data row0 col2\" >-0.172852</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col3\" class=\"data row0 col3\" >0.161133</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col4\" class=\"data row0 col4\" >-0.0844727</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col5\" class=\"data row0 col5\" >0.057373</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col6\" class=\"data row0 col6\" >0.0585938</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col7\" class=\"data row0 col7\" >-0.0825195</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col8\" class=\"data row0 col8\" >-0.0153809</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col9\" class=\"data row0 col9\" >-0.0634766</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col10\" class=\"data row0 col10\" >0.179688</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col11\" class=\"data row0 col11\" >-0.423828</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col12\" class=\"data row0 col12\" >-0.022583</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col13\" class=\"data row0 col13\" >-0.166016</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col14\" class=\"data row0 col14\" >-0.0251465</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col15\" class=\"data row0 col15\" >0.107422</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col16\" class=\"data row0 col16\" >-0.199219</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col17\" class=\"data row0 col17\" >0.15918</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col18\" class=\"data row0 col18\" >-0.1875</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow0_col19\" class=\"data row0 col19\" >-0.120117</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdblevel0_row1\" class=\"row_heading level0 row1\" >wv[\"cat\"]</th>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col0\" class=\"data row1 col0\" >0.0123291</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col1\" class=\"data row1 col1\" >0.204102</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col2\" class=\"data row1 col2\" >-0.285156</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col3\" class=\"data row1 col3\" >0.216797</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col4\" class=\"data row1 col4\" >0.118164</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col5\" class=\"data row1 col5\" >0.0830078</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col6\" class=\"data row1 col6\" >0.0498047</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col7\" class=\"data row1 col7\" >-0.00952148</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col8\" class=\"data row1 col8\" >0.220703</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col9\" class=\"data row1 col9\" >-0.125977</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col10\" class=\"data row1 col10\" >0.0805664</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col11\" class=\"data row1 col11\" >-0.585938</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col12\" class=\"data row1 col12\" >-0.00445557</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col13\" class=\"data row1 col13\" >-0.296875</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col14\" class=\"data row1 col14\" >-0.0131226</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col15\" class=\"data row1 col15\" >-0.0834961</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col16\" class=\"data row1 col16\" >0.0505371</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col17\" class=\"data row1 col17\" >0.151367</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col18\" class=\"data row1 col18\" >-0.449219</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow1_col19\" class=\"data row1 col19\" >-0.0135498</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdblevel0_row2\" class=\"row_heading level0 row2\" >wv[\"hat\"]</th>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col0\" class=\"data row2 col0\" >-0.124512</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col1\" class=\"data row2 col1\" >-0.0201416</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col2\" class=\"data row2 col2\" >-0.204102</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col3\" class=\"data row2 col3\" >-0.125</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col4\" class=\"data row2 col4\" >-0.168945</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col5\" class=\"data row2 col5\" >-0.120117</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col6\" class=\"data row2 col6\" >0.0629883</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col7\" class=\"data row2 col7\" >-0.113281</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col8\" class=\"data row2 col8\" >0.24707</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col9\" class=\"data row2 col9\" >0.130859</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col10\" class=\"data row2 col10\" >0.0300293</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col11\" class=\"data row2 col11\" >-0.104004</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col12\" class=\"data row2 col12\" >0.00320435</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col13\" class=\"data row2 col13\" >-0.191406</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col14\" class=\"data row2 col14\" >-0.523438</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col15\" class=\"data row2 col15\" >0.21582</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col16\" class=\"data row2 col16\" >-0.214844</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col17\" class=\"data row2 col17\" >0.0463867</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col18\" class=\"data row2 col18\" >0.136719</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow2_col19\" class=\"data row2 col19\" >-0.209961</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdblevel0_row3\" class=\"row_heading level0 row3\" >wv[\"the\"]</th>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col0\" class=\"data row3 col0\" >0.0800781</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col1\" class=\"data row3 col1\" >0.10498</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col2\" class=\"data row3 col2\" >0.0498047</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col3\" class=\"data row3 col3\" >0.0534668</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col4\" class=\"data row3 col4\" >-0.0673828</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col5\" class=\"data row3 col5\" >-0.120605</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col6\" class=\"data row3 col6\" >0.0351562</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col7\" class=\"data row3 col7\" >-0.118652</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col8\" class=\"data row3 col8\" >0.0439453</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col9\" class=\"data row3 col9\" >0.0301514</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col10\" class=\"data row3 col10\" >-0.0568848</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col11\" class=\"data row3 col11\" >-0.0761719</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col12\" class=\"data row3 col12\" >0.0128784</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col13\" class=\"data row3 col13\" >0.0498047</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col14\" class=\"data row3 col14\" >-0.0849609</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col15\" class=\"data row3 col15\" >-0.0634766</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col16\" class=\"data row3 col16\" >0.00628662</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col17\" class=\"data row3 col17\" >-0.0432129</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col18\" class=\"data row3 col18\" >0.0202637</td>\n",
       "                        <td id=\"T_71d43304_54b6_11e9_b063_fa163e87fcdbrow3_col19\" class=\"data row3 col19\" >0.0133057</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7f217f4916a0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import seaborn as sns    # for colors\n",
    "\n",
    "\n",
    "def heatmap(*args, **kwargs):    # this is just visualization\n",
    "    cm = sns.diverging_palette(133, 10, l=80, as_cmap=True)\n",
    "    df = DataFrame(*args, **kwargs)\n",
    "    return df.style.background_gradient(cmap=cm)\n",
    "\n",
    "\n",
    "words = ['dog', 'cat', 'hat', 'the']\n",
    "\n",
    "heatmap([en_wv[w][:20] for w in words], index=['wv[\"{}\"]'.format(w) for w in words])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... maybe the similarities make more sense?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sim(dog,cat) = 0.76\n",
      "sim(dog,hat) = 0.20\n",
      "sim(dog,the) = 0.06\n"
     ]
    }
   ],
   "source": [
    "for a, b in [['dog', 'cat'], ['dog', 'hat'], ['dog', 'the']]:\n",
    "    print('sim({},{}) = {:.2f}'.format(a, b, cos(en_wv[a], en_wv[b])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nice! Can we do that in Finnish?\n",
    "\n",
    "We'll use vectors created similarly on [http://bionlp.utu.fi/finnish-internet-parsebank.html](Finnish Internet Parsebank) texts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sim(koira,kissa) = 0.85\n",
      "sim(koira,hattu) = 0.31\n",
      "sim(koira,yksi) = 0.08\n"
     ]
    }
   ],
   "source": [
    "# This is where the word vectors are found on the server\n",
    "FI_WV_PATH = \"/course_data/textmine/wordvecs/pb34_wf_200_v2.bin\"\n",
    "\n",
    "\n",
    "fi_wv = KeyedVectors.load_word2vec_format(FI_WV_PATH, binary=True, limit=100000)\n",
    "\n",
    "for a, b in [['koira', 'kissa'], ['koira', 'hattu'], ['koira', 'yksi']]:\n",
    "    print('sim({},{}) = {:.2f}'.format(a, b, cos(fi_wv[a], fi_wv[b])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Neighbours in vector space\n",
    "\n",
    "Looking for vectors most similar to a given input can find synonyms and closely related words:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdb\" ><thead>    <tr>        <th class=\"col_heading level0 col0\" >word</th>        <th class=\"col_heading level0 col1\" >similarity</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                                <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow0_col0\" class=\"data row0 col0\" >dogs</td>\n",
       "                        <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow0_col1\" class=\"data row0 col1\" >0.868049</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow1_col0\" class=\"data row1 col0\" >puppy</td>\n",
       "                        <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow1_col1\" class=\"data row1 col1\" >0.810643</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow2_col0\" class=\"data row2 col0\" >pit_bull</td>\n",
       "                        <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow2_col1\" class=\"data row2 col1\" >0.780396</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow3_col0\" class=\"data row3 col0\" >pooch</td>\n",
       "                        <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow3_col1\" class=\"data row3 col1\" >0.762738</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow4_col0\" class=\"data row4 col0\" >cat</td>\n",
       "                        <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow4_col1\" class=\"data row4 col1\" >0.760946</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow5_col0\" class=\"data row5 col0\" >golden_retriever</td>\n",
       "                        <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow5_col1\" class=\"data row5 col1\" >0.75009</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow6_col0\" class=\"data row6 col0\" >German_shepherd</td>\n",
       "                        <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow6_col1\" class=\"data row6 col1\" >0.746517</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow7_col0\" class=\"data row7 col0\" >Rottweiler</td>\n",
       "                        <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow7_col1\" class=\"data row7 col1\" >0.743761</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow8_col0\" class=\"data row8 col0\" >beagle</td>\n",
       "                        <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow8_col1\" class=\"data row8 col1\" >0.741862</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow9_col0\" class=\"data row9 col0\" >pup</td>\n",
       "                        <td id=\"T_71d43305_54b6_11e9_b063_fa163e87fcdbrow9_col1\" class=\"data row9 col1\" >0.740691</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7f217bbeb470>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def show_similar(wv, word):    # visualization\n",
    "    return DataFrame(wv.most_similar(word), columns=['word', 'similarity']).style.hide_index()\n",
    "\n",
    "\n",
    "show_similar(en_wv, 'dog')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdb\" ><thead>    <tr>        <th class=\"col_heading level0 col0\" >word</th>        <th class=\"col_heading level0 col1\" >similarity</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                                <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow0_col0\" class=\"data row0 col0\" >pentu</td>\n",
       "                        <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow0_col1\" class=\"data row0 col1\" >0.894558</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow1_col0\" class=\"data row1 col0\" >koirakin</td>\n",
       "                        <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow1_col1\" class=\"data row1 col1\" >0.860871</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow2_col0\" class=\"data row2 col0\" >kissa</td>\n",
       "                        <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow2_col1\" class=\"data row2 col1\" >0.845947</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow3_col0\" class=\"data row3 col0\" >koiruus</td>\n",
       "                        <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow3_col1\" class=\"data row3 col1\" >0.839777</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow4_col0\" class=\"data row4 col0\" >hevonen</td>\n",
       "                        <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow4_col1\" class=\"data row4 col1\" >0.836254</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow5_col0\" class=\"data row5 col0\" >kani</td>\n",
       "                        <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow5_col1\" class=\"data row5 col1\" >0.830166</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow6_col0\" class=\"data row6 col0\" >marsu</td>\n",
       "                        <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow6_col1\" class=\"data row6 col1\" >0.795733</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow7_col0\" class=\"data row7 col0\" >eläin</td>\n",
       "                        <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow7_col1\" class=\"data row7 col1\" >0.795119</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow8_col0\" class=\"data row8 col0\" >ipana</td>\n",
       "                        <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow8_col1\" class=\"data row8 col1\" >0.792993</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow9_col0\" class=\"data row9 col0\" >poni</td>\n",
       "                        <td id=\"T_71d43306_54b6_11e9_b063_fa163e87fcdbrow9_col1\" class=\"data row9 col1\" >0.77929</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7f21743331d0>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_similar(fi_wv, 'koira')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(not quite as good.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Relations\n",
    "\n",
    "“*these representations are surprisingly good at capturing syntactic and semantic regularities in language*” (Mikolov et al. 2013)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "king - man + woman ≈ queen (0.71)\n",
      "kuningas - mies + nainen ≈ kuningatar (0.76)\n"
     ]
    }
   ],
   "source": [
    "def analogy(wv, word, minus, plus):\n",
    "    word2, sim = wv.most_similar(positive=[word, plus], negative=[minus])[0]\n",
    "    print('{} - {} + {} ≈ {} ({:.2f})'.format(word, minus, plus, word2, sim))\n",
    "\n",
    "\n",
    "analogy(en_wv, 'king', 'man', 'woman')\n",
    "analogy(fi_wv, 'kuningas', 'mies', 'nainen')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"figs/wv_relations.png\" width=\"50%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"text-align:center; font-size:80%\">(figure from <a href=\"https://www.aclweb.org/anthology/N13-1090\">Mikolov et al. 2013</a>)</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "London - UK + Finland ≈ Helsinki (0.69)\n",
      "lager - Germany + Finland ≈ vodka (0.54)\n"
     ]
    }
   ],
   "source": [
    "analogy(en_wv, 'London', 'UK', 'Finland')\n",
    "analogy(en_wv, 'lager', 'Germany', 'Finland')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's far from perfect, though (esp. for Finnish!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Lontoo - Englanti + Suomi ≈ Tukholma (0.58)\n",
      "lager - Germany + UK ≈ pub (0.57)\n"
     ]
    }
   ],
   "source": [
    "analogy(fi_wv, 'Lontoo', 'Englanti', 'Suomi')\n",
    "analogy(en_wv, 'lager', 'Germany', 'UK')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(OK, that's probably enough silly word vector tricks.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Recent work\n",
    "\n",
    "An enormous amount of work has been done on unsupervised learning of vector representations of text in recent years (partial listing):\n",
    "\n",
    "* word2vec ([Mikolov et al. 2013](https://arxiv.org/pdf/1301.3781.pdf)): https://github.com/tmikolov/word2vec\n",
    "* GloVe ([Pennington et al. 2014](https://nlp.stanford.edu/pubs/glove.pdf)): https://nlp.stanford.edu/projects/glove/\n",
    "* word2vecf ([Levy and Goldberg 2014](https://www.aclweb.org/anthology/P14-2050)): https://bitbucket.org/yoavgo/word2vecf\n",
    "* Skip-Thought ([Kiros et al. 2015](https://arxiv.org/abs/1506.06726)) https://github.com/ryankiros/skip-thoughts\n",
    "* FastText ([Joulin et al. 2016](https://arxiv.org/pdf/1607.01759.pdf)) https://fasttext.cc/\n",
    "* CoVe ([McCann et al. 2017](http://papers.nips.cc/paper/7209-learned-in-translation-contextualized-word-vectors.pdf)): https://github.com/salesforce/cove\n",
    "* ELMo ([Peters et al. 2018](https://aclweb.org/anthology/N18-1202)): https://allennlp.org/elmo\n",
    "* GPT ([Radford et al. 2018](https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf)): https://github.com/openai/finetune-transformer-lm\n",
    "* BERT ([Devlin et al. 2018](https://arxiv.org/pdf/1810.04805.pdf)): https://github.com/google-research/bert\n",
    "* MT-DNN ([Liu et al. 2019](https://arxiv.org/pdf/1901.11504.pdf)): https://github.com/namisan/mt-dnn\n",
    "\n",
    "(Recent methods extended on the capabilities of word2vec in particular in accounting for context and representing longer spans of text.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dimensionality reduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "High-dimensional data such as complex feature representations and word vectors presents a number of challenges:\n",
    "\n",
    "* Computational cost (memory and time)\n",
    "* Difficulties in learning and analysis (curse of dimensionality)\n",
    "* Humans can only really visualize data up to 3D\n",
    "\n",
    "*Dimensionality reduction* methods aim to represent data with fewer dimensions while preserving information.\n",
    "\n",
    "Example: let's say we're interested in understanding what these words look like in the vector space:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "en_words= [\n",
    "    \"dog\", \"cat\", \"computer\", \"monitor\",\n",
    "    \"eat\", \"drink\", \"walk\", \"run\",\n",
    "    \"one\", \"two\", \"1\", \"2\",\n",
    "    \"good\", \"bad\", \"ugly\",\n",
    "    \"happy\", \"unhappy\",\n",
    "    \"or\", \"what\", \"that\",\n",
    "    \"right\", \"very\", \"well\"\n",
    "]\n",
    "\n",
    "fi_words = [\n",
    "    \"koira\", \"kissa\", \"tietokone\", \"näyttö\",\n",
    "    \"syödä\", \"juoda\", \"kävellä\", \"juosta\",\n",
    "    \"yksi\", \"kaksi\", \"1\", \"2\",\n",
    "    \"hyvä\", \"paha\", \"ruma\",\n",
    "    \"onnellinen\", \"onneton\",\n",
    "    \"ja\", \"tai\", \"mitä\", \"että\",\n",
    "    \"oikein\", \"todella\",\"hyvin\"\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(We'll first set up visualization)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "def plot_words(wv, words, vectors):\n",
    "    word_indices = [wv.vocab[w].index for w in words]\n",
    "    # Drop words with indices past number of vecs (allows for truncated vecs)\n",
    "    words = [words[i] for i, idx in enumerate(word_indices) if idx < len(vectors)]\n",
    "    word_indices = [idx for idx in word_indices if idx < len(vectors)]\n",
    "    plt.scatter(vectors[word_indices,0], vectors[word_indices,1])\n",
    "    for index, word in zip(word_indices, words):\n",
    "        plt.annotate(s=word, xy=vectors[index], xytext=vectors[index]+(0.01,0.005))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can of course get 2D vectors by just discarding the rest of the dimensions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100000, 2)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAD8CAYAAACl69mTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XtcVVXe+PHPEgkJUFTMFA2x8cr9Il4QREltfpqmZVTewNSxsl5PTf6yn09lTT1SOmWZPXZxRM3GUkcmL2WT6IxaKhCIYt4yxrvhBUWBEWT9/uBwAgQEzjmcwznf9+vFq7PXWXvv78I437P23mstpbVGCCGEsIRm1g5ACCGE/ZIkI4QQwmIkyQghhLAYSTJCCCEsRpKMEEIIi5EkI4QQwmIkyQghhLAYSTJCCCEsRpKMEEIIi2lu7QBq4uXlpbt06WLtMIQQoklJT0+/oLVuZ+04ytlskunSpQtpaWnWDkMIIZoUpdS/rR1DRXK5TAghhMVIkhFCCGExkmSEEEJYjCQZIYQQFiNJRghRyfbt2/n++++tHYawE5JkhBCVSJIR5iRJRggHsWLFCgIDAwkKCmLixIls2LCBvn37EhISwn333cf58+fJyclhyZIlvPvuuwQHB7Njxw5rhy2aOJsdJyOEMJ/s7GzeeOMNvv/+e7y8vLh06RJKKXbv3o1Sik8//ZS3336bP//5z8yYMQN3d3deeOEFa4ct7IAkGSEcQEpKCuPGjcPLywuANm3asH//fuLi4jh79iw3btzA19fXylEKeySXy4RwUM888wwzZ85k//79fPTRRxQVFVk7JGGHJMkI4QCGDBnCmjVruHjxIgCXLl3iypUreHt7A7B8+XJjXQ8PD/Lz860Sp7A/kmSEsAPJGaeJTEzBd/YmIhNTSM44Xel9Pz8/5syZw6BBgwgKCuL5559n7ty5jBs3jrCwMONlNIAHHniA9evXy41/YRZKa236QZS6H3gPcAI+1VonVnn/eWAqUALkAlO01rVO4hYeHq5lgkwhbi854zQv/W0/hcU3jWWuzk7MGxvAgyHeVoxMWINSKl1rHW7tOMqZ3JNRSjkBi4HfA72Bx5RSvatUywDCtdaBwFrgbVPPK4QoM3/L4UoJBqCw+Cbztxy2UkRC/MYcl8sigGNa6+Na6xvAamB0xQpa621a6wLD5m6gkxnOK4QAzuQV1qtciMZkjiTjDZyssH3KUFaTJ4Cvq3tDKTVdKZWmlErLzc01Q2hC2L+Onq71KheiMTXqjX+l1AQgHJhf3fta64+11uFa6/B27WxmYTchbNqs4T1wdXaqVObq7MSs4T2sFJEQvzHHYMzTQOcK250MZZUope4D5gCDtNb/McN5hRBgvLk/f8thzuQV0tHTlVnDe8hNf2ETzJFkUoFuSilfypLLo8DjFSsopUKAj4D7tda/muGcQogKHgzxlqQibJLJl8u01iXATGAL8BPwpdY6Wyn1ulJqlKHafMAdWKOUylRKfWXqeYUQQtg+s8xdprXeDGyuUvZKhdf3meM8QgghmhYZ8S+EncjJycHf39/aYQhRiSQZIYSwE0lJSQD3WDmMSiTJCGFHbt68ybRp0/Dz82PYsGEUFhbyySef0KdPH4KCgnjooYcoKCgbFx0fH8+MGTMIDw+ne/fubNy4ESj7oBo9ejQxMTF069aN1157DYBXXnmFhQsXGs81Z84c3nvvvcZvpGhSJMkIYUeOHj3K008/TXZ2Np6enqxbt46xY8eSmprKvn376NWrF0uXLjXWz8nJYe/evWzatIkZM2YYp/vfu3cv69atIysrizVr1pCWlsaUKVNYsWIFAKWlpaxevZoJEyZYpZ2Oouol0AULFjB37lxiYmJ48cUXiYiIoHv37lUnMnVWSn2jlDqqlDJO4aWU+l/DYPdspdRrFcpzlFJvK6X2K6X2KqV+ZyhPUkotMexzRCk10lD+L6VUcIX9dyqlgmpqgyQZIeyIr68vwcFlf/9hYWHk5ORw4MABoqKiCAgIYNWqVWRnZxvrP/LIIzRr1oxu3brRtWtXDh06BMDQoUNp27Ytrq6ujB07lp07d9KlSxfatm1LRkYG3377LSEhIbRt29Yq7RRQUlLC3r17WbhwobG3aXAnEAcEAHFKqfJxjHMME2cGAoOUUoEV9rmitQ4APgAWVijvQtnUYSOAJUqpFsBSIB5AKdUdaKG13ldTnJJkhLAjLi4uxtdOTk6UlJQQHx/PBx98wP79+3n11VcrLU6mlKq0f/l2TeVTp04lKSmJZcuWMWXKFEs1Q9TB2LFjgd++TFRwVWt9RWtdBBwEfAzljyilfqRswmI/yiY0LvfXCv/tX6H8S611qdb6KHAc6AmsAUYqpZyBKUBSbXFKkhHCzuXn59OhQweKi4tZtWpVpffWrFlDaWkpP//8M8ePH6dHj7KpaP7xj39w6dIlCgsLSU5OJjIyEoAxY8bwzTffkJqayvDhwxu9LY6mefPmlJaWGrcrfkEo/0JR/mWigorrt9wEmhsGy78AxBpmw98EtKhhn5peA2jDZMf/oGwi5EeAVdTCLONkhBCWlZxxusHTxvzpT3+ib9++tGvXjr59+1Za9fKee+4hIiKCq1evsmTJElq0KPvciYiI4KGHHuLUqVNMmDCB8PCy5UnuuOMOBg8ejKenJ05OTtWeT5hP+/bt+fXXX7l48SLu7u5s3LiR+++/vyGHaglcB64opdpTtjTL9grvxwGJhv/+UKF8nFJqOeALdAXK14/4FNgA7NBaX67txJJkhLBxVRclO51XyEt/2w9QKdF06dKFAwcOGLdfeOEF4+snn3yy2mPfd999LFmy5JbyTp06kZycfEt5aWkpu3fvZs2aNQ1rjDCqyxcHZ2dnXnnlFSIiIvD29qZnz54NOpfWep9SKgM4RNms+buqVGmtlMoC/gM8VqH8BLCXsiQ1w3AJDq11ulLqKrDsdueWJCOEjattUbLGnK/s4MGDjBw5kjFjxtCtW7dGO689qusXB4Bnn32WZ599tsZjeXl5Ge/JxMfHk5CQcKL8Pa31yAqv42sJab7W+sVqyr/TWs+oWqiU6kjZ7ZZvazkmIElGCJtnqUXJDAP3bhEfH098fPwt5b179+b48eMmnVOUsZUvDg2hlJoEvAk8r7UuvV19STJC2LiOnq6criahyKJkTZctrWaqte5SQ3l8DeUrgBV1Pb48XSaEjZNFyeyPI61mKklGCBv3YIg388YG4O3pigK8PV2ZNzbA5i+riJo50hcHuVwmRBPQlBcly8nJYeTIkZWefGuMfW2ZI61mKklGCCGsoCl/cagPuVwmhLC4kpISxo8fT69evXj44YcpKCjg9ddfp0+fPvj7+zN9+nS0Lhtcnp6eTlBQEEFBQSxevNjKkQtTSZIRQljc4cOH6devH/feey8tW7bkww8/ZObMmaSmpnLgwAEKCwuNSw0kJCSwaNEi9u2rcc5F0YRIkhFCWFznzp0JCAgAYMKECezcuZNt27bRt29fAgICSElJITs7m7y8PPLy8oiOjgZg4sSJ1gxbmIHckxFCNEjVm/ILFizg2rVrjBgxgieeeIJmzZoxdOhQvvrqq0qzOpeWlvLdd9+xa9cufvzxR7y9vfHy8uLixYvWaoqwIOnJCCHMKiEhgY8++ojMzEzjJJonTpwwrmOzevVqIiMjKSoqwsvLiw0bNlBSUoKbmxuenp54enqyc+dOgFtmjRZNj/RkhBANdrWohMjEFM7kFaKzjhHgWUJ+fj79+5ctSfL444+zfv16evToQXJyMj/88APDhw9n0aJFREdH4+/vz5UrV4iIiDAes3ytGqUUw4YNs1bThJlIkhFCNMh3h3I5f6WAZoapUPKuXWfXxXya36g8J5ezszMHDhxg+/btLFiwgHXr1gEQEhLCrFmzmDp1Klu2bDH2esLCwird9H/77bcRTZdcLhNCNMjS9MuUXL/CzcKr6JJiCo+lUup8J9e0M3v27AHKLo3VZOrUqUyYMIFx48bJ2jR2THoyQohb1GWtk3P5xbQa8CjnVjyPk3tbnNt2AsBz2DNMmzaNZs2aMWjQIFq1alXtOUaNGkVCQgIJCQkWb4+wHkkyQohK6rrWSUdPV3T4KFqGj6q0/92usHtFFgCJiYnGVTVjYmKIiYkx1tu3bx9BQUENXohLNA1yuUwIUUlta51UVNMkjwNdThAcHIy/vz87duzgv//7v285R2JiIg899BDz5s0zfwOETVHlUznYmvDwcJ2WlmbtMIRwOL6zN1Hdp4ICfkkcUamsLpfVRONSSqVrrcOtHUc5uVwmhKikPoukOcokj6LhzHK5TCl1v1LqsFLqmFJqdjXvRyulflRKlSilHjbHOYUQluFIa50IyzM5ySilnIDFwO+B3sBjSqneVaqdAOKBz009nxDCsmSRNGFO5rhcFgEc01ofB1BKrQZGAwfLK2itcwzvlZrhfEI0Oq01WmuaNXOMZ2XkMpgwF3P8xXgDJytsnzKUCdGk5eTk0KNHDyZNmoS/v3+lAYNr164lPj4egPj4eJ599lkGDBhA165dWbt2rZUiFsL22NTXMqXUdKVUmlIqLTc319rhCMHRo0d56qmnyM7Oxs3NrcZ6Z8+eZefOnWzcuJHZs2+5LSmEwzJHkjkNdK6w3clQVm9a64+11uFa6/B27dqZITQhTOPj40O/fv1uW+/BBx+kWbNm9O7dm/PnzzdCZEI0DeZIMqlAN6WUr1LqDuBR4CszHFcIq6vYe6m4JkpRUVGlei4uLsbXtjr2TAhrMDnJaK1LgJnAFuAn4EutdbZS6nWl1CgApVQfpdQpYBzwkVIq29TzCmGK5IzTRCam4Dt7E5GJKSRn3L7z3b59e3766SdKS0tZv359I0QpRNNnlsGYWuvNwOYqZa9UeJ1K2WU0IayurnNzVZWYmMjIkSNp164d4eHhXLt2rVHiFaIpk2llhMOJTEypdkS7t6cru2YPsUJEjqvqEs7CdLY2rYxNPV0mRGM4U02Cqa1cCNFwkmSEw6luDq7aysVv/vSnP9GjRw8GDhzIY489xoIFC8jMzKRfv34EBgYyZswYLl++DFBjeXp6OkFBQQQFBbF48WJrNkc0AkkywuHI3FwNk5qayrp169i3bx9ff/015ZezJ02axFtvvUVWVhYBAQG89tprtZYnJCSwaNGiSkssC/slSUY4HJmbq2F27drF6NGjadGiBR4eHjzwwANcv36dvLw8Bg0aBMDkyZP517/+xZUrV6otz8vLIy8vj+joaAAmTpxotfaIxiFT/QuHJHNzCdE4pCcjhKiTyMhINmzYQFFREdeuXWPjxo24ubnRunVrduzYAcDKlSsZNGgQrVq1qrbc09MTT09Pdu7cCcCqVaus1h7ROKQnI4So0wqXffr0YdSoUQQGBtK+fXsCAgJo1aoVy5cvZ8aMGRQUFNC1a1eWLVsGUGP5smXLmDJlCkophg0b1uhtFY1LxskI4eCqDk6FsgchKt6nKl/qoKCgAHd3dwoKCoiOjubjjz8mNDTUWqGLatjaOBnpyQjhQGbPnk3nzp15+umnAZg7dy5Je89xpfAGBYd2om8Wc2e3/hA1nj+t/icvPvoaffv2JT09nUceeYTPP/8cV1dXioqK6N27NytXrpQkI2ol92SEcCBxcXF8+eWXxu0vv/yS6+pOSi6f4e5J79Ah4X1unD9G0ckDnL9aVGmpgz/+8Y+UlJSQmprKoUOHOHfuHFOmTLFia0RTID0ZIRxISEgIv/76K2fOnCE3N5fWrVtzJf8Ul3/J4GzSswDoG0UUXzqDd6fOOFdY6sDd3Z0hQ4awceNGevXqRXFxMQEBAdZsjmgCJMkI4WDGjRvH2rVrOXfuHHFxcWxNzWaPexwtAocb67g6O/GHfp68v6nyQm1Tp07lf/7nf+jZsycJCQmNHbpogiTJCGFH6vKUWFxcHNOmTePChQv885//pGfPnvz0xxdp03c45wuhrbrGzJiexNzbiverHL9v376cPHmSH3/8kaysrMZrmGiyJMkIYSfquoSBn58f+fn5eHt706FDBzp06MDTU3/i009n4w44u7sTNfWzGs/zyCOPkJmZSevWrS3aHmEf5BFmIexEYy1hMHLkSJ577jliY2PNdkxhPrb2CLM8XSZEA8XExBgniezSpQsXLlywajyWXsIgLy+P7t274+rqKglG1JlcLhPCTnT0dK22J2OuJQw8PT05cuSIWY4lHIf0ZITDmz9/Pu+/X3aL+7nnnmPIkLJLSykpKYwfP55vv/2W/v37Exoayrhx42x22WVZwkDYIkkywuFFRUUZJ3JMS0vj2rVrFBcXs2PHDgIDA3njjTf47rvv+PHHHwkPD+edd96xcsTVkyUMhC2Sy2XC4YWFhZGens7Vq1dxcXEhNDSUtLQ0duzYwahRozh48CCRkZEA3Lhxg/79+1s54prJEgbC1kiSEQ7P2dkZX19fkpKSGDBgAIGBgWzbto1jx47h6+vL0KFD+etf/2rtMIVokuRymbB7yRmniUxMwXf2JiITU0jOOH1LnaioKBYsWEB0dDRRUVEsWbKEkJAQ+vXrx65duzh27BgA169fl5vfQtSDJBlh18oHKJ7OK0Tz2wDFqokmKiqKs2fP0r9/f9q3b0+LFi2IioqiXbt2JCUl8dhjjxEYGEj//v05dOiQdRojRBMkgzGFXWusAYpC2AoZjClEI7L0AEUhRO0kyQi7VtNARHMNUBRC1E6SjLBrMkBRWIK7u3u96m/fvp3vv//eQtHYNkkywq7JAMXbqzgHW30/PEXdOHKSkXEywu7JAEVRX/Pnz8fFxYVnn32W5557jn379pGSkkJKSgpLly4FYM6cOWzcuBFXV1f+/ve/0759ezZs2MAbb7zBjRs3aNu2LatWraKwsJAlS5bg5OTEZ599xqJFi4iKirJyCxuPWXoySqn7lVKHlVLHlFKzq3nfRSn1heH9PUqpLuY4rxDiN7ebg+3JJ58kPDwcPz8/Xn311VqPdeHCBfr378+mTZssHrctqm2qoejoaK5fv06/fv3Yt28f0dHRfPLJJwAMHDiQ3bt3k5GRwaOPPsrbb79Nly5dmDFjBs899xyZmZkOlWDADElGKeUELAZ+D/QGHlNK9a5S7Qngstb6d8C7wFumnlcIUdntPhjffPNN0tLSyMrK4p///GeNK1ueP3+eESNG8PrrrzNixIjGbILNqDrVUP/+/Y1TDUVFRXHHHXcwcuRIY92cnBwATp06xfDhwwkICGD+/PlkZ2dbsRW2wRw9mQjgmNb6uNb6BrAaGF2lzmhgueH1WiBWKaXMcG4hhMHtPhi//PJLQkNDCQkJITs7m4MHD95yjOLiYmJjY3n77bcZOnSoFVphG6pONRQVFWWcaqhXr17cvHmTK1euAODk5ERJSQkAzzzzDDNnzmT//v24urqSm5t7y7EzMzPZvHlzo7bHmsyRZLyBkxW2TxnKqq2jtS4BrgBtzXBuIYRBbR+Mrq6uLFiwgK1bt5KVlcWIESMoKiq65RjNmzcnLCyMLVu2NDiOuXPnsmDBAlOaYlF1mWYIap5qCMDFxQVPT89b9rly5Qre3mUff+fOnTOWe3h4kJ+fD0iSsSql1HSlVJpSKq26bwBCOCpTPxivXr2Km5sbrVq14vz583z99dfV7q+U4i9/+QuHDh3irbfs76p2XacZgspTDRUWFnLu3DlOnjyJv78/BQUFxpVQ165dy/r16xk4cCAeHh4MGzaMsLAwnJ2dyc3NJSIigg8++IDPPvuMwMBAXnzxRb744guCg4P54osvGvk30PjMkWROA50rbHcylFVbRynVHGgFXKx6IK31x1rrcK11eLt27cwQmhBN36p/HWTG/5vH6bxCCk9kkfHp7Dp9MJbPwebi4kK7du0ICQmhZ8+ePP7448alC6rj5OTEX//6V1JSUvjwww/rFOObb75J9+7dGThwIIcPHwbKvrH369ePwMBAxowZw+XLlwFITU0lMDCQ4OBgZs2ahb+/fwN+Kw0zf8thCotvViorLL7J/C2Hb6kbGxtLcXExbm5uAJSUlPDBBx+QnZ2Nj48PUNaWQ4cOGRP32bNneemll0hPT+fee+9l3Lhx7N27l8WLF+Pr60tWVhZvvfUWcXFxZGZmEhcXZ/lGW5k5kkwq0E0p5auUugN4FPiqSp2vgMmG1w8DKdpWJ00Twsa8symTS2kbKpXV9YPxyJEjnDt3jjNnzpCUlMSRI0fYunUrf/vb34iPjwfKxnCEh5dNdVW+6qeLiwtbtmzhqaeeum186enprF692ngZKDU1FYBJkybx1ltvkZWVRUBAAK+99hoACQkJfPTRR2RmZuLk5FTboc3OlGmGfHx86NevX6WyXbt2MXr0aFq0aIGHhwcPPPBApffHjh0LVH44wNGYnGQM91hmAluAn4AvtdbZSqnXlVKjDNWWAm2VUseA54FbHnMWQlTv0FdLKMk7x5llz3B52zJKbxSSu/5/2Dt/MuPHj6f8+9rrr79Onz598Pf3Z/r06WitWbt2LWlpaYwfP57g4GAKC80/Z9uOHTsYM2YMd955Jy1btmTUqFFcv36dvLw8Bg0aBMDkyZP517/+RV5eHvn5+caF3x5//HGzx1MbU6YZKk/c9eHi4gJUfjjA0ZjlnozWerPWurvW+l6t9ZuGsle01l8ZXhdprcdprX+ntY7QWh83x3mFcAQ9R82guefddExYROvBCdw4f5zWsdPp88dlHD9+nF27dgEwc+ZMUlNTOXDgAIWFhWzcuJGHH36Y8PBwVq1aRWZmJq6ujj1nW3XTDDk3UxTcKLnt/a7qREZGsmHDBoqKirh27RobN2687T4VHwJwBDZ1418Icas/RHel4hP/Lh2649G2Pf/3970IDg42XobZtm0bffv2JSAggJSUlEYboxEdHU1ycjKFhYXk5+ezYcMG3NzcaN26tXHczsqVKxk0aBCenp54eHiwZ88eAFavXt0oMZarOs2Qp6szKLhcUHzbBwGq06dPH0aNGkVgYCC///3vCQgIoFWrVrXuM3jwYA4ePOgwN/5lWhkhbNwwv7tp37IF3p6u/HwCXFu4GOdf+85wGaaoqIinnnqKtLQ0OnfuzNy5c6t9RLkhkjNOM3/LYc7kFdLR05VZw3tUmqYnNDSUuLg4goKCuOuuu+jTpw8Ay5cvZ8aMGRQUFNC1a1eWLVsGwNKlS5k2bRrNmjVj0KBBt/1QNreK0wxFJqaQV1hc6f3y+10V29ilSxcOHDhg3K54f+WFF15g7ty5FBQUEB0dTVhYGFB2r6ucl5eXcZ82bdoY71s5AkkyQtg4Dw8PVHEhu2YPYfv2ZixYsOuWudjKE4qXlxfXrl1j7dq1PPzww8b9G3p5pvyR3/Inssq/6QOVYpgzZw5z5sy5Zf/du3ffUubn52ecbSAxMdH40IE1mGO9oenTp3Pw4EGKioqYPHkyoaGh5grPLkiSEcLKbtdTaNu2LZGRkfj7++Pq6kr79u1vOYanpyfTpk3D39+fu+++29ibAIiPj2fGjBm4urryww8/1Ou+TG2P/DZ00tFNmzYxb948SkpK8PHxISkpqUHHMYeOnq7Vrpxan/WGPv/8c3OGZHdk+WUhrKhqTwHK1ruxleUIfGdvorpPCAX8ktj05zWz9d9/Q8jyy0IIo/oMDrQGe19ZVNYbsjy5XCaEFZnjnoAlzRreo9pv+va0sqisN2RZ0pMRwopsvacg3/SFqaQnI4QVNYWegnzTF6aQJCOEFZV/eNf2dJkQTZkkGSGsTHoKwp7JPRkhhBAWI0lGCCGExUiSEUIIYTGSZIQQQliMJBkhhBAWI0lGCCGExUiSEUIIYTGSZIQQQliMJBkhhBAWI0lGCCGExUiSEUKYbMqUKdx11134+/tbOxRhYyTJCCFMFh8fzzfffGPtMIQNkiQjhDBZdHQ0bdq0sXYYwgZJkhFCCGExkmSEEEJYjCQZIYRdOHnyJIMHD6Z37974+fnx3nvvWTskgSQZYQMWLlxIQUGBtcMQBnl5eXz44YfWDqPemjdvzp///GcOHjzI7t27Wbx4MQcPHrR2WA5PkoywOkkytqVikknOOE1kYgq+szcRmZhCcsbpavd57LHH6N+/P4cPH6ZTp04sXbq0MUMGoEOHDoSGhgLg4eFBr169OH26+nhF45EkIyzis88+IyIiguDgYP7whz9w8+ZNnnzyScLDw/Hz8+PVV18F4P333+fMmTMMHjyYwYMHWzlqATB79mx+/vlnfHv4ER+fwNHU7Wgg4y9ziE9IIDnjNH/5y1+YM2cOAO+88w779++nbdu2zJ8/n1OnTvHEE09YtQ05OTlkZGTQt29fq8YhAK11g3+ANsA/gKOG/7auod43QB6wsa7HDgsL06JpOnjwoB45cqS+ceOG1lrrJ598Ui9fvlxfvHhRa611SUmJHjRokN63b5/WWmsfHx+dm5trtXhFZb/88ov28/PTA+Zt1V4PzNItI8Zqnxc36js6dNN3dOihB8zbquPj4/U333yj09LStL+/v7527ZrOz8/XvXv31j/++KNV48/Pz9ehoaF63bp1Vo3DWoA0bcLnurl/TO3JzAa2aq27AVsN29WZD0w08Vyiidi6dSvp6en06dOH4OBgtm7dyvHjx/nyyy8JDQ0lJCSE7OxsuV5u487kFeLS2Y+ik9ncuHAC57b34OTmyYlTp/nhhx8YMGAAO3fuZMyYMbi5ueHu7s7YsWPZsWOH1WIuLi7moYceYvz48YwdO9ZqcYjfNDdx/9FAjOH1cmA78GLVSlrrrUqpmKrlwj5prZk8eTLz5s0zlv3yyy8MHTqU1NRUWrduTXx8PEVFRVaMUtxOR09XTuNF6X+uU3Q8nRad/bhZeI3m/96Nu7s7Hh4e1g6xEq01TzzxBL169eL555+3djjCwNSeTHut9VnD63NAexOPJ+xAbGwsa9eu5ddffwXg0qVLnDhxAjc3N1q1asX58+f5+uuvjfU9PDzIz8+3VrgO53Y388v/PWYN74GrsxMuHXtwNe0rXDr708o3gPzU9URFRQEQFRVFcnIyBQUFXL9+nfXrf3uvMWMG2LVrFytXriQlJYXg4GCCg4PZvHmz2WMR9XPXUcRFAAAShklEQVTbnoxS6jvg7mremlNxQ2utlVLalGCUUtOB6QD33HOPKYcSFpKccZr5Ww5zJq+Qjp6uzBregwdDvCvV6d27N2+88QbDhg2jtLQUZ2dnFi9eTEhICD179qRz585ERkYa60+fPp3777+fjh07sm3btsZukkNJzjjNS3/bT2HxTQBO5xXy0t/2Axj/Hdu2bUtkZCT/PXE4PUIHUtw9mJycDLr43stzQ7oy+Ys5xkQSGhpKfHw8ERERAEydOpWQkJBGjxlg4MCB5feAhQ1RpvyjKKUOAzFa67NKqQ7Adq11jxrqxgAvaK1H1uXY4eHhOi0trcGxCfOr+scO4OrsxLyxAbckGmGbIhNTOJ1XeEu5t6cru2YPsUJEt9cUY7YmpVS61jrc2nGUM/Vy2VfAZMPrycDfTTyesGHztxyulGAACotvMn/LYStFJOrrTDUf1rWV24KmGLP4jalJJhEYqpQ6Ctxn2EYpFa6U+rS8klJqB7AGiFVKnVJKDTfxvE3GgAEDgLLn9j///HMrR2Ma+WNv+jp6utar3BY0xZjFb0xKMlrri1rrWK11N631fVrrS4byNK311Ar1orTW7bTWrlrrTlrrLaYG3lR8//33QMOSTElJiSVCajD5Y2/6ym/mV+Tq7MSs4dVe5bYJTTFm8RsZ8W+Qk5NDz549iY+Pp3v37owfP57vvvuOyMhIunXrxt69e7l06RIPPvgggYGB9OvXj6ysLADmzp3LlClTiImJoWvXrrz//vvG47q7uwNlo6h37NhBcHAw7777LkVFRSQkJBAQEEBISIjxhndSUhKjRo1iyJAhxMbGNv4vohbyx970PRjizbyxAXh7uqIou69h6/fUmmLMogJrjwat6aexR/z/8ssv2snJSWdlZembN2/q0NBQnZCQoEtLS3VycrIePXq0njlzpp47d67WWuutW7fqoKAgrbXWr776qu7fv78uKirSubm5uk2bNsbR7m5ublprrbdt26ZHjBhhPN+CBQt0QkKC1lrrn376SXfu3FkXFhbqZcuWaW9vb+PoeFuz/sdTesC8rbrLixv1gHlb9fofT1k7JCFEBdjYiH9TB2PaFV9fXwICAgDw8/MjNjYWpRQBAQHk5OTw73//m3Xr1gEwZMgQLl68yNWrVwEYMWIELi4uuLi4cNddd3H+/Hk6depU47l27tzJM888A0DPnj3x8fHhyJEjAAwdOtRmVxl8MMRbvkE6mPIPi2bN5MKHqD/5v6YCFxcX4+tmzZoZt5s1a3bb+yMV93VycjLpfoqbm1uD9xWiId555x38/f3x9/dn4cKF5OTk0KNHDyZNmoS/vz8nT560doiiiZIkUw9RUVGsWrUKgO3bt+Pl5UXLli3rtG/VUe0Vj3XkyBFOnDhBjx5yb0M0vvT0dJYtW8aePXvYvXs3n3zyCZcvX+bo0aM89dRTZGdn4+PjY+0wRRPlEEmmrmti3M7cuXNJT08nMDCQ2bNns3z58jrvGxgYiJOTE0FBQbz77rs89dRTlJaWEhAQQFxcHElJSZV6Q0I0lpomufTx8aFfv3417ldd76dXr15MmzYNPz8/hg0bRmFh2ePtP//8M/fffz9hYWFERUVx6NChxmqesDKTRvxbkrlG/MsodSFq995773Hx4kVef/11AF5++WXatWvHxx9/zIEDB6rdJz09nfj4eHbv3o3Wmr59+/LZZ5/Rp08f0tLSCA4O5pFHHmHUqFFMmDCB2NhYlixZQrdu3dizZw8vvfQSKSkpjdlMh2FvI/5tnoxSF46sLr34hkxyWVPvx9fXl+DgYADCwsLIycnh2rVrfP/994wbN864iN3Zs2drPb6wH3b/dJmMUheOqq4TS1Y3yWXr1q0bdM6qD8AUFhZSWlqKp6cnmZmZDW2KaMLsvicjo9SFo6pPL/7555/nwIEDHDhwgC6DxjF+9XGuj3zLLL2fli1b4uvry5o1a4CyR6L37dtnhhaKpsDuk4yMUheOqiG9+PLez+m8QjS/9X6qJpqKvZ++ffvetvezatUqli5dSlBQEH5+fvz97zKXrqOw+xv/ULc1UISwNw2ZIl+m1W/6bO3Gv93fkwEZpS4c06zhPap9srK2XrzcwxTmZveXy4RwVA2ZWFLuYQpzc4iejBCOqr69+Ib0foSojSQZIYRReUKSe5jCXCTJCCEqkXuYwpzknowQQgiLkSQjhBDCYiTJCCGEsBhJMkIIISxGkowQQgiLkSQjhBDCYiTJCCGEsBhJMkIIYSaZmZls3rzZ2mHYFEkyQghhJg1JMiUlJRaKxjZIkhFC2I0VK1YQGBhIUFAQEydOJCcnhyFDhhAYGEhsbCwnTpwAID4+nieffJJ+/frRtWtXtm/fzpQpU+jVqxfx8fHG47m7u/Pcc8/h5+dHbGwsubm5AMTExFC+FMmFCxfo0qULN27c4JVXXuGLL74gODiYL774guvXrzNlyhQiIiIICQkxrqOTlJTEqFGjGDJkCLGxsY37S2psWmub/AkLC9NCCFFXBw4c0N26ddO5ublaa60vXryoR44cqZOSkrTWWi9dulSPHj1aa6315MmTdVxcnC4tLdXJycnaw8NDZ2Vl6Zs3b+rQ0FCdkZGhtdYa0J999pnWWuvXXntNP/3001prrQcNGqRTU1O11lrn5uZqHx8frbXWy5YtM9bRWuuXXnpJr1y5Umut9eXLl3W3bt30tWvX9LJly7S3t7e+ePGi2X8PQJq2gc/w8h/pyQgh7EJKSgrjxo3Dy8sLgDZt2vDDDz/w+OOPAzBx4kR27txprP/AAw+glCIgIID27dsTEBBAs2bN8PPzIycnB4BmzZoRFxcHwIQJEyrtXxfffvstiYmJBAcHExMTQ1FRkbE3NXToUNq0aWNqs22eTJAphHBILi4uQFkiKX9dvl3TfRKlFADNmzentLQUgKKiohrPobVm3bp19OhReamEPXv24ObmZlL8TYVJPRmlVBul1D+UUkcN/71lkW+lVLBS6gelVLZSKkspFWfKOYUQojpDhgxhzZo1XLx4EYBLly4xYMAAVq9eDcCqVauIioqq1zFLS0tZu3YtAJ9//jkDBw4EoEuXLqSnpwMY3wfw8PAgPz/fuD18+HAWLVqENixzn5GR0cDWNV2mXi6bDWzVWncDthq2qyoAJmmt/YD7gYVKKU8TzyuEcCDJGaeJTEzBd/YmIhNTSM44fUsdPz8/5syZw6BBgwgKCuL5559n0aJFLFu2jMDAQFauXMl7771Xr/O6ubmxd+9e/P39SUlJ4ZVXXgHghRde4H//938JCQnhwoULxvqDBw/m4MGDxhv/L7/8MsXFxQQGBuLn58fLL79s2i+iCVLlGbZBOyt1GIjRWp9VSnUAtmuta11CTym1D3hYa320tnrh4eG6/OkNIYTjSs44Xe1qnbdbStoc3N3duXbtmkXPYW5KqXStdbi14yhnak+mvdb6rOH1OaB9bZWVUhHAHcDPJp5XCOEg5m85XCnBABQW32T+lsNWikjUx21v/CulvgPuruatORU3tNZaKVVjt8jQ01kJTNZal9ZQZzowHeCee+65XWhCCAdwJq+wXuXm1NR6MbbotklGa31fTe8ppc4rpTpUuFz2aw31WgKbgDla6921nOtj4GMou1x2u9iEEPavo6crp6tJKB09Xa0QjagvUy+XfQVMNryeDPy9agWl1B3AemCF1npt1feFEKI2s4b3wNXZqVKZq7MTs4bXevtX2AhTk0wiMFQpdRS4z7CNUipcKfWpoc4jQDQQr5TKNPwEm3heIYSDeDDEm3ljA/D2dEUB3p6ujXLTX5iHSU+XWZI8XSaEEPVnb0+XCSGEEDWSJCOEEMJiJMkIIYSwGEkyQgghLEaSjBBCCIuRJCOEEMJiJMkIIYSwGEkyQgghLEaSjBBCCIuRJCOEEMJiJMkIIYSwGEkyQgghLEaSjBBCCIuRJCOEEMJiJMkIIYSwGEkywubMnTuXBQsW3FK+ZMkSVqxYUeu+SUlJzJw501KhCSHqqbm1AxCiLkpKSpgxY4a1wxBC1JP0ZIRNePPNN+nevTsDBw7k8OHDAMTExPBf//VfhIeH895771Xq4cTExPDiiy8SERFB9+7d2bFjxy3H3LRpE/379+fChQuN2hYhxG8kyQirS09PZ/Xq1WRmZrJ582ZSU1ON7924cYO0tDT++Mc/3rJfSUkJe/fuZeHChbz22muV3lu/fj2JiYls3rwZLy8vi7dBCFE9uVwmrG7Hjh2MGTOGO++8E4BRo0YZ34uLi6txv7FjxwIQFhZGTk6OsTwlJYW0tDS+/fZbWrZsaZmghRB1Ij0ZYdPc3NxqfM/FxQUAJycnSkpKjOX33nsv+fn5HDlyxOLxCSFqJ0lGWF10dDTJyckUFhaSn5/Phg0bTDqej48P69atY9KkSWRnZ5spSiFEQ8jlMmFxyRmnmb/lMGfyCuno6cqs4T14MMTb+H5oaChxcXEEBQVx11130adPH5PP2bNnT1atWsW4cePYsGED9957r8nHFELUn9JaWzuGaoWHh+u0tDRrhyFMlJxxmpf+tp/C4pvGMldnJ+aNDaiUaIQQ5qGUStdah1s7jnJyuUxY1PwthyslGIDC4pvM33LYShEJIRqTJBlhUWfyCutVLoSwL5JkhEV19HStV7kQwr5IkhEWNWt4D1ydnSqVuTo7MWt4DytFJIRoTPJ0mbCo8pv7tT1dJoSwX5JkhMU9GOItSUUIB2XS5TKlVBul1D+UUkcN/21dTR0fpdSPSqlMpVS2Ukqm0hVCCAdh6j2Z2cBWrXU3YKthu6qzQH+tdTDQF5itlOpo4nmFEEI0AaYmmdHAcsPr5cCDVStorW9orf9j2HQxwzmFEEI0EaZ+4LfXWp81vD4HtK+uklKqs1IqCzgJvKW1PlNDvelKqTSlVFpubq6JoQkhhLC22974V0p9B9xdzVtzKm5orbVSqto5arTWJ4FAw2WyZKXUWq31+WrqfQx8DGXTytQhfiGEEDbstklGa31fTe8ppc4rpTporc8qpToAv97mWGeUUgeAKGBtvaMVQgjRpJh6uewrYLLh9WTg71UrKKU6KaVcDa9bAwMBmbhKCCEcgKlJJhEYqpQ6Ctxn2EYpFa6U+tRQpxewRym1D/gnsEBrvd/E8wohhGgCTBqMqbW+CMRWU54GTDW8/gcQaMp5hBBCNE3yOLEQQgiLsdlFy5RSucC/zXQ4L+CCmY7VFDhSex2prSDttWfmaquP1rqdGY5jFjabZMxJKZVmSyvFWZojtdeR2grSXntmr22Vy2VCCCEsRpKMEEIIi3GUJPOxtQNoZI7UXkdqK0h77ZldttUh7skIIYSwDkfpyQghhLACu0wyjraYWh3bG6yU+sHQ1iylVJw1YjVVXdpqqPeNUipPKbWxsWM0B6XU/Uqpw0qpY0qpW9ZpUkq5KKW+MLy/RynVpfGjNJ86tDfa8PdaopR62Boxmksd2vq8Uuqg4e90q1LKxxpxmotdJhkcbzG1urS3AJiktfYD7gcWKqU8GzFGc6lLWwHmAxMbLSozUko5AYuB3wO9gceUUr2rVHsCuKy1/h3wLvBW40ZpPnVs7wkgHvi8caMzrzq2NQMI11oHUjaR8NuNG6V52WuScbTF1OrS3iNa66OG12comzHbZgZs1cNt2wqgtd4K5DdWUGYWARzTWh/XWt8AVlPW7ooq/h7WArFKKdWIMZrTbdurtc7RWmcBpdYI0Izq0tZtWusCw+ZuoFMjx2hWTfmDtTZmXUytCahTe8sppSKAO4CfLR2YBdSrrU2UN2X/T5Y7ZSirto7WugS4ArRtlOjMry7ttRf1besTwNcWjcjCTJog05oaczE1W2CO9hqO0wFYCUzWWtvkt0JztVWIpkwpNQEIBwZZOxZTNNkk42iLqZmjvUqplsAmYI7WereFQjWZOf9tm6jTQOcK250MZdXVOaWUag60Ai42TnhmV5f22os6tVUpdR9lX6oGVbis3yTZ6+UyR1tMrS7tvQNYD6zQWttkIq2j27bVDqQC3ZRSvoZ/t0cpa3dFFX8PDwMpuukOeqtLe+3FbduqlAoBPgJGaa2b/pcorbXd/VB2bXorcBT4DmhjKA8HPjW8HgpkAfsM/51u7bgt3N4JQDGQWeEn2NqxW6Kthu0dQC5QSNl17+HWjr2e7fw/wBHK7pvNMZS9TtkHD0ALYA1wDNgLdLV2zBZubx/Dv+N1ynps2daO2YJt/Q44X+Hv9Ctrx2zKj4z4F0IIYTH2erlMCCGEDZAkI4QQwmIkyQghhLAYSTJCCCEsRpKMEEIIi5EkI4QQwmIkyQghhLAYSTJCCCEs5v8DPZW1ortdA6wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f217f4912e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "en_first_2d = en_wv.vectors[:,:2]\n",
    "print(en_first_2d.shape)\n",
    "plot_words(en_wv, en_words, en_first_2d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XlcVOX+wPHPw6IQLlgqCW5UKgLDJq5cl3JBw1zzpte64lI3Tb3VjdRW0xZuetVXZd1ME+1qWibumaa4ZSmoiOBG6uQvNEMT3EBZnt8f4MQybLIMMt/368XLmTMP53nOyMz3nOd8z/corTVCCCGsj42lByCEEMIyJAAIIYSVkgAghBBWSgKAEEJYKQkAQghhpSQACCGElZIAIIQQVkoCgBBCWCkJAEIIYaXsLD2A4jRs2FC3bNnS0sMQQoi7xoEDBy5qrRuVpm21DgAtW7YkJibG0sMQQoi7hlLql9K2lSkgIYSwUhIAhBDCSkkAEKKMtNZkZ2dbehhClJsEACFKwWg00qZNG/7+97/j7e2Nra2t6bVVq1YRGhoKQGhoKOPHj6dTp0488MAD7NixgzFjxtC2bVtTG4Dx48cTGBiIl5cXb775ZhVvjRA5JAAIUUqJiYlMmDCBhIQEnJycimx3+fJlfvzxR+bOncuAAQN44YUXSEhI4MiRI8TGxgLwzjvvEBMTQ1xcHDt37iQuLq6qNkMIEwkAQpRSixYt6NSpU4ntHnvsMZRSGAwGXFxcMBgM2NjY4OXlhdFoBOCrr74iICAAf39/EhISOHr0aCWPXojCqnUaqBCWsuZQErO+O8G5lDRcnR0ZZbgn316/Usr0OD09Pd/v1q5dGwAbGxvT49vPMzMzOXPmDLNnzyY6OpoGDRoQGhpaaB1CVAU5AhCigDWHkpi2+ghJKWloICkljX9vPsGV9ExTGxcXF44dO0Z2djaRkZFlWv+VK1dwcnKifv36XLhwgW+//baCt0CI0pEjACEKmPXdCdIysvItu5mZxR/Xbpqeh4eH079/fxo1akRgYCDXrl0r9fp9fX3x9/fHw8ODZs2aERQUVGFjF6IsVHW+KXxgYKC2tiuBjUYj/fv3Jz4+vlzrWbduHUePHsXV1ZUbN27w7LPPVtAIaz73qRsx96lQwJnwkKoejhBlopQ6oLUOLE1bOQKooQYMGMCAAQMsPYy7kquzI0kpaWaXC1GTyDmAaigrK4unn34aLy8v+vTpQ0JCAgEBAabXExMTCQgIYPPmzQwbNsy0fMeOHfTv3x+AZ555RvLM71BYcBsc7W3zLXO0tyUsuI2FRiRE5ZAAUA0lJiby3HPPkZCQgLOzM4cOHaJ+/fqmHPLFixczevRoevXqxb59+7h+/ToAK1euZPjw4UDOHLXkmd+ZQf5uvDfEgJuzIwpwc3bkvSEGBvm7WXpoQlQomQKqhtzd3fHz8wOgXbt2GI1Gxo0bx+LFi5kzZw4rV65k//792NnZ0bdvX9avX8/jjz/Oxo0bef/994GcPPMFCxaQmZnJ+fPnOXr0KD4+PpbcrLvKIH83+cIXNZ4EAAszl2+eN3fc1taWtLQ0hg4dyltvvcUjjzxCu3btuO+++wAYPnw4H330Effeey+BgYHUrVtX8syFEKUiU0AWVJp889scHBwIDg5m/PjxjB492rS8e/fuHDx4kM8++8w0/SN55kKI0pAAYEFF5ZtfzJNvntfIkSOxsbGhT58+pmW2trb079+fb7/91nQCOG+e+d/+9jfJMxdCmCXXAVhQWfPNZ8+eTWpqKjNnzqz0sQkh7k5yHcBdoiz55oMHD+bUqVNs3769KoYmhLACMgVkQWXJN4+MjCQuLo6GDRtW1fCEEDWcHAFY0O00w7xZQGHBbST9UAhRJSQAWJjkmwshLEWmgIQQwkpJABBCCCslAUAIIaxUhQQApVRfpdQJpdTPSqmpZl4PVUolK6Vic3/GVUS/Qggh7ly5TwIrpWyB+UBv4FcgWim1Tmtd8C7XK7XWE8vbnxBCiIpREUcAHYCftdantda3gBXAwApYrxBCiEpUEQHADfi/PM9/zV1W0FClVJxSapVSqllRK1NKPaOUilFKxSQnJ1fA8IQQQphTVSeB1wMttdY+wFZgSVENtdYLtNaBWuvARo0aVdHwhBDC+lREAEgC8u7RN81dZqK1vqS1vl3iciHQrgL6FUIIUQ4VEQCigVZKKXelVC1gOLAubwOlVJM8TwcAxyqgXyGEEOVQ7iwgrXWmUmoi8B1gC3yutU5QSs0AYrTW64DJSqkBQCbwBxBa3n6FEEKUj9wPQAghapCy3A9ArgQWQggrJQFACCGslAQAIYSwUhIAhBDCSkkAEEIIKyUBQAghrJQEACGEsFISAIQQwkpJABBCCCslAUAIIayUBAAhhLBSEgCEEMJKSQAQQggrJQFACCGslAQAIYSwUhIAhBDCSkkAEEIIKyUBQAghrJQEACGEsFISAIQQwkpJABBCCCslAUAIIayUBAAhhLBSEgCEEMJKSQAQQggrJQFACCGslAQAIYSwUhIAhBDCSlVIAFBK9VVKnVBK/ayUmmrm9dpKqZW5r+9TSrWsiH6FEELcuXIHAKWULTAf6Ad4AiOUUp4Fmo0FLmutHwLmAv8ub79CCCHKpyKOADoAP2utT2utbwErgIEF2gwEluQ+XgX0VEqpCuhbCCHEHaqIAOAG/F+e57/mLjPbRmudCaQC95lbmVLqGaVUjFIqJjk5uQKGJ4QQwpxqdxJYa71Aax2otQ5s1KiRpYcjLMRoNLJ8+XJLD0OIGq0iAkAS0CzP86a5y8y2UUrZAfWBSxXQt6ihJAAIUfkqIgBEA62UUu5KqVrAcGBdgTbrgFG5jx8HtmutdQX0LSxozpw5eHt74+3tzbx58zAajbRt25ann34aLy8v+vTpQ1paGgA9evRgypQpdOjQgdatW7N7924AsrKyCAsLo3379vj4+PDpp58CMHXqVHbv3o2fnx9z584lPT2d0aNHYzAY8Pf3JyoqCoCIiAiGDBlC3759adWqFS+//LJl3gwh7kZa63L/AI8CJ4FTwKu5y2YAA3IfOwBfAz8D+4EHSrPedu3aaVE9xcTEaG9vb33t2jV99epV7enpqQ8ePKhtbW31oUOHtNZaDxs2TH/xxRdaa627d++uX3zxRa211hs3btQ9e/bUWmv96aef6pkzZ2qttU5PT9ft2rXTp0+f1lFRUTokJMTU3+zZs/Xo0aO11lofO3ZMN2vWTKelpenFixdrd3d3nZKSotPS0nTz5s312bNnq+x9EKK6AWJ0Kb+77SooiGwCNhVY9kaex+nAsIroS1QPe/bsYfDgwTg5OQEwZMgQdu/ejbu7O35+fgC0a9cOo9Fo+p0hQ4YUWr5lyxbi4uJYtWoVAKmpqSQmJlKrVq1C/U2aNAkADw8PWrRowcmTJwHo2bMn9evXB8DT05NffvmFZs2aIYQoXoUEAFHzrTmUxKzvTnAuJQ1XZ0c8LqXg5li4Xe3atU2PbW1tTVNAeV+ztbUlMzMTyDkC/fDDDwkODs63nh07dpR6bAX7vL1uIe5mY8aMYcOGDTRu3Jj4+PhK6aPaZQGJ6mfNoSSmrT5CUkoaGkhKSWP75ftY+uXX3Lhxg+vXrxMZGUnXrl3LvO7g4GA++eQTMjIyADh58iTXr1+nbt26XL161dSua9euLFu2zNTm7NmztGnTpkK2D+Ddd981PU5JSeHjjz/O9/obb7zB999/z+TJk4mJiamwfoUoSmhoKJs3b67UPuQIQBTJaDTSv39/6j/5AWkZWfle0w3dUa170KFDBwDGjRtHgwYNzK7n0UcfLXKvfNy4cRiNRgICAtBa06hRI9asWYOPjw+2trb4+voSGhrKhAkTGD9+PAaDATs7OyIiIvLt+ZfXu+++yyuvvAL8GQAmTJhgen3GjBkA9OrVq8L6FKI43bp1yzeFWhmUrsbJOIGBgVr2tizndgC43v/fmPsrUcCZ8JCqHla5/e9//+ODDz7g1q1bdOzYkXr16jFnzhwMBgNeXl5kZWWxdu1a2rRpQ+/evXn99dcZNGgQKSkpZGZmMnPmTAYOLHixuxAV7/ZnsCxTQEqpA1rrwNK0lSMAK3fu3DkmT57MjBkzmDNnDgsXLsz3elZWFje2f8zl0/HY1r2P2s28uXFsF65j5+Pq7Mirr77Kxo0bCQ8PZ9GiRfzjH/9g165d1K5dmy+++ILjx4/TsmVLYmJiuHbtGv369eMvf/kLe/fuxc3NjbVr1+LoaOZkQiU5duwYK1eu5IcffsDe3p4JEyZgMBhwdHQkNjYWyPnQxcfHm55nZGSwZs0a6tWrx8WLF+nUqRMDBgxAqpmIu52cA7Byrq6urFq1Ck9Pz0Jf/gCJiYlMffGfPDj+U2xqO3E9IQpla4+6ZCQsuA0fffQRTzzxBL169WLfvn107tyZGTNm8Ouvv/Laa6+ZXd9zzz1HQkICzs7OfPPNN1WxmSbbtm3jwIEDtG/fHj8/P7Zt28bp06dL/L1XXnkFHx8fevXqRVJSEhcuXKiC0QprsuZQEkHh23GfupGg8O2sOVTwetqKJ0cAVsJoNJrd+16yZAmLFi3i1q1bPPTQQ3zxxRdkZWXh4+PDli1bcHd3Z+yjnQj/lzf3NG7NjZTfsL+nHpdWvcEZz1tcuXKFpUuXsmXLFvr27cvUqVOJiori5MmT1K5dmyeffDLfOIpLE60MBbOXPC9fZtSoUbz33nv52s2ePbvIdSxbtozk5GQOHDiAvb09LVu2JD09vVLHLazL7USL2+faklLSmLb6CC90cq7UfuUIwIqY2/seNmwY0dHRHD58GJsGTfEePgWfd3Zx/b42zFmymtq1a7NixQqeHD6MKSP70qKZG1Gb13F/o/t44IEHuOeee9i9ezdRUVH06tWLRYsW8cYbbxASEkJcXBxr1qzJN4aqTNk0l720LbUxS5at4Pfff6dLly788ccfrFixgps3b5oykQpmIKWmptK4cWPs7e2Jioril19+Yfr06ZU2bmF9Zn13olCixdlv3mPiiBBOnDhB06ZNWbRoUYX3K0cAVsTc3nd8fDyvvfYaZ39L5nxyCrXd/bnPE2w8erJ0+XLus8lk8eLFfPbZZ6aUtNq1axMcHMz48eOpU6eOaf12dnbY2Njw9ddfM2LECK5evcquXbsssq1g/kOV7dyUun95kj59+pCdnU3v3r0ZO3YszZs3x8fHh4CAAJYtW0ZQUBDe3t7069ePKVOm8Nhjj2EwGAgMDMTDw0MCgKhQ51LSCi1rNODlSk+0kCOAGqrgfOKWhN/M7n2Hhoby0Ucf0XTsx9QLGoHOzNkLdmjqya3UZH7/I5WsrCy8vb3zrX/kyJHY2Njg4OCQb51Nmzbl22+/pX///lWzocXI+6G6sj+Sc4smcG7RBJIvXCA2NpbTp09z4MABPD09adu2LceOHeP555/H39+fmTNnsm/fPi5dusSjjz5Keno6b7/9NosXL+aTTz5h4sSJAEyfPp0xY8bQo0cPHnjgAT744ANLba64i7k6m0+EKGp5RZEAUAOZm/r49+YTXEkvPN1y9epVmjRpQtKlq1xP2JHvtbq+wWRm3GL06NEAvPTSSxgMBq5evcqePXsYPXo09erVM02XdOjQgbS0NIxGI7Vr1+bLL7+ke/fuGI1GGjZsSMuWLfOls7300kuVuid9+8Nz87efuXbke+5/ag73P/Uf0uO3cOjQoULt9+7dy7PPPsvatWt58MEHeeedd3jkkUfYv38/UVFRhIWFcf369UK/d/z4cb777jv279/PW2+9ZZpKEqK0woLb4Ghvm2+Zo70tYcEVd7GjOTIFVAOZm/q4mZnFH9duFmo7c+ZMOnbsyKXM2tg3bkX2rT/3mp08HyZ19/8YMWKEaVloaCiPPfYYWVlZnDx5ksaNG9O3b19cXV2JiooiPDychx9+GK01ISEhFs2XDwtuw7TVR7jyawL3tO6MTS0HHO1t6TRgkKka6W3Hjh3jmWeeYcuWLbi6ugI5dYrWrVtnOkGcnp7O2bNnC/UTEhJC7dq1qV27No0bN+bChQs0bdq08jdQ1BiD/HPuoZU3YSEsuI1peWWRAFADmZtPtKvvgkvoR6bnL730kunx+PHjC2UhAGT/doyuffrj7PxnJsLQoUMZOnSo6fmkSZNMRdoARowYkS9gWELezB/ne+xxtLflehq45X6oDqz+odDvNGnShPT0dA4dOmQKAFprvvnmm0IlJwqmgEotIlERBvm7VfoXfkEyBVQD3cl84iB/N94bYsDN2REF3Ny1kOx9y1gw970if6c6Kjj9dflGBrXcvHD+PZatkzvRu7Wz2bpFzs7ObNy4kWnTppkK0QUHB/Phhx/eLnludtpIiLuZHAHUQLenPvLuzZdmPjH/HsjdV+IBzE9/matb5O/vX+h3XVxc2LBhA/369ePzzz/n9ddf5/nnn8fHx4fs7Gzc3d3ZsGFDlWyHEFVBagHVUAUvgKqK+cTqwH3qxhpVt0iIspJaQMIi84nVgauzI0lmzoFUdjqdEHcjOQcgapSw4DZkJyXw+6q3TMuKm/46d+4cjz/+eFUNT4hqRQKAMKtHjx535Y1PBvm78Uw3dxzsbFDkZP68N8RQ5NHQ7WJ4QlgjCQDirvbGG28wb9480/NXX32VujfO0eWhhpwJD2Fez7q8NaY/p06dYufOnfj5+eHn54e/vz9Xr17FaDQWuspZCGshAcBKGI1GPDw8GDlyJG3btuXxxx/nxo0bzJgxg/bt2+Pt7c0zzzxD3qSAr7/+mg4dOtC6dWvThVNGo5GuXbsSEBBAQEAAe/futdQmATn3TV26dCkA2dnZrFixwnQRVsEre2fPns38+fOJjY1l9+7dVXofAiGqIwkAVuTEiRNMmDCBY8eOUa9ePT7++GMmTpxIdHQ08fHxpKWl5UtzzMzMZP/+/cybN4+33sqZU2/cuDFbt27l4MGDrFy5ksmTJ1tqcwBo2bIl9913H4cOHWLLli34+/tz3333ma7sXb9+Pc2bNwcgKCiIF198kQ8++ICUlBTs7CQHQlg3CQBWpFmzZgQFBQHw5JNPsmfPHqKioujYsSMGg4Ht27eTkJBgaj9kyBAgf93+jIwMnn76aQwGA8OGDePo0aNVvh0FjRs3joiICBYvXsyYMWOAnCt7HRwc8l28NXXqVBYuXEhaWhpBQUEcP37cUkMWolqQXaAaquB1AKMM9xS6haFSigkTJhATE0OzZs2YPn16vhud3C5xkLe8wdy5c3FxceHw4cNkZ2fnqwZaVQpu2/OPdGDzG2+QkZHB8uXL2b17N87OzixatIjevXvj5OREjx49OHXqFAaDAYPBQHR0NMePHzeVxxbCGskRQA1UVDXQs2fP8uOPPwKwfPly/vKXvwDQsGFDrl27VqpsmNTUVJo0aYKNjY3p7mFVydy2vbH+BC0NHfjrX/+Kre2fFRVvX9n73HPPsW/fPubNm4e3tzc+Pj7Y29vTr1+/Kh27ENWNHAHUQEVVA3Vs1Iz58+czZswYPD09GT9+PJcvX8bb25v777+f9u3bl7juCRMmMHToUJYuXUrfvn1xcnKqrM0wy9y23biVwZ69P/LRe98COSmsPXr0AKB58+amaa2OHTsWWl/BEtVCWBMpBVEDmSuHkJl6gd9XvcWt5F8sMqaKUnDbbl08S/Kqt7indWdS90dabFxCVBdSCsLKFVUOwc727p/xK7httRo2x+3ZRbhJqQchyqxc3whKqXuVUluVUom5/zYool2WUio292ddefoUJTN3d6G6DV1Z/u0eC42o4ljqzklC1ETl3SWcCmzTWrcCtuU+NydNa+2X+zOgnH2KEhSs7V9SOYS7SVVvW0pKCh9//HGZfic0NLTEE+p529ytZTfE3a+8U0ADgR65j5cAO4Ap5VynqAA1uRpoVW7b7QAwYcKEKulPiKpU3iMAF631+dzHvwEuRbRzUErFKKV+UkoNKmefQlSZqVOncurUKfz8/AgLCyMsLAxvb28MBgMrV64Ecm4dOXHiRNq0aUOvXr34/fffTb9/4MABunfvTrt27QgODub8+fNFdQXk3J4zMDAQLy8v3nzzzUrdNiFKPAJQSn0P3G/mpVfzPtFaa6VUUSlFLbTWSUqpB4DtSqkjWutTRfT3DPAMYLqEXwhLCQ8PJz4+ntjYWL755hv++9//cvjwYS5evEj79u3p1q0bP/74IydOnODo0aNcuHABT09PxowZQ0ZGBpMmTWLt2rU0atSIlStX8uqrr/L5558X2d8777zDvffeS1ZWFj179iQuLg4fH58q3GJhTUoMAFrrXkW9ppS6oJRqorU+r5RqAvxurp3WOin339NKqR2AP2A2AGitFwALICcNtMQtEKKK7NmzhxEjRmBra4uLiwvdu3cnOjqaXbt2mZa7urryyCOPADm1l+Lj4+nduzcAWVlZNGnSpNg+vvrqKxYsWEBmZibnz5/n6NGjEgBEpSnvOYB1wCggPPfftQUb5GYG3dBa31RKNQSCgPfL2a8Q+XTp0qVCKpOaK6Fxp7TWeHl5ma6+LsmZM2eYPXs20dHRNGjQgNDQ0HylOYSoaOU9BxAO9FZKJQK9cp+jlApUSi3MbdMWiFFKHQaigHCtteUriFVDeWvT79ixg/79+wOwbt06wsPDLTm0aq+ivvwLlpmYvf0sv19KAaBr166sXLmSrKwskpOT2bVrFx06dKBbt26m5efPnycqKgqANm3akJycbAoAGRkZ+YrtFXTlyhWcnJyoX78+Fy5c4Ntvvy33NglRnHIdAWitLwE9zSyPAcblPt4LGMrTj7UbMGAAAwZYJns2JSWF5cuXl5gFU1F74HeqTp06/PbbbwwcOJDLly+TkZHB22+/zcCBA0u9DnNlJm7ZO6Fd2uDt7U2/fv3w8fHB19cXpRTvv/8+999/P4MHD2b79u14enrSvHlzOnfuDECtWrVYtWoVkydPJjU1lczMTJ5//nm8vLzM9u/r64u/vz8eHh75KrcKUWm01tX2p127drq6+c9//qO9vLy0l5eXnjt3rj5z5oz28PDQ48aN056enrp37976xo0bWmutu3fvrl9++WXdvn173apVK71r1y6ttdaZmZn6pZde0oGBgdpgMOj//ve/Wmutz5w5o728vLTWWkdFRemQkBCttdaLFy/Wzz33nNZa61GjRulJkybpzp07a3d3d/3111+bxvb++++b1vnGG2+Y1lnU+Eoj75iqMycnJ52RkaFTU1O11lonJyfrBx98UGdnZ5d6HS2nbNAtzPy0nLKhsoYtqqGzZ8/qHj166LZt22pPT089b948Sw+pTIAYXcrv2Lu/NkAVOnDgAIsXL2bfvn389NNPfPbZZ1y+fJnExESee+45EhIScHZ25ptvvjH9jrmbqixatIj69esTHR1NdHQ0n332GWfOnCn1OM6fP8+ePXvYsGEDU6fmXHu3ZcsWEhMT2b9/P7GxsRw4cIBdu3YBFDu+kuRNg3zhhRfo2bMnAQEBGAwG1q7985RPnTp1Sr3OyqK15pVXXsHHx4devXqRlJTEhQsXSv37rkWUkyhquaiZ7Ozs+M9//sPRo0f56aefmD9/frW470VlkFpAZbBnzx4GDx5sqoA5ZMgQdu/ejbu7u6mufN6bp9xuU3D5li1biIuLM10JmpqaSmJiIq1bty7VOAYNGoSNjQ2enp6mL7gtW7aY7ogFcO3aNRITE2nevHmx4ytJ3jTIzMxMbty4Qb169bh48SKdOnViwIABhe4zUNkKnqi9XQZi2bJlJCcnc+DAAezt7WnZsmWZTqKGBbdh2uoj+aaBpMyE9WnSpIkpW6tu3bq0bduWpKQkPD09LTyyiicBoAR5v2xIOEn7JvaF2ty+cQrk3DwlLS2t0Gt5b6qitebDDz8kODg433pK+8Wctz+dW81Va820adP4xz/+UWidxY2vLG7vYe/atQsbGxvTHvb995u7TKRy3D5Re/tLOikljWmrj5CVrUlNTaVx48bY29sTFRXFL7+UrfLp7auLCwaXmnpFtSiZ0Wjk0KFDZkuJ1wQyBVSMglkh6fe1Zt3atazc+zPXr18nMjKSrl27lnm9wcHBfPLJJ2RkZABw8uRJrl+/Xq6xBgcH8/nnn3Pt2jUAkpKS8l2RWhZrDiURFL4d96kbGTR3K+cuJAPw0UcfsXbtWg4cOEBsbCwuLi6F9rCNRiPLly8vsY+IiAgmTpxY5rGZO1GblpHFrSzNyJEjiYmJwWAwsHTpUjw8PMq8/kH+bvww9RHOhIfww9RH5Mvfil27do2hQ4cyb9486tWrZ+nhVAo5AihGwS+b2vc/xD1ePRk9pDcPNHRi3LhxNGhgtgBqscaNG4fRaCQgIACtNY0aNWLNmjXlGmufPn04duyYKQOlTp06/O9//8t3h6zSKLiHfeFKGpf/uMSaQ0nY2NgwaNCgYvewbweAv/3tb+XanqKcM1PmOivtCsqhDg0bNix1zr0Qt5mbUgzxbszQoUMZOXKkaRq3JpIbwhTD3I1VABRwJjykqodTJYLCt+ert5+89t/cOPEDytaW5m6u/P777zz44IO0a9eO9evX4+aWs4d88uRJ0tPT6dSpE8eOHcPd3Z1Ro0Yxfvx4xo8fT0xMDHZ2dsyZM4eHH36YiIgIYmJi+Oijj9i4cSNvv/0269ev59q1a4wZM4aLFy/SqFEjFi9eTPPmzQkNDaVevXpErN1GWuolnHuMxsnjL2RevcT5xZOwq+VAm+YuDB482HSyXYiSFNzhAXCws6FJ7CJ8H2rKvHnzLDi6O1OWG8LIFFAxrDErpOAedoMeodjf15QW/4pkx44dPPDAAxw5coQuXbrwwgsvEBcXR3R0NN7e3pw5c4bw8HC6du1KbGwsL7zwAvPnz0cpxZEjR/jyyy8ZNWpUvmmjyMhIwsPD2bRpEw0bNmTSpEmMGjWKuLg4Ro4cyeTJk01tz58/T8TqzTQb/hYpO5cAkHHxF+p6dGHF9gOFsp+EKIm5KcUU4xF2bPyG7du34+fnh5+fH5s2bbLQCCuXTAEVwxqzQoq6m1jBoFdUJlOtWrXytduzZw+TJk0CwMPDgxYtWnDy5EkAtm/fTkxWCtFPAAAY/0lEQVRMDFu2bDHNsf7444+sXr0agKeeeoqXX37ZtK5BgwYxpF0zbGz6M3TxCyjA7vwR9G/xvDWmP2/xZ/ZTt27dKuT9EDWbuSlFh6ZetJyygbgaepSflwSAYlhjVkhYcBvCVh0mI+vPyS+Fyg16GaZlRWUy7dixo9R9Pfjgg5w+fZqTJ08SGFjyEevtbKZB/m442ttwJjyEf/1rO61H9iyU/SREaZR2h6emkimgElhlVkieEx+qliPZt24UalJUJlPdunW5evWqqV3Xrl1ZtmyZqc3Zs2dp0ybnCKpFixZ88803/P3vfzfVyOnSpQsrVqwAcvL6S8qyqsjsJ2F9rP0Wo3IEIPKZ9d0JMrL/jAC2jvWo5ebJ3/r9hZCuf+6lF5XJ5OPjg62tLb6+voSGhjJhwgTGjx+PwWDAzs6OiIiIfNcleHh4sGzZMoYNG8b69ev58MMPGT16NLNmzTKdBC5OUdlPjRs3ruB3RtRE1niUn5dkAYl8rDHzSVSOdevWcfToUaZOncqaNWto3bq16Wrac+fOMXnyZGbMmMGcOXNYuHBhCWsTpVWWLCAJACKfgmmgt7k5O/LD1EcsMCJRE4SGhtK/f38ef/xxSw+lxpM0UHHHrH1OVJSO0WjEw8OD0NBQWrduzciRI/n+++8JCgqiVatW7N+/33S19969e1m3bh1hYWH4+flx6tQp/vvf/9K+fXt8fX0ZOnQoN24UPs8kKp8EAJHPIH833htiwM3ZEUXOnv97QwxWMycqSu/nn3/mX//6F8ePH+f48eMsX76cPXv2MHv2bN59911Tuy5dujBgwABmzZpFbGwsDz74IMOGDSM6OprDhw/Ttm1bFi1aZMEtsV5yElgUMsjfTb7wRYnc3d0xGHLu9eTl5UXPnj1RSmEwGEosbBgfH89rr71GSkoK165dK5ROLKqGBAAhRLHM1crxa5C/Kq2NjY3puY2NjanybVFCQ0NZs2YNvr6+RERElOn6EVFxZApICFEkc/dJnrb6CFsSfivTegpeH3L16lWaNGlCRkaG6ToRUfUkAAghilRU+e1Pd50u03qGDx/OrFmz8Pf359SpU8ycOZOOHTsSFBR0R2W7RcWQNFAhRJHkupC7j6SBCiEqhDVWxLUmEgCEEEWS60JqNskCEkIUydpr5dR0EgCEEMWS60JqLpkCEkIIKyUBQAghrJQEACGEsFLlCgBKqWFKqQSlVLZSqsi8U6VUX6XUCaXUz0qpqeXpUwghRMUo7xFAPDAE2FVUA6WULTAf6Ad4AiOUUp7l7FcIIUQ5lSsLSGt9DEApVVyzDsDPWuvTuW1XAAOBo+XpWwghRPlUxTkAN+D/8jz/NXeZEEIICyrxCEAp9T1wv5mXXtVar63oASmlngGeAWjevHlFr14IIUSuEgOA1rpXOftIAprled40d1lR/S0AFkBOMbhy9i2EEKIIVTEFFA20Ukq5K6VqAcOBdVXQrxBCiGKUNw10sFLqV6AzsFEp9V3uclel1CYArXUmMBH4DjgGfKW1TijfsIUQQpRXebOAIoFIM8vPAY/meb4J2FSevoQQQlQsuRJYCCGslAQAIYSwUhIAhBDCSkkAEEIIKyUBQAiB0WjE29s737KYmBgmT55soRGJqiB3BBNCmBUYGEhgYJFFfkUNIEcAQoh8Tp8+jb+/P7NmzaJ///4A7Ny5Ez8/P/z8/PD39+fq1aucP3+ebt264efnh7e3N7t37wZg/PjxBAYG4uXlxZtvvmnJTRElkCMAIYTJiRMnGD58OBEREVy+fJmdO3cCMHv2bObPn09QUBDXrl3DwcGBBQsWEBwczKuvvkpWVhY3btwA4J133uHee+8lKyuLnj17EhcXh4+PjyU3SxRBjgBEIebmg4uyY8cO015iSc6dO8fjjz9enqGJSpScnMzAgQNZtmwZvr6++V4LCgrixRdf5IMPPiAlJQU7Ozvat2/P4sWLmT59OkeOHKFu3boAfPXVVwQEBODv709CQgJHj0rl9+pKAoCoMq6urqxatcrSwxDAmkNJBIVvx33qRoLCt7Ml4Tfq169P8+bN2bNnT6H2U6dOZeHChaSlpREUFMTx48fp1q0bu3btws3NjdDQUJYuXcqZM2eYPXs227ZtIy4ujpCQENLT0y2whaI0JACIYt2eD46OjqZr164EBAQQEBDA3r17C7WNjo7G39+fU6dOmZ0zLsuRhag8aw4lMW31EZJS0tBAUkoa/958gvRsGyIjI1m6dCnLly/P9zunTp3CYDAwZcoU2rdvz/Hjx/nll19wcXHh6aefZty4cRw8eJArV67g5ORE/fr1uXDhAt9++61lNlKUipwDEEXKOx/cqlUrtm7dioODA4mJiYwYMYKYmBhT27179zJp0iTWrl1L8+bNef755wvNGYvqYdZ3J0jLyMq37GZmFn9cu4mTkxMbNmygd+/evP7666bX582bR1RUFDY2Nnh5edGvXz9WrFjBrFmzsLe3p06dOixduhR3d3f8/f3x8PCgWbNmBAUFVfXmiTJQWlffkvuBgYE675eMqBpGo5GOHTvSoEEDVq9ejaenJ6mpqUycOJHY2FhsbW05efIkN27cYMeOHYwdOxZHR0e2bNmCq6srAOHh4URGRjJy5EiGDBlC06ZNMRqN9O/fn/j4eAtvoXVzn7oRc596BZwJD6nq4YgKppQ6oLUuVf6uTAFZuYJzwWsO5dyrp+B88Ny5c3FxceHw4cPExMRw69Yt0zqaNGmCg4MDhw4dMi0zN2csqgdXZ8cyLRc1l0wBWbHbc8G3pwOSUtKYtvoIL3RyplatWkRGRhIcHEydOnVITU2ladOm2NjYsGTJErKy/pxCcHZ2ZtGiRfTu3RsnJyd69OhhmjM2GAxER0dz/Phx/Pz8LLWpIo+w4Db5/t8BHO1tCQtuY8FRCUuQIwArZm4uOC0ji093nQYwzQfPnTuXli1bsmTJEnx9fTl+/DhOTk75fs/FxYUNGzbw3HPPsW/fPubNm4e3tzc+Pj7Y29vTr1+/KtsuUbxB/m68N8SAm7MjCnBzduS9IQYG+btZemiiisk5ACsmc8FC1DxyDkCUiswFC2HdJABYsbDgNjja2+ZbJnPBQlgPOQlsxW7P+c767gTnUtJwdXYkLLiNzAULYSUkAFi5Qf5u8oUvhJWSKSAhhLBSEgCEECKPLl26VNi61qxZU62roUoAEEKIPMwVOrxTEgCEEOIuUqdOnUL3uZg4cSIREREAbNu2DX9/fwwGA2PGjOHmzZtATvkTT09PfHx8eOmll9i7dy/r1q0jLCwMPz8/Tp06xWeffUb79u3x9fVl6NChppvoWIoEACGEKKX09HRCQ0NZuXIlR44cITMzk08++YRLly4RGRlJQkICcXFxvPbaa3Tp0oUBAwYwa9YsYmNjefDBBxkyZAjR0dEcPnyYtm3bsmjRIotujwQAIYQopRMnTuDu7k7r1q0BGDVqFLt27aJ+/fo4ODgwduxYVq9ezT333GP29+Pj4+natSsGg4Fly5aRkJBQlcMvRNJAhRBWa82hpELXwQDY2dmRnZ1talfSXc3s7OzYv38/27ZtY9WqVXz00Uds3769ULvQ0FDWrFmDr68vERER7Nixo0K3p6zKdQSglBqmlEpQSmUrpYqsPaGUMiqljiilYpVSUtxHCGFx5u6MNm31EbKyNS1atODo0aPcvHmTlJQUtm3bBkCbNm0wGo38/PPPAHzxxRd0796da9eukZqayqOPPsrcuXM5fPgwAHXr1uXq1aumPq9evUqTJk3IyMhg2bJlVb7NBZX3CCAeGAJ8Woq2D2utL5azPyGEqBBFVcO9laVp1qwZf/3rX/H29jbd5QzAwcGBxYsXM2zYMDIzM2nfvj3PPvssf/zxBwMHDiQ9PR2tNXPmzAFg+PDhPP3003zwwQesWrWKmTNn0rFjRxo1akTHjh3zBQeL0FqX+wfYAQQW87oRaFjW9bZr105XlM6dO5d7HVFRUTokJKQCRiNE5Tpz5oz28vIqtHzs2LE6ISFBDx06VCclJZV73QU/E/369dOXL1+ukM9bZWs5ZYNuUeCn6eTl2rZeI0sPrVyAGF3K79iqOgeggS1KKQ18qrVeUEX9mlRkbq8Qd6uFCxcCsGrVqkpZ/6ZNm4C74/Pm6uxIUkqa6Xnm1Utc+HIazbs/YcFRVa0SzwEopb5XSsWb+RlYhn7+orUOAPoBzymluhXT3zNKqRilVExycnIZuijeneb2bt68GQ8PDwICAli9erXpd/fv30/nzp3x9/enS5cunDhxosLGKkRFOn36NP7+/uzbt4+uXbsSEBBAu3btTF/Sw4cPZ+PGjab2oaGhrFq1iqysLMLCwmjfvj0+Pj58+mnxM70//fTTXfWZKFgN167ufTz03CLmvDXVgqOqWiUGAK11L621t5mftaXtRGudlPvv70Ak0KGYtgu01oFa68BGjRqVtotyKSq3Nz09naeffpr169dz4MABfvvtN9PveHh4sHv3bg4dOsSMGTN45ZVXqmSsQpTFiRMnGDp0KBERERgMBrZu3crBgwdZsWIFkydPBuCJJ57gq6++AuDWrVts27aNkJAQFi1aRP369YmOjiY6OprPPvuMM2fOFNlX27Zt76rPhNwZrQrSQJVSToCN1vpq7uM+wIzK7rcszOX2zp8/nx49euDu7k6rVq0AePLJJ1mwIGf2KjU1lVGjRpGYmIhSioyMDIuNXwhzkpOTGThwIKtXr8bT05PU1FQmTpxIbGwstra2nDx5EoB+/frxz3/+k5s3b7J582a6deuGo6MjW7ZsIS4uzjRdlJqaSmJioulzUtCVK1cIDQ29qz4T1l4Nt1wBQCk1GPgQaARsVErFaq2DlVKuwEKt9aOACxCplLrd33Kt9eZyjrtYFZXbW5zXX3+dhx9+mMjISIxGIz169CjvsIW4YwX/5kcZ7qF+/fo0b96cPXv24Onpydy5c3FxceHw4cNkZ2fj4OAA5GS29OjRg++++46VK1cyfPhwICdB5MMPPyQ4ODhfX0aj0ewY5DNx9ynXdQBa60itdVOtdW2ttYvWOjh3+bncL3+01qe11r65P15a63cqYuBFqcjcXg8PD4xGI6dOnQLgyy+/NPWTmpqKm1vOnsPt8whCWIK5v/l/bz5BerYNkZGRLF26lOXLl5OamkqTJk2wsbHhiy++ICvrzxTIJ554gsWLF7N792769u0LQHBwMJ988olpT/7kyZNcv369yHHIZ+LuU+NKQZQ2t/evf/2r2dxeg8GAjY0Nzz77LA4ODixYsICQkBACAgJo3LixaZ0vv/wy06ZNw9/fn8zMzCrdRiHyMvc3fzMzi4vXbuLk5MSGDRuYO3cuLVu2ZMmSJfj6+nL8+HGcnJxM7fv06cPOnTvp1asXtWrVAmDcuHG4ublRt25dvL29+cc//lHs3/pjjz3GuHHj5DNxF1E5aaPVU2BgoI6JKduFw+5TN1Jwi7LSrnA+4p9kpv5ecYMTopow9zcPoIAz4SHlWrfRaKR///7Ex8ff8ToyMzOxs5OqM1VFKXVAa11kZYa8atwRgKuzY77nmVcv8dsXL1lVbq+wLgX/5ktafqdup5Pu3r2b0aNHYzAY8Pf3JyoqCiBfmvX06dN56qmnCAoK4qmnnsJoNJpSUAMCAu6K6wSsQY0Ly2HBbZi2+ojpkPh2bu97QwwWHpkQlaPg3zyAo72tKfmhIpw4cYLhw4cTERHB999/j1KKI0eOcPz4cfr06WPKKMrr6NGj7NmzB0dHR27cuMHWrVtxcHAgMTGRESNGUNaje1HxalwAuJ3SVTALyJpTvUTNVtl/8wXTSadPn86kSZOAnOthWrRoYTYADBgwAEfHnKOQjIwMsymowrJqXAAAye0V1qei/uZLk05aWnlPMheVgiosq8adAxBC3JnSppN27drVVMr45MmTnD17ljZtip9uKi4FVViOBAAhBFD6dNKHHnqI7OxsDAYDTzzxBBEREdSuXbvYdU+YMKHIFFRhOTUuDVQIcWcqM51UVB2rTgMVQtyZqkonFdWHBAAhBFC4PDJUfDqpqF5qZBaQEKLsJIXa+kgAEEKYSAq1dZEpICGEsFISAIQQwkpJACiH77//njfeeIN169YRHh5utk1ERAQTJ040Pe/SpQspKSk8+uijVTVMIYQwS84BlEOvXr3o1asXkFP3pDRuV0HctGlTpY1LCCFKQ44ACrh+/TohISH4+vri7e3NypUrGTRokOn1rVu3MnjwYCDnDmEGgwFvb2+mTJliarN48WJat25Nhw4d+OGHH0zL165dS8eOHfH396d3795cuHCh6jZMCCEKkABQwObNm3F1deXw4cPEx8fTt29fjh8/TnJyMpDz5T5mzBjOnTvHlClT2L59O7GxsURHR7NmzRrOnz/Pm2++yQ8//MCePXs4evSoad3dunXjp59+4tChQzzxxBO8//77ltpMIYSQAFCQwWBg69atTJkyhd27d1O/fn2eeuop/ve//5GSksKPP/5Iv379iI6OpkePHjRq1Ag7OztGjhzJrl272Ldvn2l5rVq1eOKJP29E8+uvvxIcHIzBYGDWrFkkJCRYcEuFENbO6s8BFCx/GxbchoMHD7Jp0yZee+01evbsybhx43jsscdwcHBg2LBhd3x7u0mTJvHiiy8yYMAAduzYwfTp0yt2Y4QQogys+gjAXPnbl5bs4PuTl3nyyScJCwvj4MGDuLq64urqyttvv83o0aMB6NChAzt37uTixYtkZWXx5Zdf0r17dzp27MjOnTu5dOkSGRkZfP3116b+UlNTcXPLuchmyZIllthkIYQwseojAHPlb6+eO8Xowa/zrks97O3t+eSTTwAYOXIkycnJtG3bFoAmTZoQHh7Oww8/jNaakJAQBg4cCOTcD7Vz5844Ozvj5+dnWvf06dMZNmwYDRo04JFHHuHMmTNVtKVCCFGYVZeDLkv524kTJ+Lv78/YsWMrbTxCCFFeUg66lEpb/rZdu3bExcXx5JNPVsWwhBCiSlh1ACht+dsDBw6wa9euEu96JIQQdxOrPgcg5W+FENbMqgMASPlbIYT1KtcUkFJqllLquFIqTikVqZRyLqJdX6XUCaXUz0qpqeXpUwghRMUo7zmArYC31toHOAlMK9hAKWULzAf6AZ7ACKWUZzn7FUIIUU7lCgBa6y1a68zcpz8BTc006wD8rLU+rbW+BawABpanXyGEEOVXkVlAY4BvzSx3A/4vz/Nfc5cJIYSwoBJPAiulvgfuN/PSq1rrtbltXgUygWXlHZBS6hngGYDmzZuXd3VCCCGKUGIA0Fr3Ku51pVQo0B/oqc1fVpwENMvzvGnusqL6WwAsgJwrgUsanxBCiDtTrlIQSqm+wBygu9Y6uYg2duScIO5Jzhd/NPA3rXWJtZCVUsnAL3c8wOqnIXDR0oOopuS9KZq8N0WT96awFlrrRqVpWN4A8DNQG7iUu+gnrfWzSilXYKHW+tHcdo8C8wBb4HOt9Tt33OldTCkVU9oaHdZG3puiyXtTNHlvyqdcF4JprR8qYvk54NE8zzcBchNcIYSoRqy6FpAQQlgzCQBVa4GlB1CNyXtTNHlviibvTTlU6/sBCCGEqDxyBCCEEFZKAkAlUkrdq5TaqpRKzP23QRHtspRSsbk/66p6nFWppMKASqnaSqmVua/vU0q1rPpRWkYp3ptQpVRynr+VcZYYZ1VTSn2ulPpdKRVfxOtKKfVB7vsWp5QKqOox3q0kAFSuqcA2rXUrYFvuc3PStNZ+uT8Dqm54VauUhQHHApdzM8zmAv+u2lFaRhmKJq7M87eysEoHaTkRQN9iXu8HtMr9eQb4pArGVCNIAKhcA4EluY+XAIMsOJbqoDSFAfO+Z6uAnkopVYVjtBQpmlgErfUu4I9imgwEluocPwHOSqkmVTO6u5sEgMrlorU+n/v4N8CliHYOSqkYpdRPSqmaHCRKUxjQ1Ca30mwqcF+VjM6ySls0cWjuNMcqpVQzM69bIyk4eYes/o5g5VVcsby8T7TWWilVVMpVC611klLqAWC7UuqI1vpURY9V3PXWA19qrW8qpf5BzpHSIxYek7iLSQAop+KK5SmlLiilmmitz+cekv5exDqScv89rZTaAfgDNTEAlKYw4O02v+bWkarPn6VGarIS3xutdd73YSHwfhWM625QpoKT4k8yBVS51gGjch+PAtYWbKCUaqCUqp37uCEQBBytshFWrWiglVLKXSlVCxhOznuUV9737HFgexFVZmuaEt+bAvPaA4BjVTi+6mwd8PfcbKBOQGqeqVdRDDkCqFzhwFdKqbHkVDX9K4BSKhB4Vms9DmgLfKqUyiYnIIdrrWtkANBaZyqlJgLf8WdhwASl1AwgRmu9DlgEfJFbaPAPcr4Ia7xSvjeTlVIDyLn3xh9AqMUGXIWUUl8CPYCGSqlfgTcBewCt9X/JqTP2KPAzcAMYbZmR3n3kSmAhhLBSMgUkhBBWSgKAEEJYKQkAQghhpSQACCGElZIAIIQQVkoCgBBCWCkJAEIIYaUkAAghhJX6f4UvNVSnjn+oAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f21722ee278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fi_first_2d = fi_wv.vectors[:,:2]\n",
    "plot_words(fi_wv, fi_words, fi_first_2d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... but this doesn't make much sense: 99% of the information is just discarded, and there isn't anything special about the first two dimensions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Principal component analysis\n",
    "\n",
    "* General statistical analysis method, standard tool for data analysis in many fields\n",
    "* Fast and (comparatively) easy to understand: identify axes of highest variance and drop the rest\n",
    "* Convex objective: the result is always the same"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xtc1FX++PHXAW8EKt6yQhI0r9wvgoiDFzJtLTLTrK+aqOmqWb+1tc2+7ZZd3CxdU/vWam2KleZ1ZSst20IL7wwLgng3SUPzGioCCnJ+fwCzgCADDMwwvJ+PBw9mPvOZz+d9Bnzz8XzOeR+ltUYIIYR9cbB2AEIIISxPkrsQQtghSe5CCGGHJLkLIYQdkuQuhBB2SJK7EELYIUnuQghhhyS5CyGEHZLkLoQQdqiRtU7ctm1b7eHhYa3TCyFEvZSYmHhBa92usv2sltw9PDwwGo3WOr0QQtRLSqmfzdlPumWEEMIOSXIXQgg7JMldCCHskCR3IYSwQ5LchaiiU6dOMWDAAHr27ImXlxeLFi2ydkhC3MJqo2WEqK8aNWrE3/72NwIDA7l69SpBQUEMGjSInj17Wjs0IUzkyl2IKrr77rsJDAwEoHnz5vTo0YOMjAwrRyVEaZLchaiB9PR0kpKSCA0NtXYoQpQiyV2IasrKyuKxxx5j4cKFtGjRwtrhCFGKJHchqiEvL4/HHnuM0aNHM3z4cGuHI8QtJLkLUUVaayZOnEiPHj14/vnnrR2OEOWS0TJClBGblMG8LYc5nZnDPa5OvDC4G8MC3Eyv79ixg08//RQfHx/8/f0B+Otf/8rvfvc7a4UsxC0kuQtRQmxSBi/9M5WcvJsAZGTm8NI/UwFMCb5v375ora0WoxDmkG4ZIUqYt+WwKbEXy8m7ybwth60UkRDVI8ldiBJOZ+ZUabsQtkqSuxAl3OPqVKXtomJ9+vQBCucCrFq1ysrRNDyS3IUo4YXB3XBq7Fhqm1NjR14Y3M1KEdVfO3fuBKqX3PPz82sjpAZFkrsQJQwLcOOt4T64uTqhADdXJ94a7lNqtIy9SU9Pp3v37kRHR9O1a1dGjx7Nd999R3h4OF26dGHv3r1cunSJYcOG4evrS+/evUlJSQFg9uzZTJgwgf79+9OpUycWL15sOq6LiwsAs2bNIj4+Hn9/f959911yc3MZP348Pj4+BAQEsHXrVgBiYmKIiopi4MCBREZG1v0HYW+01lb5CgoK0kII6ztx4oR2dHTUKSkp+ubNmzowMFCPHz9eFxQU6NjYWP3II4/o6dOn69mzZ2uttf7++++1n5+f1lrrV199VYeFhenc3Fx9/vx53bp1a33jxg2ttdbOzs5aa623bt2qhw4dajrf/Pnz9fjx47XWWh88eFC7u7vrnJwcvXz5cu3m5qYvXrxYl82vdwCjNiPHmnXlrpQaopQ6rJQ6ppSadZv9HlNKaaVUsMX++gghap2npyc+Pj44ODjg5eVFZGQkSil8fHxIT09n+/btjB07FoCBAwdy8eJFrly5AsDQoUNp2rQpbdu25c477+Ts2bO3Pdf27dsZM2YMAN27d6djx44cOXIEgEGDBtG6detabGnDUWlyV0o5Au8DDwI9gSeVUrfUNlVKNQf+H7DH0kEKIWpX06ZNTY8dHBxMzx0cHCrt/y75XkdHxxr1lzs7O1f7vaI0c67cQ4BjWuuftNY3gNXAI+Xs9wbwNpBrwfiEEDbAYDCwcuVKALZt20bbtm3NLpbWvHlzrl69Wu6xjhw5wsmTJ+nWTW5YW5o5M1TdgFMlnv8ClKpvqpQKBNy11puUUi9YMD4hRA1VVk7BHMU3Tn19fbnjjjtYsWKF2e/19fXF0dERPz8/oqOjmTZtGlOnTsXHx4dGjRoRExNT6upfWIbSlUyjVkqNAIZorZ8uej4WCNVaTy967gDEAdFa63Sl1DZgptbaWM6xJgOTAe69996gn3/+2ZJtEUKUUbacAhQO7bT3EUD2TCmVqLWu9L6mOd0yGYB7iecdirYVaw54A9uUUulAb+CL8m6qaq0/1FoHa62D27VrZ8aphRA1IeUUGi5zknsC0EUp5amUagI8AXxR/KLW+rLWuq3W2kNr7QHsBqLKu3IXQtQtKafQcFWa3LXW+cB0YAtwEFirtU5TSr2ulIqq7QCFENUn5RQaLrNK/mqtNwOby2x7pYJ9+9c8LCGEJbwwuFu5fe5STsH+ST13IexY8U3Tmo6WEfWPJHch7NywADdJ5g2QFA4TdSYmJobp06dbOwwhGgRJ7kIIYYckuYtqS09Px9vb2/R8/vz5zJ49m/79+/Piiy8SEhJC165diY+PN+1z+vRphgwZQpcuXfjTn/5k2j516lSCg4Px8vLi1VdfNW338PDgT3/6Ez4+PoSEhHDs2DEAoqOjmTJlCsHBwXTt2pWvvvoKgIiICJKTk03v79u3L/v27au1z0AIWyXJXdSK/Px89u7dy8KFC3nttddM25OTk1mzZg2pqamsWbOGU6cKK1vMmTMHo9FISkoKP/zwg6leOEDLli1JTU1l+vTp/OEPfzBtT09PZ+/evWzatIkpU6aQm5vLxIkTiYmJAQrrluTm5uLn51c3jRbChkhyF7Vi+PDhAAQFBZGenm7aHhkZScuWLWnWrBk9e/akuATF2rVrCQwMJCAggLS0NA4cOGB6z5NPPmn6vmvXLtP2xx9/HAcHB7p06UKnTp04dOgQI0eO5KuvviIvL49ly5YRHR1d+40VwgbJaBlRbY0aNaKgoMD0PDf3vwVBiwtBlS0BW1552BMnTjB//nwSEhJo1aoV0dHRpY6llKr0cfHzO+64g0GDBvGvf/2LtWvXkpiYaIGWClH/yJW7qFBsUgbhc+PwnLWJ8LlxxCZllHq9ffv2nDt3josXL3L9+nVTv3dVXblyBWdnZ1q2bMnZs2f5+uuvS72+Zs0a0/ewsDDT9nXr1lFQUMDx48f56aefTGVjn376aZ577jl69epFq1atqhWTEPWdXLmLcpWtJpiRmcNL/0wF/jsxpnHjxrzyyiuEhITg5uZG9+7dq3UuPz8/AgIC6N69O+7u7oSHh5d6/bfffsPX15emTZvy+eefm7bfe++9hISEcOXKFZYsWUKzZs2Awq6gFi1aMH78+GrFI4Q9qLTkb20JDg7WRqPUFrNV4XPjyCinuJSbqxM7Zg2sszg8PDwwGo20bdu21Pbo6GgeeughRowYcct7Tp8+Tf/+/Tl06BAODvKfU2FfLFnyVzRA9bWa4CeffEJoaChz5syRxC4aNLlyF+WylSt3IURpcuUuauSFwd1wauxYaptUExSi/pAbqqJcUk1QiPpNkruokFQTFKL+km4ZIYSwQ5LchRDCDklyF6IWlK2YKURdk+QuhBB2SJK7EMCECRO48847LXq1ffPmTSZNmoSXlxcPPPAAOTk5fPTRR/Tq1Qs/Pz8ee+wxsrOzgYrr08fExPDII4/Qv39/unTpYiqf/Morr7Bw4ULTuV5++WUWLVpksdhF/SfJXQgKk+s333xj0WMePXqUZ555hrS0NFxdXdmwYQPDhw8nISGBffv20aNHDz7++GPT/uXVpwfYu3cvGzZsICUlhXXr1mE0GpkwYQKffPIJAAUFBaxevZoxY8ZYNH5Rv0lyF4LCFZxat25t0WN6enri7+8P/Leu/f79+zEYDPj4+LBy5UrS0tJM+5dXnx5g0KBBtGnTBicnJ4YPH8727dvx8PCgTZs2JCUl8e233xIQEECbNm0sGr+o32ScuxC1pGzt+pycHKKjo4mNjcXPz4+YmBi2bdtm2qe8+vS32/70008TExPDr7/+yoQJE2qpFaK+kit3IerQ1atXufvuu8nLy2PlypWlXquoPv2///1vLl26RE5ODrGxsaaSyI8++ijffPMNCQkJDB48uM7bImybXLkLUQ2xSRnVKs3wxhtvEBoaSrt27QgNDeXq1aum1yqqTx8SEsJjjz3GL7/8wpgxYwgOLqwZ1aRJEwYMGICrqyuOjo7lnk80XJLchagicxYy8fDwYP/+/ab3zJw50/R46tSp5R73/vvvZ8mSJbds79ChA7GxsbdsLygoYPfu3axbt676jRF2S7plhN2rbLlAKFx8OywsjMOHD9OhQ4dSo1jKmrflsCmxF8vJu8m8LYctHntFDhw4wH333UdkZCRdunSps/OK+kPquQu7VvYqGwpLF7813KfaRdE8Z22ivH81Cjgxd2j1AhXCTFLPXQhq5yr7HlenKm0XwhokuQu7VhvLBcpCJqI+kOQu7FptXGUPC3DjreE+uLk6oShcerAm3TxC1AYZLSPs2guDu5Xb517Tq2xZyETYOrOu3JVSQ5RSh5VSx5RSs8p5fYpSKlUplayU2q6U6mn5UIWoOrnKFg1VpaNllFKOwBFgEPALkAA8qbU+UGKfFlrrK0WPo4BpWushtzuujJaxP1prtNY4OEhvnxC1xZKjZUKAY1rrn7TWN4DVwCMldyhO7EWcodyRYsIOpaen061bN5566im8vb1xdHREa01BQQHr168nOjoaKKy6+Nxzz9GnTx86derE+vXrrRu4EHbOnD53N+BUiee/AKFld1JKPQM8DzQBBlokOmFVs2bNwt3dnWeeeQaA2bNn4+LigtaatWvXcv36dQYMGMDRo0eZO3cue/bsoVGjRnh7e/P444+zd+9e2rVrB8CRI0c4f/48hw8f5tChQ0RFRTFixAhrNk8Iu2ax/z9rrd/XWncGXgT+XN4+SqnJSimjUsp4/vx5S51a1JJRo0axdu1a0/O1a9fSrl07jh49yt69e0lOTmb//v20b9+egIAAjh49SqNGjUhLS+OPf/wjiYmJFBQUAHDs2DEmTpyIg4MDPXv25OzZs9ZqlhANgjlX7hmAe4nnHYq2VWQ18PfyXtBafwh8CIV97mbGKKwkICCAc+fOcfr0ac6fP0+rVq1ITU011Q8HyMzMNPWxd+zYkQsXLgDg4uJCjx49OHXqFIcOHaKgoID77rvPdGxrzYwWoqEw58o9AeiilPJUSjUBngC+KLmDUqpkcYuhwFHLhSiK9e/fn+Kb0B4eHqZEWptGjhzJ+vXrWbNmDaNGjUJrzUsvvURycjLJycls27aNVq1aAeDs7Ez79u05ePCg6Yr9+PHjLF++vFRitzeyGLawRZUmd611PjAd2AIcBNZqrdOUUq8XjYwBmK6USlNKJVPY7z6u1iIWFmNOQa22vv3589+WMm/JCj799W6adw5i2bJlZGVlAfDrr7+Sn59v2n/u3Lk89NBD9OnTBy8vL65du8aqVavw9PSss3YJIcycxKS13gxsLrPtlRKP/5+F47Jr8+bNo2nTpjz33HPMmDGDffv2ERcXR1xcHB9//DHjxo3j1Vdf5fr163Tu3Jnly5fj4uJi0RjMKVsbm5TB3/fdIDf7Go7N23C+4A7WnHGkT8TvCAsLAwq7XzZt2mQ67ogRI0rdKO3QoQPJycl8/vnnpc5f/MfBGt544w0+++wz2rVrh7u7O0FBQdx///1MmTKF7OxsOnfuzLJly2jVqhXJycnlbk9MTDStfvTAAw9YrS1CVEQGJFuBwWAgPj4eAKPRSFZWFnl5ecTHx+Pr68ubb77Jd999x3/+8x+Cg4NZsGCBxWMwp6BW8T73THyfu558y7TPoTYGUlNTSU1NZdeuXXTu3PmW+uXFtm/fzqRJkywef3UlJCSwYcMG9u3bx9dff23q5nrqqad4++23SUlJwcfHh9dee+2228ePH897773Hvn37rNYWIW5HkrsVBAUFkZiYyJUrV2jatClhYWEYjUbi4+NxcnLiwIEDhIeH4+/vz4oVK/j5558tHoM5BbVqUnQrMzOTrl274uTkRGRkZPWCrAU7duzgkUceoVmzZjRv3pyHH36Ya9eukZmZSb9+/QAYN24cP/74I5cvXy53e2ZmJpmZmURERAAwduxYq7VHiIpIbRkraNy4MZ6ensTExNCnTx98fX3ZunUrx44dw9PTk0GDBt3SjWFp97g6kVFOki5ZUMucfSri6urKkSNHahakEKLa5MrdSgwGA/PnzyciIgKDwcCSJUsICAigd+/e7Nixg2PHjgFw7dq1WkmS5pSttcfStuHh4Xz55Zfk5uaSlZXFV199hbOzM61atTJ1lX366af069ePli1blrvd1dUVV1dXtm/fDnDLQtdC2AK5cq8F5iyebDAYmDNnDmFhYTg7O9OsWTMMBgPt2rUjJiaGJ598kuvXrwPw5ptv0rVrV4vGWBzP7eI0Zx9bU9ln36tXL6KiovD19aV9+/b4+PjQsmVLVqxYYbpx2qlTJ5YvXw5Q4fbly5czYcIElFJyQ1XYJFlmz8JqY1k3YR5zP/usrCxcXFzIzs4mIiKCDz/8kMDAQGuELESVyTJ7VmILiyc3VOZ+9pMnT8bf35/AwEAee+wxSezCLkm3jIXVxrJuwjzmfvarVq2qi3CEsKiFCxcyefJks/eXK3cLk8WTrUc+e2HPFi5cSHZ2ttn7S3K3MHscYVJfyGcv6pvPPvuMkJAQ/P39+f3vf8/NmzeZOnUqwcHBeHl58eqrrwKwePFiTp8+zYABAwDMGl0hyd3CZFk365HPXtQnBw8eZM2aNezYsYPk5GQcHR1ZuXIlc+bMwWg0kpKSwg8//EBKSgrPPfcc99xzD1u3boXClfEqJX3utUAWT7Ye+exFffH999+TmJhIr169AMjJyeHOO+9k7dq1fPjhh+Tn53PmzBkOHDiAr69vlY8vyV0IIaxAa824ceN46623TNtOnDjBoEGDSEhIoFWrVkRHR5Obm1ut40u3jBBCWEFkZCTr16/n3LlzAFy6dImTJ0/i7OxMy5YtOXv2LF9//bVp/+bNm3P16lWzjy9X7kIIUQsqmy3ds2dP3nzzTR544AEKCgpo3Lgx77//PgEBAXTv3h13d3fCw8NN+0+ePJkhQ4aAmTdUZYaqEEJYWG3OVJcZqkIIYSW2MFNdkrsQQliYLcxUl+QuhBAWZguzpSW5CyGEhdnCbGkZLSOEEBZmC2shSHK3MZmZmaxatYpp06ZZOxQhRA1Ye7a0dMvYmMzMTD744ANrhyGEqOckuduYWbNmcfz4cfz9/Rk/fjxffPEFAI8++igTJkwAYNmyZbz88ssALFiwAG9vb7y9vVm4cKHV4hZC2BZJ7jZm7ty5dO7cmeTkZAYPHmxanDkjI4MDBw4AEB8fT0REBImJiSxfvpw9e/awe/duPvroI5KSkqwZvhDCRkhyt2EGg4H4+HgOHDhAz549ad++PWfOnGHXrl306dOH7du38+ijj+Ls7IyLiwvDhw83/TEQQjRsckPVhrm5uZGZmck333xDREQEly5dYu3atbi4uNC8eXNrhyeEsGFy5W4FsUkZhM+Nw3PWJsLnxhGblGF6rWzlt969e7Nw4UIiIiIwGAzMnz8fg8EAFF7Zx8bGkp2dzbVr19i4caPpNSFEwyZX7nWsbEGhjMwcXvpnKlA4dKpNmzaEh4fj7e3Ngw8+iMFg4Ntvv+W+++6jY8eOXLp0yZTAAwMDiY6OJiQkBICnn36agIAA6zRMCGFTpCpkHQufG0dGOfUl3Fyd2DFroBUiEkLUJ1IV0kbZQkEhIYT9k+Rex2yhoJAQwv5Jcq9jtlBQSAhh/+SGah2zhYJCQgj7Z1ZyV0oNARYBjsA/tNZzy7z+PPA0kA+cByZorX+2cKx2w9oFhYQQ9q/SbhmllCPwPvAg0BN4UinVs8xuSUCw1toXWA+8Y+lAhbAV/fv3p3ikl4uLi5WjEaJ85vS5hwDHtNY/aa1vAKuBR0ruoLXeqrXOLnq6G+hg2TCFEEJUhTnJ3Q04VeL5L0XbKjIR+LomQQlRF+bNm8fixYsBmDFjBgMHFs4ziIuLY/To0UydOpXg4GC8vLx49dVXb3usCxcuEBYWxqZNm2o9biHMYdHRMkqpMUAwMK+C1ycrpYxKKeP58+cteWohqqy4MBuA0WgkKyuLvLw8U9XNOXPmYDQaSUlJ4YcffiAlJaXc45w9e5ahQ4fy+uuvM3To0LpsghAVMie5ZwDuJZ53KNpWilLqfuBlIEprfb28A2mtP9RaB2utg9u1a1edeIWwmKCgIBITE7ly5QpNmzYlLCwMo9FIfHw8BoOBtWvXEhgYSEBAAGlpaaaSyyXl5eURGRnJO++8w6BBg6zQCiHKZ05yTwC6KKU8lVJNgCeAL0ruoJQKAJZSmNjPWT5MISyvcePGeHp6EhMTQ58+fTAYDGzdupVjx47h5OTE/Pnz+f7770lJSWHo0KHk5ubecoxGjRoRFBTEli1brNACISpWaXLXWucD04EtwEFgrdY6TSn1ulIqqmi3eYALsE4playU+qKCwwlhU4orbRZX3VyyZAkBAQFcuXIFZ2dnWrZsydmzZ/n66/JvIymlWLZsGYcOHeLtt9+u4+iFqJhZ49y11puBzWW2vVLi8f0WjkuIGotNyqh0spjBYGDOnDmEhYXh7OxMs2bNMBgM+Pn5ERAQQPfu3XF3dyc8PLzC8zg6OvL5558TFRVF8+bNZXFzYROkKqSwS2VLK0NhmYe3hvvIBDJRr0lVSNGgzdtyuFRiB8jJu8m8LYetFJEQdUuSu7BLtV1aedu2bTz00ENm7Zueno63t3e1zlOT94qGTZK7sEtSWlk0dJLchV2qamnlAQMG0KNHD9Pz+fPnM3v2bBISEvD19cXf3x93d3c6d+5c6n0FBQXce++9fP7556bn9913H2Un6eXn5zN69Gh69OjBiBEjyM7O5vXXX6dXr154e3szefJkiu9/JSYm4ufnh5+fH++//36NPwvRMElyF3ZpWIAbbw33wc3VCUXhMoYV3UzVWvPxxx/j6Oh4y2vjx49n6dKlJCcno5S65XUHBwcCAwP54IMPAPjuu+/w8/Oj7CS9w4cPM23aNA4ePEiLFi344IMPmD59OgkJCezfv5+cnBy++uor0znfe+899u3bZ4FPQjRUUs9d2K3blVZOT09n8ODBhIaGkpiYyIEDB+jWrfCq/o033uC9996jUaNGXL58mR07dhAWFkb79u05ceIEISEhnDlzBnd3d27cuEFCQgLnzp3D39+fO+64g//93/+95Xwlh1OOGTOGxYsX4+npyTvvvEN2djaXLl3Cy8sLg8FAZmYmERERAIwdO7bCMfZC3I5cuYsG6+jRo0ybNo20tDQ6dOhAQUEBCQkJbNiwgalTpzJs2DBu3Lhxy/v27t3LM888w5EjR2jSpAlz5szhnnvuYcGCBfz66688+OCDt7yn7FW/Uopp06axfv16UlNTmTRpUrkzYIWoLknuwu7FJmUQPjcOz1mbCJ8bR2xSYWmkjh070rt3b6BwItKFCxf49ttvGTp0KFu2bOHOO++kZcuWnDx5EoBz587RokULALp27UpOzn9H3vTs2ZMxY8YwcuTIcrt3Tp48ya5duwBYtWoVffv2BaBt27ZkZWWxfv16AFxdXXF1dWX79u0ArFy5sjY+EtEASLeMsGtlJzNlZObw0j9TmdHbFWdn51L7zpw5kwULFuDs7MyAAQMAGDRoEOvWrePHH3/k5s2buLq6AoV/DIpvgP7n599I+q0x5y5m8m1eD8KSMm7pDurWrRvvv/8+EyZMoGfPnkydOpXffvsNb29v7rrrLnr16mXad/ny5UyYMAGlFA888ECtfTbCvskMVWGXiksPZFQwrr2Nvsz1TX9l//79AHh4eGA0Gjlx4gS///3v2blzJ/n5+fj7+zNlyhRmzpxJp06dCAkJYfXq1Vy4cIHg4GAWbtzBM3P+zqWkLegb2dw1+h2ZCStqlcxQLceCBQvw9vbG29ubhQsXkp6eTo8ePZg0aRJeXl488MADpv9qHz9+nCFDhhAUFITBYODQoUNWjl6Yq/hqvaLEDnD2Svn927169SIqKgpfX18efPBB2rRpw6JFi/D29uby5ctMnDix1P7zthwm68xxctOTuXntN64d/FFmwgqb0GCu3BMTE4mOjmb37t1orQkNDeWzzz6jV69eGI1G/P39efzxx4mKimLMmDFERkayZMkSunTpwp49e3jppZeIi4urs3hF9YXPjbttYofCoZE7Zg0s97WsrCxcXFzIzs4mIiKCDz/8kMDAwHL39Zy1ifL+BSngxFxZuENYnrlX7g2mz3379u08+uijpn7W4cOHEx8fj6enJ/7+/kDh4g3p6elkZWWxc+dORo4caXr/9evlrj8ibFBlJQZuN5kJYPLkyRw4cIDc3FzGjRtXYWKHwhmv5f0hkZmwwtoaTHKvSNOmTU2PHR0dycnJoaCgAFdXV5KTk60YmaiuihIuFF6xl1f6t6RVq1aZfa4XBncrt/rk7f54CFEXGkyfu8FgIDY2luzsbK5du8bGjRsxGAzl7tuiRQs8PT1Zt24dUDiDUWYL1h8VlR5YOMqfHbMGWvRGZ1VmwgpRl+zmyr2yhRkCAwOJjo4mJCQEgKeffppWrVpVeLyVK1cydepU3nzzTfLy8njiiSfw8/Or9XaImiv+uVe2UIclzyfJXNgau7ihKgszCCEaigY1FFIWZhBCiNLsIrnX9sIMQghR39hFcpeFGWyHi4tLlfbftm0bO3furKVohGi47CK5V3VhBmE7JLkLUTvsIrnLcLS6M2/ePBYvXgzAjBkzGDiwcJZnXFwco0ePBuDll1/Gz8+P3r17c/bsWQC+/PJLQkNDCQgI4P777+fs2bOkp6ezZMkS3n33Xfz9/YmPj7dOo4SwQ3aR3KEwwe+YNZATc4dafCyz+C+DwWBKwkajkaysLPLy8oiPjyciIoJr167Ru3dv9u3bR0REBB999BEAffv2Zffu3SQlJfHEE0/wzjvv4OHhwZQpU5gxYwbJyckVzjsQQlSd3YxzF3UjKCiIxMRErly5QtOmTQkMDMRoNBIfH8/ixYtp0qQJDz30kGnff//73wD88ssvjBo1ijNnznDjxg08PT2t2Qwh7J7dXLmLutG4cWM8PT2JiYmhT58+GAwGtm7dyrFjx+jRoweNGzc2rTrk6OhIfn4+AM8++yzTp08nNTWVpUuXyqpDQtQyuXIXpVQ20xcKu2bmz5/PsmXL8PHx4fnnnycoKKjcBaSLXb61d8RZAAAQvklEQVR8GTe3wuOsWLHCtL158+ZcuXKldhojRAMmV+7CpGQddM1/Vy0qXpaumMFg4MyZM6ZFo5s1a1Zpf/ns2bMZOXIkQUFBtG3b1rT94YcfZuPGjQ3ihqrWmoKCAmuHIRoIuyg/ICyjojrot6t9LkpbsGABy5YtAwrrFw0bNozBgwcTGhpKYmIimzdvpmPHjlaOUtRnUs9dVJnM9K2ZxMREli9fzp49e0wLwvTr14+jR4+yYsUK02LcQtQF6ZYRJjLTt2ZKLgjj4uJiWhCmY8eOkthFnZPkLkxkpm/tKF79S4i6JMldmMhM35qpyoIwQtQ26XMXpcjCE+UzZ4hoVReEEaI2mTVaRik1BFgEOAL/0FrPLfN6BLAQ8AWe0Fqvr+yYMlpG1BeyGIywJRZbrEMp5Qi8DzwI9ASeVEr1LLPbSSAaMH9lYSHqCVkMRtRH5nTLhADHtNY/ASilVgOPAAeKd9Bapxe9JjM0hN2RIaKiPjLnhqobcKrE81+KtlWZUmqyUsqolDKeP3++OoeosszMTD744AOgsHZ4cVErc8XExHD69OnaCE3UEzJEVNRHdTpaRmv9odY6WGsd3K5duzo5Z8nkXh2S3IUMERX1kTndMhmAe4nnHYq21QuzZs3i+PHj+Pv707hxY5ydnRkxYgT79+8nKCiIzz77DKUUr7/+Ol9++SU5OTn06dOHpUuXsmHDBoxGI6NHj8bJyYldu3bh5CRXaw1N8U3TykbLCGFTtNa3/aLwD8BPgCfQBNgHeFWwbwwworJjaq0JCgrSdeHEiRPay8tLa6311q1bdYsWLfSpU6f0zZs3de/evXV8fLzWWuuLFy+a3jNmzBj9xRdfaK217tevn05ISKiTWIUQojKAUZuRYyvtltFa5wPTgS3AQWCt1jpNKfW6UioKQCnVSyn1CzASWKqUSrP0HyFLCQkJoUOHDjg4OODv7096ejoAW7duJTQ0FB8fH+Li4khLs9kmCCFEpcyaxKS13gxsLrPtlRKPEyjsrrF5TZs2NT0uXkwiNzeXadOmYTQacXd3Z/bs2bKYhBCiXqv35QdikzIInxuH56xNhM+Nu6X2ePPmzbl69eptj1GcyNu2bUtWVhbr1/93DpY57xdCCFtTr8sPlJ05WLy4BPz3JlibNm0IDw/H29sbJycn2rdvf8txXF1dmTRpEt7e3tx111306tXL9Fp0dDRTpkyRG6pCiHqlXi/WIYtLCCEaGouVH7BlMnNQCCHKV6+Tu8wcFEKI8tXr5C4zB4UQonz1+oaqzBwUQojy1evkDrK4hBBClKded8sIIYQonyR30aBs27aNnTt3WjsMIWqdJHfRoEhyFw2FJHdhFz755BN8fX3x8/Nj7NixfPnll4SGhhIQEMD999/P2bNnSU9PZ8mSJbz77rv4+/sTHx9v7bCFqDX1/oaqEGlpabz55pvs3LmTtm3bcunSJZRS7N69G6UU//jHP3jnnXf429/+xpQpU3BxcWHmzJnWDluIWiXJXdR7cXFxjBw5krZt2wLQunVrUlNTGTVqFGfOnOHGjRt4enpaOUoh6pZ0ywi79OyzzzJ9+nRSU1NZunSplHAWDY4kd1HvDRw4kHXr1nHx4kUALl26xOXLl3FzK5z/sGLFCtO+UsJZNBSS3IXNq6xmv5eXFy+//DL9+vXDz8+P559/ntmzZzNy5EiCgoJM3TUADz/8MBs3bpQbqsLu1euSv8L+la3ZD4X1g94a7iMzk0WD1CBK/gr7N2/L4VKJHSAn7ybzthy2UkR1a/bs2cyfP/+W7UuWLOGTTz657XtjYmKYPn16bYUmbJyMlhE2TWr23yo/P58pU6ZYOwxh4+TKXdi0hlizf86cOXTt2pW+ffty+HDh/1D69+/PH/7wB4KDg1m0aFGpK/r+/fvz4osvEhISQteuXcu9l7Bp0ybCwsK4cOFCnbZFWI8kd2HTGlrN/sTERFavXk1ycjKbN28mISHB9NqNGzcwGo388Y9/vOV9+fn57N27l4ULF/Laa6+Vem3jxo3MnTuXzZs3l7q5LOybdMsIm9bQavbHx8fz6KOPcscddwAQFRVlem3UqFEVvm/48OEABAUFkZ6ebtoeFxeH0Wjk22+/pUWLFrUTtLBJktyFzZOa/YWcnZ0rfK1p06YAODo6kp+fb9reuXNnfvrpJ44cOUJwcKUDLIQdkW4ZIepQZWP2IyIiiI2NJScnh6tXr/Lll1/W6HwdO3Zkw4YNPPXUU6SlpdXoWKJ+keQuRB2JTcpgxgexHEv8EQ1kZObw0j9TSyX4wMBARo0ahZ+fHw8++CC9evWq8Xm7d+/OypUrGTlyJMePH6/x8UT9IJOYhKgj4XPjOBz/JTd+PUrrQVNN291cndgxa2CF78vPz6dRI+lBFYVkEpMQ5Shb9z09PZ2BAwfi6+tLZGQkJ0+eBCA6OpqpU6fSu3dvOnXqxLZt25gwYQI9evQgOjradDwXFxdmzJiBl5cXkZGRnD9/Higcnlh88XLhwgU8PDzIuHiFzPiVXDsYz+nlz3Lt4I8U3MglZdVbhISEEBAQwL/+9S+gcAJSVFQUAwcOJDIysm4/JGEXJLmLBqO47ntcXBz79u1j0aJFPPvss4wbN46UlBRGjx7Nc889Z9r/t99+Y9euXbz77rtERUUxY8YM0tLSSE1NJTk5GYBr164RHBxMWloa/fr1u2UYYklubVrgahiNcw8D94x/D+ceEVzetYa7ewSzd+9etm7dygsvvMC1a9cA+M9//sP69ev54YcfaveDEXZJkrtoMMqr+75r1y7+53/+B4CxY8eyfft20/4PP/wwSil8fHxo3749Pj4+ODg44OXlZRpu6ODgYBqiOGbMmFLvL+uFwd1o7Fj6n9yNn5PJTSwsZNa/f39yc3NN/3sYNGgQrVu3tlj7RcMiHXlCVKB4eKGDg4PpcfHzksMNS1JKAdCoUSMKCgoATLXkhwW4ERfUgXXnj6MonGXr2LIZm2PX0a1b6UlZe/bsue3QRyEqI1fuosEor+57nz59WL16NQArV67EYDBU6ZgFBQWsX78egFWrVtG3b18APDw8SExMBDC9DtDP+14e6NqSE3OHsmPWQEY9+hDvvfcexQMbkpKSatZIIYpIchd2ozp139977z2WL1+Or68vn376KYsWLarSOZ2dndm7dy/e3t7ExcXxyiuvADBz5kz+/ve/ExAQUKqey4ABAzhw4AD+/v6sWbOGv/zlL+Tl5eHr64uXlxd/+ctfav5BCIGZQyGVUkOARYAj8A+t9dwyrzcFPgGCgIvAKK11+u2OKUMhhSVZq+67i4sLWVlZtXZ8Icqy2FBIpZQj8D7wINATeFIp1bPMbhOB37TW9wHvAm9XPWQhqm/elsOc2foJl/f807StIdV9F6Isc7plQoBjWuuftNY3gNXAI2X2eQQoXqhyPRCpiu8sCVEHrFX3Xa7aha0yZ7SMG3CqxPNfgNCK9tFa5yulLgNtACkeLWrVnDlzWLFiBRdvNOXmHa1p0v4+bpz9iYtb3kfnX6d5Ozd+e7EPrVq1IiEhgYkTJ+Lg4MCgQYP4+uuv2b9/v7WbIEStqNMbqkqpyUopo1LKWDyTT4jqKln7fMmn68j79SgAFzYtoFX/aDr//u9EhgebJhaNHz+epUuXkpycjKOj4+0OLUS9Z05yzwDcSzzvULSt3H2UUo2AlhTeWC1Fa/2h1jpYax3crl276kUsRJGStc//x9CdIb97iOaONynIvUZn3xDeGu7DWy8+y48//khmZiZXr14lLCwMwDRxSQh7ZU5yTwC6KKU8lVJNgCeAL8rs8wUwrujxCCBOW6simWiw7rvThYkGT+5xbcaOWQOlBrxo0CpN7lrrfGA6sAU4CKzVWqcppV5XShUvE/Mx0EYpdQx4HphVWwELUay82ufOzs60atXKtI7op59+Sr9+/XB1daV58+bs2bMHwDRxSQh7ZVb5Aa31ZmBzmW2vlHicC4y0bGiioYtNyrjt8nola5/feeedptrnK1asYMqUKWRnZ9OpUyeWL18OwMcff8ykSZNwcHCgX79+tGzZ0irtEqIuSD13YZNqY1JSVlYWLi4uAMydO5czZ85UeUaqENYm9dxFvTZvy+FSiR1qPilp06ZN+Pv74+3tTXx8PH/+859rGqYQNkuqQgqbVBuTkkaNGmUqzyuEvZMrd2GT7nF1qtJ2IURpktyFTXphcDecGpeeaOTU2JEXBner4B1CiJKkW0bYpOKbprcbLSOEqJgkd2GzhgW4STIXopqkW0YIIeyQJHchhLBDktyFEMIOSXIXQgg7JMldCCHskCR3IYSwQ5LchRDCDklyF0IIOyTJXQgh7JAkdyGEsEOS3IUQwg5ZbSUmpdR54GernLz62gIXrB1ELZG21U/23Daw7/ZVt20dtdbtKtvJasm9PlJKGc1Z3qo+krbVT/bcNrDv9tV226RbRggh7JAkdyGEsEOS3KvmQ2sHUIukbfWTPbcN7Lt9tdo26XMXQgg7JFfuQghhhyS534ZSqrVS6t9KqaNF31tVsN+9SqlvlVIHlVIHlFIedRtp1VWhbTeVUslFX1/UdZzVYW7bivZtoZT6RSn1f3UZY3WZ0zalVEel1H+KfmZpSqkp1oi1Osxsn79SaldR21KUUqOsEWtVVeHf3DdKqUyl1Fc1OZ8k99ubBXyvte4CfF/0vDyfAPO01j2AEOBcHcVXE+a2LUdr7V/0FVV34dWIuW0DeAP4sU6isgxz2nYGCNNa+wOhwCyl1D11GGNNmNO+bOAprbUXMARYqJRyrcMYq8vc38t5wNgan01rLV8VfAGHgbuLHt8NHC5nn57AdmvHWhttK3oty9qx1mLbgoDVQDTwf9aO25JtK7F/G+AkcI+1Y6+N9hXttw/oYu3YLdk2oD/wVU3OJ1fut9dea32m6PGvQPty9ukKZCql/qmUSlJKzVNKOdZdiNVmTtsAmimljEqp3UqpYXUUW01V2jallAPwN2BmXQZmAWb93JRS7kqpFOAU8LbW+nRdBVhD5v5eAqCUCgGaAMdrOzALqFLbaqpRbR68PlBKfQfcVc5LL5d8orXWSqnyhhY1AgxAAIVXSGsovBL82LKRVp0F2gaFU50zlFKdgDilVKrW2ur/kCzQtmnAZq31L0qp2gix2izxc9NanwJ8i7pjYpVS67XWZy0fbdVZ6PcSpdTdwKfAOK11gWWjrB5Ltc0SGnxy11rfX9FrSqmzSqm7tdZnin6RyutL/wVI1lr/VPSeWKA3NpDcLdA2tNYZRd9/Ukpto/CPmNWTuwXaFgYYlFLTABegiVIqS2t9u/75OmGJn1uJY51WSu2n8AJkvYVDrRZLtE8p1QLYBLystd5dS6FWmSV/djUl3TK39wUwrujxOOBf5eyTALgqpYoL+QwEDtRBbDVVaduUUq2UUk2LHrcFwrGTtmmtR2ut79Vae1DYNfOJLSR2M5jzc+uglHIqetwK6Ethf299YE77mgAbKfyZ2cQfLDOZk08sx9o3GWz5i8KbUd8DR4HvgNZF24OBf5TYbxCQAqQCMUATa8duibYBfYratK/o+0Rrx23Jn1uJ/aOpPzdUzfm5Ff8+7iv6PtnacVu4fWOAPCC5xJe/tWO3RNuKnscD54EcCnsGBlfnfDJDVQgh7JB0ywghhB2S5C6EEHZIkrsQQtghSe5CCGGHJLkLIYQdkuQuhBB2SJK7EELYIUnuQghhh/4//ny6Nf+xZqsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f21722707b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import sklearn.decomposition\n",
    "\n",
    "\n",
    "pca2 = sklearn.decomposition.PCA(2)\n",
    "\n",
    "en_vectors_pca2 = pca2.fit_transform(en_wv.vectors)\n",
    "plot_words(en_wv, en_words, en_vectors_pca2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XlYVeXa+PHvA6KYE5VD4JDUSRzYwEZQk5dETbHkaE5NNqCpqZm9bycSs1JPpw6lJ/055CkzUQ+WHU00HHJEJSsBQUREydwNaIQlOIECPr8/gB2TAgJ7M9yf6+KKvfbaa91Lbd9rPcP9KK01QgghRFE21g5ACCFE7SPJQQghRCmSHIQQQpQiyUEIIUQpkhyEEEKUIslBCCFEKZIchBBClCLJQQghRCmSHIQQQpTSyBonbd26te7cubM1Ti2EEHVWbGzsOa11G0ucyyrJoXPnzsTExFjj1EIIUWcppX601LmkWUkIIUQpkhyEEHXG+PHjadu2La6urtYOpd6T5CCEqDMCAwPZvn27tcNoECQ5CCGsKiMjgw8++KDc/fr27csDDzzAHXfcYYGohCQHIYRVVTQ5HDx40ALRiEJWGa0khBCFgoODOXXqFB4eHvTv35+EhATOnz9PTk4O//jHPxg+fDgAzZs359KlS1aOtuGQ5CCEsKqQkBASExOJj48nNzeXK1eu0LJlS86dO0efPn243rEn83ec5Mq1PHxC9vCs4TZrh9wgSHIQQtQaWmtee+019u/fj42NDT//8guvrtlPTpNWAKRmZPHu9p/Jyc61cqT1nyQHIYTFhcelMu+rE5zJyOIOncmFgi/7sLAw0tPTiY2Nxc7ODvvb7yIrO5tGBckhffN7XP3pKNezLtChQwfmzp3Lc889Z81LqbckOQghLCo8LpWZXxwlKycPgN+yFb+mnyc8LpXMzEzatm2LnZ0de/fu5WpGWrHPthn2KgAKOB0y1NKhNyiSHIQQFjXvqxPmxABg27Qljdt348mH/ofHHvIjOTkZg8GAl5cXTdt0KvMYTg5NLRVugyXJQQhhUWcyskptazMsCAWsLPE0MLzIU0anl9cD0NTOliB/F0uE2qDJPAchhEXd6K7/909fLbXtEWN7/jnSQHuHpiigvUNT/jnSwCPG9jUcpZAnByGERQX5uxTrc4D8p4HV4TvK3P8RY3tJBlYgTw5CCIu60dPAU74uXLp0iYEDB+Lp6YnBYGDTpk3WDrfBUlpri5/Uy8tLy3oOQoiimjdvTkZGRqlJcCkpKSilavz8P//8M8888wxpaWkopZg0aRIvvfRSjZ+3MpRSsVprL0ucS5qVhBC1RslJcKmpqaSlpXHXXXfV+LkbNWrEv/71Lzw9Pbl48SI9e/Zk0KBBdO/evcbPXRtJchBCWEXRiXBODk3Ju65LTYLr3Lkz2dnZFonH0dERR0dHAFq0aEG3bt1ITU2V5CCEEJZSciJcakYWV3Ovc+DYj8Umwf34o8VWxSzGZDIRFxdH7969rXL+2kCSgxDC4kpOhANAKY7au2Eb9Z55ElzXrl0tHtulS5cYNWoUCxcupGXLlhY/f20hyUEIYXElJ8LlZV3Axr456TmNOf3NNxaLo2TT1v8NuIflbzzP2LFjGTlypMXiqI1kKKsQwuKKToTLvfg7v655hZa9Rlq0LEZh01ZqRhYa+OX8FZ6bMIHGd3bk5ZdftlgctZUkByGExQX5u9DUzhaARi3upP2kj2jX5xGLlsUo2bR1NTWJC0d3s2fvHjw8PPDw8GDr1q0Wi6e2kWYlIYTFFc54LtqkE+TvYtGZ0CWbtuw79ODuGREoIF4qvkpyEEJYh7XLYjg5NCW1jCKAUvE1nzQrCSEapKJNW4Wk4uufqiU5KKWGKKVOKKW+V0oFV8cxhRCiJknF15urcrOSUsoWWAoMAn4BopVSm7XWSVU9thBC1CRrN23VZtXx5NAL+F5r/YPW+hrwGTC8Go4rhBDCSqojObQHfi7y+peCbcUopSYppWKUUjHp6enVcFohhBA1xWId0lrrj7TWXlprrzZt2ljqtEKIOiwyMpKAgIAK7XvmzBlGjx5dwxE1HNWRHFKBjkVedyjYJoQQFuPk5MT69eutHUa9UR3JIRq4TynlrJRqDDwObK6G4wohGpA333yThQsXml/PmjWLI0eOmF9HR0djNBo5deoU+/btM89iNhqNXLx4EZPJhKurqzVCr5eqnBy01rnANOAr4Djwudb6WFWPK4RoWMaPH8/q1asBuH79Op999hkdOnQA4ODBg0yePJlNmzZx7733Mn/+fJYuXUp8fDwHDhygaVOZuFbdqmWGtNZ6K9Bwi5AIIaqsc+fO3HnnncTFxZGWlobRaOTOO+/k+PHjTJo0iR07duDk5ASAj48PL7/8srl6amESEdVHymcIIaymZMns+weOJDQ0lF9//ZXx48cD+Su0ZWdnExcXZ04OwcHBDB06lK1bt+Lj48NXX32Fvb29NS+l3pHkIISwirJWg4uw6cDFTfNpYqNZu3YtBw4cwMHBgRUrVjBo0CCaNWuGn58fp06dwmAwYDAYiI6OJjk5GQ8PDytfUf0itZWEEFZR1mpw2ddtyGvXnUcffRRb2z/rHrVr146IiAheeOEFvvvuOxYuXIirqytubm7Y2dnx0EMPWTr8ek9prS1+Ui8vLx0TE2Px8wohag/n4C2U/PbR+jpnQ1/iWNRX3HfffVaJqzZTSsVqrb0scS55chDiJgonViUlJTFhwoRy9w8NDeXMmTPm1wsXLuTKlSvm17t27eLNN99k8+bNhISE1EjMdUXJ0tjXzv3EmQ8n0rpLzxpLDJUZ7trQJ+DJk4MQ1cjPz4/58+fj5ZV/c9e5c2diYmJo3bq1lSOrfUr2OUB+yeyarIxqMpkICAggMTGx3H0jIyOZP38+ERERNRLLrZAnByEszGQy0a1bNyZOnEiPHj0YPHgwWVlZ/Pvf/8bb2xt3d3dGjRrFlStXuHjxIs7OzuTk5ABw4cIFnJ2d+e9//0tMTAxjx47Fw8OD//f//h9nzpyhf//+9O/fH4BPP/0Ug8GAq6srM2bMsOYlW521S2b/8MMPGI1GoqOj8fX1xdPTE09PTw4ePFhq3wY5AU9rbfGfnj17aiFqk9OnT2tbW1sdFxentdZ6zJgxes2aNfrcuXPmfWbNmqUXLVqktdY6MDBQb9y4UWut9YcffqhffvllrbXW/fr109HR0ebP3H333To9PV1rrXVqaqru2LGj/u2333ROTo7u37+/+RjCMk6fPq179Oihk5OTtYeHh46Pj9eXL1/WWVlZWmutT548qQu/n/bu3auHDh2qv/76a+3p6al//PFHrbXWAQEBOioqSmut9cWLF3VOTo75uDUNiNEW+p6WJwdRTEZGBh988AFQsXZUk8nE2rVryz1uaGgo06ZNq5YYa4qzs7N5OGTPnj0xmUwkJibi6+uLwWAgLCyMY8fyJ/9PmDCBlStXArBy5UrGjRtX7vGjo6Px8/OjTZs2NGrUiLFjx7J///6auyBRpvT0dIYPH05YWBju7u7k5OQwceJEDAYDY8aMISnpz6VoCifgffnll3Tq1An4cwLeokWLyMjIoFGj+jkjQJKDKKZocqhIIbOKJofaKDwuFZ+QPTgHb2HUsoNc1X8OnbS1tSU3N5fAwECWLFnC0aNHmT17NtnZ2UD+F4TJZCIyMpK8vLz616RQTxT9O/YJ2cOOY7/SqlUrOnXqRFRUFAALFiygXbt2HDlyhJiYGK5du2b+vKOjI/b29sTFxZm3BQcH8/HHH5OVlYWPjw/JyckWvy5LkOQgigkODubUqVN4eHgwZswY85deXl4eQUFBeHt74+bmxocffmje/8CBA3h4eLBgwQKys7MZN24cBoMBo9HI3r17S51jy5Yt3H///Zw7dw6TycSAAQNwc3Nj4MCB/PTTTwAEBgYyffp0+vbtyz333FMsSc2bN88cx+zZs2/pOgs7Q1MzstBA2oVs0i5kEx5XvKDwxYsXcXR0JCcnh7CwsGLvPfPMMzz55JPFnhpatGjBxYsXy3zdq1cv9u3bx7lz58jLy+PTTz+lX79+txS/KF/Jv+PUjCze3X6C7Os2bNy4kdWrV7N27VoyMzNxdHTExsaGNWvWkJf3Zwe5g4MDW7ZsYebMmURGRgKYJ+DNmDEDb29vSQ6iYQgJCeHee+8lPj6eefPmmbevWLGCVq1aER0dTXR0NMuXL+f06dOEhITg6+tLfHw8//d//8fSpUtRSnH06FE+/fRTnn32WfPdNsDGjRsJCQlh69attG7dmhdffJFnn32WhIQExo4dy/Tp0837nj17lqioKCIiIggOzl+afMeOHaSkpHDo0CHi4+OJjY29paaZsiZgaa2Z99WJYtveeustevfujY+PD127di323tixYzl//jxPPPGEeVtgYCCTJ0/Gw8ODrKwsJk2axJAhQ+jfvz+Ojo6EhITQv39/3N3d6dmzJ8OHy6KJNaWsv+OruXmcu3SVZs2aERERwYIFC+jcuTOrVq3C3d2d5ORkmjVrVuwzDXUCngxlFcUUHepX9PfRo0eTkJDAbbfdBkBmZiYffvghjRs3Ljbcb8SIEbz44osMGDAAAF9fX5YuXcrhw4d57733aNmyJTt27KBly5YAtG7dmrNnz2JnZ0dOTg6Ojo6cO3eOwMBABg0axNixY4E/78BfeeUV1q9fj4ODAwCXLl1i5syZPPfcc5W6zrImYAEo4HTI0AodY/369WzatIk1a9ZU6tzCMqrj77i2seRQ1vrZkyIqpWjxszt0Jheyc0vto7Vm8eLF+Pv7F9te+KhdEffeey8//PADJ0+eNM8DuJkmTZoUO3/hf2fOnMnzzz9f4fOWxcmhKakZWWVur4gXX3yRbdu2sXWrFCOurar6d9zQSbNSA1eyXfa3bMXZ9POl2t79/f1ZtmyZeWz/yZMnuXz5cqk2dl9fX3Pb/MmTJ/npp59wcXEB4O6772bDhg0888wz5lE/ffv25bPPPgMgLCwMX1/fm8br7+/PJ598wqVLlwBITU3lt99+q/R1B/m70NTOtti2pna2BPm7VOjzixcv5vvvv6dLly6VPrewjKr+HTd08uTQwJVsl7Vt2pLG7bvx5EP/w1DfP+/uJ0yYgMlkwtPTE601bdq0ITw8HDc3N2xtbXF3dycwMJCpU6cyZcoUDAYDjRo1IjQ0tNgTQNeuXQkLC2PMmDF8+eWXLF68mHHjxjFv3jzatGljHh56I4MHD+b48ePcf//9ADRv3pz//Oc/tG3btlLXXTjRqmi56CB/F4tNwBI1T/6Oq0b6HBq4+tguK0R9JeUzhMXcqP21IbbLVlcJhIcffpiMjIxqiEgI65Hk0EC98847QH67bOO8LC4e3mJ+r6mdLe1/2MyuXbuYPn068pRXOVu3bjWPphKirpLk0EAVJodHjO15xa8DWQnbihU/+/yjhTz44IMsWrSoQiOL6ou8vLxixfeOHTuGp6en+f2UlBQ8PT3Zvn07Y8aMMW8vWt65c+fO5gl+ZRXzE6IukOTQAPznP/+hV69eeHh48PzzzxMUFERWVhYeHh6MHTuWPWsWojN/pdX2WfT9fRsD7m3BgAED8PT0xM3NjU2bNln7EiwmJSWFF154gWPHjuHg4EBcXBytWrUiPj4e+LOO0oMPPsh3333H5cuXAVi3bh2PP/54ucfbsGGDRa9HiFslo5XquePHj7Nu3Tq+/vpr7OzsmDp1KgaDgaZNm5q/8AoLzBW+zsnJITw8nJYtW3Lu3Dn69OnDsGHDUEpZ81Isoqzie4VF9t5//33WrVvHoUOHaNSoEUOGDOHLL79k9OjRbNmyhffee69CxxOiLpDkUM/t3r2b2NhYvL29AcjKyqrQsM/XXnuN/fv3Y2NjQ2pqKmlpadx11101Ha7FlZwAWLL4XlZWFqNGjWLu3LkMGDCAnj17cueddwLw+OOPs2TJEu644w68vLxo0aJFqeMXHcZbeDwh6oIqJQel1BhgDtAN6KW1lp7LWqDoF55KOsH9Q0ay4ZMlxfaZP3/+DT8fFhZGeno6sbGx2NnZ0blz52L1keqLkiuRpV3IJr2g+F7RsfD29vb4+/szZcoUVqxYYd7er18/xo8fz/Lly8tsUhKiLqtqn0MiMBKQovS1RMkZz9ltu/Nl+EZCdx8B4I8//uDHH3801zKC0pVEMzMzadu2LXZ2duzdu5cff/zRGpdS4ypafA/yi+zZ2NgwePBg8zZbW1sCAgLYtm1bhdcaFqKuqJZJcEqpSOCVij45yCS4muMTsqdUPZnLx/dzJXoD97a+DTs7O5YuXcrGjRvZvHkznp6ehIWF8eSTT5KQkMBDDz3EjBkz+Otf/8qlS5fw8vLi22+/Zdu2bXTu3Nk6F1VDKjMBcP78+WRmZvLWW29ZJDYhyiKF98QtO1NGobFm3R6gebcHSCjyhdenTx/effdd8+uSC/Z88803NRdkLVHRwmwjRozg1KlT7Nmzx1KhCWF15TYrKaV2KaUSy/ipVCF6pdQkpVSMUiomPT391iMWNyUzniuuooXZNm7cSEJCAq1bt7ZkeEJYVbnJQWv9oNbatYyfSg1+11p/pLX20lp7tWnT5tYjFjcllSgr7hFje/450kB7h6bFJgBWR2G2osutVlRgYGC5y7IW3cfPz09mr4saI81K9YxUoqycR4zta+TPpjA5TJ06tdqPLYQlVHUo6whgMdAG2KKUitda+5fzMVHDauoLT1Rc0bW4Bw0aBMC2bdtQSvH666/z2GOPobXmxRdfZOfOnXTs2JHGjRubPx8bG8vLL7/MpUuXaN26NaGhoTg6Ot7wfFOmTCE6OpqsrCxGjx7N3Llza/waRf1WpaGsWuuNWusOWusmWut2khiEyFd0Le4+ffoQHx/PkSNH2LVrF0FBQZw9e5aNGzdy4sQJkpKSWL16NQcPHgTyZ6i/+OKLrF+/ntjYWMaPH8+sWbNuer63336bmJgYEhIS2LdvHwkJCZa4TFGPSbOSEDUsKiqKJ554AltbW9q1a0e/fv2Ijo5m//795u1OTk7mdbdPnDhBYmKi+YkjLy/vpk8NAJ9//jkfffQRubm5nD17lqSkJNzc3Gr82kT9JclBiGpSkbW4K0JrTY8ePSo8nPj06dPMnz+f6Ohobr/9dgIDA+vljHZhWVKVVYhqcLO1uH19fVm3bh15eXmkp6ezf/9+evXqxQMPPGDefvbsWfbu3QuAi4sL6enp5uSQk5NjXnO7LBcuXKBZs2a0atWKtLQ0tm3bZolLFvWcPDkIUQ1uthb3C0+Pxs3NDXd3d5RSvPfee9x1112MGDGCPXv20L17dzp16mReF7tx48asX7+e6dOnk5mZSW5uLv/7v/9Ljx49yjy3u7s7RqORrl270rFjR3x8fCxyzaJ+kzWkhagGtWEtbpPJREBAAImJiVU6zubNm0lKSsLJyYkrV64wefLkaopQVJWUzxCijqloKY66YNiwYQwbNszaYQgrq9d9Dps3byYkJASA8PBwkpKSzO+dOXOG0aNHk5SUxIQJE6wVoqgnasvM9OpY5nTSpEl4eXnRo0cPZs+ebdH4Re1Rr5PDsGHDCA4OBkonBycnJ9avX0/37t35+OOPrRWiqCdqshRHZVTHMqchISEyZ0LU3T4Hk8nEkCFD6NOnDwcPHsTb25tx48Yxe/ZsfvvtN8LCwkhKSiImJoYnn3ySgIAAWrVqRatWrdiwYQM7d+5kxYoVXLt2jb/85S+sWbOG2267rZquUAjLM5lMDBo0iJSUFADeffddcnJycHZ25tChQ7z//vt06dKFQ4cOceeddzJp0iQGDBjA6NGjueeeezh27BgtWrTg3//+d7E5E4sXL5bFjGoJS/Y51Oknh++//56//e1vJCcnk5yczNq1a4mKimL+/Pm888475v369u3LsGHDmDdvHvHx8dx7772MGTOG6Ohojhw5Qrdu3Yqt8CVEXREel4pPyB6cg7cwatnBUsuc5ubmMmrUKLZt20ZERESpZU4///xz9uzZY17mtHDOxO7du0lISGDo0KEyZ6KBqtPJwdnZGYPBgI2NDT169GDgwIEopTAYDOUu5J6YmIivry8Gg4GwsLCbjiMXojYqObci7UI2aQXLnBZVdJnTcePGmbf369ePw4cPF1vmVOZMiEJ1arTSzRaDt7GxMS/mbmNjQ27uzWenBgYGEh4ejru7O6GhoURGRtZk6EJUu5stc1qyr2Ps2LFs3LixzGVOQ0NDWbVqFSBzJsSf6kxyqOhi8DdScp3kixcv4ujoSE5ODmFhYbRvL1VMRd1SctW/Rq3a4fTcB+btr7zyivm9qKgoxo0bh61t8RFVS5YsYcmSJcW2hYaG1kzAok6pM81KlVkMviyPP/448+bNw2g0curUKd566y169+6Nj48PXbt2rYmQhahRFV31b8SIEaxevZqXXnrJEmGJeqLOjFaqDTNQhahNSj5NQ/7cCmsMoRWWIaOVyiBrIwtRXG2ZWyHqpzrT5xDk71LmXZKsjSwaMln1T9SUOpMcZG1kIYSwnDqTHEDukoQQwlLqTJ+DEEIIy5HkIIQQohRJDkIIIUqR5CCEEKIUSQ5CCCFKqVJyUErNU0olK6USlFIblVIO1RWYEEII66nqk8NOwFVr7QacBGZWPSQhhBDWVqXkoLXeobUurI39LdCh6iEJIYSwturscxgP3HBlEKXUJKVUjFIqJj09vRpPK4QQorqVO0NaKbULuKuMt2ZprTcV7DMLyAXCbnQcrfVHwEeQX5X1lqIVQghhEeUmB631gzd7XykVCAQAA7U16n8LIYSodlWqraSUGgK8CvTTWl+pnpCEEEJYW1X7HJYALYCdSql4pdS/qyEmIYQQVlalJwet9V+qKxAhhBC1h8yQFkIIUYokByGEEKVIchBCCFGKJAchhBClSHIQQghRiiQHIYQQpUhyEEIIUYokByFuUd++fQGIjIwkICCgwp+LiYlh+vTpNRWWENVCkoMQt+jgwYO39DkvLy8WLVpUzdGI+sZkMrF27VqrnV+SgxAV8P777+Pq6oqrqysLFy4EoHnz5qX2i46Oxmg0curUKS5fvsz48ePp1asXRqORTZs2AcWfNObMmcP48ePx8/PjnnvukaQhzCQ5CFHLxcbGsnLlSr777ju+/fZbli9fTlxcXKn9Dh48yOTJk9m0aRP33nsvb7/9NgMGDODQoUPs3buXoKAgLl++XOpzycnJfPXVVxw6dIi5c+eSk5NjicsS1aDkTYPJZKJbt25MnDiRHj16MHjwYLKysgDw8/NjxowZ9OrViy5dunDgwAEA8vLyCAoKwtvbGzc3Nz788EMAgoODOXDgAB4eHixYsIDs7GyAzkqpo0qpOKVUf8ivjK2U+kIptV0plaKUeq86rq1KtZWEqK/C41KZ99UJzmRkwbGteN8/kGbNmgEwcuRI8//YhY4fP86kSZPYsWMHTk5OAOzYsYPNmzczf/58ALKzs/npp59KnWvo0KE0adKEJk2a0LZtW9LS0ujQQRZVtCatNVprbGxufP9c9KZBa03v3r3p168fKSkpfPrppyxfvpxHH32UDRs28NRTTwGQm5vLoUOH2Lp1K3PnzmXXrl2sWLGCVq1aER0dzdWrV/Hx8WHw4MGEhIQwf/58IiIiAPjXv/5VGJtBKdUV2KGU6lIQjgdgBK4CJ5RSi7XWP1flz0CSgxAlhMelMvOLo2Tl5AFwISuHPcfPEx6XyiPG9mV+xtHRkezsbOLi4szJQWvNhg0bcHFxKbZvWlpasddNmjQx/25ra0tubi7C8kwmE/7+/vTu3ZvY2FiSkpIoXKJm/fr1RERE8MhLbzNpwniyr9ty1XQYu7wsoqOjWb16NWlpaUyePBlnZ2c8PDyYMmUKBw8eJDIykpSUFCD/xgKgZ8+emEwmIP8mIiEhgfXr1wOQmZlJSkoKjRs3LhZfVFQUwO8AWutkpdSPQGFy2K21zgRQSiUBdwOSHISoTvO+OmFODABNOvTg960LCfnyCIO6OLBx40bWrFlT7DMODg6sWLGCQYMG0axZM/z8/PD392fx4sUsXrwYpRRxcXEYjUZLX46ohJSUFFatWkWfPn1K9Sn99McVZn5xlCvX8tC5WdzmOYy8s8k8HPBXvvvmIE5OTqxatcqc7N9++23uvfdeLly4wL59+7h8+bL5vaI3AVprFi9ejL+/f7HzRUZGVib0q0V+z6Mavtulz0GIEs5kZBV73eSuv9DcdSCHF0+hd+/eTJgwocwv+Xbt2hEREcELL7zAd999xxtvvEFOTg5ubm706NGDN954w1KXICogPC4Vn5A9OAdvwSdkDzuO/crdd99Nnz59ytw/MTXTfNPQ9C+9sO/Yg6y0H7jepAX33HMP4eHhuLu7c+3aNQA+//xzFixYwIcffsixY8fK7G8C8Pf3Z9myZea+ppMnT3L58mVatGjBxYsXzfv5+voC3AFQ0JzUCThRPX8apcmTgxAlODk0JbVEgmjZawTdBj/J18EDzNsuXboE5Hc0+vn5AdCpUyeOHTtm3qewc7GoovvPmTOn2HuJiYnVcAWiPCWbDlMzsnh3+8/k2f7ZxKeUMv+enZ3NlWt53Fb4nq0dTe76C7e5+HDh0EbzTUNB0w+nT59m/vz5PP/881y/fh2TycS3335bZiwTJkzAZDLh6emJ1po2bdoQHh6Om5sbtra2uLu7ExgYyNSpU/nb3/6mlFJHgVwgUGt9tWic1UmSgxAlBPm7FPviAGhqZ0uQv8tNPiXqkpJNhwBXc/P449KfrTPt2rXj+PHjuLi4sHHjRm5rbFvqOC3c/cn54ZA5qcfHx/P+++9z4cIFmjVrxuuvv056ejpubm68++67eHl5AdC6dWtzn4ONjQ3vvPMO77zzTqnj79mzp+Qmk9baq+gGrXUoEFrkdcVnZN6EJAchSijsdC4creTk0JQgf5cbdkaLuqdk02Gh3Lzr5t9DQkIICAigTZs2eHl54ZrzG2ftiieIJo1sad28ScnD4O7ujtFopGvXrnTs2BEfH5/qvQALUIW98Zbk5eWlY2JiLH5eIYQA8AnZU6rpEKC9Q9NiTYclFR3ibI2bBqVUbMknh5qyvZ8ZAAAYdklEQVQiTw5CiAbnVpsOHzG2bzBPkJIchBANjjQdlk+SgxCiQWpITwG3QuY5CCGEKKVKyUEp9ZZSKkEpFa+U2qGUcqquwIQQ1uPn54cMGmnYqvrkME9r7aa19gAigDerISYhhBBWVqXkoLW+UORlM8Dy42KFEOUymUx07dqVsWPH0q1bN0aPHs2VK1f4+9//jre3N66urkyaNImiQ9v/+9//liovbTKZ8PX1xdPTE09Pz1te8EjUflXuc1BKva2U+hkYizw5CFFrnThxgqlTp3L8+HFatmzJBx98wLRp04iOjiYxMZGsrCxzeWj4s7z0woULmTt3LgBt27Zl586dHD58mHXr1slyp/VYuclBKbVLKZVYxs9wAK31LK11RyAMmHaT40xSSsUopWLS09Or7wqEEBVSdKbuU089RVRUFHv37qV3794YDAb27NlTrC5UWeWlc3JymDhxIgaDgTFjxpCUlGTx6xCWUe5QVq31gxU8VhiwFZh9g+N8BHwE+TOkKxqgEKLySs7kfdZwGyULtCmlmDp1KjExMXTs2JE5c+YUrjYGUGZ56QULFtCuXTuOHDnC9evXsbe3t9xFCYuq6mil+4q8HA4kVy0cIURVFVYcTc3IQlNYcfQEP/30E9988w0Aa9eu5X/+53+A/CJwly5dMi82czOZmZk4OjpiY2PDmjVryMvLK/czom6q6iS4EKWUC3Ad+BGYXPWQhBBVcaOKo03bdGTp0qWMHz+e7t27M2XKFM6fP4+rqyt33XUX3t7e5R576tSpjBo1itWrVzNkyBDz0qmi/pHCe0LUM87BW0oNG8zNTOO39XO5lv6jVWIqymQyERAQQGJiIpGRkeZ1kjdv3kxSUhLBwcHWDrHWksJ7QohbVtZiRQCNbGt3QYRhw4YxbNgwa4chCtTufy1CiEoL8nehaYl1B1q0dmLttqgKH+P999/H1dUVV1dXFi5ciMlkolu3bkycOJEePXowePBgsrLyE5Cfnx8zZswoNSciLy+PoKAgvL29cXNzK3NVvKJCQ0OZNi1/wGNgYCDTp0+nb9++3HPPPcX6Q+bNm2c+5uzZ+eNfbhafuDWSHISoZx4xtuefIw20d2iKIn+Ngn+ONFS4yFxsbCwrV67ku+++49tvv2X58uWcP3+elJQUXnjhBY4dO4aDgwMbNmwwf6asORErVqygVatWREdHEx0dzfLlyzl9+nSFr+Ps2bNERUURERFhbmrasWMHKSkpHDp0iPj4eGJjY9m/fz/ATeMTlSfNSkLUQ5WtOFp06CvHtuJ9/0BzZ/PIkSM5cOAAzs7OeHh4AMXnPhTuU3L7jh07SEhIMN/1Z2ZmkpKSQpcuXSp2DY88go2NDd27dyctLc18zB07dmA0GoH8dbxTUlLo1KnTTeMTlSfJQYgGrnDoa+EIpwtZOew5fp7wuNRiCaZw3gPkz30o2mxT1pwIrTWLFy/G39+/2Pkq+qVd9HyFA2e01sycOZPnn3++1DFvFp+oPGlWEqKBKzn0tUmHHlw48Q0hXx7h8uXLbNy4EV9f30of19/fn2XLlpGTkwPAyZMnuXz5cpVi9ff355NPPuHSpUsApKam8ttvv1XpmKJs8uQgRAN3psTIpiZ3/YXmrgM5vHgKvT9vwYQJE7j99tsrfdwJEyZgMpnw9PREa02bNm0IDw+vUqyDBw/m+PHj3H///QA0b96cq1ev8sUXX1TpuEWH1Ip8Ms9BiAbOJ2RPmUNf2zs05evgAVaIyPLqSnKw5DwHaVYSooEra+hrUztbgvxdrBRR5TRv3pzIyEgCAgLM26ZNm0ZoaCgAu3fvxmg0YjAYGD9+PFevXgVg+/btdO3aFU9Pz2JPHocOHeL+++/HaDTSt29fTpw4YdHrqS0kOQjRwFV16Gttlp2dTWBgIOvWrePo0aPk5uaybNkysrOzmThxIl9++SWxsbH8+uuv5s907dqVAwcOEBcXx9///ndee+01K16B9UifgxCi0kNfraVktdnynm5OnDiBs7Ozefjss88+y9KlS/Hz88PZ2Zn77suvHfrUU0/x0UcfAflDbp999llSUlJQSpk71BsaeXIQQtQJZVWbnfnFUfKuaxo1asT169fN+xYtPV5Zb7zxBv379ycxMZEvv/yySseqyyQ5CCHqhLKqzWbl5HEt7zp33303SUlJXL16lYyMDHbv3g2Ai4sLJpOJ77//HoA1a9bQr18/unbtislk4tSpUwB8+umn5mNmZmbSvn3+U1Rhv0VDJMlBCFEnlBxyW0ij6NixI48++iiurq48+uij5hnU9vb2rFy5kjFjxmAwGLCxsWHy5MnY29vz0UcfMXToUDw9PWnbtq35eK+++iozZ87EaDSaJ/Q1RDKUVQhRJ5Q15DYv6wK/rf5frp5Ps1JUliVDWYUQooSSQ25zL/5O2n+CeGbSDZeuF1UgyUEIUSeUHHJ7d8cOfL7rO5a/+4a1QwNg165dvPnmm2zevJmQkJAy9ylalhygb9++ZGRk8PDDD1sqzAqTZiUhhLCQ0NBQYmJiWLJkyS19XpqVhBCiFrh8+TJDhw7F3d0dV1dX1q1bxyOPPGJ+f+fOnYwYMQLIH/FkMBhwdXVlxowZ5n1WrlxJly5d6NWrF19//bV5+6ZNm+jduzdGo5FBgwaZy5LXFpIchBDiBrZv346TkxNHjhwhMTGRIUOGkJycTHp6OpD/xT9+/HjOnDnDjBkz2LNnD/Hx8URHRxMeHs7Zs2eZPXs2X3/9NVFRUSQlJZmP/cADD/Dtt98SFxfHY489xnvvvWetyyyTzJAWQogiis7Cvj3nEr9s2c4dM2YQEBCAr68vTz/9NP/5z38YN24c33zzDatXr2bLli34+fnRpk0bAMaOHWteoa7o9scee4yTJ08C8Msvv/DYY49x9uxZrl27hrOzs3Uu+AYkOQghRIGSCx/9Ydcahyff52qLs7z++usMHDiQCRMm8Ne//hV7e3vGjBlDo0a39jX64osv8vLLLzNs2DAiIyOZM2dONV5J1UmzkhBCFCg5Czv34u9cpRHRjVwJCgri8OHDODk54eTkxD/+8Q/GjRsHQK9evdi3bx/nzp0jLy+PTz/9lH79+tG7d2/27dvH77//Tk5ODv/973/Nxy46E3vVqlWWvdAKkCcHIYQoUHIWdk66id8iV3JWKeZ2upNly5YB+c1G6enpdOvWDQBHR0dCQkLo378/WmuGDh3K8OHDAZgzZw73338/Dg4O5jWuC7ePGTOG22+/nQEDBnD69GkLXWXFVMtQVqXU34D5QBut9bny9pehrEKI2qiiCx9NmzYNo9HIc889Z8nw6tZQVqVUR2Aw8FPVwxFCCOupyMJHPXv2JCEhgaeeesrS4VlUdTQrLQBeBTZVw7GEEMJqCte0KLlmRNG1LmJjY60VnkVVKTkopYYDqVrrI0qpagpJCCGsp64sfFTTyk0OSqldwF1lvDULeI38JqVyKaUmAZMAOnXqVIkQhRBCWNotd0grpQzAbuBKwaYOwBmgl9b61xt+EOmQFkKIW1EnOqS11ke11m211p211p2BXwDP8hKDEDXBZDLh6upabFtMTAzTp0+3UkRC1G0yz0HUW15eXnh5WeQmS4h6p9pmSBc8QZQ7x0GImvbDDz9gNBqZN28eAQEBAOzbtw8PDw88PDwwGo1cvHiRs2fP8sADD+Dh4YGrqysHDhwAYMqUKXh5edGjRw9mz55tzUsRwmrkyUHUKydOnODxxx8nNDSU8+fPs2/fPgDmz5/P0qVL8fHx4dKlS+Y1hP39/Zk1axZ5eXlcuZLfffb2229zxx13kJeXx8CBA0lISMDNzc2alyWExUltJVFvpKenM3z4cMLCwnB3dy/2no+PDy+//DKLFi0iIyODRo0a4e3tzcqVK5kzZw5Hjx6lRYsWAHz++ed4enpiNBo5duxYsTLLQjQUkhxEnRUel4pPyB6cg7cwatlBbO2b0alTJ6KiokrtGxwczMcff0xWVhY+Pj4kJyfzwAMPsH//ftq3b09gYCCrV6/m9OnTzJ8/n927d5OQkMDQoUPJzs62wtUJYV2SHESdVFhaOTUjCw2kXcjm96zrjJ+zlNWrV7N27dpi+586dQqDwcCMGTPw9vYmOTmZH3/8kXbt2jFx4kQmTJjA4cOHuXDhAs2aNaNVq1akpaWxbds261ygqLfKGlkHMGHCBJKSkhg9ejRnzpy5pWMrpTorpRILfvdTSkUUeW+rUspBKXWwIseSPgdRJ5UsrQygtWbx/p/ZEhHBoEGDeOONPxeeX7hwIXv37sXGxoYePXrw0EMP8dlnnzFv3jzs7Oxo3rw5q1evxtnZGaPRSNeuXenYsSM+Pj6WvjTRQH388ccArF+/vkaOr7V+uODXvhXZv1qqslaWTIITVeUcvIWy/uUq4HTIUEuHI0SFmUwmAgICSExM5IcffmDUqFH8+9//5pVXXuHy5csopVi8eDF9+/bl8ccf5+mnn2bo0Px/00qp34HJwEYgBPADmgBLtdYfKqU6AxFaa1ellB/witY6QCnVh/w6ePZAFjBOa33iZnFKs5Kok5wcmlZquxC1zYkTJxg1ahShoaEYDAZ27tzJ4cOH+eyzz8yTNx977DE+//xzAK5duwbQEtgCPAdkaq29AW9golLqZuuMHgd8tdZG4E3gnfLik2YlUScF+bsUW84RSpdWFqK2KhxZ98UXX9C9e3cyMzOZNm0a8fHx2NramteZfuihh3jppZe4evUq27dvB7iotc5SSg0G3JRSowsO2Qq4Dzh5g1O2BEKVUvcBGrArL0ZJDrVQ0cfO8sTExLB69WoWLVpkgchqj4qUVhaitgiPSzX/W71DZxYbWde9e3cWLFhAu3btOHLkCNevX8fe3h4Ae3t7/Pz8+Oqrr1i3bh3AHwWHVMCLWuuvip6noFmpLG8Be7XWIwr2iSwvZkkOddyNSkTk5ube8sLndYWUVhZ1QeHIusKn3KIj65a8Oo7mzZuTmZlJhw4dsLGxYdWqVeTl/flE/Nhjj/Hxxx9T0E97oWDzV8AUpdQerXWOUqoLkHqTMFoVeT+wInFLn0MtV1gK4sCBA4wbNw6DwYDRaGTv3r0AREZGmktEzJkzh6effhofHx+efvppTCYTvr6+eHp64unpycGDFRrBJoSoRjcbWRcREcGCBQvo3Lkzq1atwt3dneTkZJo1a2bed/Dgwezbt48HH3wQMI/D+BhIAg4XDF39kJvf7L8H/FMpFVfOfmb1+9ayjitaCmLXrl0opTh69CjJyckMHjzY3C5ZVFJSElFRUTRt2pQrV66wc+dO7O3tSUlJ4YknnkBGiQlhWWdKrEndqFU7nJ77gDMZWTg4OBAdHQ3ASy+9ZN7n3XffNf9uZ2fHH3/ktyaFhoYCoLW+Tv56Oq+VOF0m4FqwTyQFzUda62+ALkX2e728uOXJoZYqWQoiKirKvGZt165dufvuu8tMDsOGDaNp0/wROzk5OUycOBGDwcCYMWOkDIQQVlBXR9bJk0MtUV6HVUUVfRy9USeXEMJy6urIOnlyqAUqUgrC19eXsLAwAE6ePMlPP/2Ei8vN/3FlZmbi6OiIjY0Na9asKdbJJYSwjEeM7fnnSAPtHZqigPYOTfnnSEOtH0whTw61QEVLQRw9ehSDwUCjRo0IDQ2lSZMmNz3u1KlTGTVqFKtXr2bIkCHFniqEEJZTF0fWSfmMWkBKQQghKqJOrCEtqk9d7bASQtRfkhxqgSB/F5ra2RbbVhc6rIQQ9Zf0OdQCUgpCCFHbSHKoJepih5UQov6SZiUhhBClSHIQQghRiiQHIYQQpVQpOSil5iilUpVS8QU/D5f/KSGEELVddXRIL9Baz6+G4wghhKglpFlJCCFEKdWRHKYppRKUUp8opW6/0U5KqUlKqRilVEx6eno1nFYIIURNKbe2klJqF3BXGW/NAr4FzpG/OtFbgKPWenx5J5XaSkIIUXmWrK1Ubp+D1vrBihxIKbUciKhyREIIIayuqqOVHIu8HAEkVi0cIYQQtUFVRyu9p5TyIL9ZyQQ8X+WIhBBCWF2VkoPW+unqCkQIIUTtIUNZhRCiGvTt27fajhUeHk5SUlK1He9WSHIQQohqcPDgwWo7liQHIYSoJ5o3b05kZCQBAQHmbdOmTSM0NBSA3bt3YzQaMRgMjB8/nqtXrwIQHBxM9+7dcXNz45VXXuHgwYNs3ryZoKAgPDw8OHXqFMuXL8fb2xugu1Jqg1Lqtpq+HlnPQQghalh2djaBgYHs3r2bLl268Mwzz7Bs2TKefvppNm7cSHJyMkopMjIycHBwYNiwYQQEBDB69GgAHBwcmDhxIkqpJOA48BywuCZjlicHIYS4BeFxqfiE7ME5eAs+IXvIu37jCcUnTpzA2dmZLl26APDss8+yf/9+WrVqhb29Pc899xxffPEFt91W9gNBYmIivr6+AN2BsUCPar+gEiQ5CCFEJYXHpTLzi6OkZmShgdSMLK7mXufgD+e5fv26eb/s7OybHqdRo0YcOnSI0aNHExERwZAhQ8rcLzAwkCVLlgAkAXMB++q6lhuR5CCEEJU076sTZOXkldr+xcmrJCUlcfXqVTIyMti9ezcALi4umEwmvv/+ewDWrFlDv379uHTpEpmZmTz88MMsWLCAI0eOANCiRQsuXrxoPu7FixdxdHQEUOQ/OdQ46XMQQohKOpORVXqjUvxOCx599FFcXV1xdnbGaDQCYG9vz8qVKxkzZgy5ubl4e3szefJk/vjjD4YPH052djZaa95//30AHn/8cSZOnMiiRYtYv349b731Fr179wboCnwCtKjpayy38F5NkMJ7Qoi6zCdkD6lFEkRe1gXOhr5E75mf8XXwgBo7ryUL70mzkhBCVFKQvwtN7WwByL34O7+ueYU77x9FkL+LlSOrPtKsJIQQlfSIsT2Q3/dwhjvp9eoagvxdzNvrA0kOQghxCx4xtq9XyaAkaVYSQghRiiQHIYQQpUhyEEIIUYokByGEEKVIchBCCFGKJAchhBClWGWGtFLqInDC4ieuutbAOWsHcYvqaux1NW6ou7HX1bih7sZe0bjv1lq3qelgwHrzHE5Yagp4dVJKxdTFuKHuxl5X44a6G3tdjRvqbuy1MW5pVhJCCFGKJAchhBClWCs5fGSl81ZVXY0b6m7sdTVuqLux19W4oe7GXuvitkqHtBBCiNpNmpWEEEKUYrXkoJR6USmVrJQ6ppR6z1pxVJZSao5SKlUpFV/w87C1Y6ospdTflFJaKdXa2rFUhFLqLaVUQsGf9w6llJO1Y6oopdS8gn/nCUqpjUopB2vHVBFKqTEF/29eV0rVqlE0ZVFKDVFKnVBKfa+UCrZ2PBWllPpEKfWbUirR2rGUZJXkoJTqDwwH3LXWPYD51oijChZorT0KfrZaO5jKUEp1BAYDP1k7lkqYp7V201p7ABHAm9YOqBJ2Aq5aazfgJDDTyvFUVCIwEthv7UDKo5SyBZYCDwHdgSeUUt2tG1WFhQJDrB1EWaz15DAFCNFaXwXQWv9mpTgaogXAq0Cd6WzSWl8o8rIZdSv2HVrr3IKX3wIdrBlPRWmtj2ut68pE1V7A91rrH7TW14DPyL/5rPW01vuBP6wdR1mslRy6AL5Kqe+UUvuUUt5WiuNWTStoJvhEKXW7tYOpKKXUcCBVa33E2rFUllLqbaXUz8BY6taTQ1HjgW3WDqIeag/8XOT1LwXbRBXU2AxppdQu4K4y3ppVcN47gD6AN/C5UuoeXUuGTpUT+zLgLfLvXt8C/kX+//S1Qjmxv0Z+k1Ktc7O4tdabtNazgFlKqZnANGC2RQO8ifJiL9hnFpALhFkytpupSNyi4aqx5KC1fvBG7ymlpgBfFCSDQ0qp6+TXFkmvqXgq42axF6WUWk5+G3itcaPYlVIGwBk4opSC/OaNw0qpXlrrXy0YYpkq+mdO/pfrVmpRcigvdqVUIBAADKwtN0BQqT/z2i4V6FjkdYeCbaIKrNWsFA70B1BKdQEaU0eKZSmlHIu8HEF+x12tp7U+qrVuq7XurLXuTP6jt2dtSAzlUUrdV+TlcCDZWrFUllJqCPl9PMO01lesHU89FQ3cp5RyVko1Bh4HNls5pjrPWlVZGwOfAB7ANeAVrfUeiwdyC5RSa8iPWwMm4Hmt9VmrBnULlFImwEtrXeuTslJqA+ACXAd+BCZrrevEnaFS6nugCfB7waZvtdaTrRhShSilRgCLgTZABhCvtfa3blQ3VjCkfCFgC3yitX7byiFViFLqU8CP/JaTNGC21nqFVYMqIDOkhRBClCIzpIUQQpQiyUEIIUQpkhyEEEKUIslBCCFEKZIchBBClCLJQQghRCmSHIQQQpQiyUEIIUQp/x/WWMOALkZxAwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2172270860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fi_vectors_pca2 = pca2.fit_transform(fi_wv.vectors)\n",
    "plot_words(fi_wv, fi_words, fi_vectors_pca2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a reasonable visualization, but it's still hard to make sense of the structure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## t-SNE\n",
    "\n",
    "t-Distributed Stochastic Neighbor Embedding (t-SNE) ([van der Maaten and Hinton, 2008](https://lvdmaaten.github.io/publications/papers/JMLR_2008.pdf))\n",
    "\n",
    "* Designed specifically for visualization (reduction to 2D/3D)\n",
    "* Non-convex: re-running can product different results\n",
    "* Preserves local relationships (neighbors) but not global structure\n",
    "* Tends to work great in practice, but can be slow (PCA first can help)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD8CAYAAACGsIhGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XlclNX+wPHPkRSJRTTX0Fx+KSr7ouCCouZyy9zKzDRFf2Zmdrt6s+y2YcvN0spsuVY/AzVLLdPcKjO00FKBQFTSNB2voZkbCgopcn5/zDANCI7IwCx836/XvJw5zzPPfI8Pw5fnnPOco7TWCCGEEFdTy94BCCGEcHySLIQQQlglyUIIIYRVkiyEEEJYJclCCCGEVZIshBBCWCXJQgghhFWSLIQQQlglyUIIIYRVN9g7gGvRsGFD3apVK3uHIYQQTiUtLe2k1rqRLY7lFMmiVatWpKam2jsMIYRwKkqpw7Y6ljRDCSGEsEqShRClbN68mYEDB9o7DCEciiQLIYQQVkmyEC7PYDAQGBhofj1nzhzi4+NJSUkhODiY0NBQpk+fXmIfgKKiItq2bcuJEyfMr2+99VbzayFqEkkWosYaN24c7733HhkZGbi5uV2xvVatWowePZolS5YAsHHjRkJCQmjUyCaDS4RwKpIshMtalZ5Nt1lJdH8liYMnz7MqPdu8LScnh9zcXLp06QLAfffdV+Yxxo8fz6JFiwD48MMPGTduXNUHLoQDcoqhs0JU1Kr0bJ78fBf5ly6DcqOw8DJPfr4LgIKCgms+TosWLWjSpAlJSUns2LHDfJUhRE0jVxbCJc3+ep8xUQBunr5cvnCWvHNneGXdLtauXYuvry/e3t5s374dgKVLl5Z7rAkTJjB69GiGDx9eZnOVEDWBJAvhko7m5JufK7cbqNf1Xn5fNI309x6jffv2ACxYsIAHHniA0NBQzp8/T7169co81qBBg8jLy5MmKFGjSTOUcEk3+3qQbZEwfCIH4RM5CD9fDxJn9AYgLy+PzMxMAGbNmkVkZCQAsbGxxMbGmt+7c+dOQkJCzElGiJpIriyES5re3x+P2iWbjDxquzG9v7/59bp16wgNDSUwMJDk5GSefvrpK44za9Ys7rrrLl5++eUqj1kIR6a01vaOwarIyEgtc0OJilqVns3sr/dxNCefm309mN7fnyFhfvYOS4hqo5RK01pH2uJY0gxVSZs3b6ZOnTp07drV3qGIUoaE+UlyEFfIyMjg6NGj3H777fYOxalIM1Qlbd68mR9++MHeYQghrlFGRgbr16+v0HsKCwurKBrnIcmiHIsWLSI4OJiQkBDuv/9+1qxZQ1RUFGFhYdx2220cP34cg8HA/PnzeeONNwgNDSU5OdneYQvh8Ep/twwGA7179yY4OJg+ffrw3//+F4C4uDgeeughoqOjadOmDZs3b2b8+PF06NCBuLg48/G8vLyYOnUqAQEB9OnTxzwdS2xsrHlpg5MnT9KqVSsuXrzIs88+y7JlywgNDWXZsmWcP3+e8ePH07lzZ8LCwvjiiy8ASExMZNCgQfTu3Zs+ffpU73+SI9JaO/wjIiJCV6fdu3frtm3b6hMnTmittT516pQ+ffq0Lioq0lpr/cEHH+hp06ZprbV+7rnn9OzZs6s1PiGcVVnfrYEDB+rExESttdYLFizQgwcP1lprPXbsWD1ixAhdVFSkV61apb29vXVmZqa+fPmyDg8P1+np6VprrQH90Ucfaa21njlzpn744Ye11lr37NlTp6SkaK21PnHihG7ZsqXWWuuEhATzPlpr/eSTT+rFixdrrbU+c+aMbtu2rc7Ly9MJCQnaz89Pnzp1qor/V6oOkKpt9HtY+izKkJSUxPDhw2nYsCEADRo0YNeuXYwYMYJjx45x8eJFWrdubecohXA+ZX23fvzxRz7//HMA7r//fh5//HHz/nfeeSdKKYKCgmjSpAlBQUEABAQEYDAYCA0NpVatWowYMQKA0aNHM2zYsArFtGHDBlavXs2cOXMA4x3+xVc3ffv2pUGDBpWrtIuQZGFiOXJGZf1CeCNVYvsjjzzCtGnTGDRoEJs3byY+Pt4+gQrhhIq/X3u/3YNHYS6d0rOvafCBu7s7YJzUsfh58evy+hGUMn53b7jhBoqKioCrT/GitWbFihX4+/uXKN++fTuenp5WY6wppM+Cv+YRys7JRwMFjTqwetXnLNq0G4DTp09z9uxZ/PyMP9wLFy40v9fb25vc3Fx7hC2EU7D8frnfEszxnZt5/KOtrErP5vTp03Tt2tU83cqSJUuIiYmp0PGLior47LPPAPj444/p3r07YFyOOS0tDcC8Ha78zvbv35+33noLbbqNID09/for68IkWVByHiGAOo1a4hN9D5NG3klISAjTpk0jPj6e4cOHExERYb6EBuNl8sqVK6WDW4hyWH6/6jRqSb0uIzAsms6o23swbdo03nrrLRISEggODmbx4sW8+eabFTq+p6cnO3bsIDAwkKSkJJ599lkAHnvsMf7zn/8QFhbGyZMnzfv36tWLrKwscwf3M888w6VLlwgODiYgIIBnnnnGdpV3IXJTHtB6xjrK+l9QwKFZd1TZ5wpRE1T198vLy4u8vLxKH8cV2fKmPLmywDiPUEXKhRDXTr5frkGSBdc2j5AQ4vpU9fdLriqqh4yGAvOoDJlHSAjbk++Xa5A+CyGEcFHSZyGEEKJaSbIQQghhlSQLIYQQVkmyEEIIYZUkCyGEEFZJshBCCGGVJAshhBBWVTpZKKVaKKU2KaWylFJ7lFKPmsobKKW+UUrtN/1b31SulFLzlFIHlFKZSqnwysYghBCiatniyqIQ+KfWuiMQDTyslOoIzAC+1Vq3Bb41vQb4G9DW9JgI/McGMQghhKhClU4WWutjWuufTM9zgZ8BP2AwULzww0JgiOn5YGCRadW/bYCvUqpZZeMQQghRdWzaZ6GUagWEAduBJlrrY6ZNvwNNTM/9gCMWb/vNVFZt4uPjzUsoCudgMBgIDAys9vcKIYxsliyUUl7ACuAfWutzlttMC4dXaBIqpdREpVSqUir1xIkTtgpTCCHEdbBJslBK1caYKJZorT83FR8vbl4y/fuHqTwbaGHx9uamshK01u9rrSO11pGNGjWqdIwvvfQS7dq1o3v37uzbtw+AjIwMoqOjCQ4OZujQoZw5cwaAlJQUgoODCQ0NZfr06fJXqYMoLCxk1KhRdOjQgbvvvpsLFy7w/PPP06lTJwIDA5k4caJ5acy0tDRCQkIICQnhnXfesXPkQjg/W4yGUsAC4Get9esWm1YDY03PxwJfWJSPMY2KigbOWjRXVYm0tDSWLl1KRkYG69evJyUlBYAxY8bwyiuvkJmZSVBQEDNnzgRg3LhxvPfee2RkZODm5na1Q4tqtG/fPiZPnszPP/+Mj48P7777LlOmTCElJYXdu3eTn5/P2rVrAeM5fOutt9i5c6edoxbCNdjiyqIbcD/QWymVYXrcDswC+iql9gO3mV4DrAcOAgeAD4DJNoihTKvSs+k2K4m+j79HTuNQNuw7g4+PD4MGDeL8+fPk5OTQs2dPAMaOHcv3339PTk4Oubm5dOnSBYD77ruvqsITVhSfv9Yz1nHXf36gYdOb6datGwCjR49my5YtbNq0iaioKIKCgkhKSmLPnj3k5OSQk5NDjx49ALj//vvtWQ0hXEKlFz/SWm/BuJxuWfqUsb8GHq7s51qzKj2bJz/fZV4oPregkCc/31XVHytspPT5O36ugJwLhaxKzzYvmqOUYvLkyaSmptKiRQvi4+MpKCiwZ9hCuCyXvYN79tf7zL9o3FsEcGH/Ns5fuMCs1emsWbMGT09P6tevT3JyMgCLFy+mZ8+e+Pr64u3tzfbt2wFYunSp3epQk1mev2KF5/7g2feNXWIff/wx3bt3B6Bhw4bk5eXx2WefAeDr64uvry9btmwBYMmSJdUYuRCuyWWXVT2ak29+7t70Vjzbx3As4RH+uNGXYT06AbBw4UImTZrEhQsXaNOmDQkJCQAsWLCABx54gFq1atGzZ0/q1atnlzrUZJbnr9gNDZpz8PvP6dDhXTp27MhDDz3EmTNnCAwMpGnTpnTq1Mm8b0JCAuPHj0cpRb9+/aozdCFckssuq9ptVhLZZfzC8fP1YOuM3ld9b15eHl5eXgDMmjWLY8eO8eabb1bo80XlVOb8CSGMZFnVazC9vz8etUuOZPKo7cb0/v5W37tu3TpCQ0MJDAwkOTmZp59+uqrCFOWozPkTQtiey15ZgLGTdPbX+ziak8/Nvh5M7+9v7hwVjk/OnxCVY8srC5dOFkIIUZNJM5QQQogyde3aFTDOiQY0sNVxJVkI4WRycnJ499137R2GcFA//PADcH3JQilV7ghZSRZCOBlJFq7BYDDQvn174uLiaNeuHaNGjWLjxo1069aNtm3bsmPHDk6fPs2QIUMIDg4mOjqazMxMwDhz9vjx44mNjaVNmzbMmzfPfNzikZwzZswA8DLNqjFVKVVXKZWglNqllEpXSvUCUErFKaVWK6WSMK49VDattcM/IiIitBDCaMSIEbpu3bo6JCREx8XF6S+++EJrrfWQIUP0uHHjtNZaL1iwQP/rX//SWmv92muv6YCAAB0QEKDfeOMNu8UtSjp06JB2c3PTmZmZ+vLlyzo8PFyPGzdOFxUV6VWrVunBgwfrKVOm6Pj4eK211t9++60OCQnRWmv93HPP6S5duuiCggJ94sQJ3aBBA33x4kWttdaenp5aa603bdqkgRxt+j0K/BP40PS8PfBfoC4Qh3GpiAb6Kr+HXfamPCFc1axZs9i9ezcZGRksXbqU5ORkBg0aRHZ2NseOGefkTE5O5t577yUtLY2EhAS2b9+O1pqoqCh69uxJWFiYnWtRM1mO8Gugz9L45hYEBQUBEBAQQJ8+fVBKERQUhMFg4PDhw6xYsQKA3r17c+rUKc6dM64Acccdd+Du7o67uzuNGzfm+PHjNG/e/Gof3x14C0BrvVcpdRhoZ9r2jdb69NXeLM1QQjixmJgYkpOTycrKomPHjjRp0oRjx47x448/0rVrV7Zs2cLQoUPx9PTEy8uLYcOGmae4EdWreL6z7Jx8NMb5zk4VaFalG1doqFWrFu7u7ubnhYWFVz1e8b4Abm5uVve34ry1HSRZCOEkimfh7f5KEgdPnmdVejZ+fn7k5OTw1Vdf0aNHD2JiYli+fDleXl54e3vbO2Rhoaz5zrTWzP56X7nviYmJMc9ttnnzZho2bIiPj881fZ7p/Fve2ZoMjAJQSrUDbgHK//BSpBlKCCdgOQuvquPBxfzz5lmUo6OjmTt3LklJSZw6dYq7776bu+++GzD+somLi2PGjBlorVm5ciWLFy+2Z1VqrLLmO7taOfzVkR0cHMyNN97IwoULr/nzgoODAbRSaieQCLwL/EcptQsoBOK01n8alySyTm7KE8IJlJ4r68Tq2Vw6cYhGHaJ5fmw/nnnmGY4ePcqlS5fw9fVl8eLFDBs2DIDXX3+dDz/8EIAJEybwj3/8wy51qOnsMd+Z3MEtRA3Tesa6MhexV8ChWXdUdzjiOpReowWM8529PCyoyqaxkTu4hahhbvb1qFC5cDxDwvx4eVgQfr4eKIxXFFWZKGxN+iyEcALT+/uX+VepzMLrXIaE+TlNcihNkoUQTqD4F4zMwivsRZKFEE7Cmf8qFc5P+iyEEEJYJclCCBODwUBgYKC9wxDCIUmyEEIIYZUkC+G0XnjhBfz9/enevTsjR45kzpw5ZGRkEB0dTXBwMEOHDuXMmTMA5ZanpaUREhJCSEgI77zzjj2rI2wgNjaW4nuyWrVqxcmTJ+0ckeuQZCGcUkpKCitWrGDnzp18+eWX5l8QY8aM4ZVXXiEzM5OgoCBmzpx51fJx48bx1ltvsXPnTrvVRQhnIMlCOKWtW7cyePBg6tati7e3N3feeSfnz58nJyeHnj17AjB27Fi+//57zp49W2Z5Tk4OOTk59OjRA4D777/fbvURJc2ePdu8oM/UqVPp3ds4HUZSUhKjRo1iw4YNdOnShfDwcIYPH05eXp49w60RJFkIp1I88+rza7L4cMsh8/TOwrUUT70OkJqaSl5eHpcuXSI5OZng4GBefPFFNm7cyE8//URkZCSvv/66nSN2fZIshNOwXA/AvXkH/tjzA08sT+OTrb+wdu1aPD09qV+/vvmXzOLFi+nZsyf16tUrs9zX1xdfX1+2bNkCYJ4KWthfREQEaWlpnDt3Dnd3d7p06UJqairJycl4eHiQlZVFt27dCA0NZeHChRw+fNjeIbs8uSlPOA3L9QDcm7XD49bOHHzvIR5c2oC+4UHUq1ePhQsXMmnSJC5cuECbNm1ISEgAKLc8ISGB8ePHo5SiX79+dqubMLJcSe5MLV+mvTiXrl27EhwczKZNmzhw4ACtW7emb9++fPLJJ/YOt0aRZCGcRul5/306D8O3+yj0pQIOf/cyERERhIaGsm3btiveW155REREic7tV1991faBi2tyxaysTduz8L23efqVecTExDBt2jQiIiKIjo7m4Ycf5sCBA9x6662cP3+e7Oxs2rVrd/UPEJUizVDCaZSeYfXUV29zNOERTiyeyl133UV4eLidIhO2UHolOffmARTmnebLP7xp0qQJdevWJSYmhkaNGpGYmMjIkSMJDg6mS5cu7N27146R1wyynoVwGvZYD0BUH1mzw/ZkPQtRIzn7egDi6mTNDscmfRbCqcjMq65L1uxwbJIshBAOQdbscGySLIQQDkOuHB2X9FkI4eJef/11AgMDCQwMZO7cuRgMBjp06MADDzxAQEAA/fr1Iz/fOCz5119/ZcCAAURERBATEyOjjISZJAshXFhaWhoJCQls376dbdu28cEHH3DmzBn279/Pww8/zJ49e/D19WXFihUATJw4kbfeeou0tDTmzJnD5MmT7VwD4Sjs1gyllBoAvAm4Af+ntZ5lr1iEcFVbtmxh6NCheHp6AjBs2DCSk5Np3bo1oaGhgPHGRIPBQF5eHj/88APDhw83v//PP/+0S9zC8dglWSil3IB3gL7Ab0CKUmq11jrLHvEI4WqKp83Yu3EPN+p8wtOzS/QFuLu7m5+7ubmRn59PUVERvr6+ZGRk2CNk4eDs1QzVGTigtT6otb4ILAUG2ykWIVyK5YSLdZoHcHzXFp5YlsLSH/azcuVKYmJiynyfj48PrVu35tNPPwVAay3rfAgzeyULP+CIxevfTGVCiEoqMeFi01vxCuzDoQWPMn5YPyZMmED9+vXLfe+SJUtYsGABISEhBAQE8MUXX1RX2MLB2WW6D6XU3cAArfUE0+v7gSit9RSLfSYCEwFuueWWCJmCWIhrI9NmiGKuMN1HNtDC4nVzU5mZ1vp9rXWk1jqyUaNG1RqcEM5Mps0QVcFeySIFaKuUaq2UqgPcC6y2UyxCuJTp/f3xqO1WokymzRCVZZfRUFrrQqXUFOBrjENnP9Ra77FHLEK4Gpk2Q1QFmaJcCCFclCv0WQghhHAikiyEEEJYJclCCCGEVZIshBBCWCXJQgghhFWSLIQQQlglyUIIIYRVkiyEcEBaa4qKiuwdhhBmkiyEsJOyljv19/dnzJgxBAYGcuTIEesHEaKaSLIQ181gMBAYGGjvMJzS1ZY7nTx5Mnv27KFly5b2DrOEnJwc3n33XQA2b97MwIEDK/T+xMREjh49WhWhiWogyUIIO7Bc7tTLy8u83GnLli2Jjo62d3hlskwW10OShXOTZCEq5fLlyzzwwAMEBATQr18/8vPz+eCDD+jUqRMhISHcddddXLhwAYC4uDgmTZpEZGQk7dq1Y+3atYDxl8jgwYOJjY2lbdu2zJw5E4Bnn32WuXPnmj/rqaee4s0336z+StrIqvRsus1KovWMdczd+At7fz93xT7Fa2U7ohkzZvDrr78SGhrK9OnTycvL4+6776Z9+/aMGjWK4nnmnn/+eTp16kRgYCATJ05Ea81nn31Gamoqo0aNIjQ0lPz8fDvXRlSY1trhHxEREVo4nkOHDmk3Nzednp6utdZ6+PDhevHixfrkyZPmfZ566ik9b948rbXWY8eO1f3799eXL1/Wv/zyi/bz89P5+fk6ISFBN23aVJ88eVJfuHBBBwQE6JSUFH3o0CEdFhamtdb68uXLuk2bNiWO7UxW/vSbbv/0l7rlE2t1yyfW6qZj52r3xq300q37dV5eng4ICNA//fSTDggIsHeo5Tp06JA5vk2bNmkfHx995MgRffnyZR0dHa2Tk5O11lqfOnXK/J7Ro0fr1atXa6217tmzp05JSan+wGswIFXb6PewXFmISmndujWhoaEAREREYDAY2L17NzExMQQFBbFkyRL27Plr9vl77rmHWrVq0bZtW9q0acPevXsB6Nu3LzfddBMeHh4MGzaMLVu20KpVK2666SbS09PZsGEDYWFh3HTTTXapZ2VZLnUKxuVObwzow7hhfYmKirK63Kkj6ty5M82bN6dWrVqEhoZiMBgA2LRpE1FRUQQFBZGUlFTi/AvnZZf1LITzWpWebV4noYE+y5/6r0V23NzcyM/PJy4ujlWrVhESEkJiYiKbN28276OUKnG84tfllU+YMIHExER+//13xo8fX0W1qnpHc65sdvHpPJR6nYey22Kp0927d1dnWNek+JwfPmzg9MnzrErPxhdwd3c37+Pm5kZhYSEFBQVMnjyZ1NRUWrRoQXx8PAUFBfYLXtiMXFmIa7YqPZsnP99Fdk4+Gjh+roDj5wpYlV5iRVxyc3Np1qwZly5dYsmSJSW2ffrppxQVFfHrr79y8OBB/P2Nq7d98803nD59mvz8fFatWkW3bt0AGDp0KF999RUpKSn079+/WupZFZx1qVPLc67qeHAx/zxPfr6LLftPlLl/cWJo2LAheXl5fPbZZ+Zt3t7e5ObmVkvcwvbkykJcs9JNKWDs85r99b4Sq7C98MILREVF0ahRI6Kiokr8grjlllvo3Lkz586dY/78+dStWxcwNmncdddd/Pbbb4wePZrISON6LXXq1KFXr174+vri5lZyqVBnMr2/P09+vqvE/58zLHVqec7dPHxw9+vIr/MfZJa7B7Ght16xv6+vLw888ACBgYE0bdqUTp06mbcVD3Dw8PDgxx9/xMPD9onSy8uLvLy8a95/8+bN1KlTh65du9o8FlcjK+WJa9Z6xjrK+mlRwCGLppTyxMXFMXDgQO6+++4S5YmJiaSmpvL2229f8Z6ioiLCw8P59NNPadu27XVG7hgsm/CcZanTyp7z6lbRZBEfH4+XlxePPfZYFUZlP7JSnrCL6m5KycrK4tZbb6VPnz5OnyjAuDb21hm9OTTrDrbO6O3wiQIcr/ls9uzZzJs3D4CpU6fSu3dvAJKSkhg1ahRgHGIdEhJCdHQ0x48fB2DNmjVERUURFhbGbbfdxvHjxzEYDMyfP5833niD0NBQkpOT7VInZyHJQlyz6f398ahdsimoIk0piYmJV1xVgPGKo6yrio4dO3Lw4EFee+216wtYVFplz7mtxcTEmH+pp6amkpeXx6VLl0hOTqZHjx6cP3+e6Ohodu7cSY8ePfjggw8A6N69O9u2bSM9PZ17772XV199lVatWjFp0iSmTp1KRkYGMTExdqmTs5A+C3HNiv8SdramFHH9HOWcFzfhZZ/K5fdvt/Jx8l7c3d0JDw8nNTWV5ORk5s2bR506dczTkERERPDNN98A8NtvvzFixAiOHTvGxYsXad26dbXG7wokWYgKGRLmJ8mhhrH3OS8ekZV/6TK43YDyacTUF9+kZ5tAYmK6s2nTJg4cOECHDh2oXbu2edh18XBegEceeYRp06YxaNAgNm/eTHx8vN3q46ykGUoI4dCuuKGxeQCnflzBniI/YmJimD9/PmFhYVfcq2Pp7Nmz+PkZE97ChQvN5TKc99pJshBCOLTSNzS6Nw/g8vnT5Pm0oUmTJtStW9dqf0N8fDzDhw8nIiKChg0bmsvvvPNOVq5cKR3c10CGzgohHFq3WUlkl3EHvJ+vB1tn9LZDRM5Dhs4KIWoMRxuRVVNJB7cQwqE5yoismk6ShRDC4dl7RJaQZighhBDXQJKFMEtMTGTKlCn2DkMIYQOxsbEANwIopa59wqxySLIQQghhlSQLF2YwGAgMDDS/njNnDvHx8cTGxvLEE0/QuXNn2rVrV2J8+dGjRxkwYABt27bl8ccfN5c/9NBDREZGEhAQwHPPPWcub9WqFY8//jhBQUF07tyZAwcOAOWvt92jRw8yMjLM7+/evTs7d+6ssv8DIZydtckTy/tulkUp1VAp9aNSqsJTBkuyqKEKCwvZsWMHc+fOZebMmebyjIwMli1bxq5du1i2bBlHjhwB4KWXXiI1NZXMzEy+++47MjMzze+pV68eu3btYsqUKfzjH/8wlxsMBnbs2MG6deuYNGkSBQUF/O///i+JiYkA/PLLLxQUFBASElI9lRbCCVmbPPFq301LSqkmwDrgWa31uorGIcmihho2bBjw17rZxfr06UO9evWoW7cuHTt25PDhwwAsX76c8PBwwsLC2LNnD1lZWeb3jBw50vzvjz/+aC4va73t4cOHs3btWi5dusSHH35IXFxc1VdWCCcWERFBWloa586dw93dnS5dupgnT4yJibnqd9NCbeBb4HGt9TfXE4cMnXUxlgvs3KTyOHvhonmb5VrIxesnW062Zlluue3QoUPMmTOHlJQU6tevT1xcXIljWc7JU97z4tc33ngjffv25YsvvmD58uWkpaXZoNZCuB7L7/KZWr5Me3EuXbt2JTg42Dx5ooeHx1W/mxYKgTSgP/Dd9cQjVxYupPQa2ScK63Ls9+Ms2rSbP//809xvUFHnzp3D09OTevXqcfz4cb788ssS25ctW2b+t0uXLuby8tbbnjBhAn//+9/p1KkT9evXv77KCuHCSn+Xadqehe+9jdvNHUtMnmjtu2lBA+OB9kqpJ64nJrmycCGlZ+dUbjfg0/VeJt7dj/8LuJX27dtf13FDQkIICwujffv2tGjRgm7dupXYfubMGYKDg3F3d+eTTz4xl5e33nZERAQ+Pj6MGzfuuuIRwtWVNdPu2R+X8+Uf3jxuBA8SAAAS+ElEQVRnMXmite+mJa31ZaXUSGC1UipXa/1uRWKSiQRdiD3WS27VqhWpqaklZvKE8tfbBuOIq9jYWPbu3UutWnJxK0RptvouO8xEgkqp2UqpvUqpTKXUSqWUr8W2J5VSB5RS+5RS/S3KB5jKDiilZlTm80VJjrZeclkWLVpEVFQUL730kiQKB3L77beTk5Nz1X1iY2Mp64+2jIwM1q9fX1Wh1UiO+F2u7Lf1GyBQax0M/AI8CaCU6gjcCwQAA4B3lVJuSik34B3gb0BHYKRpX2ED9pid02AwXHFVAeWvtz1mzBiOHDnC8OHDqywmUTFaa9auXYuvr6/1ncsgycL2HHGm3UolC631Bq118VCabUBz0/PBwFKt9Z9a60PAAaCz6XFAa31Qa30RWGraV9jAkDA/Xh4WhJ+vBwrjfP8vDwuSCdjEFQwGA/7+/owZM4bAwEDc3Nw4efIkAC+88AL+/v50796dkSNHMmfOHPP7Pv300xI3c168eJFnn32WZcuWERoaah7sICrHEb/LtuzgHg8U/6T4YUwexX4zlQEcKVUeVdbBlFITgYlg7CgV10Zm5xTXav/+/SxcuJDo6GhatWoFQEpKCitWrGDnzp1cunSJ8PBwIiIizO8pvplz/fr1zJw5k40bN/L888+TmprK22+/baeauCZH+y5bTRZKqY1A0zI2PaW1/sK0z1MYx/EusVVgWuv3gffB2MFtq+MKIYxatmxJdHR0ibKtW7cyePBg6tatS926dbnzzjtLbC/vZk7h+qwmC631bVfbrpSKAwYCffRfQ6uygRYWuzU3lXGVciFEFbK8yauBPstlN3frbyqlvJs5heur7GioAcDjwCCt9QWLTauBe5VS7kqp1kBbYAeQArRVSrVWStXB2Am+ujIxCCGsK32T1/FzBRw/V8Cq9JJ/q3Xr1o01a9ZQUFBAXl7eNd3I6e3tTW5ubhVFLhxFZUdDvQ14A98opTKUUvMBtNZ7gOVAFvAV8LDW+rKpM3wK8DXwM7DctK8QTk9rTVFRkb3DKFPpm7zAGO/sr/eVKOvUqRODBg0iODiYv/3tbwQFBVGvXr2rHrtXr15kZWVJB7eLk5vyhKgEg8FA//79iYqKIi0tjaysLIq/U5999hlr164lMTGRuLg4fHx8SE1N5ffff+fVV18tc2hxVanITV55eXl4eXlx4cIFevTowfvvv094eHi1xClsy5Y35cl0H0JUkuWoIi8vr3L3O3bsGFu2bGHv3r0MGjSoWpPFzb4eZOfkl1le2sSJE8nKyqKgoICxY8dKohCAJAshKq2sUUVlGTJkCLVq1aJjx44cP368GiL7y/T+/jz5+a4STVHl3eT18ccfV2dowklIshCigq42qshyWvbSU0VbTv9e3c2/xeP1i+O+2deD6f39HWocv3BskiyEqIDiUUXFf6EfP1fACdOooiFhfjRp0oSff/4Zf39/Vq5cibe3t50j/ouj3eQlnIvM5CZEBVgbVTRr1iwGDhxI165dadasmT1CFKJKyGgoISrAHtPAC3G9HGaKciFqGkecOlqI6iDJQogKcMSpo4WoDtLBLUQFyKgiUVNJshCigmRUkaiJpBlKCOEyHHl+LmcnVxZCCIczY8YMWrRowcMPPwxAfHw8Xl5eaK1Zvnw5f/75J0OHDmXmzJlXzM91zz33cObMGebOnQvABx98QFZWFm+88YY9q+T05MpCCOFwRowYwfLly82vly9fTqNGjdi/fz87duwgIyODtLQ0vv/+e8A4P9fkyZPZs2cP//znP1mzZg2XLl0CICEhgfHjx9ulHq5EriyEEA4nLCyMP/74g6NHj3LixAnq16/Prl272LBhA2FhYYBxdtz9+/dzyy23lJify8vLi969e7N27Vo6dOjApUuXCAoKsmd1XIIkCyGEw7Ccd6vw5kienft/NL6hgBEjRnD48GGefPJJHnzwwRLvMRgMeHp6liibMGEC//73v2nfvj3jxo2rziq4LEkWQgiHUHrercutuvDxJ2/jW6uAtG1b2bVrF8888wyjRo3Cy8uL7OxsateuXeaxoqKiOHLkCD/99BOZmZnVWQ2XJclCCOEQSs+7VadRSwoLLnDex5dmzZrRrFkzfv75Z7p06QIYm5s++ugj3NzcyjzePffcQ0ZGBvXr16+W+F2dJAshhEM4WtbiTP/7Dsri9aOPPsqjjz56xX67d+++omzLli1MnTrVliHWaDIaSgjhEGw171ZOTg7t2rXDw8ODPn362CI0gVxZCCEcREVW87saX19ffvnlF1uHV+NJshBCOASZd8uxSbIQQjgMmXfLcUmfhRBCCKskWQghhLBKkoUQQgirJFkIIYSwSpKFEEIIqyRZCCGEsEqShRBCCKskWQghhLBKkoUQQgirJFkIIYSwSpKFEEIIqyRZCCGEsEqShRBCuJi5c+dy4cIFmx5TkoUQQrgYSRZCCFFDffTRR3Tu3JnQ0FAefPBBLl++zEMPPURkZCQBAQE899xzAMybN4+jR4/Sq1cvgHa2+nxJFkII4eB+/vlnli1bxtatW8nIyMDNzY0lS5bw0ksvkZqaSmZmJt999x2ZmZn8/e9/5+abb2bTpk0ANlsy0CaLHyml/gnMARpprU8qpRTwJnA7cAGI01r/ZNp3LPC06a0vaq0X2iIGIYRwJavSs82rBqqsr8nZlkKnTp0AyM/Pp3Hjxixfvpz333+fwsJCjh07RlZWFsHBwVUST6WThVKqBdAP+K9F8d+AtqZHFPAfIEop1QB4DogENJCmlFqttT5T2TiEEMJVrErPLrEe+dn8i6h2PYl/+3XzSoKHDh2ib9++pKSkUL9+feLi4igoKKiymGzRDPUG8DjGX/7FBgOLtNE2wFcp1QzoD3yjtT5tShDfAANsEIMQQriM2V/vMycKgLotQzj3czIvrdgGwOnTp/nvf/+Lp6cn9erV4/jx43z55Zfm/b29vcnNzbVpTJVKFkqpwUC21npnqU1+wBGL17+ZysorL+vYE5VSqUqp1BMnTlQmTCGEcCpHc/JLvK7T8BZ8Y+5n5wfTCQ4Opm/fvri7uxMWFkb79u2577776Natm3n/iRMnMmDAALBhB7fVZiil1EagaRmbngL+hbEJyua01u8D7wNERkZqK7sLIYTLuNnXg+xSCcOzQw/adenP1hm9zWXR0dFlvv+RRx7hkUceQSllsw5uq1cWWuvbtNaBpR/AQaA1sFMpZQCaAz8ppZoC2UALi8M0N5WVVy6EEMJken9/PGq7lSjzqO3G9P7+doqoEs1QWutdWuvGWutWWutWGJuUwrXWvwOrgTHKKBo4q7U+BnwN9FNK1VdK1cd4VfJ15ashhBCuY0iYHy8PC8LP1wMF+Pl68PKwIHPntj3YZOhsGdZjHDZ7AOPQ2XEAWuvTSqkXgBTTfs9rrU9XUQxCCOG0hoT52TU5lGazZGG6uih+roGHy9nvQ+BDW32uEEKIqid3cAshhLBKkoUQQgirJFkIIYSwSpKFEFYcOXKEXr160bFjRwICAnjzzTftHZIQ1a6qRkMJ4TJuuOEGXnvtNcLDw8nNzSUiIoK+ffvSsWNHe4cmRLWRKwshrGjWrBnh4eGAcc6dDh06kJ0t95KKmkWShRAVYDAYSE9PJyoqyt6hCFGtJFkIcY3y8vK46667mDt3Lj4+PtXymePHj6dx48YEBgZWy+cJUR7psxCiHJaLzzT1rk3B2n9z/6hRDBs2rNpiiIuLY8qUKYwZM6baPlOIskiyEKIMlovPaK3Z9cks6nj60qbXiGqNo0ePHhgMhmr9TCHKIs1QQpTBcvGZP7OzOL9nE3mHdjLqjp6Ehoayfv16O0coRPWSKwshymC5+Ezd5gG0fGItAArImHWHnaISwn4kWQhRhrIWnykur2qWfSU3+3owNujGKv9MIayRZighymCvxWeK+0qyc/LRQHZOPq98tY9zBYVV+rlCWCPJQogy2GvxGcu+EoATq1/lcMJUfjt0gObNm7NgwYIq/XwhyiPNUEKUwx6Lzxwt1fTVaNDjgLGv5JD0lQg7kisLIRxIeX0i1dFXIuwvPj6eOXPmXFE+f/58Fi1adNX3JiYmMmXKlKoKTa4shHAk0/v7m+/vKFYdfSXCcRUWFjJp0iR7hyHJQghHUtzsZTkaanp/f4dai1nY1ksvvcTChQtp3LgxLVq0ICIigtjYWEJDQ9myZQsjR44kNzcXLy8vHnvsMWJjY4mKimLTpk3k5OSwYMECYmJiShxz3bp1vPjii2DD3/GSLIRwMPboKxH2kZaWxtKlS8nIyKCwsJDw8HAiIiIAuHjxIqmpqYCxecpSYWEhO3bsYP369cycOZONGzeat61cuZLXX3+d9evX06BBA5sNo5NkIYQQdpKcnMzQoUO58UbjvTSDBg0ybxsxovypZYrnJ4uIiCgxHUxSUhKpqals2LDB5pNdSge3EEJUs1Xp2XSblcTza7JI2HqIVelXro/i6elZ7vvd3d0BcHNzo7Dwr4uH//mf/yE3N5dffvnF5jFLshBCiGpkeeOle4sAju/awhPLUvlkyz7WrFlTqWO3bNmSFStWMGbMGPbs2WOjiI2kGUoIIaqR5Y2X7k1vxbN9DAffn8yDSxswsGunSh+/ffv2LFmyhOHDhwO4V/qAJkprbatjVZnIyEhd3NEjhBDOrPWMdZT1W7cqbrxUSqVprSNtcSxphhJCiGrkrDdeSrIQQohqZK9JKitL+iyEEKIaOeuNl5IshBCimjnjjZfSDCWEEMIqSRZCCCGskmQhhBDCKkkWQgghrJJkIYQQwiqnuINbKXUCOFzFH9MQOFnFn2FvNaGOUDPqKXV0HVVZz5Za60a2OJBTJIvqoJRKtdVt8Y6qJtQRakY9pY6uw1nqKc1QQgghrJJkIYQQwipJFn95394BVIOaUEeoGfWUOroOp6in9FkIIYSwSq4shBBCWFVjk4VS6hGl1F6l1B6l1KsW5U8qpQ4opfYppfpblA8wlR1QSs2wT9QVp5T6p1JKK6Uaml4rpdQ8Uz0ylVLhFvuOVUrtNz3G2i/qa6OUmm06h5lKqZVKKV+LbS51Hi25Qh0AlFItlFKblFJZpu/ho6byBkqpb0w/h98opeqbysv92XV0Sik3pVS6Umqt6XVrpdR2U12WKaXqmMrdTa8PmLa3smfcJWita9wD6AVsBNxNrxub/u0I7MS4FGFr4FfAzfT4FWgD1DHt09He9biGerYAvsZ4j0pDU9ntwJcYF+aKBrabyhsAB03/1jc9r2/vOlipXz/gBtPzV4BXXPE8lqqz09fBoi7NgHDTc2/gF9O5exWYYSqfYXFey/zZdYYHMA34GFhrer0cuNf0fD7wkOn5ZGC+6fm9wDJ7x178qKlXFg8Bs7TWfwJorf8wlQ8Glmqt/9RaHwIOAJ1NjwNa64Na64vAUtO+ju4N4HEosYrjYGCRNtoG+CqlmgH9gW+01qe11meAb4AB1R5xBWitN2itC00vtwHNTc9d7TxacoU6AKC1Pqa1/sn0PBf4GfDDWJ+Fpt0WAkNMz8v72XVoSqnmwB3A/5leK6A38Jlpl9J1LK77Z0Af0/52V1OTRTsgxnSZ951SqniVdD/giMV+v5nKyit3WEqpwUC21npnqU0uU8dSxmP8qxNct47gGnW4gqm5JQzYDjTRWh8zbfodaGJ67qx1n4vxj7Yi0+ubgByLP3Qs62Guo2n7WdP+dueyix8ppTYCTcvY9BTGejfAeCnbCViulGpTjeHZhJU6/gtjM41Tu1odtdZfmPZ5CigEllRnbMI2lFJewArgH1rrc5Z/SGuttVLKaYdsKqUGAn9ordOUUrH2jqcyXDZZaK1vK2+bUuoh4HNtbBjcoZQqwjg/SzbGdv5izU1lXKXcbsqro1IqCGNb/U7TF6858JNSqjPl1zEbiC1VvtnmQVfQ1c4jgFIqDhgI9DGdT3Cy81hBV6ub01FK1caYKJZorT83FR9XSjXTWh8zNTMVNxM7Y927AYOUUrcDdQEf4E2MTWg3mK4eLOtRXMfflFI3APWAU9Ufdhns3WlijwcwCXje9Lwdxss+BQRQsmP0IMYOxRtMz1vzV6digL3rUYH6Gvirg/sOSnYS7jCVNwAOYezcrm963sDesVup1wAgC2hUqtwlz6Opbk5fB4u6KGARMLdU+WxKdnC/anpe5s+uszww/jFW3MH9KSU7uCebnj9MyQ7u5faO2xy/vQOw00mrA3wE7AZ+AnpbbHsK42iTfcDfLMpvxzha41eMTSB2r0cF6muZLBTwjqkeu4BIi/3GY+wMPgCMs3fc11CvA6ZEn2F6zHfl8+hKdTDVozvGwReZFufwdoxt9N8C+zGOWmxg2r/cn11neJRKFm2AHaaf4U/5a2RmXdPrA6btbewdd/FD7uAWQghhVU0dDSWEEKICJFkIIYSwSpKFEEIIqyRZCCGEsEqShRBCCKskWQghhLBKkoUQQgirJFkIIYSw6v8B7qL07sYYcoIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f21722c58d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.manifold import TSNE\n",
    "\n",
    "\n",
    "def plot_all_words(words, vectors):\n",
    "    plt.scatter(vectors[:,0], vectors[:,1])\n",
    "    for i, word in enumerate(words):\n",
    "        plt.annotate(s=word, xy=vectors[i], xytext=vectors[i]+(0.01,0.005))\n",
    "\n",
    "\n",
    "# To reduce computational cost, we'll just work with the word vectors we're plotting\n",
    "en_word_indices = [en_wv.vocab[w].index for w in en_words]\n",
    "en_vectors_words = en_wv.vectors[en_word_indices]\n",
    "\n",
    "# For this small data, needed to tweak perplexity\n",
    "en_vectors_tsne = TSNE(perplexity=10.0, random_state=1234).fit_transform(en_vectors_words)\n",
    "plot_all_words(en_words, en_vectors_tsne)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAD8CAYAAABZ/vJZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xtc1FX++PHX4aKw3ihTEy/JlonACIMoKUuSNyxZNS9puSWaWppddzVst3u7S+kv28ra9GugaaWZoqKl5t2sBAMRUURyuqCRWiAqGOD5/cEwOyAqDpeZgffz8ZiHM+dze38Q5v0553M+5yitNUIIIcS1crF3AEIIIZyTJBAhhBA2kQQihBDCJpJAhBBC2EQSiBBCCJtIAhFCCGETSSBCCCFsIglECCGETSSBCCGEsImbvQOojhtuuEF36dLF3mEIIYRT2bdv3ymtdZu62n+tJRCllCuQDORoraOUUj7Ax0BrYB9wv9b6d6VUU2AJ0BM4DYzVWpuutO8uXbqQnJxcW6EKIUSjoJT6vi73X5tNWI8Dh6w+vwrM01rfAvwGPGgufxD4zVw+z7yeEEIIJ1MrCUQp1REYCvyf+bMC+gMrzassBkaY3w83f8a8fIB5fSGEcBgmk4mAgAAAtm/fTlRUFABr164lNjbWnqE5jNpqwnoDmAW0MH9uDeRprUvMn38COpjfdwB+BNBalyil8s3rn6qlWIQQos4MGzaMYcOG2TsMh1DjGohSKgr4RWu9rxbisd7vVKVUslIq+eTJk7W5a6fw448/cscdd+Dn54e/vz//+c9/7B2SEA7n9ddfJyAggICAAN544w1MJhPdu3dnypQp+Pv7M3jwYAoLCwGIiIjg6aefpnfv3tx6663s2rULgNLSUmbOnEmvXr3o0aMH77333hWPGR8fz4wZMwCIjo7mscceo2/fvvzxj39k5cqVlvXmzJlj2efzzz8PcMX4nFFtNGGFAcOUUibKbpr3B/4DeCmlyms4HYEc8/scoBOAeXkrym6mV6C1XqC1DtFah7RpU2edCByWm5sb/+///T8yMjL4+uuvmT9/PhkZGfYOSwiHsW/fPuLi4vjmm2/4+uuvWbhwIb/99htZWVk88sgjHDx4EC8vLz799FPLNiUlJezdu5c33niDF198EYBFixbRqlUrkpKSSEpKYuHChRw7dqzacZw4cYLdu3eTmJhITEwMAJs2bSIrK4u9e/eSmprKvn372LlzJ8AV43M2NU4gWuvZWuuOWusuwDhgq9Z6PLANGG1ebQKwxvx+rfkz5uVbtcxqdYn27dsTHBwMQIsWLejevTs5OTlX2Uo0ZlprLl68aO8w6lxCSg5hsVsZ/PR75LUNYvORPJo3b87IkSPZtWsXPj4+BAUFAdCzZ09MJpNl25EjR15SvmnTJpYsWUJQUBChoaGcPn2arKysasczYsQIXFxc8PPzIzc317LPTZs2YTQaCQ4O5vDhw5Z9Xik+Z1OXz4E8DXyslHoFSAEWmcsXAR8opY4Cv1KWdMQVmEwmUlJSCA0NtVsMkyZNIjExkbZt25Kenm63OERFJpOJyMhIQkND2bdvHxkZGZRfj61cuZLExETi4+OJjo7G09OTlJQUfvnlF95//32WLFnCV199RWhoKPHx8QBMmzaNpKQkCgsLGT16tOUq3VEkpOQwe9UBCotL0UBBUQmzVx2osE7Tpk0t711dXSs0EZUvc3V1paSk7Bat1pq33nqLyMjICvup7he79fHKf/Zaa2bPns1DDz10yT6vFJ+zqdUn0bXW27XWUeb332mte2utb9Faj9FaXzCXF5k/32Je/l1txuDMyq+sfGLWExa7lYSUHM6ePcuoUaN44403aNmypd1ii46O5vPPP7fb8cXlZWVlMX36dA4ePEizZs0uu95vv/3GV199xbx58xg2bBhPPvkkBw8e5MCBA6SmpgLwz3/+k+TkZNLS0tixYwdpaWn1dRrVMmdjJoXFpQA07ejP+ayvOXf+HLHrUlm9ejXh4eHXvM/IyEjeffddiouLAThy5Ajnzp2rUZyRkZG8//77nD17FoCcnBx++eWXGu3TETnFk+iNgfWVFUBOXiExn6TguW0u948fb6l628vtt9/u1FXthuymm27itttuu+p6f/7zn1FKYTAYaNeuHQaDAQB/f39MJhNBQUGsWLGCBQsWUFJSwokTJ8jIyKBHjx51fQrVdjzPqjZx4y00DxjAz0ue4mfgtb8/yXXXXXfN+5w8eTImk4ng4GC01rRp04aEhIQaxTl48GAOHTpEnz59AGjevDlLly7F1dW1Rvt1NMoZbj+EhITohv4keljsVnKs/ji01pxe/zotW13H8S8d4yabyWQiKipKmrDsLCElhzkbMzmeV8j1Op+Tn77ED0cPA2X3ywoKCgBYunQpX3zxhaUJKyoqitGjR1/y/1i+rGfPngwaNIikpCSuu+46oqOjiYiIIDo62l6neonKfyflOnh58mVMfztE5NiUUvu01iF1tX8ZTNFBHK/0R3EhJ4NzB7dxKutbgoKCCAoKYsOGDfUaU+UmtU0Hf67X44tLlddUc/IK0UDumSJyzxSRkFLWwaJdu3YcOnSIixcvsnr16mva95kzZ2jWrBmtWrUiNzeXzz77rA7OoGZmRnbD073iVbynuyszI7vZKaLGrdEkEOunSq/G+qnTqzl+/DijR4+++opX4e3lWeGzR0d/bno6kV5PLSI1NZXU1FTuuuuuGh+nuip/UeXkFfLq55mcKSq56rai7ljfAyintWbOxkwAYmNjiYqKom/fvrRv3/6a9h0YGIjRaMTX15f77ruPsLCwWou7towwduDfIw108PJEUVbz+PdIAyOMHa66rah9jaYJ61qaX7Zv387cuXNJTEys0TGvReV7IFB2ZWWvP46qmgpK8nP5dfXLnP+5+n3kRe3yiVlPVX+xCjgWO7S+wxEOTpqw6sB3332H0WgkKSmJ8PBwgoODCQ4OZs+ePZesm5SUhNFoJDs7mx07dliak4xGIwUFBddUs7kSR7uyqtykdnLta/z8wd8oPPkjHTt2ZNGiRZfZUtSlyjXVq5ULUZcaXS+szMxMxo0bR3x8PF27dmXz5s14eHiQlZXFvffeW2HY+D179vDoo4+yZs0aOnfuzBNPPMH8+fMJCwvj7NmzeHh41GpsI4wdHKYq7u3lWaEG0mbYLEBuVtrbzMhuVdZU5R6AsIdGVQM5efIkw4cPZ9myZQQGBlJcXMyUKVMwGAyMGTOmwlAhhw4dYurUqaxbt47OnTsDEBYWxlNPPcWbb75JXl4ebm4NN//KzUrH5Gg1VdG4NdxvQC7t7ujq0YzOnTuze/du/Pz8mDdvHu3atWP//v1cvHixQo2iffv2FBUVkZKSgre3NwAxMTEMHTqUDRs2EBYWxsaNG2u9FuIoyr+Qyn9+3l6ezIzsJl9UDsCRaqqicWuwCaTyTencM0WcLrzIpBfm8/asiTRv3pz8/Hw6duyIi4sLixcvprT0f80CXl5eLFq0iEGDBtGsWTMiIiLIzs7GYDBgMBhISkri8OHDljFtGiL5ohJCXEmDbcK6XHfHt3b+SGJiIvPmzaNLly4sXryYwMBADh8+fMkwEO3atSMxMZFHHnmEb775hjfeeIOAgAB69OiBu7s7d955Z32ekhBCOJQG241XujsK4RxMJhN79uzhvvvus3coDY5047WRdHcUwjmYTCY+/PBDe4chbNBgE4j0IhKi9tTlzH8xMTHs2rWLoKAg5s2bR1FRERMnTsRgMGA0Gtm2bRtQNhPgyJEjGTJkCF27dmXWrFn2+WEIiwabQKS7oxC1o65n/ouNjSU8PJzU1FSefPJJ5s+fj1KKAwcO8NFHHzFhwgSKiooASE1NZfny5Rw4cIDly5fz448/2uVnIso02F5YIL2IhKgNu3fv5u6777Z0MqnJzH9paWmWecPz8/PJysqiSZMmlxzv0UcfBcDX15ebbrqJI0eOADBgwABatWoFgJ+fH99//z2dOnWqmxMXV9WgE4gQwjbWz1Bx8Ai92rtfsk5tzfy3ffv2asdV+Zjl+xb20WCbsIQQtqk8EnNR61tZu2YNy/cc5dy5c7U+85/1HCYA4eHhLFu2zLLODz/8QLducu/SEUkNRAhRQeVnqJreeAt/8B/AxJGD+OMNzZg8eXKtzvzXo0cPXF1dCQwMJDo6munTpzNt2jQMBgNubm7Ex8dXqHkIx9FgnwMRQthGnqFqOOQ5ECFEvZJnqER1SQIRQlQgz1CJ6pJ7IEKICmQkZlFdkkCEEJeQZ6hEdUgTlhCiUYmIiEA65dQOSSBCCCFsIglECOHUTCYTvr6+jB8/nu7duzN69GjOnz/PSy+9RK9evQgICGDq1KlYP7LwySefXDLYo8lkIjw8nODgYIKDg9mzZ4+9TslpSAIRQji9zMxMpk+fzqFDh2jZsiXvvPMOM2bMICkpifT0dAoLC0lMTLSsX9Vgj23btmXz5s18++23LF++nMcee8xep+M05Ca6EMLpderUibCwMAD+8pe/8Oabb+Lj48Nrr73G+fPn+fXXX/H39+fPf/4zUPVgj8XFxcyYMYPU1FRcXV0tAziKy5MEIoRwOtaDPV6v8ykqvlhhuVKK6dOnk5ycTKdOnXjhhRcsQ8JD1YM9zps3j3bt2rF//34uXryIh4dH/Z2Qk5ImLCFqwfbt24mKiqrWusePH2f06NF1HFHDVXmwx9wzRZz8OYfY+LUAfPjhh/zpT38C4IYbbuDs2bOWIeSvJD8/n/bt2+Pi4sIHH3xAaWnpVbdp7CSBCFHPvL29q/WFJqpWebBHALfrO/L//vMm3bt357fffmPatGlMmTKFgIAAIiMj6dWr11X3O336dBYvXkxgYCCHDx+2zH8iLk8GUxTiGj333HNcf/31PPHEEwD8/e9/t9yATUxMJCkpialTp7Jy5Up++uknHn/8caCsWWXnzp2cPn2aqKgo0tPT7XkaTqvyYI8l+bn8svJFOjz4jgz2WIkMpiicWl5eHu+8885V1+vbt289RFM7Jk2axJIlSwC4ePEiH3/8MR07dgRgz549PPzww6xZs4abb76ZuXPnMn/+fFJTU9m1axeenjIgYU3JYI+OQxKIqFPVTSDO1Oe+S5cutG7dmpSUFDZt2oTRaKR169YcOnSIqVOnsm7dOjp37gxAWFgYTz31FG+++SZ5eXm4uUm/lZqqPNijW6t23PzwezLYox3Ib7OoUzExMWRnZxMUFMQdd9xBWloav/32G8XFxbzyyisMHz4cgObNm3P27Fk7R3tl1j1/mrS5jWfnvE2z0rNMmjQJgPbt21NUVERKSgre3t5A2fkPHTqUDRs2EBYWxsaNG6V3Tw3JYI+OQxKIqFOxsbGkp6eTmppKSUkJ58+fp2XLlpw6dYrbbruNYcOGoZSyd5hXVd7zp/zmbVGHnmx+P47rPF358MMP2bVrF15eXixatIhBgwbRrFkzIiIiyM7OxmAwYDAYSEpK4vDhwwQFBdn5bJyfDPboGCSBiDpRfrX+/fcmfj11joSUHIYGtOWZZ55h586duLi4kJOTQ25uLjfeeKO9w72qyj1/lKs7TTobcGvlhavr/5pT2rVrR2JiInfeeSfvv/8+S5cuZdu2bbi4uODv78+dd97JiRMn7HEKQtS6GicQpVQnYAnQDtDAAq31f5RS1wPLgS6ACbhHa/2bKrvc/A9wF3AeiNZaf1vTOITjqHy1XlJ6kdmrDrB17SFOnjzJvn37cHd3p0uXLhUe7nJkx/MKK3zW+iIXjmdCrxigbITXiIgIADp37szBgwcBCA0NvWRfXbp0cfoeWCaTqVZ6kq1du5aMjAy8vb05f/48Dz/8cC1FKOpDbdRASoC/aq2/VUq1APYppTYD0cAWrXWsUioGiAGeBu4EuppfocC75n9FA2F9ta6aeHLx90IKi0tZl3qUqC5tcXd3Z9u2bXz//fd2jrT6vL08yTEnkd9P/cDJlS/ieWsfbvrjLXaOzLkNGzaMYcOG2TsMYaMa98LSWp8or0ForQuAQ0AHYDiw2LzaYmCE+f1wYIku8zXgpZRqX9M4hOOwvlp39WxJ0w5+HF80nVOmIyQnJ2MwGFiyZAm+vr52jPLaWPf8aXJDZzo8vIgOkQ816p4/paWlTJkyBX9/fwYPHszBgwcJDg62LM/KyiI4OJjPP/+cMWPGWMqtn9qfOnUqISEh+Pv78/zzz9f7OYiaqdV7IEqpLoAR+AZop7Uub+z9mbImLihLLj9abfaTuUwahhsI66t1gDbDZgLQwcuTL2P6V7mNo/fAkp4/l8rKyuKjjz5i4cKF3HPPPaSkpNCqVStSU1MJCgoiLi6OiRMnMnDgQKZOncq5c+do1qwZy5cvZ9y4cUBZJ4vrr7+e0tJSBgwYQFpaGj169LDzmYnqqrUEopRqDnwKPKG1PmPds0ZrrZVS1/TIu1JqKjAVsPSpF85hZmS3CvdAADzdXZ3+ar2x9/ypPIBhW+9Olh5l5aPaTp48mbi4OF5//XWWL1/O3r17cXNzY8iQIaxbt47Ro0ezfv16XnvtNQBWrFjBggULKCkp4cSJE2RkZEgCcSK18iChUsqdsuSxTGu9ylycW940Zf73F3N5DtDJavOO5rIKtNYLtNYhWuuQNm3a1EaYop6MMHbg3yMNdPDyRFFW8/j3SEOj/vJ1dlUNYHi6SJOQUvanWz6q7ahRo/jss89ITEykZ8+etG7dGoBx48axYsUKtm7dSkhICC1atODYsWPMnTuXLVu2kJaWxtChQ52mU4UoUxu9sBSwCDiktX7datFaYAIQa/53jVX5DKXUx5TdPM+3auoSDURjv1pvaKoawFBrzZyNmRX+nz08PIiMjGTatGksWrTIUt6vXz8mTZrEwoULLc1XZ86coVmzZrRq1Yrc3Fw+++wzS0824RxqowYSBtwP9FdKpZpfd1GWOAYppbKAgebPABuA74CjwEJgei3EIISoQ5W7MV+pfPz48bi4uDB48GBLmaurK1FRUXz22WeWG+iBgYEYjUZ8fX257777LBNCCecho/EKIa4qLHZrhY4R5arqGDF37lzy8/N5+eWX6ys8cRl1PRqvPIkuhLiq6naMuPvuu8nOzmbr1q31HaKwA0kgQoirqm435tWrV9sjPGEnkkCE0+jbt69TDfve0EjHCFGZzAcinIYkDyEciyQQ4TTK5wwZMGAAwcHBGAwG1qxZc/UNhRB1QpqwhFPx8PBg9erVTjmniBANjSQQ4VS01k47p4gQDY0kEOHQrMdfKiwu5a//ettp5xQRoqGReyDCYVUef0lrWL4nk7PqD045p4gQDY3UQITDumT8JaVo0u12dia8gsFgICQkxKnmFBGioZEEIhyW9ThLpYVncPFojusfWnHDfXM4EDvUjpEJIUCasIQD8/byBKCk4DQ/f/A3WvYeWaFcCGFfkkCEwyqfRtatRWs6TF1Ay55/bhATUwnRUEgCaYBMJhMBAQE13s9dd91FXl5eLURkG5mYSgjHJvdAxGVt2LDB3iHI+EtCODCpgTRQpaWlTJkyBX9/fwYPHszBgwcJDg62LM/KyiI4OJjPP/+cMWPGWMq3b99umfCnS5cunDp1CpPJRPfu3Svsr7Cw6gmGhBCNhySQBiorK4tHHnmEgwcP4uXlRUpKCq1atSI1NRWAuLg4Jk6cyMCBA/nmm284d+4cAMuXL7dMOXql/X366af1ej5CCMcjCaQBSUjJISx2K396dSvuXjdi0m0A6NmzJyaTicmTJxMXF0dpaSnLly/nvvvuw83NjSFDhrBu3TpKSkpYv349w4cPv2TfPj4+BAUFVdifEKJxk3sgDUT5U9vlD96VKldmrzoAlM1HXVhYyKhRo3jxxRfp378/PXv2pHXr1gCMGzeOt99+m+uvv56QkBBatGhxyf6bNm1qeV++PyFE4yY1kAbikqe2gcLiUuZszLR89vDwIDIykmnTpjFx4kRLeb9+/fj2229ZuHBhlc1XQghRFUkgDYT1U9tXKh8/fjwuLi4MHjzYUubq6kpUVBSfffaZ5Qa6EEJcjdJa2zuGqwoJCdHJycn2DsOhhcVuJaeKJNLBy5MvY/pbPs+dO5f8/Hxefvnl+gxPCGEHSql9WuuQutq/3ANpIGZGdqtwDwS45Kntu+++m+zsbLZu3WqPEIUQDYw0YTUQ1Xlqe/Xq1aSlpXHDDTfUa2x5eXm8884717RNdHQ0K1eurPY6ERERSC1ViPolNZAGxFGf2i5PINOnT7d3KEKIWiQ1EFHnYmJiyM7OJigoiJkzZzJz5kwCAgIwGAwsX74cKJuqdsaMGXTr1o2BAwfyyy+/WLbft28f/fr1o2fPnkRGRnLixIkrHm/atGmEhITg7+/P888/X6fnJoQ9HT9+nNGjR5ORkcHkyZOvur5SKlop5W31+Qml1B+sPg9USr2klBqmlIq56g611g7/6tmzpxbO69ixY9rf319rrfXKlSv1wIEDdUlJif755591p06d9PHjx/Wnn35qKc/JydGtWrXSn3zyif799991nz599C+//KK11vrjjz/WEydO1FprPWHCBP3JJ59orbXu16+fTkpK0lprffr0aa211iUlJbpfv356//799X3KQjgEIFlbfZcC24EQq88m4AZt43ez1EBEvdq9ezf33nsvrq6utGvXjn79+pGUlMTOnTst5d7e3vTvX9ZzLDMzk/T0dAYNGkRQUBCvvPIKP/300xWPsWLFCoKDgzEajRw8eJCMjIz6ODUh6szlxqP773//S69evQgMDGTUqFGcP3+egoICfHx8KC4uBkAp1VIpdUwpNQYIAZYppVKVUo8D3sA2pdQ287r3KqUOKKXSlVKvXi0uSSCizlgPrfLdqXMkpORc8z601vj7+5OamkpqaioHDhxg06ZNl13/2LFjzJ07ly1btpCWlsbQoUMpKiqqyWkI4RCqGo9uzJgxJCUlsX//frp3786iRYto0aIFERERrF+/vnzTccAqrfUnQDIwXmsdpLX+D3AcuENrfYe5aetVoD8QBPRSSo24UkySQESdKB9aJSevENXEk98LzzF71QGadPBj+fLllJaWcvLkSXbu3Env3r25/fbbLeUnTpxg27ZtAHTr1o2TJ0/y1VdfAVBcXMzBgwcve9wzZ87QrFkzWrVqRW5uLp999lm9nK8Qda2q8ejS09MJDw/HYDCwbNkyy99G+bh3ZhOBuCp3WlEvYLvW+qTWugRYBtx+pQ2kF5aoE9ZDq7h6tqRpBz+y//sQS7vfxn2hPQgMDEQpxWuvvcaNN97I3XffzdatW/Hz86Nz58706dMHgCZNmrBy5Uoee+wx8vPzKSkp4YknnsDf37/K4wYGBmI0GvH19aVTp06EhYXV2zkLUZsSUnKYszGT43mFXK/zuaBdLcvKx6OLjo4mISGBwMBA4uPj2b59OwBhYWHlA562APK11ul1EaMkEFEnKg+h0mbYTAAUMCd2KHPmzKmwXCnF22+/XeW+goKC2Llz5yXl8fHxlvflfziVy4VwRpUHR809U8TJM0UkpORU6KpfUFBA+/btKS4uZtmyZXTo8L9lDzzwAH/72998gMetdl1AWVKp/PkUsBd4Uyl1A/AbcC/w1pXilCYsUSe8vTyvqdxR9O3bF6g4sVZ1JCcn89hjj9VVWKKRqWpwVK11hcFRAV5++WVCQ0MJCwvD19e3wrLx48dDWSXhI6vieOC/5pvonsAC4HOl1Dat9QkgBtgG7Af2aa3XXClOGQtL1InKV1BQNrSKs8xpvn37dubOnUtiYqK9QxGNkE/Meqr6ZlbAsdih1drHypUrGTNmzK9a69a1GpwVqYGIOlGdoVXs7fXXXycgIICAgADeeOMNAJo3b37JeklJSRiNRrKzszl37hyTJk2id+/eGI1G1qwpu0CzrrG88MILTJo0iYiICP74xz/y5ptv1t9JiQahpjX4Rx99lJiYGCjrZVVn5B6IqDOOOrQKlD3dHhcXxzfffIPWmtDQUPr163fJenv27OHRRx9lzZo1dO7cmWeeeYb+/fvz/vvvk5eXR+/evRk4cOAl2x0+fJht27ZRUFBAt27dmDZtGu7u7vVxaqIBqM7gqFfy1ltlty6UUhfqJEAzSSCi0bDu1cLBDfTqM4BmzZoBMHLkSHbt2lVh/UOHDjF16lQ2bdqEt3fZ6A+bNm1i7dq1zJ07F4CioiJ++OGHS441dOhQmjZtStOmTWnbti25ubl07Nixjs9QNBTlF17lv6/eXp7MjOzmcBdkkkBEo1D5nsyZwmK2Hvrtkl4t1tq3b09RUREpKSmWBKK15tNPP6Vbt4pXgrm5uRU+V54CuKSkpDZPRzQCjlyDL2e3eyBKqSFKqUyl1NFqDdolRA1U7tXStKM/ZzK/Inbdfs6dO8fq1asJDw+vsI2Xlxfr169n9uzZlm7CkZGRvPXWW+XjCJGSklJv5yDqn/VUBOUDF16JyWTiww8/vOp+4+PjmTFjRq3EaE92SSBKKVdgPnAn4Afcq5Tys0csonGo/FxK0xtvoXnAAL59axqhoaFMnjwZo9F4yXbt2rUjMTGRRx55hG+++YZnn32W4uJievTogb+/P88++2x9nYKwA+sE4u3tfdU5aqqbQBoKu3TjVUr1AV7QWkeaP88G0Fr/u6r1pRuvqKnqTvkrhLVx48axZs0aunXrRteuXTl06BDp6emUlpYSExPD9u3buXDhAo888ggPPfQQt912G4cOHcLHx4cJEyYwbdo0pk2bRnJyMm5ubrz++uvccccdxMfHk5yczNtvv8369et55ZVXWLduHWfPnmXSpEmcOnWKNm3aEBcXR+fOnYmOjqZly5YkJyfz888/89prr1lqQ3PmzGHFihVcuHCBu+++mxdffNESf11PaWuvJqwOwI9Wn38ylwlRJ2ZGdsPT3bVC2bX0amlMTCYTAQEBFcoa64OSsbGx3HzzzaSmplYYPWHRokW0atWKpKQkkpKSWLhwIceOHSM2Npbw8HBSU1N58sknmT9/PkopDhw4wEcffcSECRMqDO65evVqYmNj2bBhAzfccAOPPvooEyZMIC0tjfHjx1f4mZ84cYLdu3eTmJhY3kWXTZsqbTTuAAAdSklEQVQ2kZWVxd69e0lNTWXfvn1VjtpQVxz2JrpSaiowFaBz5852jkY4O2fp1eKoQkJCCAmpswtZp7Np0ybS0tIsTVr5+flkZWXRpEmTCuvt3r2bRx99FABfX19uuukmjhw5AsDWrVtJTk5m06ZNtGzZEoCvvvqKVatWAXD//fcza9Ysy75GjBiBi4sLfn5+lk4bmzZtYtOmTZbm17Nnz5KVlcXtt19xDMRaY68EkgN0svrc0VxmobVeQNlj9oSEhDj+4/LC4TlDrxZH89133zFq1Cjuu+8+duzYQWJiIjt27ODxx8uGV1JKsXPnTs6ePcvYsWM5c+YMJSUlvPvuu4SHhzNt2jSSkpIoLCxk9OjRFZpXHFl5l+/vvzfxq3kqgqDr/rdca81bb71FZGRkhe2sx2S7mptvvpnvvvuOI0eOVCs5W/fsK7/1oLVm9uzZPPTQQ9U+bm2yVxNWEtBVKeWjlGpC2Xj1a+0UixCiCpmZmYwaNYr4+Hh69eplKZ87dy7z588nNTWVXbt24enpyYcffkhkZCSpqans37/fMuz4P//5T5KTk0lLS2PHjh2kpaXZ63Sq7XJTEWw6+LNlncjISN59913LpE1Hjhzh3LlztGjRgoKCAst64eHhLFu2zLLODz/8YOkCftNNN/Hpp5/ywAMPWIZh79u3Lx9//DEAy5Ytu6RnYGWRkZG8//77nD17FoCcnJwK00HXNbvUQLTWJUqpGcBGwBV4X2t9+UkehBB1qvLQ4T+dyGX48OGsWrUKPz+/ClfWYWFhPPXUU4wfP56RI0fSsWNHevXqxaRJkyguLmbEiBGWBLJixQoWLFhASUkJJ06cICMjgx49etjpLKvnclMRPJPQhRvN60yePBmTyURwcDBaa9q0aUNCQgI9evTA1dWVwMBAoqOjmT59OtOmTcNgMODm5kZ8fHyFmoSvry/Lli1jzJgxrFu3jrfeeouJEycyZ84cy030Kxk8eDCHDh2yTH/QvHlzli5dStu2bevkZ1OZDKYoLvGvf/2LZ555Bijrxvjhhx8yffp0y/LnnnuO22+/nbVr1/LAAw9I27iTq/yQZUl+Lr+seA6/W29m+sTxTJ069ZLBJQ8cOMCGDRt455132LhxI76+vhw/fpz169czf/58nnrqKcLDwxk0aBBJSUlcd911REdHExERQXR0tB3P9upqYyBDR1HXvbAc9ia6sJ/KCeSdd96pkEBeeuklgCrHgBLOp6qhw3FxxePOWSxZ8m+aN29ueRIfIDs7G4PBgMFgICkpicOHD+Pp6UnHjh2ZMmUKFy5c4NtvvyUwMPCS2SEjIiLq9+Rs4O3lWWWXb0efisAeJIE0ckuXLuXNN9/k999/JzQ0lJYtW1JYWEhQUBD+/v6UlpaSnZ1NUFAQgwYN4tlnn2XEiBHk5eVRUlLCyy+/zPDhw+19GqIGKj9kWS73PKQkJlr+38u98cYbbNu2DRcXF/z9/bnzzjv5+OOPmTNnDu7u7jRv3pwlS5bg4+PjlLND1nQgw8ZEmrAasUOHDjFr1ixWrVqFu7s706dP57bbbmP69OmWm3Imk4moqCjS08tmxCwuLqawsJCWLVty6tQpbrvtNrKyslBK2fNURA3IQ5aXsr4n5MxdvqUJS9SZLVu2sG/fPksPm8LCwmrdfHvmmWfYuXMnLi4u5OTkkJuby4033njV7YRjkivuS0mX7+qRBNIIlV9dHd6SjuetEbwwb06FP5byocqrsmzZMk6ePMm+fftwd3enS5cuFZ6sFc5HHrIUtpIE0shY97hpelMguateZuYHu4Bwbr/Jk4KCAtzd3SkuLsbd3f2Sfu35+fm0bdsWd3d3tm3bxvfff2+/kxG1Rq64hS1kSttGxrrHTZMbOuMVfj/fL3uG8XeVdbk8ceIEU6dOpUePHowfP57WrVsTFhZGQEAAM2fOZPz48SQnJ2MwGFiyZAm+vr52PiMhhL3ITfRGpiH1cRdCXFlDHY1XVLJ27VpiY2MBSEhIICMjw7KsfCKbjIwMJk+eXKPjXK4vu/RxF0JcK6mBOKDo6GiioqKuOvuZLSo/dQxlPW7+PdIgbeBCNDBSA2kATCYTvr6+REdHc+uttzJ+/Hi++OILwsLC6Nq1K3v37rVMcblnzx7Wrl3LzJkzCQoKIjs7m//+97/06tWLwMBARo0axfnz522OZYSxA/8eaaCDlyeKsr7+kjwarqrm9ricxjrnh7Cd1EDqgclk4pZbbiElJQV/f39LMli0aBFr164lLi6OESNGWGYoq1wDOX36NK1btwbgH//4B+3atbPMMSDElVR+ENQWJSUluLlJh01nJDWQBsLHxweDwWAZ/mHAgAEopTAYDJhMpitum56eTnh4OAaDgWXLllmGfhbiWnz33XcYjUZ27drFxIkTMRgMGI1Gtm3bBpTNZREVFQXACy+8wP33309YWBj3338/JpOJ8PBwgoODCQ4OZs+ePfY8FeEg5LKijlQeHvuC/t90qi4uLpYhnV1cXCgpKbnivqKjo0lISCAwMJD4+PhrmrRGCCib22PcuHHEx8fzxRdfWKZZPXz4MIMHD7bMkmctIyOD3bt34+npyfnz59m8eTMeHh5kZWVx77334sytAqJ2SA2kDlhPSKOB3DNF5J4pIiEl56rbApc8vFdQUED79u0pLi62TE4jRHWdPHmS4cOHs2zZMgIDA9m9ezd/+ctfgEunWbU2bNgwPD3LeucVFxczZcoUDAYDY8aMqdBLUDReUgOpA1UNj621Zs7GzGrdrB43bhxTpkzhzTffZOXKlbz88suEhobSpk0bQkNDKyQXISqrXPt19WhG586d2b17N35+ftXeT7NmzSzv582bR7t27di/fz8XL17Ew8OjLkIXTkYSSB2oPDy2W6t2eD/4jqU8Pj7esqxLly6WG5zlE+2EhYVVuMKbNm0a06ZNq9ugRYNQuZt27pkiThdeZNIL83l71kSaN29umWa1f//+FaZZ/eqrry673/z8fDp27IiLiwuLFy+mtLT0suuKxkOasOqAPKwn7OVytd+3dv5IYmIi8+bN45ZbbuHixYsYDAbGjh17yTSrVZk+fTqLFy8mMDCQw4cPV6idiMZLuvHWAXlYT9iLDFUjrEk3XickD+sJe5Har6hPcg+kjsjw2MIeZHIoUZ8kgQjRgMjkUKI+SQIRooGR2q+oL3IPRAghhE0kgQghhLCJJBAhhBA2kQQihBDCJpJAhBBC2EQSiBBCCJtIAhFCCGETSSBCCCFsIglECCGETSSBCCGEsIkkECGEEDaRBCKEEMImkkCEEELYRBKIEEIIm0gCEUIIYZMaJRCl1Byl1GGlVJpSarVSystq2Wyl1FGlVKZSKtKqfIi57KhSKqYmxxdCCGE/Na2BbAYCtNY9gCPAbACllB8wDvAHhgDvKKVclVKuwHzgTsAPuNe8rhBCCCdTowSitd6ktS4xf/wa6Gh+Pxz4WGt9QWt9DDgK9Da/jmqtv9Na/w58bF5XCCGEk6nNeyCTgM/M7zsAP1ot+8lcdrnySyilpiqlkpVSySdPnqzFMIUQQtSGq86JrpT6ArixikV/11qvMa/zd6AEWFZbgWmtFwALAEJCQnRt7VcIIUTtuGoC0VoPvNJypVQ0EAUM0FqXf9HnAJ2sVutoLuMK5UIIIZxITXthDQFmAcO01uetFq0FximlmiqlfICuwF4gCeiqlPJRSjWh7Eb72prEIIQQwj6uWgO5ireBpsBmpRTA11rrh7XWB5VSK4AMypq2HtFalwIopWYAGwFX4H2t9cEaxiCEEMIO1P9anRxXSEiITk5OtncYQgjhVJRS+7TWIXW1f3kSXQghhE0kgQghhLCJJBAhhBA2kQQihBDCJpJAhBBC2EQSiBBCCJtIAhFCCGETSSBCCCFsIglECCGETSSBCCGEsIkkECGEEDaRBCKEEMImkkCEEELYRBKIEEIIm0gCEUIIYRNJIEIIIWwiCUQIIYRNJIEI4SBMJhMBAQGXlE+ePJmMjAxGjx7N8ePHa7zv7du3ExUVZVl21113kZeXR9++fW0LXDRaNZ0TXQhRx/7v//4PgJUrV9bJ/jds2ADAnj176mT/ouGSGogQDui7777DaDTyzTffEB4eTnBwMD179rR8yY8bN47169db1o+OjmblypWUlpYyc+ZMevXqRY8ePXjvvfeueJyvv/6aPn36YDQa6du3L5mZmXV6XqJhkQQiBNRq801CQgIZGRk2b5+ZmcmoUaOIj4/HYDCwefNmvv32Wz7++GMee+wxAMaOHcuKFSsA+P3339myZQtDhw5l0aJFtGrViqSkJJKSkli4cCHHjh277LG6d+/Orl27SElJ4aWXXuKZZ56xOW7R+EgTlhDUbvNNQkICUVFR+Pn5XfO2J0+eZPjw4axatQo/Pz/y8/OZMWMGqampuLq6cuTIEQDuvPNOHn/8cS5cuMDnn3/O7bffjqenJ5s2bSItLc3S3JWfn09WVha33nprlcc7c+YM0dHRZGVloZSiuLjY9hMXjY7UQIQAmjdvfsnN5RkzZhAfHw/Ali1bMBqNGAwGJk2axIULFwCIiYnBz8+PHj168Le//Y09e/awdu1aZs6cSVBQENnZ2SxcuJBevXoRGBjIqFGjOH/+fIVjJ6TkEBa7lT+9upUzpe54XNeO3bt3AzBv3jzatWvH/v37SU5O5vfffwfAw8ODiIgINm7cyPLlyxk7diwAWmveeustUlNTSU1N5dixYwwePPiy5/3ss89yxx13kJ6ezrp16ygqKqq1n6lo+CSBCHEVRUVFREdHs3z5cg4cOEBJSQnvvvsup0+fZvXq1Rw8eJC0tDT+8Y9/0LdvX4YNG8acOXNITU3l5ptvZuTIkSQlJbF//366d+/OokWLLPtOSMlh9qoD5OQVAlCqXCm8/Unmvft/fPjhh+Tn59O+fXtcXFz44IMPKC0ttWw7duxY4uLi2LVrF0OGDAEgMjKSd99911KTOHLkCOfOnbvsueXn59OhQwcAS7IUorokgQhxFZmZmfj4+FiagSZMmMDOnTtp1aoVHh4ePPjgg6xatYo//OEPVW6fnp5OeHg4BoOBZcuWcfDgQcuyORszKSwurbD+BeVOy2H/YN68eXTp0oXFixcTGBjI4cOHadasmWW9wYMHs2PHDgYOHEiTJk2Asi6/fn5+BAcHExAQwEMPPURJScllz23WrFnMnj0bo9F4xfWEqIrcAxGNUkJKDnM2ZnI8rxBvL09KL2rc3Ny4ePGiZZ2rNee4ubmxd+9etmzZwsqVK3n77bfZunXrJetFR0eTkJBAYGAg8fHxbN++3bLsuLnmAeDWqh3eD74DwC8XXDmWlATA448/blnn1Vdftbx3d3fn119/rXAsFxcX/vWvf/Gvf/2rQnmrVq1IT08HICIigoiICAD69Oljua8C8Morr1zxnIWwJjUQ0ehYNxtpICevkAslFzlc0JSMjAwuXLhAXl4eW7ZsAaBbt26YTCaOHj0KwAcffEC/fv04e/Ys+fn53HXXXcybN4/9+/cD0KJFCwoKCizHKygooH379hQXF7Ns2bIKsXh7eVYZ4+XKhXAkkkBEo1NVsxFKEbe/gHvuuYeAgADuuecejEYjUHbDOi4ujjFjxmAwGHBxceHhhx+moKCAqKgoevTowZ/+9Cdef/11oOwZjTlz5mA0GsnOzubll18mNDSUsLAwfH19Kxx2ZmQ3PN1dK5R5ursyM7Jb3f0AhKglSmtt7xiuKiQkRCcnJ9s7DNFA+MSsx/q3vrTwDCfiH6fTtDiOxQ6t93gqN6fNjOzGCGOHeo9DNDxKqX1a65C62r/cAxGNjreXp6XXU0nBaXI/mk3L3iPt1mw0wthBEoZwStKEJRod62Yjtxat6TB1Ae1uGyHNRkJcI6mBiEan/Gpfmo2EqBlJIKJRkmYjIWpOmrCEEELYRBKIEEIIm0gCEUIIYRNJIEIIIWwiCUQIIYRNaiWBKKX+qpTSSqkbzJ+VUupNpdRRpVSaUirYat0JSqks82tCbRxfCCFE/atxN16lVCdgMPCDVfGdQFfzKxR4FwhVSl0PPA+EABrYp5Raq7X+raZxCCGEqF+1UQOZB8yCCsMLDQeW6DJfA15KqfZAJLBZa/2rOWlsBobUQgxCCCHqWY0SiFJqOJCjtd5faVEH4Eerzz+Zyy5XXtW+pyqlkpVSySdPnqxJmEIIIerAVZuwlFJfADdWsejvwDOUNV/VOq31AmABlI3GWxfHEEIIYburJhCt9cCqypVSBsAH2K+UAugIfKuU6g3kAJ2sVu9oLssBIiqVb7chbiGEEHZmcxOW1vqA1rqt1rqL1roLZc1RwVrrn4G1wAPm3li3Afla6xPARmCwUuo6pdR1lNVeNtb8NIQQQtS3uhpMcQNwF3AUOA9MBNBa/6qUehlIMq/3ktb616p3IYQQwpHVWgIx10LK32vgkcus9z7wfm0dVwghhH3Ik+hCCCFsIglEiEamb9++Nd7H9u3biYqKqoVohDOTBCKEA/jiiy947rnnWLt2LbGxsVWuEx8fz4wZMyyf+/btS15eHnfdddc1HWvPnj01ilWIcpJAhHAAAwcO5KWXXmLYsGHExMRUa5s9e/bg5eXFhg0brulYzZs3v6QGMWPGDOLj4wHYsmULRqMRg8HApEmTuHDhAgCff/45vr6+BAcHs2rVKsu2e/fupU+fPhiNRvr27UtmZuY1xSOclyQQIerIuXPnGDp0KIGBgQQEBLB8+XJGjBhhWb5582buvvtuAD766CMMBgMBAQE8/fTTlnXi4uK49dZb6d27N19++aWlfM2aNYSGhmI0Ghk0aBC5ubm1EnNRURHR0dEsX76cAwcOUFJSwrvvvktRURFTpkxh3bp17Nu3j59//tmyja+vL7t27SIlJYWXXnqJZ555plZiEY5PEogQdeTzzz/H29ub/fv3k56ezpAhQzh8+DDlQ/PExcUxadIkjh8/ztNPP83WrVtJTU0lKSmJhIQETpw4wfPPP8+XX37J7t27ycjIsOz79ttv5+uvvyYlJYWxY8fy2muv1UrMmZmZ+Pj4cOuttwIwYcIEdu7cyeHDh/Hx8aFr164opfjLX/5i2SY/P58xY8YQEBDAk08+ycGDB2slFuH46uo5ECEaPYPBwF//+leefvppoqKiCA8P5/7772fp0qVMnDiRr776iiVLlrB+/XoiIiJo06YNAOPHj2fnzp0AFcrHjh3LkSNHAPjpp58YO3YsJ06c4Pfff8fHx+eycSSk5DBnYybH8wrx9vKk9KLGzc2NixcvWtYpKiqy+TyfffZZ7rjjDlavXo3JZCIiIsLmfQnnIjUQIWpZQkoOYbFbiXw/i7YPvMGFFh34xz/+wUsvvcTEiRNZunQpH330EWPGjMHNzbZruEcffZQZM2Zw4MAB3nvvvcsmgISUHGavOkBOXiEayMkr5ELJRQ4XNCUjI4MLFy6Ql5fHli1bAOjWrRsmk4mjR48C8MEHH9CvXz98fX0xmUxkZ2cDZU1u5fLz8+nQoWxM1PL7KKJxkAQiRC2y/sIuLjhN7nnNxgu38qeRk/j222/x9vbG29ubV155hYkTJwLQu3dvduzYwalTpygtLeWjjz6iX79+hIaGsmPHDk6fPk1xcTGffPKJ5TjWX9qLFy++bDxzNmZSWFxasVAp4vYXcM899xAQEMA999yD0WgEwMPDg7i4OMaMGYPBYMDFxYWHH34YDw8PFixYwNChQwkODqZt27aW3c2aNYvZs2djNBopKSmprR+lcALShCVELbL+wi4+aeKX7XGgFP9xb8L2hKVAWRPVyZMn6d69OwDt27cnNjaWO+64A601Q4cOZfjw4QC88MIL9OnTBy8vL4KCgizHeeGFFxgzZgzXXXcd/fv359ixY1XGczyvsMLn0sIzuHg053heIa+99lqV904GDBhASkrKJeXl93Aq69Onj6VpDeCVV1654s9INByqbNQRxxYSEqKTk5PtHYYQV+UTs56q/qIUcCx2KFDWZdZoNPLggw/WeTxhsVvJMSeRkoLT5H40mxY9/0z3AffwZUz/Oj++sC+l1D6tdUhd7V+asISoRd5enlcs79mzJ2lpaRV6MdWlmZHd8HR3BcCtRWs6TF1Au9tGMDOyW70cXzRs0oQlRC2aGdmN2asOVLjv4OnuavnC3rdvX73GM8JYdp/EuhfWzMhulnIhakISiBC1yBG/sEcYO0jCEHVCEogQtUy+sEVjIfdAhBBC2EQSiBBCCJtIAhFCCGETSSBCCCFsIglECCGETZziSXSl1Enge3vHUYUbgFP2DqIGnDl+Z44dnDt+id1+rjX+m7TWbeoqGKdIII5KKZVcl8ME1DVnjt+ZYwfnjl9itx9Hi1+asIQQQthEEogQQgibSAKpmQX2DqCGnDl+Z44dnDt+id1+HCp+uQcihBDCJlIDEUIIYRNJINWklHpZKZWmlEpVSm1SSnmby5VS6k2l1FHz8mCrbSYopbLMrwl2jH2OUuqwOb7VSikvq2WzzbFnKqUircqHmMuOKqVi7BO5JZYxSqmDSqmLSqmQSsscPn5rjhqXNaXU+0qpX5RS6VZl1yulNpt/lzcrpa4zl1/2998elFKdlFLblFIZ5t+Zx50lfqWUh1Jqr1Jqvzn2F83lPkqpb8wxLldKNTGXNzV/Pmpe3qXeg9Zay6saL6Cl1fvHgP+a398FfEbZpHO3Ad+Yy68HvjP/e535/XV2in0w4GZ+/yrwqvm9H7AfaAr4ANmAq/mVDfwRaGJex8+OP/vuQDdgOxBiVe4U8VvF65BxVRHn7UAwkG5V9hoQY34fY/U7VOXvvx1jbw8Em9+3AI6Yf08cPn5zDM3N792Bb8wxrQDGmcv/C0wzv59u9T00Dlhe3zFLDaSatNZnrD42A8vMpcOBJbrM14CXUqo9EAls1lr/qrX+DdgMDKnXoM201pu01iXmj18DHc3vhwMfa60vaK2PAUeB3ubXUa31d1rr34GPzevahdb6kNY6s4pFThG/FUeNqwKt9U7g10rFw4HF5veLgRFW5VX9/tuF1vqE1vpb8/sC4BDQASeI3xzDWfNHd/NLA/2BlebyyrGXn9NKYIBSStVTuIA0YV0TpdQ/lVI/AuOB58zFHYAfrVb7yVx2uXJ7m0TZFRc4X+yVOVv8jhpXdbTTWp8wv/8ZaGd+77DnZG7SMVJ2Je8U8SulXJVSqcAvlF10ZgN5VheA1vFZYjcvzwda12e8kkCsKKW+UEqlV/EaDqC1/rvWuhOwDJhh32grulrs5nX+DpRQFr9DqU78wjHosjYTh+6+qZRqDnwKPFGp9cCh49dal2qtgyhrJegN+No5pCuSGQmtaK0HVnPVZcAG4HkgB+hktayjuSwHiKhUvr3GQV7G1WJXSkUDUcAA8x8QXD52rlBeJ67hZ2/NYeKvpivF6+hylVLttdYnzE08v5jLHe6clFLulCWPZVrrVeZip4kfQGudp5TaBvShrFnNzVzLsI6vPPaflFJuQCvgdH3GKTWQalJKdbX6OBw4bH6/FnjA3JvjNiDfXFXeCAxWSl1n7vEx2FxW75RSQ4BZwDCt9XmrRWuBcebeHD5AV2AvkAR0Nff+aELZDbq19R13NThb/I4aV3WsBcp7Ek4A1liVV/X7bxfmewCLgENa69etFjl8/EqpNsrcQ1Ip5QkMouwezjZgtHm1yrGXn9NoYKvVxWH9qO+79s76ouyKJh1IA9YBHfT/ek7Mp6yt8gAVewlNouzG7lFgoh1jP0pZW2mq+fVfq2V/N8eeCdxpVX4XZT1YsoG/2/lnfzdlbb8XgFxgozPFX+lcHDKuSjF+BJwAis0/9wcpa1vfAmQBXwDXm9e97O+/nWL/E2XNU2lWv+93OUP8QA8gxRx7OvCcufyPlF0YHQU+AZqayz3Mn4+al/+xvmOWJ9GFEELYRJqwhBBC2EQSiBBCCJtIAhFCCGETSSBCCCFsIglECCGETSSBCCGEsIkkECGEEDaRBCKEEMIm/x9jK0RCMju/QQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f216edf57f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fi_word_indices = [fi_wv.vocab[w].index for w in fi_words]\n",
    "fi_vectors_words = fi_wv.vectors[fi_word_indices]\n",
    "\n",
    "fi_vectors_tsne = TSNE(perplexity=10.0, random_state=1234).fit_transform(fi_vectors_words)\n",
    "plot_all_words(fi_words, fi_vectors_tsne)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(Note: getting a better-looking visualization doesn't mean it's (more) correct!)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Clustering\n",
    "\n",
    "Unsupervised classification: identify coherent groups (clusters) in data without the help of examples of correct grouping.\n",
    "\n",
    "* Hard clustering: assign each item to one cluster\n",
    "* Soft clustering - degree of belonging to a cluster\n",
    "* Hierarchical clustering - hierarchy of clusters, usually constructed bottom-to-top\n",
    "\n",
    "(Focus here on hard clustering.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"figs/sklearn_cluster_comparison.png\" width=\"80%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"text-align:center; font-size:80%\">Illustration of clustering algorithms (figure from <a href=\"http://scikit-learn.org/stable/modules/clustering.html\">scikit-learn documentation</a>)</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Depending on the clustering algorithm, we might need:\n",
    "\n",
    "* A similarity metric on the items being clustered (function or matrix)\n",
    "* A way to establish cluster centres\n",
    "* A way to measure between cluster distance\n",
    "* A way to measure cluster coherence\n",
    "\n",
    "A notable difference between clustering algorithms is whether they can work with a matrix of pairwise similarities between arbitrary objects, or if they require a vector representation of the data items."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## K-means\n",
    "\n",
    "* Simple and popular clustering algorithm operating on vectors\n",
    "* Requires number of clusters (k) to be specified in advance\n",
    "\n",
    "Iterative algorithm (sketch):\n",
    "\n",
    "1. Randomly identify k vectors as cluster centers\n",
    "2. Assign each vector to the nearest center\n",
    "3. Re-calculate cluster centers\n",
    "4. Repeat 2-3 until the assignment no longer changes.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/e/ea/K-means_convergence.gif\" width=\"40%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"text-align:center; color:gray; font-size:80%\">(Figure from <a href=\"https://en.wikipedia.org/wiki/K-means_clustering#/media/File:K-means_convergence.gif\">Wikimedia Commons</a>)</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: word vector clustering\n",
    "\n",
    "Word vectors can be clustered given a measure of their similarity.\n",
    "\n",
    "**Sidenote**: We would generally prefer cosine similarity but `sklearn` only supports Eucledian distance for K-means. We'll normalize the vectors to unit length, which makes these equivalent for K-means ($\\sqrt{\\sum (a_i b_i)^2} = \\sqrt{2-\\sum 2 a_i b_i}$ since $\\sum a_i^2 =\\sum b_i^2 = 1$)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.00\n",
      "1.00\n"
     ]
    }
   ],
   "source": [
    "# init_sims(replace=True) normalizes word vectors\n",
    "en_wv.init_sims(replace=True)\n",
    "fi_wv.init_sims(replace=True)\n",
    "\n",
    "print('{:.2f}'.format(np.dot(en_wv['dog'],en_wv['dog'])))\n",
    "print('{:.2f}'.format(np.dot(fi_wv['koira'],fi_wv['koira'])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll use `MiniBatchKMeans` for clustering. This is a faster approximation of K-means."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100000, 20)\n",
      "[('koira', 6),\n",
      " ('kissa', 6),\n",
      " ('tietokone', 5),\n",
      " ('näyttö', 5),\n",
      " ('syödä', 19),\n",
      " ('juoda', 6),\n",
      " ('kävellä', 6),\n",
      " ('juosta', 6),\n",
      " ('yksi', 3),\n",
      " ('kaksi', 7),\n",
      " ('1', 7),\n",
      " ('2', 7),\n",
      " ('hyvä', 3),\n",
      " ('paha', 3),\n",
      " ('ruma', 3),\n",
      " ('onnellinen', 3),\n",
      " ('onneton', 3),\n",
      " ('ja', 12),\n",
      " ('tai', 2),\n",
      " ('mitä', 12),\n",
      " ('että', 12),\n",
      " ('oikein', 12),\n",
      " ('todella', 6),\n",
      " ('hyvin', 12)]\n"
     ]
    }
   ],
   "source": [
    "import sklearn.cluster\n",
    "\n",
    "from pprint import pprint as pretty_print\n",
    "\n",
    "\n",
    "k = sklearn.cluster.MiniBatchKMeans(batch_size=5000, n_clusters=20, random_state=1234)\n",
    "distances = k.fit_transform(fi_wv.vectors)\n",
    "\n",
    "print(distances.shape)\n",
    "\n",
    "pretty_print(list(zip(fi_words, k.labels_[fi_word_indices]))) # Cluster label for each word"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's a bit hard to interpret, so let's see that with the words grouped into clusters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdb\" ><thead>    <tr>        <th class=\"col_heading level0 col0\" >label</th>        <th class=\"col_heading level0 col1\" >words</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                                <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow0_col0\" class=\"data row0 col0\" >2</td>\n",
       "                        <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow0_col1\" class=\"data row0 col1\" >['tai']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow1_col0\" class=\"data row1 col0\" >3</td>\n",
       "                        <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow1_col1\" class=\"data row1 col1\" >['yksi', 'hyvä', 'paha', 'ruma', 'onnellinen', 'onneton']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow2_col0\" class=\"data row2 col0\" >5</td>\n",
       "                        <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow2_col1\" class=\"data row2 col1\" >['tietokone', 'näyttö']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow3_col0\" class=\"data row3 col0\" >6</td>\n",
       "                        <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow3_col1\" class=\"data row3 col1\" >['koira', 'kissa', 'juoda', 'kävellä', 'juosta', 'todella']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow4_col0\" class=\"data row4 col0\" >7</td>\n",
       "                        <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow4_col1\" class=\"data row4 col1\" >['kaksi', '1', '2']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow5_col0\" class=\"data row5 col0\" >12</td>\n",
       "                        <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow5_col1\" class=\"data row5 col1\" >['ja', 'mitä', 'että', 'oikein', 'hyvin']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow6_col0\" class=\"data row6 col0\" >19</td>\n",
       "                        <td id=\"T_7772c834_54b6_11e9_b063_fa163e87fcdbrow6_col1\" class=\"data row6 col1\" >['syödä']</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7f217bbeba58>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from collections import defaultdict\n",
    "from pandas import DataFrame\n",
    "\n",
    "\n",
    "def group_by_label(words, labels):\n",
    "    grouped = defaultdict(list)\n",
    "    for word, label in zip(words, labels):\n",
    "            grouped[label].append(word)\n",
    "    return grouped\n",
    "\n",
    "\n",
    "def show_grouped(grouped):\n",
    "    return DataFrame(list(sorted(grouped.items())), columns=['label', 'words']).style.hide_index()\n",
    "\n",
    "\n",
    "grouped = group_by_label(fi_words, k.labels_[fi_word_indices])\n",
    "show_grouped(grouped)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the same for English:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdb\" ><thead>    <tr>        <th class=\"col_heading level0 col0\" >label</th>        <th class=\"col_heading level0 col1\" >words</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                                <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow0_col0\" class=\"data row0 col0\" >0</td>\n",
       "                        <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow0_col1\" class=\"data row0 col1\" >['computer', '1', '2']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow1_col0\" class=\"data row1 col0\" >4</td>\n",
       "                        <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow1_col1\" class=\"data row1 col1\" >['dog']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow2_col0\" class=\"data row2 col0\" >6</td>\n",
       "                        <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow2_col1\" class=\"data row2 col1\" >['cat', 'eat', 'drink']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow3_col0\" class=\"data row3 col0\" >7</td>\n",
       "                        <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow3_col1\" class=\"data row3 col1\" >['monitor', 'happy', 'or', 'what', 'that', 'right']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow4_col0\" class=\"data row4 col0\" >9</td>\n",
       "                        <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow4_col1\" class=\"data row4 col1\" >['very', 'well']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow5_col0\" class=\"data row5 col0\" >12</td>\n",
       "                        <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow5_col1\" class=\"data row5 col1\" >['walk', 'run', 'one', 'two']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow6_col0\" class=\"data row6 col0\" >15</td>\n",
       "                        <td id=\"T_7868533a_54b6_11e9_b063_fa163e87fcdbrow6_col1\" class=\"data row6 col1\" >['good', 'bad', 'ugly', 'unhappy']</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7f216da0c5f8>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "k = sklearn.cluster.MiniBatchKMeans(batch_size=5000, n_clusters=20, random_state=1234)\n",
    "distances = k.fit_transform(en_wv.vectors)\n",
    "\n",
    "\n",
    "grouped = group_by_label(en_words, k.labels_[en_word_indices])\n",
    "show_grouped(grouped)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some unexpected groupings there, but at least part of the expected structure is correct.\n",
    "\n",
    "The clusters above are built on 100K words, how about we try just with our example words? This makes it easier to pick an appropriate value of k:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdb\" ><thead>    <tr>        <th class=\"col_heading level0 col0\" >label</th>        <th class=\"col_heading level0 col1\" >words</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                                <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow0_col0\" class=\"data row0 col0\" >0</td>\n",
       "                        <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow0_col1\" class=\"data row0 col1\" >['syödä', 'juoda', 'kävellä', 'juosta']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow1_col0\" class=\"data row1 col0\" >1</td>\n",
       "                        <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow1_col1\" class=\"data row1 col1\" >['koira', 'kissa', 'oikein', 'todella', 'hyvin']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow2_col0\" class=\"data row2 col0\" >2</td>\n",
       "                        <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow2_col1\" class=\"data row2 col1\" >['kaksi', '1', '2']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow3_col0\" class=\"data row3 col0\" >3</td>\n",
       "                        <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow3_col1\" class=\"data row3 col1\" >['onnellinen']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow4_col0\" class=\"data row4 col0\" >4</td>\n",
       "                        <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow4_col1\" class=\"data row4 col1\" >['ja', 'tai', 'mitä', 'että']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow5_col0\" class=\"data row5 col0\" >5</td>\n",
       "                        <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow5_col1\" class=\"data row5 col1\" >['tietokone', 'näyttö']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow6_col0\" class=\"data row6 col0\" >6</td>\n",
       "                        <td id=\"T_7868533b_54b6_11e9_b063_fa163e87fcdbrow6_col1\" class=\"data row6 col1\" >['yksi', 'hyvä', 'paha', 'ruma', 'onneton']</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7f216da0c780>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "k = sklearn.cluster.MiniBatchKMeans(batch_size=5000, n_clusters=7, random_state=1234)\n",
    "distances = k.fit_transform(fi_wv.vectors[fi_word_indices])\n",
    "    \n",
    "grouped = group_by_label(fi_words, k.labels_)\n",
    "show_grouped(grouped)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdb\" ><thead>    <tr>        <th class=\"col_heading level0 col0\" >label</th>        <th class=\"col_heading level0 col1\" >words</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                                <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow0_col0\" class=\"data row0 col0\" >0</td>\n",
       "                        <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow0_col1\" class=\"data row0 col1\" >['ugly', 'happy', 'unhappy', 'very', 'well']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow1_col0\" class=\"data row1 col0\" >1</td>\n",
       "                        <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow1_col1\" class=\"data row1 col1\" >['dog', 'cat', 'computer', 'monitor', 'or']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow2_col0\" class=\"data row2 col0\" >2</td>\n",
       "                        <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow2_col1\" class=\"data row2 col1\" >['one', 'two', '1', '2']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow3_col0\" class=\"data row3 col0\" >3</td>\n",
       "                        <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow3_col1\" class=\"data row3 col1\" >['walk', 'run']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow4_col0\" class=\"data row4 col0\" >4</td>\n",
       "                        <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow4_col1\" class=\"data row4 col1\" >['eat', 'drink']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow5_col0\" class=\"data row5 col0\" >5</td>\n",
       "                        <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow5_col1\" class=\"data row5 col1\" >['what', 'that', 'right']</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                                <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow6_col0\" class=\"data row6 col0\" >6</td>\n",
       "                        <td id=\"T_7868533c_54b6_11e9_b063_fa163e87fcdbrow6_col1\" class=\"data row6 col1\" >['good', 'bad']</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7f216da0c978>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "k = sklearn.cluster.MiniBatchKMeans(batch_size=5000, n_clusters=7, random_state=1234)\n",
    "distances=k.fit_transform(en_wv.vectors[en_word_indices])\n",
    "\n",
    "grouped = group_by_label(en_words, k.labels_)\n",
    "show_grouped(grouped)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not too bad given that the methods are told nothing about the words.\n",
    "\n",
    "---"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}