4.4.2_NLP_Alice_v_Emma.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction \n",
    "This notebook imports the popular stories 'Alice in Wonderland' and 'Emma' and tries to predict if any given sentence is from \"Alice\" or \"Emma\".  In these models, 10% of Emma was used instead of the approximately 2% in the course curriculum.  As a result, the accuracy scores a significantly better.  \n",
    "\n",
    "The first step is to import the necessary modules and import the text data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to\n",
      "[nltk_data]     C:\\Users\\Fred\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import scipy\n",
    "import sklearn\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import re\n",
    "from nltk.corpus import gutenberg, stopwords\n",
    "from collections import Counter\n",
    "\n",
    "import nltk\n",
    "nltk.download('punkt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import the nlp library, spacy\n",
    "import spacy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package gutenberg to\n",
      "[nltk_data]     C:\\Users\\Fred\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package gutenberg is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.download('gutenberg')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: en_core_web_sm==2.0.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz#egg=en_core_web_sm==2.0.0 in c:\\users\\fred\\anaconda3\\lib\\site-packages (2.0.0)\n",
      "\n",
      "    Linking successful\n",
      "    C:\\Users\\Fred\\Anaconda3\\lib\\site-packages\\en_core_web_sm -->\n",
      "    C:\\Users\\Fred\\Anaconda3\\lib\\site-packages\\spacy\\data\\en\n",
      "\n",
      "    You can now load the model via spacy.load('en')\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# this takes a long time\n",
    "\n",
    "### need to run as administrator from Anaconda3 promplt\n",
    "\n",
    "!python -m spacy download en"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['austen-emma.txt', 'austen-persuasion.txt', 'austen-sense.txt', 'bible-kjv.txt', 'blake-poems.txt', 'bryant-stories.txt', 'burgess-busterbrown.txt', 'carroll-alice.txt', 'chesterton-ball.txt', 'chesterton-brown.txt', 'chesterton-thursday.txt', 'edgeworth-parents.txt', 'melville-moby_dick.txt', 'milton-paradise.txt', 'shakespeare-caesar.txt', 'shakespeare-hamlet.txt', 'shakespeare-macbeth.txt', 'whitman-leaves.txt']\n"
     ]
    }
   ],
   "source": [
    "print(gutenberg.fileids())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Clean and explore the data  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Utility function for standard text cleaning.\n",
    "def text_cleaner(text):\n",
    "    # Visual inspection identifies a form of punctuation spaCy does not\n",
    "    # recognize: the double dash '--'.  Better get rid of it now!\n",
    "    text = re.sub(r'--',' ',text)\n",
    "    text = re.sub(\"[\\[].*?[\\]]\", \"\", text)\n",
    "    text = ' '.join(text.split())\n",
    "    return text\n",
    "    \n",
    "# Load and clean the data.\n",
    "emma = gutenberg.raw('austen-emma.txt')\n",
    "alice = gutenberg.raw('carroll-alice.txt')\n",
    "\n",
    "# The Chapter indicator is idiosyncratic\n",
    "emma = re.sub(r'Chapter \\d+', '', emma)\n",
    "alice = re.sub(r'CHAPTER .*', '', alice)\n",
    "    \n",
    "emma = text_cleaner(emma[:int(len(emma)/10)])\n",
    "alice = text_cleaner(alice[:int(len(alice)/10)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "87962\n",
      "14139\n"
     ]
    }
   ],
   "source": [
    "print(len(emma))\n",
    "print(len(alice))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parse the cleaned novels. This can take a bit.\n",
    "# had to do this in an admin terminal.......\n",
    "nlp = spacy.load('en')\n",
    "alice_doc = nlp(alice)\n",
    "emma_doc = nlp(emma)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>(Alice, was, beginning, to, get, very, tired, of, sittin...</td>\n",
       "      <td>Carroll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>(So, she, was, considering, in, her, own, mind, (, as, w...</td>\n",
       "      <td>Carroll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>(There, was, nothing, so, VERY, remarkable, in, that, ;,...</td>\n",
       "      <td>Carroll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>(Oh, dear, !)</td>\n",
       "      <td>Carroll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>(I, shall, be, late, !, ')</td>\n",
       "      <td>Carroll</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                             0        1\n",
       "0  (Alice, was, beginning, to, get, very, tired, of, sittin...  Carroll\n",
       "1  (So, she, was, considering, in, her, own, mind, (, as, w...  Carroll\n",
       "2  (There, was, nothing, so, VERY, remarkable, in, that, ;,...  Carroll\n",
       "3                                                (Oh, dear, !)  Carroll\n",
       "4                                   (I, shall, be, late, !, ')  Carroll"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Group into sentences.\n",
    "alice_sents = [[sent, \"Carroll\"] for sent in alice_doc.sents]\n",
    "emma_sents = [[sent, \"Austen\"] for sent in emma_doc.sents]\n",
    "\n",
    "# Combine the sentences from the two novels into one data frame.\n",
    "sentences = pd.DataFrame(alice_sents + emma_sents)\n",
    "\n",
    "pd.set_option('max_colwidth', 60)\n",
    "\n",
    "sentences.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Utility function to create a list of the 2000 most common words.\n",
    "def bag_of_words(text):\n",
    "    \n",
    "    # Filter out punctuation and stop words.\n",
    "    allwords = [token.lemma_\n",
    "                for token in text\n",
    "                if not token.is_punct\n",
    "                and not token.is_stop]\n",
    "    \n",
    "    # Return the most common words.\n",
    "    return [item[0] for item in Counter(allwords).most_common(2000)]\n",
    "    \n",
    "\n",
    "# Creates a data frame with features for each word in our common word set.\n",
    "# Each value is the count of the times the word appears in each sentence.\n",
    "def bow_features(sentences, common_words):\n",
    "    \n",
    "    # Scaffold the data frame and initialize counts to zero.\n",
    "    df = pd.DataFrame(columns=common_words)\n",
    "    df['text_sentence'] = sentences[0]\n",
    "    df['text_source'] = sentences[1]\n",
    "    df.loc[:, common_words] = 0\n",
    "    \n",
    "    # Process each row, counting the occurrence of words in each sentence.\n",
    "    for i, sentence in enumerate(df['text_sentence']):\n",
    "        \n",
    "        # Convert the sentence to lemmas, then filter out punctuation,\n",
    "        # stop words, and uncommon words.\n",
    "        words = [token.lemma_\n",
    "                 for token in sentence\n",
    "                 if (\n",
    "                     not token.is_punct\n",
    "                     and not token.is_stop\n",
    "                     and token.lemma_ in common_words\n",
    "                 )]\n",
    "        \n",
    "        # Populate the row with word counts.\n",
    "        for word in words:\n",
    "            df.loc[i, word] += 1\n",
    "        \n",
    "        # This counter is just to make sure the kernel didn't hang.\n",
    "        if i % 50 == 0:\n",
    "            print(\"Processing row {}\".format(i))\n",
    "            \n",
    "    return df\n",
    "\n",
    "# Set up the bags.\n",
    "alicewords = bag_of_words(alice_doc)\n",
    "emmawords = bag_of_words(emma_doc)\n",
    "\n",
    "# Combine bags to create a set of unique words.\n",
    "common_words = set(alicewords + emmawords)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing row 0\n",
      "Processing row 50\n",
      "Processing row 100\n",
      "Processing row 150\n",
      "Processing row 200\n",
      "Processing row 250\n",
      "Processing row 300\n",
      "Processing row 350\n",
      "Processing row 400\n",
      "Processing row 450\n",
      "Processing row 500\n",
      "Processing row 550\n",
      "Processing row 600\n",
      "Processing row 650\n",
      "Processing row 700\n",
      "Processing row 750\n",
      "Processing row 800\n",
      "Processing row 850\n",
      "Processing row 900\n",
      "Processing row 950\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>repent</th>\n",
       "      <th>fortnight</th>\n",
       "      <th>flavour</th>\n",
       "      <th>repeat</th>\n",
       "      <th>key</th>\n",
       "      <th>luxury</th>\n",
       "      <th>inch</th>\n",
       "      <th>competence</th>\n",
       "      <th>irksomeness</th>\n",
       "      <th>distance</th>\n",
       "      <th>...</th>\n",
       "      <th>push</th>\n",
       "      <th>basin</th>\n",
       "      <th>apply</th>\n",
       "      <th>earnestly</th>\n",
       "      <th>illiterate</th>\n",
       "      <th>8th</th>\n",
       "      <th>personage</th>\n",
       "      <th>screw</th>\n",
       "      <th>text_sentence</th>\n",
       "      <th>text_source</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(Alice, was, beginning, to, get, very, tired, of, sittin...</td>\n",
       "      <td>Carroll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(So, she, was, considering, in, her, own, mind, (, as, w...</td>\n",
       "      <td>Carroll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(There, was, nothing, so, VERY, remarkable, in, that, ;,...</td>\n",
       "      <td>Carroll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(Oh, dear, !)</td>\n",
       "      <td>Carroll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(I, shall, be, late, !, ')</td>\n",
       "      <td>Carroll</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 1973 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  repent fortnight flavour repeat key luxury inch competence irksomeness  \\\n",
       "0      0         0       0      0   0      0    0          0           0   \n",
       "1      0         0       0      0   0      0    0          0           0   \n",
       "2      0         0       0      0   0      0    0          0           0   \n",
       "3      0         0       0      0   0      0    0          0           0   \n",
       "4      0         0       0      0   0      0    0          0           0   \n",
       "\n",
       "  distance  ... push basin apply earnestly illiterate 8th personage screw  \\\n",
       "0        0  ...    0     0     0         0          0   0         0     0   \n",
       "1        0  ...    0     0     0         0          0   0         0     0   \n",
       "2        0  ...    0     0     0         0          0   0         0     0   \n",
       "3        0  ...    0     0     0         0          0   0         0     0   \n",
       "4        0  ...    0     0     0         0          0   0         0     0   \n",
       "\n",
       "                                                 text_sentence text_source  \n",
       "0  (Alice, was, beginning, to, get, very, tired, of, sittin...     Carroll  \n",
       "1  (So, she, was, considering, in, her, own, mind, (, as, w...     Carroll  \n",
       "2  (There, was, nothing, so, VERY, remarkable, in, that, ;,...     Carroll  \n",
       "3                                                (Oh, dear, !)     Carroll  \n",
       "4                                   (I, shall, be, late, !, ')     Carroll  \n",
       "\n",
       "[5 rows x 1973 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create our data frame with features. This can take a while to run.\n",
    "word_counts = bow_features(sentences, common_words)\n",
    "word_counts.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>repent</th>\n",
       "      <th>fortnight</th>\n",
       "      <th>flavour</th>\n",
       "      <th>repeat</th>\n",
       "      <th>key</th>\n",
       "      <th>luxury</th>\n",
       "      <th>inch</th>\n",
       "      <th>competence</th>\n",
       "      <th>irksomeness</th>\n",
       "      <th>distance</th>\n",
       "      <th>...</th>\n",
       "      <th>basin</th>\n",
       "      <th>apply</th>\n",
       "      <th>earnestly</th>\n",
       "      <th>illiterate</th>\n",
       "      <th>8th</th>\n",
       "      <th>personage</th>\n",
       "      <th>screw</th>\n",
       "      <th>text_sentence</th>\n",
       "      <th>text_source</th>\n",
       "      <th>num_words</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(Alice, was, beginning, to, get, very, tired, of, sittin...</td>\n",
       "      <td>Carroll</td>\n",
       "      <td>67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(So, she, was, considering, in, her, own, mind, (, as, w...</td>\n",
       "      <td>Carroll</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(There, was, nothing, so, VERY, remarkable, in, that, ;,...</td>\n",
       "      <td>Carroll</td>\n",
       "      <td>33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(Oh, dear, !)</td>\n",
       "      <td>Carroll</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(I, shall, be, late, !, ')</td>\n",
       "      <td>Carroll</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 1974 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  repent fortnight flavour repeat key luxury inch competence irksomeness  \\\n",
       "0      0         0       0      0   0      0    0          0           0   \n",
       "1      0         0       0      0   0      0    0          0           0   \n",
       "2      0         0       0      0   0      0    0          0           0   \n",
       "3      0         0       0      0   0      0    0          0           0   \n",
       "4      0         0       0      0   0      0    0          0           0   \n",
       "\n",
       "  distance  ... basin apply earnestly illiterate 8th personage screw  \\\n",
       "0        0  ...     0     0         0          0   0         0     0   \n",
       "1        0  ...     0     0         0          0   0         0     0   \n",
       "2        0  ...     0     0         0          0   0         0     0   \n",
       "3        0  ...     0     0         0          0   0         0     0   \n",
       "4        0  ...     0     0         0          0   0         0     0   \n",
       "\n",
       "                                                 text_sentence text_source  \\\n",
       "0  (Alice, was, beginning, to, get, very, tired, of, sittin...     Carroll   \n",
       "1  (So, she, was, considering, in, her, own, mind, (, as, w...     Carroll   \n",
       "2  (There, was, nothing, so, VERY, remarkable, in, that, ;,...     Carroll   \n",
       "3                                                (Oh, dear, !)     Carroll   \n",
       "4                                   (I, shall, be, late, !, ')     Carroll   \n",
       "\n",
       "  num_words  \n",
       "0        67  \n",
       "1        63  \n",
       "2        33  \n",
       "3         3  \n",
       "4         6  \n",
       "\n",
       "[5 rows x 1974 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "word_counts['num_words'] = word_counts['text_sentence'].apply(lambda x: len(x))\n",
    "word_counts.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following functions count the number of adverbs, interjections and pronouns and adds the columns to the word_counts dataframe in hopes that these additional features will improve the accuracy of the 4 models:  Random Forest, Logistic Regression, Gradient Boosting, and Support Vector Machines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Count the adverbs and add the column to the word_counts dataframe\n",
    "\n",
    "def count_verbs(txt):\n",
    "    sentences = nltk.sent_tokenize(str(txt))\n",
    "    count = 0\n",
    "    for sentence in sentences:    \n",
    "        text = nltk.word_tokenize(sentence)\n",
    "        tag = nltk.pos_tag(text)\n",
    "        a = pd.Series(tag)\n",
    "        a = a.map(lambda x: 1 if x[1] == \"RB\" else 0).sum()\n",
    "#        count = count + a\n",
    "    return a\n",
    "\n",
    "word_counts['count_adverbs'] = word_counts['text_sentence'].apply(lambda x: count_verbs(x))\n",
    "\n",
    "# -------------------------------------------------------------------------------------------------------\n",
    "# Count the interjections and add the column to the dataframe\n",
    "\n",
    "def count_verbs(txt):\n",
    "    sentences = nltk.sent_tokenize(str(txt))\n",
    "    count = 0\n",
    "    for sentence in sentences:    \n",
    "        text = nltk.word_tokenize(sentence)\n",
    "        tag = nltk.pos_tag(text)\n",
    "        a = pd.Series(tag)\n",
    "        a = a.map(lambda x: 1 if x[1] == \"UH\" else 0).sum()\n",
    "#        count = count + a\n",
    "    return a\n",
    "\n",
    "word_counts['count_inter'] = word_counts['text_sentence'].apply(lambda x: count_verbs(x))\n",
    "\n",
    "# ------------------------------------------------------------------------------------------------------\n",
    "# Count the pronouns and add the column to the dataframe\n",
    "\n",
    "def count_verbs(txt):\n",
    "    sentences = nltk.sent_tokenize(str(txt))\n",
    "    count = 0\n",
    "    for sentence in sentences:    \n",
    "        text = nltk.word_tokenize(sentence)\n",
    "        tag = nltk.pos_tag(text)\n",
    "        a = pd.Series(tag)\n",
    "        a = a.map(lambda x: 1 if x[1] == \"PRP\" else 0).sum()\n",
    "#        count = count + a\n",
    "    return a\n",
    "\n",
    "word_counts['count_pronoun'] = word_counts['text_sentence'].apply(lambda x: count_verbs(x))\n",
    "\n",
    "# ------------------------------------------------------------------------------------------------------\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the first 5 lines of the dataframe to make sure the knew features are captured."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>repent</th>\n",
       "      <th>fortnight</th>\n",
       "      <th>flavour</th>\n",
       "      <th>repeat</th>\n",
       "      <th>key</th>\n",
       "      <th>luxury</th>\n",
       "      <th>inch</th>\n",
       "      <th>competence</th>\n",
       "      <th>irksomeness</th>\n",
       "      <th>distance</th>\n",
       "      <th>...</th>\n",
       "      <th>illiterate</th>\n",
       "      <th>8th</th>\n",
       "      <th>personage</th>\n",
       "      <th>screw</th>\n",
       "      <th>text_sentence</th>\n",
       "      <th>text_source</th>\n",
       "      <th>num_words</th>\n",
       "      <th>count_adverbs</th>\n",
       "      <th>count_inter</th>\n",
       "      <th>count_pronoun</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(Alice, was, beginning, to, get, very, tired, of, sittin...</td>\n",
       "      <td>Carroll</td>\n",
       "      <td>67</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(So, she, was, considering, in, her, own, mind, (, as, w...</td>\n",
       "      <td>Carroll</td>\n",
       "      <td>63</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(There, was, nothing, so, VERY, remarkable, in, that, ;,...</td>\n",
       "      <td>Carroll</td>\n",
       "      <td>33</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(Oh, dear, !)</td>\n",
       "      <td>Carroll</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>(I, shall, be, late, !, ')</td>\n",
       "      <td>Carroll</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 1977 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  repent fortnight flavour repeat key luxury inch competence irksomeness  \\\n",
       "0      0         0       0      0   0      0    0          0           0   \n",
       "1      0         0       0      0   0      0    0          0           0   \n",
       "2      0         0       0      0   0      0    0          0           0   \n",
       "3      0         0       0      0   0      0    0          0           0   \n",
       "4      0         0       0      0   0      0    0          0           0   \n",
       "\n",
       "  distance  ... illiterate 8th personage screw  \\\n",
       "0        0  ...          0   0         0     0   \n",
       "1        0  ...          0   0         0     0   \n",
       "2        0  ...          0   0         0     0   \n",
       "3        0  ...          0   0         0     0   \n",
       "4        0  ...          0   0         0     0   \n",
       "\n",
       "                                                 text_sentence text_source  \\\n",
       "0  (Alice, was, beginning, to, get, very, tired, of, sittin...     Carroll   \n",
       "1  (So, she, was, considering, in, her, own, mind, (, as, w...     Carroll   \n",
       "2  (There, was, nothing, so, VERY, remarkable, in, that, ;,...     Carroll   \n",
       "3                                                (Oh, dear, !)     Carroll   \n",
       "4                                   (I, shall, be, late, !, ')     Carroll   \n",
       "\n",
       "  num_words count_adverbs count_inter count_pronoun  \n",
       "0        67             2           0             3  \n",
       "1        63             5           0             3  \n",
       "2        33             4           0             2  \n",
       "3         3             0           1             0  \n",
       "4         6             1           0             1  \n",
       "\n",
       "[5 rows x 1977 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "word_counts.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Build the models  \n",
    "The \"text_source\" is what the models are trying to predict."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Fred\\Anaconda3\\lib\\site-packages\\sklearn\\ensemble\\forest.py:246: FutureWarning: The default value of n_estimators will change from 10 in version 0.20 to 100 in 0.22.\n",
      "  \"10 in version 0.20 to 100 in 0.22.\", FutureWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.9774305555555556\n",
      "\n",
      "Test set score: 0.8880208333333334\n"
     ]
    }
   ],
   "source": [
    "from sklearn import ensemble\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "rfc = ensemble.RandomForestClassifier()\n",
    "Y = word_counts['text_source']\n",
    "X = np.array(word_counts.drop(['text_sentence','text_source'], 1))\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, \n",
    "                                                    Y,\n",
    "                                                    test_size=0.4,\n",
    "                                                    random_state=0)\n",
    "# normalize the training data\n",
    "X_train = sklearn.preprocessing.normalize(X_train)\n",
    "\n",
    "# normalize the test data\n",
    "X_test = sklearn.preprocessing.normalize(X_test)\n",
    "\n",
    "train = rfc.fit(X_train, y_train)\n",
    "\n",
    "print('Training set score:', rfc.score(X_train, y_train))\n",
    "print('\\nTest set score:', rfc.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Fred\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(576, 1975) (576,)\n",
      "Training set scorae: 0.8611111111111112\n",
      "\n",
      "Test set score: 0.8671875\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "lr = LogisticRegression(penalty='l2') # No need to specify l2 as it's the default. But we put it for demonstration.\n",
    "train = lr.fit(X_train, y_train)\n",
    "print(X_train.shape, y_train.shape)\n",
    "print('Training set scorae:', lr.score(X_train, y_train))\n",
    "print('\\nTest set score:', lr.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Gradient Boosting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.9809027777777778\n",
      "\n",
      "Test set score: 0.9192708333333334\n"
     ]
    }
   ],
   "source": [
    "clf = ensemble.GradientBoostingClassifier()\n",
    "train = clf.fit(X_train, y_train)\n",
    "\n",
    "print('Training set score:', clf.score(X_train, y_train))\n",
    "print('\\nTest set score:', clf.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Support Vector Machines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set score: 0.8611111111111112\n",
      "\n",
      "Test set score: 0.8671875\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "\n",
    "sv_c = SVC(gamma='auto')\n",
    "train = sv_c.fit(X_train, y_train)\n",
    "\n",
    "print('Training set score:', sv_c.score(X_train, y_train))\n",
    "print('\\nTest set score:', sv_c.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluation and Conclusion  \n",
    "\n",
    "The final accuracy scores (for test data set) for the 4 models are as follows:  \n",
    "\n",
    "Random Forest:  0.89  \n",
    "Logistic Regression:  0.87  \n",
    "Gradient Boosting:  0.92  \n",
    "Support Vector Machines:  0.86  \n",
    "\n",
    "Including 10% of 'Emma' (instead of using just 2% before) increased the accuracy significantly.  Using only 10% of Alice and only about 2% of Emma (in the previous model) produced accuracy scores in the range of 55% to 67%.  This is much, much worse than the scores listed above.  To reiterate, the new features implemented in this model are as follows:\n",
    "\n",
    "- the number of words in each sentence  \n",
    "- the number of adverbs\n",
    "- the number of interjections  \n",
    "- the number of pronouns  "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}