Basic Three Layer Neural Network in Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As part of understanding neural networks I was reading Make Your Own Neural Network by Tariq Rashid. The book itself can be painful to work through, as it is written for a novice, not just in algorithms and data analysis, but also in programming. Although the code is a verbatim transcription from the text (see Source section), I published it to better understand how neural networks are designed, made easy by the use of a Jupyter Notebook, not to present this as my own work, although I do hope that this helps others develop their talents with data analytics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code itself  develops as follows:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Constructor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- set number of nodes in each input, hidden, output layer\n",
    "- link weight matrices, wih and who \n",
    "- weights inside the arrays are w_i_j, where link is from node i to node j in the next layer \n",
    "- learning rate \n",
    "- activation function is the sigmoid function "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the Training Function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- convert inputs list to 2d array \n",
    "- calculate signals into hidden layer \n",
    "- calculate the signals emerging from hidden layer \n",
    "- calculate signals into final output layer \n",
    "- calculate the signals emerging from final output layer \n",
    "- output layer error is the (target ­ actual) \n",
    "- hidden layer error is the output_errors, split by weights, recombined at hidden nodes \n",
    "- update the weights for the links between the hidden and output layers \n",
    "- update the weights for the links between the input and hidden layers "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the Query Function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- convert inputs list to 2d array \n",
    "- calculate signals into hidden layer \n",
    "- calculate the signals emerging from hidden layer \n",
    "- calculate signals into final output layer \n",
    "- calculate the signals emerging from final output layer "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# for scientific computing with Python\n",
    "import numpy\n",
    "\n",
    "# for the sigmoid function expit()\n",
    "import scipy.special"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Develop Neural Network Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#neural network class definition\n",
    "class neuralNetwork:\n",
    "    \n",
    "    #  for scientific computing with Python\n",
    "    import numpy\n",
    "\n",
    "    # for the sigmoid function expit()\n",
    "    import scipy.special\n",
    "\n",
    "    #initialise the neural network \n",
    "    def __init__(self, inputnodes, hiddennodes, outputnodes, learningrate):\n",
    "    \n",
    "        #set number of nodes in each input, hidden, output layer\n",
    "        self.inodes = inputnodes \n",
    "        self.hnodes = hiddennodes \n",
    "        self.onodes = outputnodes \n",
    "\n",
    "        #link weight matrices, wih and who \n",
    "        #weights inside the arrays are w_i_j, where link is from node i to node j in the next layer \n",
    "        #w11 w21 \n",
    "        #w12 w22 etc \n",
    "        self.wih = numpy.random.normal(0.0, pow(self.hnodes, -0.5), (self.hnodes, self.inodes)) \n",
    "        self.who = numpy.random.normal(0.0, pow(self.onodes, -0.5), (self.onodes, self.hnodes)) \n",
    "\n",
    "        #learning rate \n",
    "        self.lr = learningrate \n",
    "\n",
    "        #activation function is the sigmoid function \n",
    "        self.activation_function = lambda x: scipy.special.expit(x) \n",
    "\n",
    "        pass \n",
    "\n",
    "    #train the neural network \n",
    "    def train(self, inputs_list, targets_list): \n",
    "        \n",
    "        #convert inputs list to 2d array \n",
    "        inputs = numpy.array(inputs_list, ndmin=2).T \n",
    "        targets = numpy.array(targets_list, ndmin=2).T \n",
    "\n",
    "        #calculate signals into hidden layer \n",
    "        hidden_inputs = numpy.dot(self.wih, inputs) \n",
    "\n",
    "        #calculate the signals emerging from hidden layer \n",
    "        hidden_outputs = self.activation_function(hidden_inputs) \n",
    "\n",
    "        #calculate signals into final output layer \n",
    "        final_inputs = numpy.dot(self.who, hidden_outputs) \n",
    "\n",
    "        #calculate the signals emerging from final output layer \n",
    "        final_outputs = self.activation_function(final_inputs)\n",
    "\n",
    "        #output layer error is the (target ­ actual) \n",
    "        output_errors = targets - final_outputs \n",
    "\n",
    "        #hidden layer error is the output_errors, split by weights, recombined at hidden nodes \n",
    "        hidden_errors = numpy.dot(self.who.T, output_errors) \n",
    "\n",
    "        #update the weights for the links between the hidden and output layers \n",
    "        self.who += self.lr * numpy.dot((output_errors * final_outputs * (1.0 - final_outputs)), numpy.transpose(hidden_outputs)) \n",
    "\n",
    "        #update the weights for the links between the input and hidden layers \n",
    "        self.wih += self.lr * numpy.dot((hidden_errors * hidden_outputs * (1.0 - hidden_outputs)), numpy.transpose(inputs)) \n",
    "        \n",
    "        pass \n",
    "\n",
    "    #query the neural network \n",
    "    def query(self, inputs_list): \n",
    "        \n",
    "        #convert inputs list to 2d array \n",
    "        inputs = numpy.array(inputs_list, ndmin=2).T \n",
    "\n",
    "        #calculate signals into hidden layer \n",
    "        hidden_inputs = numpy.dot(self.wih, inputs) \n",
    "\n",
    "        #calculate the signals emerging from hidden layer \n",
    "        hidden_outputs = self.activation_function(hidden_inputs) \n",
    "\n",
    "        #calculate signals into final output layer \n",
    "        final_inputs = numpy.dot(self.who, hidden_outputs) \n",
    "\n",
    "        #calculate the signals emerging from final output layer \n",
    "        final_outputs = self.activation_function(final_inputs) \n",
    "        \n",
    "        return final_outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run and Outptut Result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.4353349 ],\n",
       "       [ 0.64146723],\n",
       "       [ 0.3650745 ]])"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nn = neuralNetwork(3, 3, 3, .5)\n",
    "nn.train([1.0, 0.5, -1.5], [1.0, 0.5, -1.5])\n",
    "nn.query([1.0, 0.5, -1.5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Source"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tariq's GitHub: https://github.com/makeyourownneuralnetwork/makeyourownneuralnetwork\n",
    "\n",
    "Book: Make Your Own Neural Network by Tariq Rashid: https://www.goodreads.com/book/show/29746976-make-your-own-neural-network"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6",
   "language": "python",
   "name": "python36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}