02_Convolutional_Neural_Network_CN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow 教程 #02\n",
    "# 卷积神经网络\n",
    "\n",
    "by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)/[GitHub中文](https://github.com/Hvass-Labs/TensorFlow-Tutorials-Chinese)\n",
    "/ [GitHub](https://github.com/Hvass-Labs/TensorFlow-Tutorials) / [Videos on YouTube](https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ)\n",
    "\n",
    "中文翻译 [thrillerist](https://zhuanlan.zhihu.com/insight-pixel)修订[ZhouGeorge](https://github.com/ZhouGeorge)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 介绍\n",
    "\n",
    "先前的教程展示了一个简单的线性模型，对MNIST数据集中手写数字的识别率达到了91%。\n",
    "\n",
    "在这个教程中，我们会在TensorFlow中实现一个简单的卷积神经网络，它能达到大约99%的分类准确率，如果你做了一些建议的练习，准确率还可能更高。\n",
    "\n",
    "卷积神经网络在一张输入图片上移动一个小的滤波器。这意味着在遍历整张图像来识别模式时，要重复使用这些滤波器。这让卷积神经网络在拥有相同数量的变量时比全连接网络（Fully-Connected）更强大，也让卷积神经网络训练得更快。\n",
    "\n",
    "你应该熟悉基本的线性代数、Python和Jupyter Notebook编辑器。如果你是TensorFlow新手，在本教程之前应该先学习第一篇教程。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 流程图"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面的图表直接显示了之后实现的卷积神经网络中数据的传递。\n",
    "\n",
    "![Flowchart](images/02_network_flowchart.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "输入图像在第一层卷积层里使用权重过滤器处理。结果在16张新图里，每张代表了卷积层里一个过滤器（的处理结果）。图像经过降采样，分辨率从28x28减少到14x14。\n",
    "\n",
    "16张小图在第二个卷积层中处理。这16个通道以及这层输出的每个通道都需要一个过滤权重。总共有36个输出，所以在第二个卷积层有16 x 36 = 576个滤波器。输出图再一次降采样到7x7个像素。\n",
    "\n",
    "第二个卷积层的输出是36张7x7像素的图像。它们被转换到一个长为7 x 7 x 36 = 1764的向量中去，它作为一个有128个神经元（或元素）的全连接网络的输入。这些又输入到另一个有10个神经元的全连接层中，每个神经元代表一个类别，用来确定图像的类别，即图像上的数字。\n",
    "\n",
    "卷积滤波一开始是随机挑选的，因此分类也是随机完成的。根据交叉熵（cross-entropy）来测量输入图预测值和真实类别间的错误。然后优化器用链式法则自动地将这个误差在卷积网络中传递，更新滤波权重来提升分类质量。这个过程迭代了几千次，直到分类误差足够低。\n",
    "\n",
    "这些特定的滤波权重和中间图像是一个优化结果，和你执行代码所看到的可能会有所不同。\n",
    "\n",
    "注意，这些在TensorFlow上的计算是在一部分图像上执行，而非单独的一张图，这使得计算更有效。也意味着在TensorFlow上实现时，这个流程图实际上会有更多的数据维度。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 卷积层"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面的图片展示了在第一个卷积层中处理图像的基本思想。输入图片描绘了数字7，这里显示了它的四张拷贝，我们可以很清晰的看到滤波器是如何在图像的不同位置移动。在滤波器的每个位置上，计算滤波器以及滤波器下方图像像素的点乘，得到输出图像的一个像素。因此，在整张输入图像上移动时，会有一张新的图像生成。\n",
    "\n",
    "红色的滤波权重表示滤波器对输入图的黑色像素有正响应，蓝色的代表有负响应。\n",
    "\n",
    "在这个例子中，很明显这个滤波器识别数字7的水平线段，在输出图中可以看到它对线段的强烈响应。\n",
    "\n",
    "![Convolution example](images/02_convolution.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "滤波器遍历输入图的移动步长称为stride。在水平和竖直方向各有一个stride。\n",
    "\n",
    "在下面的源码中，两个方向的stride都设为1，这说明滤波器从输入图像的左上角开始，下一步移动到右边1个像素去。当滤波器到达图像的右边时，它会返回最左边，然后向下移动1个像素。持续这个过程，直到滤波器到达输入图像的右下角，同时，也生成了整张输出图片。\n",
    "\n",
    "当滤波器到达输入图的右端或底部时，它会用零（白色像素）来填充。因为输出图要和输入图一样大。\n",
    "\n",
    "此外，卷积层的输出可能会传递给修正线性单元（ReLU），它用来保证输出是正值，将负值置为零。输出还会用最大池化(max-pooling)进行降采样，它使用了2x2的小窗口，只保留像素中的最大值。这让输入图分辨率减小一半，比如从28x28到14x14。\n",
    "\n",
    "第二个卷积层更加复杂，因为它有16个输入通道。我们想给每个通道一个单独的滤波，因此需要16个。另外，我们想从第二个卷积层得到36个输出，因此总共需要16 x 36 = 576个滤波器。要理解这些如何工作可能有些困难。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 导入"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/magnus/anaconda3/envs/tf-gpu/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "from sklearn.metrics import confusion_matrix\n",
    "import time\n",
    "from datetime import timedelta\n",
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用Python3.6（Anaconda）开发，TensorFlow版本是："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.9.0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 神经网络的配置\n",
    "\n",
    "方便起见，在这里定义神经网络的配置，你可以很容易找到或改变这些数值，然后重新运行Notebook。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convolutional Layer 1.\n",
    "filter_size1 = 5          # Convolution filters are 5 x 5 pixels.\n",
    "num_filters1 = 16         # There are 16 of these filters.\n",
    "\n",
    "# Convolutional Layer 2.\n",
    "filter_size2 = 5          # Convolution filters are 5 x 5 pixels.\n",
    "num_filters2 = 36         # There are 36 of these filters.\n",
    "\n",
    "# Fully-connected layer.\n",
    "fc_size = 128             # Number of neurons in fully-connected layer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 载入数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MNIST数据集大约12MB，如果没在文件夹中找到就会自动下载。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mnist import MNIST\n",
    "data = MNIST(data_dir=\"data/MNIST/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在已经载入了MNIST数据集，它由70,000张图像和对应的标签（比如图像的类别）组成。数据集分成三份互相独立的子集。我们在教程中只用训练集和测试集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of:\n",
      "- Training-set:\t\t55000\n",
      "- Validation-set:\t5000\n",
      "- Test-set:\t\t10000\n"
     ]
    }
   ],
   "source": [
    "print(\"Size of:\")\n",
    "print(\"- Training-set:\\t\\t{}\".format(data.num_train))\n",
    "print(\"- Validation-set:\\t{}\".format(data.num_val))\n",
    "print(\"- Test-set:\\t\\t{}\".format(data.num_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了方便起见，复制一些数据的维度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The number of pixels in each dimension of an image.\n",
    "img_size = data.img_size\n",
    "\n",
    "# The images are stored in one-dimensional arrays of this length.\n",
    "img_size_flat = data.img_size_flat\n",
    "\n",
    "# Tuple with height and width of images used to reshape arrays.\n",
    "img_shape = data.img_shape\n",
    "\n",
    "# Number of classes, one class for each of 10 digits.\n",
    "num_classes = data.num_classes\n",
    "\n",
    "# Number of colour channels for the images: 1 channel for gray-scale.\n",
    "num_channels = data.num_channels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用来绘制图片的帮助函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个函数用来在3x3的栅格中画9张图像，然后在每张图像下面写出真实类别和预测类别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_images(images, cls_true, cls_pred=None):\n",
    "    assert len(images) == len(cls_true) == 9\n",
    "    \n",
    "    # Create figure with 3x3 sub-plots.\n",
    "    fig, axes = plt.subplots(3, 3)\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Plot image.\n",
    "        ax.imshow(images[i].reshape(img_shape), cmap='binary')\n",
    "\n",
    "        # Show true and predicted classes.\n",
    "        if cls_pred is None:\n",
    "            xlabel = \"True: {0}\".format(cls_true[i])\n",
    "        else:\n",
    "            xlabel = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n",
    "\n",
    "        # Show the classes as the label on the x-axis.\n",
    "        ax.set_xlabel(xlabel)\n",
    "        \n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "    \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制几张图像来看看数据是否正确"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUMAAAD5CAYAAAC9FVegAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAHihJREFUeJzt3XmUFNXZx/HvA0LYVQQFFWdOwAVCFBWDu0aBKCogccG4EGM0osEtAaNx1xglKBzRE7YD4QQNigKCUVFAEV8EJIIi4wYiCsRlhLggIsJ9/5i5XdUzPXtXVU/7+5zjmequ6qpnvPSdp27dxZxziIj80DVIOgARkVygylBEBFWGIiKAKkMREUCVoYgIoMpQRARQZSgiAqgyFBEBVBmKiACwS00ObtOmjSssLIwolNzzwQcfUFxcbEnHESeVcf5TGWdWo8qwsLCQZcuW1T6qeqZ79+5JhxA7lXH+UxlnpttkERFUGYqIAKoMRUQAVYYiIoAqQxERoIZPk0Vqa8SIEQBs3boVgDfeeAOAxx9/vNyxgwcPBuCoo44C4MILL4wjRPmBU2YoIoIyQ4nYueeeC8C0adMy7jcr3xd2zJgxAMydOxeAE044AYD99tsvihAlQe+++y4ABx54IAAPPPAAAEOGDIk9FmWGIiIoM5QI+GwQKs4IDzroIABOOeUUAN5///3UvlmzZgGwevVqAKZMmQLAjTfemP1gJVHLly8HoEGDkrxsn332SSwWZYYiIigzlCzy411nzJhRbl/Xrl2BIOtr06YNAC1atADgu+++Sx3bo0cPAF5//XUAPv/884gilqStWLECCP4dDBgwILFYlBmKiBBDZuj7kY0fPx6AvffeO7WvSZMmAJx//vkAtGvXDoBOnTpFHZZE4L///S8AzrnUez4jnDNnDgDt27fP+FnfDxHgrbfeStt3+umnZzVOSd7KlSsBGD16NAAXXXRRkuEAygxFRIAYMsOhQ4cCJRMsVsT3K2vVqhUAXbp0ycq1O3ToAMCwYcOAH+bcdXE644wzgOApMEDLli0BaN26daWfffTRR1Pb4fZDyU/vvPMOAFu2bAHSeyAkRZmhiAiqDEVEgBhukydMmAAE3STCt8BFRUVA0PHyxRdfBGDx4sVAMPzqww8/rPD8jRo1AoKuGr4RP3wef7us2+R4FBQUVPvYv/3tb0AwLCvMd7HxPyV/DB8+HChZggBy47upzFBEhBgyw5NPPjntZ5gfiuVt3rwZCDJF/9fi1VdfrfD8P/rRj4BgoLcf5gWwadMmADp27Fir2CU6Tz31FAC33HILANu2bUvt22uvvQC45557AGjWrFnM0UkUwg9R/Xfaf2+bN2+eREhplBmKiJBjw/F23313AE466aS09zNllWU98cQTQJBdAhx88MEADBw4MFshSpb4oXvhjNDz3Sz81F2SHxYsWFDuvbZt2yYQSWbKDEVEyLHMsDY+/fRTAK644gogfSiYb4+qqsOvxKd///5AMDzPGzRoUGr7rrvuijUmiYdf6iHMD4jIBcoMRUTIg8zwoYceAoIMcbfddkvt80+qJHm+/+eiRYuAoK3QtxnddNNNqWP9dE6SH1555RUAJk2alHrv0EMPBaBXr16JxJSJMkMREepxZvjyyy8DQV8078knn0xt++mjJHl+0s7i4uK09/30beoLmr/mzZsHpPf08H2M/TR+uUCZoYgIqgxFRIB6fJv89NNPA8Hcdz179gTgqKOOSiwmKc+veeKHWHonnngiAHfccUfcIUnM/CQtYWeffXYCkVROmaGICPUwM9y6dSsAzz77LBBM1HD77bcDwZRekpzwanZ33303UH726m7dugHqRpPPPv74YwAWLlwIpE+icuaZZyYSU2WUGYqIUA8zQz8ZqG+DOvXUUwE4+uijE4tJ0t13332p7aVLl6bt88Px1FaY//7xj38A8MknnwDBdzVXKTMUEaGeZIZ+IlCAO++8E4Bdd90VgJtvvjmRmKRi999/f4X7/PBJtRXmv3Xr1qW99lP05SplhiIi5Hhm6J9KXnXVVan3vv/+ewD69OkDqF9hfePLtDpP/X3274/dvn07AF988UW5Y/1Qr5EjR2Y8V8OGDVPb9957L6DlBKI2e/bstNenn356QpFUjzJDERFUGYqIADl6m7xjxw4gmNli7dq1qX2dOnUCggcpUr/4dWmq45xzzgGgffv2QNBFY+rUqXWKwa++F55DUbLHd7L25VVfKDMUESFHM8M1a9YAwQpqYb7bhua/y13+4RbAzJkza32exx57rMpj/MOVBg3S/6737dsXCNbeDjv22GNrHZNUbcaMGUDwsNPPap3rqx0qMxQRIccyQ99Js3fv3mnvjxgxIrWd64/nBaZPn57aHj58OFB+ogavqKgIqLwd8JJLLgGgoKCg3L5f/vKXAHTu3Ll2wUrWfPPNNwA888wzae/76brC3ZtykTJDERFyLDMcO3YsUH4YT7itwcxijUnqprrr4j7yyCMRRyJR8+23foXKfv36AXD11VcnFlNNKDMUESFHMkPfL+nBBx9MOBIRqS2fGfp1kusbZYYiIuRIZujXQP7qq6/S3vejTTTdk4hETZmhiAiqDEVEgBy5TS7Lr5w2b948AFq3bp1kOCLyA6DMUESEHMkMb7jhhrSfIiJxU2YoIgKYc676B5t9Bqyr8sD8UeCca5t0EHFSGec/lXFmNaoMRUTylW6TRURQZSgiAkT8NNnM9gDmlb5sB+wAPit9/TPnXOYZP+t2zS5AeD6ojsANzjnNAhGBhMq4AJgM7Ak44O8q3+gkUcal150M9AE2OOe6RXGNtOvF1WZoZrcBXzvnRpR530rj2BnBNRsBG4DDnHPrs31+SRdXGZvZ3sCezrkVZtYKWA6c6px7Nxvnl4rF+T02sxOArcC4OCrDRG6TzayTmRWZ2cPAKqCDmf0vtH+gmU0o3d7LzKab2TIzW2pmR9bgUr2At1QRxi/KMnbObXTOrSjd/hJ4G9gnut9GMon6e+ycWwBsiuwXKCPJNsODgJHOuS6UZG8VeQAY7pzrDpwD+P+5PcxsTBXXGAj8KxvBSq1EXsZm9mOgK/BqdkKWGorjexyLJEegrHHOlV8LtLyewIGh6f53N7OmzrklwJKKPmRmTYDTgOvqHKnUVtRl3Ap4AhjinPu6ztFKbURaxnFKsjLcEtreCYQXN2kS2jZq10h7GrDEOVdcy/ik7iIrYzNrDEwHJjnnZtUpSqmLqL/HscmJrjWlja6bzWx/M2sAnBnaPRe40r8ws+o2pJ6HbpFzRjbLuLSx/h/ACufcAxGEK7UQ0fc4NjlRGZa6HpgDLALCDzyuBI4xszfMrAi4FCpvazCzlsDPgZnRhiw1lK0yPoGSP3a9zGxF6X+/iDh2qZ5sfo+nAQuBLma23sx+HWXgGo4nIkJuZYYiIolRZSgigipDERFAlaGICKDKUEQEqGGn6zZt2rjCwsKIQsk9H3zwAcXFxVb1kflDZZz/VMaZ1agyLCwsZNmy6oy8yQ/du3dPOoTYqYzzn8o4M90mi4igylBEBFBlKCICqDIUEQFUGYqIAKoMRUSAZCd3rdCWLSXzRQ4dOhSAMWOCGX78Y/Jp06YBUFBQEHN0IpKPlBmKiJCjmeHGjRsBGD9+PAANGzZM7fOdRWfPng3A73//+5ijk9p47bXXABgwYABQMiqgtp577rnUdufOnQHo0KFD7YOTxPjvcd++fQEYPXo0AIMHD04dE/7+R0mZoYgIOZYZfvbZZwAMGjQo4Ugk2+bMmQPAtm3b6nyuWbOC9Z8mTpwIwNSpU+t8XonP559/DqRngABDhgwB4JJLLkm917Rp01hiUmYoIkKOZIYPPFCywNnMmSXrN736atXrgS9cuBAAv4bLIYccAsDxxx8fRYhSS99//z0ATz/9dNbOGR54f//99wNBD4TmzZtn7ToSnZdeegmADRvS150/77zzAGjSpEm5z0RNmaGICDmSGV5zzTVAzZ4aTZ8+Pe3nfvvtB8Bjjz2WOubwww/PVohSSy+88AIAixYtAuD666+v8zk3bdqU2l61ahUA33zzDaDMMJeF24vvuuuujMdceOGFAJQsjR0vZYYiIqgyFBEBEr5N7tOnDxA8BNmxY0eVn2nTpg0Q3A6tW7cOgLVr1wJwxBFHpI7duXNn9oKValu5cmVqe+DAgQB06tQJgBtvvLHO5w93rZH644033kht+0743i67lFRFp556aqwxhSkzFBEhgcxwwYIFqe23334bCBpLK3qAcvnll6e2e/fuDcCuu+4KwPz58wH4y1/+Uu5zf//734HyHTslWuGy8A82pkyZAkCLFi1qfV7/4CT8byiJhnapHf+wM5NevXrFGElmygxFRIgxM/QD830bEkBxcXHGY303mbPOOguAW2+9NbWvWbNmacf6KbzGjh1b7pzDhg0D4NtvvwWCSR0aNWpUu19CKvX4448D6R2sfVthuC23tnx3jHA2eOKJJwKw22671fn8Eq1wRu81btwYgLvvvjvucMpRZigiQoyZ4fbt24GKs0EIhtI9+uijQPDkuDI+M/RPKa+77rrUPj9Ey2eIfpqgjh071ih2qR4/4a7//w7Zaa/1dxWPPPIIEDx5BLjpppsAZfu5zHe4f+WVV8rt83d63bp1izWmTJQZioiQI8PxfHvSpEmTgOplhGX5rO/hhx9Ovbd06dIsRCdV+eKLLwBYvHhxuX1XXHFFnc8/btw4IJjirUuXLql9J510Up3PL9GqbOKVXOrpocxQRIQEMsNMo0yWLFlS5/P6USzhUSdlR7b4p9K+z5tkhx+Av379eiCYhilb1qxZk/a6a9euWT2/RCtTZuif/mfjziFblBmKiKDKUEQEiPE22a99HNVKV36VreXLl6feKzvM7/bbb4/k2j90LVu2BILuEeGJGvwQutatW9f4vJ9++ikQdNnxjjnmmFrFKfF6+eWXgaBLVJgfTrvvvvvGGlNllBmKiBBjZvjUU09l9Xy+m0VRURFQ+XAe31VHHXOj4Vcv80Pv/LA8gNNOOw1I7wyfyZtvvpna9g9M/PRsZSdjaNBAf8PrA78Cnn+QGZYLEzOUpX9VIiLkSKfr2vDTRD300EMVHlNYWAjA5MmTgWACCInGbbfdBqRnAv6OIDxBRyZt27ZNbftMsKKhmxdffHFdwpSYlG3rDU+mcdlll8UdTpWUGYqIUA8zQ79UgJ8YtjJ+2NZxxx0XaUxSonPnzkD6CoX+6X7ZjtNl+enawgYNGgSU7yTv2yglN/nO92WfIoefHGdjSrdsU2YoIkKMmWFliz4988wzaa8vvfRSADZu3Fjheaoz3Xu2n2BLzR166KFpP2vixz/+ccb3w/0Yf/rTn9YuMImMn7Kr7FPkfv36JRFOtSkzFBFBlaGICBDjbbKft8zPOh3mO+aWHaqXaeiev82uzkp6Ur/526yyt1u6Nc5tvrO15wc9XHPNNUmEU23KDEVEiDEzHDBgAADDhw9PvVfZeihV8X9tfHeO8ePHA9C+fftan1Nyi39IprWR65c5c+akve7QoQMQTM6Qq5QZiogQY2boV7HzK98BzJw5E4BRo0bV+Hx//vOfgWAtZMk/fr1rT52tc5tfAXP16tVp7zdp0gTI/YlSlBmKiJDAcDy/NnJ4u3fv3kCwCpqfqPWMM84A4He/+13qM/7JYniFNMlPfrVEP8D/lltuSTIcqYKfWs0PtVu1ahUA+++/f2Ix1YQyQxERcmSihlNOOSXtpwgEGca1114LaI3kXOf7/vrp9XwvgMMOOyyxmGpCmaGICDmSGYpk4tuOpX7Ze++9AZg4cWLCkdSMMkMREVQZiogAqgxFRABVhiIigCpDERFAlaGICACWabX7Cg82+wxYF104OafAOde26sPyh8o4/6mMM6tRZSgikq90mywigipDERFAlaGICBDx2GQz2wOYV/qyHbAD+Kz09c+cc99FdN0+wEigITDWOfe3KK4jyZVx6bV3AV4D3nfO9Y/qOj90CX6PJwN9gA3OuW5RXCPtenE9QDGz24CvnXMjyrxvpXHszNJ1GgHvAD8HPgaWAb90zr2bjfNLxeIq49B5hwHdgGaqDOMRZxmb2QnAVmBcHJVhIrfJZtbJzIrM7GFgFdDBzP4X2j/QzCaUbu9lZtPNbJmZLTWzI6s4/ZHAW865dc65bcBjQL+ofhfJLOIyxswKgF7ApKh+B6lc1GXsnFsAbIrsFygjyTbDg4CRzrkuwIZKjnsAGO6c6w6cA/j/uT3MbEyG4/cBPgq9Xl/6nsQvqjIGGAUMBdQ3LFlRlnGskpzPcI1zblk1jusJHBhaO3d3M2vqnFsCLIksOsmGSMrYzPoDHznnVphZz+yFK7WQN9/jJCvDLaHtnUB4pfAmoW2jZo20G4AOodf7UvlfLIlOVGV8NDDAzPqWnqeVmU12zg2qU7RSG1GVcexyomtNaaPrZjPb38waAGeGds8FrvQvzKyqhtTFQBczKzCzH1GSks/KdsxSM9ksY+fcMOfcvs65QuAC4DlVhMnL8vc4djlRGZa6HpgDLKKknc+7EjjGzN4wsyLgUqi4rcE5tx24CngeKAKmOOfeiTp4qZaslLHktKyVsZlNAxZSktysN7NfRxm4xiaLiJBbmaGISGJUGYqIoMpQRARQZSgiAtSwn2GbNm1cYWFhRKHkng8++IDi4mKr+sj8oTLOfyrjzGpUGRYWFrJsWXU6m+eH7t27Jx1C7FTG+U9lnJluk0VEUGUoIgKoMhQRAVQZiogAqgxFRABVhiIigCpDEREg2cldRUQA2Lx5MwAffvhhhccUFBQAMHLkSAC6du0KwAEHHADAIYccUqcYlBmKiJBwZvjpp58CcM455wBw9NFHA3DZZZcBJT3ls+GLL74A4KWXXgLglFNOAaBRo0ZZOb+I1MxTTz0FwOzZswF48cUXAXjvvfcq/MyBBx4IlAyvA9i2bVva/p0767ZKqTJDERESyAx92wDAT37yEyDI3Pbaay8g+xnhYYcdBkBxcTFAalzm/vvvn5XrSPV9+eWXAPzpT38CYNWqVQDMnTs3dYwy9vywZs0aAB566CEAxo0bl9q3detWAGoy0/4770S7eocyQxERYswMfVbm2wcBPv/8cwCuvLJk0azRo0dn9Zp33XUXAGvXrgWCv0zKCOM3ZcoUAG666Sag/FNDnzEC7LHHHvEFJpFZv75kPahRo0bV6TwHHXQQEDw9jooyQxERYswMX3vtNSB4ahR2yy23ZO06b775Zmp7xIgRAJx5Zsnyreeee27WriPV47ODa6+9FgjuEMzS59ocMmRIavvBBx8EoHXr1nGEKLXgyxGCzO/YY48Fgt4ajRs3BmDXXXcFoEWLFqnPfP311wD84he/AIKsr0ePHgAceuihqWObNm0KQPPmzbP8W6RTZigigipDEREghttk37H6iSeeKLdv4sSJALRt27bO1/G3x7169Sq3b8CAAQC0bNmyzteRmvFNFf5hWUWmTp2a2n7mmWeA4GGLv4X2t12SnC1btgDp37PXX38dgJkzZ6Yde9RRRwGwfPlyIL3LnH+Atu+++wLQoEHyeVnyEYiI5IDIM8M//OEPQNC1wneABjj77LOzdp2XX34ZgI8//jj13sUXXwzABRdckLXrSNXWrVuX2p40aVLaPj+Y3newf/7558t93neW91nl+eefD0C7du2yH6xUy3fffQfAr371KyDIBgFuvPFGAHr27Jnxs5kGUey3335ZjrDulBmKiBBDZui7UPif++yzT2pfXdqA/HCeu+++GwiG/IS7bPg2SYnXihUrUtu+M/Xxxx8PwIIFCwD49ttvAXjkkUcA+Otf/5r6zOrVq4Egy+/Xrx8QtCWqy018fBcY/z3zEyuE2/mHDh0KQLNmzWKOLruUGYqIkMBEDX7qHoDevXsDsNtuuwEwePDgKj/vO237n4sXL07bn812SKmd8NRKPlP3na69Jk2aAPCb3/wGgMcffzy1zw/w94P4fcahp8nx80+I77nnHiCYYHXhwoWpY3yn6vpOmaGICDFkhldffTUA8+fPB2Djxo2pfb79yGcATz75ZJXn88eWHc7VsWNHIGjbkOT861//Kvfev//9bwD69++f8TN+WrVMjjzySCB9OJfEY9GiRWmv/TA53z8wnygzFBEhhszw8MMPB2DlypVA+pPGZ599FoDhw4cDsOeeewIwaNCgCs934YUXAnDwwQenve+XDPAZoiTnvPPOS237bP/VV18F4O233waCfw8zZswA0if99W3I/j0/9Zov+y5dukQWu6QLt+VC8ET/9ttvT73Xt29fIH1yhfpImaGICKoMRUQAsJqsQdC9e3dXWUN3HN5//30guB3u1q0bAM899xyQnUkfvO7du7Ns2TKr+sj8kY0y3rRpU2rbl5MfYlfRA7DwwH/fgf70008H4N133wWCVRPHjBlTp/jCVMaVKztoIpOGDRsCcPnllwPBnIQfffQRAJ06dQKCNY/C/Bo4flKHKB7MVLeMlRmKiJDwusm1cccddwDBXyr/8CWbGaHUTXi43LRp0wA466yzgPIZ4lVXXQXAvffem/qM75Dtp17zQ/XmzJkDBJ2yQQ/MovbHP/4RgPvuu6/CY3bs2AEEGb3/WRP+4emJJ54IpE/pFhdlhiIi1JPM0GcXAJMnTwagVatWgFZSy3V+WiffRcNPzOC7z/hM32eDYTfffDMAb731FhB00/GfgeDfg0TDD8Pzq1r66dS2b9+eOsavc+MzxNrwk0D773p4JTw/yW/UlBmKiFBPMkPf0TPstNNOA9Ini5Xc5TPEiiYAzcSviuZXNfSZ4QsvvJA6xj+51rRe0fBPio844gggeLIfNm/ePCDIFm+77TYAli5dWuPr+bbk//znPzX+bF0pMxQRoR5mhn7tVP+US/Kfb6+aNWsWkP6k0a+xnM21t6VmTj755LTXfsitzwwbNWoEBMtwAFx66aUAjBw5EgjakpOkzFBEBFWGIiJAjt8m+2FX4RXv/KpqenDyw+HX1B02bBiQvj6vb6wfOHAgAAcccEC8wUk5fgZ7v2qef7DiZx8CeO+994BgxvqywmslxUWZoYgI9SQzDA8S79OnT9oxX331FRDMfZeL67FKdvhJOe68887Ue/5B2g033AAE63P7bjkSv86dOwNBl6hHH3203DHh7lEAu+xSUhX5LnPh4ZlxUWYoIkKOZ4aZ+L8gPgPwj+b98B0Nz8p/F110UWp77NixAEyfPh0I2qLKzoQu8fFZ+ahRo4Dg7i3ckfqTTz4BoLCwEAjK1LcBJ0GZoYgI9TAzHD9+PAATJkwA4Le//S0QDOqX/Beerm3u3LlAsJ6vn1ggFzrx/tD5nh9+rfR//vOfqX2vvPIKEGSCfgqvJCkzFBEhxzPD0aNHA3Drrbem3jv++OMBGDx4MAC77747AI0bN445OskFvveAXzbAD9krKioCtJJeLvGrG5bdzhXKDEVEyPHM8LjjjgNg/vz5CUciuc5PHnvIIYcAsHr1akCZoVSfMkMREVQZiogAOX6bLFJdfk2ctWvXJhyJ1FfKDEVEUGUoIgKoMhQRAcD8alTVOtjsM2BddOHknALnXNuqD8sfKuP8pzLOrEaVoYhIvtJtsogIqgxFRICI+xma2R7AvNKX7YAdwGelr3/mnPsuwmvvArwGvO+c6x/VdX7okipjM7sOuKT05Rjn3OgoriOJlvF6YHPp9bY553pEcZ3U9eJqMzSz24CvnXMjyrxvpXHszPL1hgHdgGaqDOMRVxmbWTdgMnAk8D3wHPAb55x6XEcszu9xaWXY1Tn3v2ydszKJ3CabWSczKzKzh4FVQAcz+19o/0Azm1C6vZeZTTezZWa21MyOrMb5C4BewKSofgepXMRl3BlY7Jzb6pzbDrwEnBnV7yKZRf09jluSbYYHASOdc12ADZUc9wAw3DnXHTgH8P9ze5jZmAo+MwoYCuhRebKiKuOVwAlm1trMmgOnAh2yG7pUU5TfYwfMN7P/mNklFRyTNUmOTV7jnFtWjeN6AgeGlgvd3cyaOueWAEvKHmxm/YGPnHMrzKxn9sKVWoikjJ1zb5rZ/cBc4GtgOSXtShK/SMq41JHOuQ1m1g543szecs4tykLMGSVZGW4Jbe8ELPS6SWjbqFkj7dHAADPrW3qeVmY22Tk3qE7RSm1EVcY458YB4wDMbDiwug5xSu1FWcYbSn9+bGZPAj8DIqsMc6JrTWmj62Yz29/MGpDe/jMXuNK/KG08r+xcw5xz+zrnCoELgOdUESYvm2VcesyepT8Lgb7A1GzGKzWXzTI2sxZm1qJ0uzklzwDezH7UgZyoDEtdD8yhpOZfH3r/SuAYM3vDzIqAS6HKtgbJTdks45mlx84ELnfOfRlh3FJ92Srj9sD/mdnrwFJghnNubpSBazieiAi5lRmKiCRGlaGICKoMRUQAVYYiIoAqQxERQJWhiAigylBEBFBlKCICwP8D3P5bzM0W5d8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f894a0bd7b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get the first images from the test-set.\n",
    "images = data.x_test[0:9]\n",
    "\n",
    "# Get the true classes for those images.\n",
    "cls_true = data.y_test_cls[0:9]\n",
    "\n",
    "# Plot the images and labels using our helper-function above.\n",
    "plot_images(images=images, cls_true=cls_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TensorFlow图\n",
    "\n",
    "TensorFlow的全部目的就是使用一个称之为计算图（computational graph）的东西，它会比直接在Python中进行相同计算量要高效得多。TensorFlow比Numpy更高效，因为TensorFlow了解整个需要运行的计算图，然而Numpy只知道某个时间点上唯一的数学运算。\n",
    "\n",
    "TensorFlow也能够自动地计算需要优化的变量的梯度，使得模型有更好的表现。这是由于图是简单数学表达式的结合，因此整个图的梯度可以用链式法则推导出来。\n",
    "\n",
    "TensorFlow还能利用多核CPU和GPU，Google也为TensorFlow制造了称为TPUs（Tensor Processing Units）的特殊芯片，它比GPU更快。\n",
    "\n",
    "一个TensorFlow图由下面几个部分组成，后面会详细描述：\n",
    "\n",
    "* 占位符变量（Placeholder）用来改变图的输入。\n",
    "* 模型变量（Model）将会被优化，使得模型表现得更好。\n",
    "* 模型本质上就是一些数学函数，它根据Placeholder和模型的输入变量来计算一些输出。\n",
    "* 一个cost度量用来指导变量的优化。\n",
    "* 一个优化策略会更新模型的变量。\n",
    "\n",
    "另外，TensorFlow图也包含了一些调试状态，比如用TensorBoard打印log数据，本教程不涉及这些。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建新变量的帮助函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "函数用来根据给定大小创建TensorFlow变量，并将它们用随机值初始化。需注意的是在此时并未完成初始化工作，仅仅是在TensorFlow图里定义它们。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def new_weights(shape):\n",
    "    return tf.Variable(tf.truncated_normal(shape, stddev=0.05))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def new_biases(length):\n",
    "    return tf.Variable(tf.constant(0.05, shape=[length]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建卷积层的帮助函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个函数为TensorFlow在计算图里创建了新的卷积层。这里并没有执行什么计算，只是在TensorFlow图里添加了数学公式。\n",
    "\n",
    "假设输入的是四维的张量，各个维度如下：\n",
    "\n",
    "1. 图像数量\n",
    "2. 每张图像的Y轴\n",
    "3. 每张图像的X轴\n",
    "4. 每张图像的通道数\n",
    "\n",
    "输入通道可能是彩色通道，当输入是前面的卷积层生成的时候，它也可能是滤波通道。\n",
    "\n",
    "输出是另外一个4通道的张量，如下：\n",
    "\n",
    "1. 图像数量，与输入相同\n",
    "2. 每张图像的Y轴。如果用到了2x2的池化，是输入图像宽高的一半。\n",
    "3. 每张图像的X轴。同上。\n",
    "4. 卷积滤波生成的通道数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def new_conv_layer(input,              # The previous layer.\n",
    "                   num_input_channels, # Num. channels in prev. layer.\n",
    "                   filter_size,        # Width and height of each filter.\n",
    "                   num_filters,        # Number of filters.\n",
    "                   use_pooling=True):  # Use 2x2 max-pooling.\n",
    "\n",
    "    # Shape of the filter-weights for the convolution.\n",
    "    # This format is determined by the TensorFlow API.\n",
    "    shape = [filter_size, filter_size, num_input_channels, num_filters]\n",
    "\n",
    "    # Create new weights aka. filters with the given shape.\n",
    "    weights = new_weights(shape=shape)\n",
    "\n",
    "    # Create new biases, one for each filter.\n",
    "    biases = new_biases(length=num_filters)\n",
    "\n",
    "    # Create the TensorFlow operation for convolution.\n",
    "    # Note the strides are set to 1 in all dimensions.\n",
    "    # The first and last stride must always be 1,\n",
    "    # because the first is for the image-number and\n",
    "    # the last is for the input-channel.\n",
    "    # But e.g. strides=[1, 2, 2, 1] would mean that the filter\n",
    "    # is moved 2 pixels across the x- and y-axis of the image.\n",
    "    # The padding is set to 'SAME' which means the input image\n",
    "    # is padded with zeroes so the size of the output is the same.\n",
    "    layer = tf.nn.conv2d(input=input,\n",
    "                         filter=weights,\n",
    "                         strides=[1, 1, 1, 1],\n",
    "                         padding='SAME')\n",
    "\n",
    "    # Add the biases to the results of the convolution.\n",
    "    # A bias-value is added to each filter-channel.\n",
    "    layer += biases\n",
    "\n",
    "    # Use pooling to down-sample the image resolution?\n",
    "    if use_pooling:\n",
    "        # This is 2x2 max-pooling, which means that we\n",
    "        # consider 2x2 windows and select the largest value\n",
    "        # in each window. Then we move 2 pixels to the next window.\n",
    "        layer = tf.nn.max_pool(value=layer,\n",
    "                               ksize=[1, 2, 2, 1],\n",
    "                               strides=[1, 2, 2, 1],\n",
    "                               padding='SAME')\n",
    "\n",
    "    # Rectified Linear Unit (ReLU).\n",
    "    # It calculates max(x, 0) for each input pixel x.\n",
    "    # This adds some non-linearity to the formula and allows us\n",
    "    # to learn more complicated functions.\n",
    "    layer = tf.nn.relu(layer)\n",
    "\n",
    "    # Note that ReLU is normally executed before the pooling,\n",
    "    # but since relu(max_pool(x)) == max_pool(relu(x)) we can\n",
    "    # save 75% of the relu-operations by max-pooling first.\n",
    "\n",
    "    # We return both the resulting layer and the filter-weights\n",
    "    # because we will plot the weights later.\n",
    "    return layer, weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 转换一个层的帮助函数\n",
    "\n",
    "卷积层生成了4维的张量。我们会在卷积层之后添加一个全连接层，因此我们需要将这个4维的张量转换成可被全连接层使用的2维张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def flatten_layer(layer):\n",
    "    # Get the shape of the input layer.\n",
    "    layer_shape = layer.get_shape()\n",
    "\n",
    "    # The shape of the input layer is assumed to be:\n",
    "    # layer_shape == [num_images, img_height, img_width, num_channels]\n",
    "\n",
    "    # The number of features is: img_height * img_width * num_channels\n",
    "    # We can use a function from TensorFlow to calculate this.\n",
    "    num_features = layer_shape[1:4].num_elements()\n",
    "    \n",
    "    # Reshape the layer to [num_images, num_features].\n",
    "    # Note that we just set the size of the second dimension\n",
    "    # to num_features and the size of the first dimension to -1\n",
    "    # which means the size in that dimension is calculated\n",
    "    # so the total size of the tensor is unchanged from the reshaping.\n",
    "    layer_flat = tf.reshape(layer, [-1, num_features])\n",
    "\n",
    "    # The shape of the flattened layer is now:\n",
    "    # [num_images, img_height * img_width * num_channels]\n",
    "\n",
    "    # Return both the flattened layer and the number of features.\n",
    "    return layer_flat, num_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建一个全连接层的帮助函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个函数为TensorFlow在计算图中创建了一个全连接层。这里也不进行任何计算，只是往TensorFlow图中添加数学公式。\n",
    "\n",
    "输入是大小为`[num_images, num_inputs]`的二维张量。输出是大小为`[num_images, num_outputs]`的2维张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def new_fc_layer(input,          # The previous layer.\n",
    "                 num_inputs,     # Num. inputs from prev. layer.\n",
    "                 num_outputs,    # Num. outputs.\n",
    "                 use_relu=True): # Use Rectified Linear Unit (ReLU)?\n",
    "\n",
    "    # Create new weights and biases.\n",
    "    weights = new_weights(shape=[num_inputs, num_outputs])\n",
    "    biases = new_biases(length=num_outputs)\n",
    "\n",
    "    # Calculate the layer as the matrix multiplication of\n",
    "    # the input and weights, and then add the bias-values.\n",
    "    layer = tf.matmul(input, weights) + biases\n",
    "\n",
    "    # Use ReLU?\n",
    "    if use_relu:\n",
    "        layer = tf.nn.relu(layer)\n",
    "\n",
    "    return layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 占位符 （Placeholder）变量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Placeholder是作为图的输入，每次我们运行图的时候都可能会改变它们。将这个过程称为feeding placeholder变量，后面将会描述它。\n",
    "\n",
    "首先我们为输入图像定义placeholder变量。这让我们可以改变输入到TensorFlow图中的图像。这也是一个张量（tensor），代表一个多维向量或矩阵。数据类型设置为float32，形状设为`[None, img_size_flat]`，`None`代表tensor可能保存着任意数量的图像，每张图象是一个长度为`img_size_flat`的向量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = tf.placeholder(tf.float32, shape=[None, img_size_flat], name='x')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "卷积层希望`x`被编码为4维张量，因此我们需要将它的形状转换至`[num_images, img_height, img_width, num_channels]`。注意`img_height == img_width == img_size`，如果第一维的大小设为-1， `num_images`的大小也会被自动推导出来。转换运算如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_image = tf.reshape(x, [-1, img_size, img_size, num_channels])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们为输入变量`x`中的图像所对应的真实标签定义placeholder变量。变量的形状是`[None, num_classes]`，这代表着它保存了任意数量的标签，每个标签是长度为`num_classes`的向量，本例中长度为10。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_true = tf.placeholder(tf.float32, shape=[None, num_classes], name='y_true')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们也可以为class-number提供一个placeholder，但这里用argmax来计算它。这里只是TensorFlow中的一些操作，没有执行什么运算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_true_cls = tf.argmax(y_true, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 卷积层 1\n",
    "\n",
    "创建第一个卷积层。将`x_image`当作输入，创建`num_filters1`个不同的滤波器，每个滤波器的宽高都与 `filter_size1`相等。最终我们会用2x2的max-pooling将图像降采样，使它的尺寸减半。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "layer_conv1, weights_conv1 = \\\n",
    "    new_conv_layer(input=x_image,\n",
    "                   num_input_channels=num_channels,\n",
    "                   filter_size=filter_size1,\n",
    "                   num_filters=num_filters1,\n",
    "                   use_pooling=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "检查卷积层输出张量的大小。它是（？,14, 14, 16），这代表着有任意数量的图像（？代表数量），每张图像有14个像素的宽和高，有16个不同的通道，每个滤波器各有一个通道。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'Relu:0' shape=(?, 14, 14, 16) dtype=float32>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layer_conv1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 卷积层 2\n",
    "\n",
    "创建第二个卷积层，它将第一个卷积层的输出作为输入。输入通道的数量对应着第一个卷积层的滤波数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "layer_conv2, weights_conv2 = \\\n",
    "    new_conv_layer(input=layer_conv1,\n",
    "                   num_input_channels=num_filters1,\n",
    "                   filter_size=filter_size2,\n",
    "                   num_filters=num_filters2,\n",
    "                   use_pooling=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "核对一下这个卷积层输出张量的大小。它的大小是（？， 7， 7， 36）,其中？也代表着任意数量的图像，每张图有7像素的宽高，每个滤波器有36个通道。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'Relu_1:0' shape=(?, 7, 7, 36) dtype=float32>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layer_conv2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 转换层\n",
    "\n",
    "这个卷积层输出一个4维张量。现在我们想将它作为一个全连接网络的输入，这就需要将它转换成2维张量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "layer_flat, num_features = flatten_layer(layer_conv2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个张量的大小是（？， 1764），意味着共有一定数量的图像，每张图像被转换成长为1764的向量。其中1764 = 7 x 7 x 36。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'Reshape_1:0' shape=(?, 1764) dtype=float32>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layer_flat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1764"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "num_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 全连接层 1\n",
    "\n",
    "往网络中添加一个全连接层。输入是一个前面卷积得到的被转换过的层。全连接层中的神经元或节点数为`fc_size`。我们可以用ReLU来学习非线性关系。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "layer_fc1 = new_fc_layer(input=layer_flat,\n",
    "                         num_inputs=num_features,\n",
    "                         num_outputs=fc_size,\n",
    "                         use_relu=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "全连接层的输出是一个大小为（？，128）的张量，？代表着一定数量的图像，并且`fc_size` == 128。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'Relu_2:0' shape=(?, 128) dtype=float32>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layer_fc1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 全连接层 2\n",
    "\n",
    "添加另外一个全连接层，它的输出是一个长度为10的向量，它确定了输入图是属于哪个类别。这层并没有用到ReLU。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "layer_fc2 = new_fc_layer(input=layer_fc1,\n",
    "                         num_inputs=fc_size,\n",
    "                         num_outputs=num_classes,\n",
    "                         use_relu=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'add_3:0' shape=(?, 10) dtype=float32>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layer_fc2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 预测类别"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二个全连接层估算了输入图有多大的可能属于10个类别中的其中一个。然而，这是很粗略的估计并且很难解释，因为数值可能很小或很大，因此我们会对它们做归一化，将每个元素限制在0到1之间，并且相加为1。这用一个称为softmax的函数来计算的，结果保存在`y_pred`中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = tf.nn.softmax(layer_fc2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "类别数字是最大元素的索引。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_cls = tf.argmax(y_pred, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 优化损失函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了使模型更好地对输入图像进行分类，我们必须改变`weights`和`biases`变量。首先我们需要对比模型`y_pred`的预测输出和期望输出的`y_true`，来了解目前模型的性能如何。\n",
    "\n",
    "交叉熵（cross-entropy）是在分类中使用的性能度量。交叉熵是一个常为正值的连续函数，如果模型的预测值精准地符合期望的输出，它就等于零。因此，优化的目的就是通过改变网络层的变量来最小化交叉熵。\n",
    "\n",
    "TensorFlow有一个内置的计算交叉熵的函数。这个函数内部计算了softmax，所以我们要用`layer_fc2`的输出而非直接用`y_pred`,因为`y_pred`上已经计算了softmax。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-31-2dd067a7547b>:2: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "\n",
      "Future major versions of TensorFlow will allow gradients to flow\n",
      "into the labels input on backprop by default.\n",
      "\n",
      "See @{tf.nn.softmax_cross_entropy_with_logits_v2}.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=layer_fc2,\n",
    "                                                        labels=y_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们为每个图像分类计算了交叉熵，所以有一个当前模型在每张图上表现的度量。但是为了用交叉熵来指导模型变量的优化，我们需要一个额外的标量值，因此简单地利用所有图像分类交叉熵的均值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "cost = tf.reduce_mean(cross_entropy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 优化方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "既然我们有一个需要被最小化的损失度量，接着就可以建立优化一个优化器。这个例子中，我们使用的是梯度下降的变体`AdamOptimizer`。\n",
    "\n",
    "优化过程并不是在这里执行。实际上，还没计算任何东西，我们只是往TensorFlow图中添加了优化器，以便之后的操作。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = tf.train.AdamOptimizer(learning_rate=1e-4).minimize(cost)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 性能度量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们需要另外一些性能度量，来向用户展示这个过程。\n",
    "\n",
    "这是一个布尔值向量，代表预测类型是否等于每张图片的真实类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "correct_prediction = tf.equal(y_pred_cls, y_true_cls)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的计算先将布尔值向量类型转换成浮点型向量，这样子False就变成0，True变成1，然后计算这些值的平均数，以此来计算分类的准确度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 运行TensorFlow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建TensorFlow会话（session）\n",
    "\n",
    "一旦创建了TensorFlow图，我们需要创建一个TensorFlow会话，用来运行图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "session = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 初始化变量\n",
    "\n",
    "我们需要在开始优化weights和biases变量之前对它们进行初始化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "session.run(tf.global_variables_initializer())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用来优化迭代的帮助函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在训练集中有50,000张图。用这些图像计算模型的梯度会花很多时间。因此我们利用随机梯度下降的方法，它在优化器的每次迭代里只用到了一小部分的图像。\n",
    "\n",
    "如果内存耗尽导致电脑死机或变得很慢，你应该试着减少这些数量，但同时可能还需要更优化的迭代。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_batch_size = 64"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "函数执行了多次的优化迭代来逐步地提升网络层的变量。在每次迭代中，从训练集中选择一批新的数据，然后TensorFlow用这些训练样本来执行优化器。每100次迭代会打印出相关信息。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Counter for total number of iterations performed so far.\n",
    "total_iterations = 0\n",
    "\n",
    "def optimize(num_iterations):\n",
    "    # Ensure we update the global variable rather than a local copy.\n",
    "    global total_iterations\n",
    "\n",
    "    # Start-time used for printing time-usage below.\n",
    "    start_time = time.time()\n",
    "\n",
    "    for i in range(total_iterations,\n",
    "                   total_iterations + num_iterations):\n",
    "\n",
    "        # Get a batch of training examples.\n",
    "        # x_batch now holds a batch of images and\n",
    "        # y_true_batch are the true labels for those images.\n",
    "        x_batch, y_true_batch, _ = data.random_batch(batch_size=train_batch_size)\n",
    "\n",
    "        # Put the batch into a dict with the proper names\n",
    "        # for placeholder variables in the TensorFlow graph.\n",
    "        feed_dict_train = {x: x_batch,\n",
    "                           y_true: y_true_batch}\n",
    "\n",
    "        # Run the optimizer using this batch of training data.\n",
    "        # TensorFlow assigns the variables in feed_dict_train\n",
    "        # to the placeholder variables and then runs the optimizer.\n",
    "        session.run(optimizer, feed_dict=feed_dict_train)\n",
    "\n",
    "        # Print status every 100 iterations.\n",
    "        if i % 100 == 0:\n",
    "            # Calculate the accuracy on the training-set.\n",
    "            acc = session.run(accuracy, feed_dict=feed_dict_train)\n",
    "\n",
    "            # Message for printing.\n",
    "            msg = \"Optimization Iteration: {0:>6}, Training Accuracy: {1:>6.1%}\"\n",
    "\n",
    "            # Print it.\n",
    "            print(msg.format(i + 1, acc))\n",
    "\n",
    "    # Update the total number of iterations performed.\n",
    "    total_iterations += num_iterations\n",
    "\n",
    "    # Ending time.\n",
    "    end_time = time.time()\n",
    "\n",
    "    # Difference between start and end-times.\n",
    "    time_dif = end_time - start_time\n",
    "\n",
    "    # Print the time-usage.\n",
    "    print(\"Time usage: \" + str(timedelta(seconds=int(round(time_dif)))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用来绘制错误样本的帮助函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "函数用来绘制测试集中被误分类的样本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_example_errors(cls_pred, correct):\n",
    "    # This function is called from print_test_accuracy() below.\n",
    "\n",
    "    # cls_pred is an array of the predicted class-number for\n",
    "    # all images in the test-set.\n",
    "\n",
    "    # correct is a boolean array whether the predicted class\n",
    "    # is equal to the true class for each image in the test-set.\n",
    "\n",
    "    # Negate the boolean array.\n",
    "    incorrect = (correct == False)\n",
    "    \n",
    "    # Get the images from the test-set that have been\n",
    "    # incorrectly classified.\n",
    "    images = data.x_test[incorrect]\n",
    "    \n",
    "    # Get the predicted classes for those images.\n",
    "    cls_pred = cls_pred[incorrect]\n",
    "\n",
    "    # Get the true classes for those images.\n",
    "    cls_true = data.y_test_cls[incorrect]\n",
    "    \n",
    "    # Plot the first 9 images.\n",
    "    plot_images(images=images[0:9],\n",
    "                cls_true=cls_true[0:9],\n",
    "                cls_pred=cls_pred[0:9])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制混淆（confusion）矩阵的帮助函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_confusion_matrix(cls_pred):\n",
    "    # This is called from print_test_accuracy() below.\n",
    "\n",
    "    # cls_pred is an array of the predicted class-number for\n",
    "    # all images in the test-set.\n",
    "\n",
    "    # Get the true classifications for the test-set.\n",
    "    cls_true = data.y_test_cls\n",
    "    \n",
    "    # Get the confusion matrix using sklearn.\n",
    "    cm = confusion_matrix(y_true=cls_true,\n",
    "                          y_pred=cls_pred)\n",
    "\n",
    "    # Print the confusion matrix as text.\n",
    "    print(cm)\n",
    "\n",
    "    # Plot the confusion matrix as an image.\n",
    "    plt.matshow(cm)\n",
    "\n",
    "    # Make various adjustments to the plot.\n",
    "    plt.colorbar()\n",
    "    tick_marks = np.arange(num_classes)\n",
    "    plt.xticks(tick_marks, range(num_classes))\n",
    "    plt.yticks(tick_marks, range(num_classes))\n",
    "    plt.xlabel('Predicted')\n",
    "    plt.ylabel('True')\n",
    "\n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 展示性能的帮助函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "函数用来打印测试集上的分类准确度。\n",
    "\n",
    "为测试集上的所有图片计算分类会花费一段时间，因此我们直接用这个函数来调用上面的结果，这样就不用每次都重新计算了。\n",
    "\n",
    "这个函数可能会占用很多电脑内存，这也是为什么将测试集分成更小的几个部分。如果你的电脑内存比较小或死机了，就要试着降低batch-size。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split the test-set into smaller batches of this size.\n",
    "test_batch_size = 256\n",
    "\n",
    "def print_test_accuracy(show_example_errors=False,\n",
    "                        show_confusion_matrix=False):\n",
    "\n",
    "    # Number of images in the test-set.\n",
    "    num_test = data.num_test\n",
    "\n",
    "    # Allocate an array for the predicted classes which\n",
    "    # will be calculated in batches and filled into this array.\n",
    "    cls_pred = np.zeros(shape=num_test, dtype=np.int)\n",
    "\n",
    "    # Now calculate the predicted classes for the batches.\n",
    "    # We will just iterate through all the batches.\n",
    "    # There might be a more clever and Pythonic way of doing this.\n",
    "\n",
    "    # The starting index for the next batch is denoted i.\n",
    "    i = 0\n",
    "\n",
    "    while i < num_test:\n",
    "        # The ending index for the next batch is denoted j.\n",
    "        j = min(i + test_batch_size, num_test)\n",
    "\n",
    "        # Get the images from the test-set between index i and j.\n",
    "        images = data.x_test[i:j, :]\n",
    "\n",
    "        # Get the associated labels.\n",
    "        labels = data.y_test[i:j, :]\n",
    "\n",
    "        # Create a feed-dict with these images and labels.\n",
    "        feed_dict = {x: images,\n",
    "                     y_true: labels}\n",
    "\n",
    "        # Calculate the predicted class using TensorFlow.\n",
    "        cls_pred[i:j] = session.run(y_pred_cls, feed_dict=feed_dict)\n",
    "\n",
    "        # Set the start-index for the next batch to the\n",
    "        # end-index of the current batch.\n",
    "        i = j\n",
    "\n",
    "    # Convenience variable for the true class-numbers of the test-set.\n",
    "    cls_true = data.y_test_cls\n",
    "\n",
    "    # Create a boolean array whether each image is correctly classified.\n",
    "    correct = (cls_true == cls_pred)\n",
    "\n",
    "    # Calculate the number of correctly classified images.\n",
    "    # When summing a boolean array, False means 0 and True means 1.\n",
    "    correct_sum = correct.sum()\n",
    "\n",
    "    # Classification accuracy is the number of correctly classified\n",
    "    # images divided by the total number of images in the test-set.\n",
    "    acc = float(correct_sum) / num_test\n",
    "\n",
    "    # Print the accuracy.\n",
    "    msg = \"Accuracy on Test-Set: {0:.1%} ({1} / {2})\"\n",
    "    print(msg.format(acc, correct_sum, num_test))\n",
    "\n",
    "    # Plot some examples of mis-classifications, if desired.\n",
    "    if show_example_errors:\n",
    "        print(\"Example errors:\")\n",
    "        plot_example_errors(cls_pred=cls_pred, correct=correct)\n",
    "\n",
    "    # Plot the confusion matrix, if desired.\n",
    "    if show_confusion_matrix:\n",
    "        print(\"Confusion Matrix:\")\n",
    "        plot_confusion_matrix(cls_pred=cls_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 优化之前的性能\n",
    "\n",
    "测试集上的准确度很低，这是由于模型只做了初始化，并没做任何优化，所以它只是对图像做随机分类。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on Test-Set: 10.6% (1059 / 10000)\n"
     ]
    }
   ],
   "source": [
    "print_test_accuracy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1次迭代后的性能\n",
    "\n",
    "做了一次优化后，此时优化器的学习率很低，性能其实并没有多大提升。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization Iteration:      1, Training Accuracy:   4.7%\n",
      "Time usage: 0:00:00\n"
     ]
    }
   ],
   "source": [
    "optimize(num_iterations=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on Test-Set: 11.2% (1123 / 10000)\n"
     ]
    }
   ],
   "source": [
    "print_test_accuracy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 100次迭代优化后的性能\n",
    "\n",
    "100次优化迭代之后，模型显著地提升了分类的准确度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time usage: 0:00:00\n"
     ]
    }
   ],
   "source": [
    "optimize(num_iterations=99) # We already performed 1 iteration above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on Test-Set: 67.3% (6729 / 10000)\n",
      "Example errors:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8FNWZ//HPo4CgKIqYyKhw/SkCLhENAu4a3KKOoBLAiIlxcB8SXOJoXMYtidEIxBgVdCZRcdCIivsCLiSjKEJARNyNC0xQUFExCgLn90fX01XNXbrr3l4v3/frdV9dXX2q+sC5de5Tp85iIQRERKQw61U6AyIitUSVpohICqo0RURSUKUpIpKCKk0RkRRUaYqIpKBKU0QkBVWaIiIpqNIUEUmhTUsO7tKlS6irqytSVmrD7Nmzl4YQtqh0PspFZdz6qYzTaVGlWVdXx6xZs1pyippjZu9VOg/lpDJu/VTG6ej2XEQkBVWaIiIpqNIUEUlBlaaISAqqNEVEUlClKSKSgipNEZEUWtRPU6QSPv30UwDef//9RtN0794dgLFjxwKw8847A7DDDjsAsOuuu5Yyi9KKKdIUEUlBkaZUvYceegiABx98EIBnnnkGgDfffLPRY3r27AnAu+++C8CKFStyPl+zZk2RcynrCkWaIiIpVHWk+fnnnwNw/vnnA/DKK68AMG3atGyatm3blj9jUnRvv/02AH/4wx8AmDBhQvazr776CoA0y02//vrrRcydSEyRpohIClUZaU6cOBGAiy66CKj/lNQjUIDNN9+8fBmTklm4cCEA48aNa9F5evXqBcRPy6X6vPXWWwAsXbo0u+++++4D4vbq9dbLxHOnnXYaAHvttVc2bY8ePcqRzUYp0hQRSaGqIk2PNs466ywg/ktkZjnpRo0ald2+/vrrAejcuXM5sijNkIwoPJLcZ599ADjssMMAaNeuHQCdOnUCoGPHjtljli9fDsChhx4KxFFk//79Adhtt92yaTt06ADARhttVOR/hTTXyy+/DMTt1ffeey8AS5YsyXvs888/D+Q+u/CeEf479Lvf/Q6If4dKTZGmiEgKqjRFRFKoqtvz3/72twB8/PHHTaa78847s9uPPvooED808lv3coXq0rgvv/wSgIMPPji776WXXgJgypQpOWn33HNPAObMmQNklmBw/iBw6623BuKHBFKd5s2bB8S343fddRcAn332WU46L0+AfffdF4jL/ZprrgHgu9/9LgAvvPBCNq3XD4888ggQD4n1h0alpt8+EZEUKh5pvvdevL7RH//4x5zP/C/It7/9bQCmTp1a73j/6+VR6vHHHw/AlltuWfzMSkFWrlwJwA9/+EMgji4BfvGLXwBw0EEHNXhsQ6siduvWrcg5lGI79dRTs9vefWjtBz1e5rvssgsAv/rVr7KftW/fPiftjBkzALjxxhsB+MlPfpL9bO7cuUB8jZ9xxhkAHHvssQBssUVpFxJVpCkikkLFI03/qwFxp/X99tsPgOnTpwPw9ddfA/A///M/APz617/OHuMdZRcvXgzAoEGDgLitU12Ryse7BnkE4RNsJP/y//znPwdgww03LHPupJj8mrz66qsBuPnmm7Of+XDXb33rWwCcfvrpQFz2hXQH83bLVatWAXDZZZdlP/OuZz4ZS7kp0hQRSaHikWZyyi7vxO6d2523d5x00kkATJ48OfuZT/Tgf908gtHT8/LzJ+JXXXUVEE8E/Ne//jWbxjuvS23z4Y7+lDs5mcpWW20FxJ3Y+/Xrl/d8q1evBuCDDz4A4Ec/+hEARxxxBBBPPN2QE044AYBNN9204Py3hCJNEZEUKh5pTpo0qd6+hx9+GIDBgwc3eMysWbMaPd+AAQOA3GF4Uh7PPfdcznsf3pjsjyetg7c1rr/++vU+8yGP3rfS7wxfe+21nHQ+5BXg1VdfzXnt0qULED+raIj3qvE+2uWaJlKRpohIChWPNI877rjs9v333w/Aiy++CMR/mXzAv/f/SrZveDuG7/PJa72dY8cddyxZ3iVXsq0Z4h4MySefRx11FJA7yYbUnoEDBwJw4IEHArl9qL3v9U9/+tMGj23TJlPteLTakLUjzOQosGOOOQaA6667DoCuXbumyntLKdIUEUlBlaaISAqWZt2VtfXt2zc09VCmEJ988kl2e7vttgPioZGet7Xn00xOAOGTAhx55JEAvPHGGwCccsopANx0000tyt/azGx2CKFvUU9axdKUsZfT2uWV5A8OfHIFnxPTu5psv/32AOy00071jvU1onxyj1I9YFIZp7ds2bLstnc5e/bZZ4F4dQUfDuvdDJPDa5MTcjTEO8hDPHiiJV2MWlLGijRFRFKo+IOg5DDHu+++G4AhQ4YA9SNOb1j+zW9+kz3GO75747APsXz88ceBuPM7xJGslMa5554LwLXXXttoGu/E7HcI/pqGD8874IADgNypAqUyklGfR5r5eAd2qB9pbrLJJgCMGTMGgBNPPDH7WUPdnMpJkaaISAoVjzSTfOoo77riE3T4X7HLL78cqD+NFMDFF18MxJ1jvfuSHwNw6623liLbEvEIY+jQoUA8Td8333yTTePrQHnE2RwfffQREN+ZJFee9I7OUr18ko+m7hB8SjifXrCaKNIUEUmhqiJN5xFnYxPVNsSHZA0bNgyII82nn346m8af1Gu6uNLwtqY99tgDiHsyJD355JNAHH1eeumlAMycOTP193lb9+zZs1MfK+V3yy23AHDllVcCuXcgzu8afELhaqRIU0QkhaqMNFvC29MeeOABILfdxNdIv+SSS8qfMQHi4XfOJ6H2SNMnXUgub3DyyScDMHbsWCBu65ba4GV7zjnnAPDFF1/US7PxxhsDcVvmBhtsUKbcpadIU0QkBVWaIiIptLrbc58N5bzzzgNy19f2hw7Dhw8HYIcddihv5qSeQw45BIhXqfSHAz5bFcCbb74JxLOFr81nCpfq5GtF+RpgLrlWkDen7bPPPuXLWDMp0hQRSaHVRZquT58+AFxxxRXZfT7M74ILLgBg4sSJQO4M0lJevXv3BuKuYnfddVe9NMluYxDPx+jrxySH1Ur18Ac+3pl9bSNGjMhu+5DYWqBIU0QkhVYbabrkpADjx48H4lXyvK3sO9/5TvkzJkAc5Y8bNw6Io5Nkh/UPP/wQgLq6OiAuU2+jluqyfPlyIL6LWLlyZc7nu+66KxCXea1RpCkikkKrjzS32GKL7Pa0adOAeD1un2BCnaUrz1cWfOihhwC4/fbbs5/NmDEDiCNLnxpOqtNTTz0FwKJFixr83Kd7a2jinVqgSFNEJIVWH2km+XT7vlyG9w1bsGABoJUrq4mvJrr2tlQ/n6Zxbd53+nvf+145s1N0ijRFRFJYpyJN55Mc+1O8t956C1CkKVIMycUSIW6DHj16dCWyU3SKNEVEUlClKSKSwjp5e+4r3f3973+vcE5EWp+zzz4759UfDHXt2rVieSomRZoiIimsk5GmiJTOWWedlfPa2ijSFBFJwXxFv2YdbLYEeK942akJ3UMIW+RP1jqojFs/lXE6Lao0RUTWNbo9FxFJQZWmiEgKTVaaZra5mc2Nfhab2aLE+3alypSZnW1mr0Q/owpIP9LMlkT5etXMTmrh9080s8EFpt3TzFYXmr7aVLCMF5rZy9H3vFBAepVxM+k6bjLN+Yn/i1fMbJWZdWrqmCa7HIUQPgb6RCe/FFgeQvjtWl9qZNpG1xTyD8nHzPoAPwb6AquAJ8zsoRBCvp7od4QQRpvZlsB8M3sghLA0cd42IYRVxchj8pzAr4CpxTxvOVWijBP2DSEsS5FeZdwMuo4bF0K4CrgqOvfRwOkhhM+aOqZZt+dmtr2ZLTCzO4BXgG3MbFni8+Fmdku0/W0zu9fMZpnZTDMbkOf0vYHnQwhfhRC+Af4CHF1o3kIIi4F3gW5mdqWZ3WZmzwJ/MrM2ZjYmysc8MxsZ5XE9M7vBzF4zs6lAlwK/bjRwJ7A0X8JaU+IybhGVcXHoOq7nOGBSvkQtadPsBYwNIewINDxFc8Z1wNUhhL7AUMALob+Z3dRA+peB/c2ss5ltBHwf2KbQTJnZ9kB34J1EPgeGEEYApwAfhRD6AXsAZ5pZN2AIsC2wI/ATYK/E+X5pZoc38D3dgCOAmwvNWw0qVRkDBOApM5ttZv+WJlMq46Jap6/jxOcdgYOAe/PlrSUjgt4OIcwqIN1BQM9M9A/AZmbWIYTwAlCvLSuEMN/MxgDTgOXAHGB1Ad9zvJkdAKwARoYQlkXfeX8I4esozSFAbzMbHr3vBPQA9gMmRbcmC83smUR+Lmzk+8YB54UQ1iT+ba1NSco4MiCEsCi6DZtqZq+GEJ7L8z0q4+Jb169jNwiYnu/WHFpWaX6Z2F4DJH+rkot/GNAvhJC7JF0TQggTgAkAZnY18FYBh90RQmhowr5kPg04I4TwZDJB1JaRVl/g7qhAuwCHmNnqEMKDzThXtSplGS+KXheb2f1APyBfpakyLr51/Tp2w4Hb86aiSF2Oopr9UzPrYWbrkdt2MQ04099YpoG4SWb2rei1DjiKTJsSZvYzMzutBVl9HDjDMo37mFlPM+tApr1lWNQmshWwf74ThRC6hRDqQgh1wBTglFZ2MeUoZhmbWcfodojo1u1gYH70XmVcIevidRwdvxmZW/mCyraY/TT/g8w/5jlgYWL/mcDeUYPtAuDkKKNNtXdNidJOAU4LIXwe7e8NfNyCPI4H3gTmmtl84EYy0fZk4H1gAfBHYIYfkK8tZB1TrDLuCjxrZi8BM4H7QgjTos9UxpW1Ll7HxwKPhhC+KuTLa2oYpZk9DAwqdrcSqR4q49av1su4pipNEZFK0zBKEZEUVGmKiKSgSlNEJAVVmiIiKbRojaAuXbqEurq6ImWlNsyePXvpujSrt8q49VMZp9OiSrOuro5ZswoZgdV6mNk6tSyAyrj1Uxmno9tzEZEUVGmKiKSgSlNEJAVVmiIiKajSFBFJQZWmiEgKLepyJFIss2fPBuC+++4D4J577sl+9vrrrwPgk8v47OHf/e53Aejdu3c27QUXXFBvn0gxKdIUEUlBkaaU3IQJE7Lbr732GgB//etfc9J4pOlRZHLKQt936qmnAnD00ZkJxQ855JAS5VikcYo0RURSUKQpJecRIsRR44YbbgjEbY+jR2fW0urVqxcAXbrES1Yfc8wxZcmnlMYzzzwDwL33ZlbHnTx5MgD/+Mc/sml22203AIYOHQrA+eefX8YcpqNIU0QkhZqINOfMmQPAxRdfDMAjjzyS/WztJ6o/+MEPAPjlL38JQNeuXbNpn376aQAGDhwIQIcOHUqZbYkkI8UpU6YAcYT54osvViRPUjqLFy8G4rbnmTNnAvG1us022wDQs2fP7DEffPABABdemFmevHv37gAcd9xxZchxOoo0RURSqMpI85tvvgFg+vTpAJx44olA3AbiUWWS7/P2Eo8i33///Wwab1u57bbbABgxYkSRcy4NuemmeIXXv/3tbwC8915mZi4vn27dupU/Y1I0S5cuzW4ffnhmpdy5c+cCcdQ4fvx4APr37w9Ap06dssd4pHnUUUcBcPfddwMwbNiwnPcQt3/26NEDaLg+KCVFmiIiKajSFBFJoSpvz/0W7tBDD83Z/y//8i8AXH/99dl93nXF+W2f7x81alT2sw022ADIfTgkpbfFFvGqAieffDIAF110ERDf1un2vLZdc8012W2/Ld9qq62AeBhsu3btGj3eHw5585pfq/7Qt6EHQl9++SVQ/ge6ijRFRFKoqkhz/vz5QNwY7A466CAAfv3rXwOw++67N3qO//u//wNg0KBBACxbtiz72XnnnQfEXY6k/NasWQPE3U8WLFiQ874h3j1p7bsKqbw777wTgDFjxmT3bb755gC8+uqrQNMR5tq22247IP69OOGEE+qlGTx4MADt27dvRo5bTpGmiEgKVRVpXnnllQAsWbIEgCOPPBKAa6+9Foi7GDTFo1VvF0067LDDipJPScfLE+C//uu/gLibyI9//GOg/iCFZOTpnaSPP/54QMMqq8m8efMAWL16dXbfTjvtBEDHjh2bfd6tt9660c823nhjoPxdjZwiTRGRFCoeafrTVIA///nPQPwX6qqrrgIKizC9Q7y3e3qkcsABB2TT7L///i3PsBTMI8z99tsvu897N6w9gfA+++yTc+zNN9+c3fa7Bp/wwSMMH4KZnHBY7Z7l9fbbb9fb588OWuLxxx8H4Ouvv673mQ+VrhRFmiIiKVQ80pw1a1Z22yOIjTbaCIAdd9wx7/EeYfpkHn/5y19yznXJJZcUL7OSik847P30AI499lggd1hcQ0455ZTstvflnDhxIhBP+rHHHnsAub8nfl4td1Fa//znP4F4eZIk75/ZHCtXrgTgF7/4BQArVqwA4nZMgF122aXZ5y8GRZoiIilUPNJsjnfffTe7fcMNNwDxE3bno4f69OlTtnxJrn333ReI+2Y2l09I7BMV+6svo5Fs//R260cffRSI206lNFatWlWU8/gd41NPPQXUbys96aSTsts+AUilKNIUEUlBlaaISAoVvz1PNth7R9lPPvkEiOfNW1uys7QPm1y7o6sPldx0002Ll1mpKv6wKNnZ3bs3HXHEEUDcfKMO8cXVpk2m6qirqwNym8yeeOIJAHbdddcmz5FcI+j2228HGl8byOfUrQaKNEVEUqh4pOnD6gC++OILAB5++GEgjjyb8sADDwDxXyqfWuq0004raj6leiVXrvRZ4s855xwg/j3wGeL9IZK0jE/C4V38kt2+vHO7R5zezcwn4fDr3I8F+PDDD4F4NnefaMcf+vjUcdVAkaaISAoVjzSTE4g++OCDQLyWT7LjO8R/zXwNEoAzzjgDiDs1+wp3PsWUrFu8TdO7HPl7jzwVaRaXT6zhAw8gXgn2ySefzHn16HTbbbcFcoc4//CHPwTiSXr8GcX3vvc9ADp37lyS/DeHIk0RkRQqHmk2xP8CJf8SNcbbsPwvkw+tSy6xIOseb+f0DvY+pFNKIzlx+Pe//30AZs+enZPGI82GJhF/4403gHjYpBsyZEhR81kMijRFRFKoykgzn2SfMOcD+tVmJRAvteCTexQy+YsUR9u2bQEYMGBAwccsXLiwwf1pzlEuijRFRFJQpSkikkJN3p5ffvnl9fZ5V4WmVqqUyhg7dmx22x/QjRgxoiTf5TPDX3jhhUC8Nvb06dNL8n1SHD4opRYo0hQRSaGmIk1fadLXiknSSpPVx8vJO5YDnHrqqUDzIk2fqGXt2cKT7309IY9ofXhtr169Un+flJYPbQWYNGlSzmc+L+omm2xS1jwVQpGmiEgKNRVpzpkzB4DPP/88u887tbdv374ieZL8kmuYjx8/HoB77rkHiKds8zTeCX3zzTfPHuPdhhpbGz05vaCvje5rzCQn85Dq8tZbb2W3P/vss5zPBg0aBMRT0FUTRZoiIilUXzXeBG/TSk44vPPOOwPVOdxqXedR5GOPPZbd51Gj8/bIjz76CIg7oSfL2NtBPWo8+uijc86RbK/Uuue1IzmZuPPyGzVqVLmzUzBFmiIiKdRUpOlPQpNOOOGECuRE0jj00EMb3Aa48cYby50dqRLerp3ka5qvv/765c5OwRRpioikUFORpj8lLWQZDBGpbj5xOMRt2I0tplhNFGmKiKSgSlNEJIWauj33GaHfeeed7D6fqV1Eakty0EMtUaQpIpJCTUWa3r1I3YxEpFIUaYqIpGAtaVcwsyXAe8XLTk3oHkJYZ5a6VBm3firjdFpUaYqIrGt0ey4ikoIqTRGRFFRpioik0GSlaWabm9nc6GexmS1KvG9XqkyZ2dlm9kr0k3diPTMbaWZLony9amYntfD7J5rZ4ALT7mlmqwtNX20qWMYLzezl6HteKCB92cvYzI4xs3nRd75oZnu15DsrRddxQWkLvo6b7KcZQvgY6BOd9FJgeQjht2t9mZF5oLSmkMzlY2Z9gB8DfYFVwBNm9lAI4e95Dr0jhDDazLYE5pvZAyGEpYnztgkhrCpGHpPnBH4FTC3mecupEmWcsG8IYVmK9OUu4yeA+0IIwcx2B24Ddi7i+ctC13HevKa6jpt1e25m25vZAjO7A3gF2MbMliU+H25mt0Tb3zaze81slpnNNLMBeU7fG3g+hPBVCOEb4C/A0XmOyQohLAbeBbqZ2ZVmdpuZPQv8yczamNmYKB/zzGxklMf1zOwGM3vNzKYChS4sMxq4E1iaL2GtKXEZt0i5yjiEsDzE3Us2AlpVVxNdx1mpruOWtGn2AsaGEHYEFjWR7jrg6hBCX2Ao4IXQ38xuaiD9y8D+ZtbZzDYCvg9sU2imzGx7oDvgA9R7AQNDCCOAU4CPQgj9gD2AM82sGzAE2BbYEfgJsFfifL80s8Mb+J5uwBHAzYXmrQaVqowhUwE9ZWazzezf0mSqXGUcfTbEzF4HpgAj0+SzRug6Tnkdt2QY5dshhFkFpDsI6Gnxmi+bmVmHEMILQL22rBDCfDMbA0wDlgNzgNUFfM/xZnYAsAIYGUJYFn3n/SGEr6M0hwC9zWx49L4T0APYD5gU3ZosNLNnEvm5sJHvGwecF0JYk/i3tTYlKePIgBDCoug2bKqZvRpCeC7P95S7jAkhTAYmm9mBwBXR+VsTXccpr+OWVJpfJrbXAMlvTK6na0C/EMLKQk8cQpgATAAws6uBt5o+AojaQvLk04AzQghPJhOYWcG3DQl9gbuj/+guwCFmtjqE8GAzzlWtSlnGi6LXxWZ2P9APyFdplruMk/l92sxuNbNNU7bDVjtdxymv46J0OYpq9k/NrIeZrUdu28U04Ex/Y5kG4iaZ2bei1zrgKDLtDZjZz8zstBZk9XHgDMs0/GJmPc2sA5n2lmFRm8hWwP75ThRC6BZCqAsh1JG5dTullVWYOYpZxmbW0cw6RtsbAQcD86P3VVPGUZufRdt9yTwoaU0VZg5dx4Vdx8Xsp/kfZP4xzwELE/vPBPaOGmwXACdD3vauKVHaKcBpIYTPo/29gY9bkMfxwJvAXDObD9xIJtqeDLwPLAD+CMzwA5pq71oHFauMuwLPmtlLwEwyT6inRZ9VUxkPJfMEdy6ZNr1hLchXrdB1nEdNjT03s4eBQcXuciDVQ2Xc+tV6GddUpSkiUmkaRikikoIqTRGRFFRpioik0KI1grp06RLq6uqKlJXaMHv27KXr0qzeKuPWT2WcTosqzbq6OmbNKmQwQethZuvUsgAq49ZPZZyObs9FRFJQpSkikoIqTRGRFFRpioikoEpTRCQFVZpSVmPGjGHMmDGYGWbGjBkzmDFjRv4DRaqEKk0RkRRa1E9TJK1x48ZVOgsiLaJIU0QkBUWaUnIffPBBve1rr70WgD333LMieZLqNHVqZhXde+65B4A///nPAHz66ad5j11vvUwM+MILmSWL+vbtW4osKtIUEUmj5iPN997LDCG97rrrALJjaP/whz8AsPPOO1cmY5J1991319u39dZbVyAnUg3uuusuAB58MF6K55FHHgFg2bLMEkw+OXqPHj0AGDkyXj25f//+QHxt+13LzTdnVuH1KFWRpohIFaipSPONN94A4Prrr8/uu+222wD47LPPctIedthhADz00EPZfd6e1r17dwC+853vlC6zktVQpKm2zHXHeeedB8Dvf/97AFasWAHE0SRAz549ATj00EMBOOusswDYbbfdAGjbtm2j5+/Xrx8Ab775JgBXXnll0fLeEEWaIiIpVHWkuWbNGgAWLFgAwMEHHwzA4sWL8x67aNEiAPbfP176+PPPMyuIepTzv//7v0D81E2KyyP7559/Prtvm222yXmV1u/WW28F4OuvvwZg6NChAJx77rnZNLvuuisA7dq1S33+Aw88EIARI0YAsP766zc/swVQbSEikoIqTRGRFKry9nzJkiVA3HB8xRVXNJp20003BeJbb7+ld74/6bXXXstJq9vz0hg7dmy9fUOGDGn2+Xxij2Rneci9/f/BD34A6EFTNdl7770BuO+++wA44ogjANhjjz2Kcv7tttuuKOcplGoLEZEUqjLSvPDCC4G4s6rzRuLf/e532X3bbrstAJdeeimQG3WsbYstMovP3X///QC0aVOV//xWY+HChfX2DRgwIPV5PMIcNmwYUD/STPLo9rnnngMUcVaSdxF87LHHgPhaPfrooyuWp2JQpCkikkLFQ61kG6S3d3kk6G2N3gn9lltuAeJB/QCjR48G4nbKpuy+++6Aoo9a49Hj2hFmQ5N+eDR69tlnA2iC4wq66aabAPjqq6+AeMDJxhtvXLE8FYMiTRGRFCoeafpEGxA/XXO9evUC4Pzzzwdgn332AeJOsoXYYYcdstvjx49vdj6lvJIR4trDMH3CB+8kneRtpg0N3ZTy8gjTJa/FWqZIU0QkhYpFmt988w0Av/nNbxpN4+2Uw4cPz9nfuXPn7PaoUaMAmDZtGgDPPvtsTtqTTjopu+0TdUj1a6iPZ1MRZmN8Ets0x0hxPPzww0Dchjl48OBKZqdoFGmKiKRQsUjTn4x73y2oPxFHhw4dANhggw0A+Pd//3cgfjIK8RPVtSNWb9s6/fTTi5ltSaGhiYYb6ruZ5OWZbJP0slS0WBuWL18OxKPxfNo33z9//vxGj91+++0BaN++fSmz2CKKNEVEUlClKSKSQsVuz33OO18bBOJZ1n14Y58+fYC465HzMB/i4ZPeDckbnX0Ov0022aTYWZcC+ezbyYc655xzDpDbxJLknyf5JByF8Nt7n69Tt/Tl50NYfTUFX7drl112yXusz9R+wQUXAPCv//qvQHXdrivSFBFJoeKd231qN4hnXs7HV5uD+h3ifRhda+lIW8s82ktO0uETqowZMwZoPOJMyrdypXcrSp7fh1hK+fnDXV/vp1OnTkD9KdySDwX/9re/ATBnzhwgvkM47rjjAPjv//7vbNpKR52KNEVEUqh4pJnGJ598AjQcRXTr1g2I1zuX6uFtmxDfCXjbpUeGnibNypUeYSbXmvHoNk07qBSXr1XuU8IVwodc+npgl19+OQCTJk0CoHfv3tm0F198cVHy2VyKNEVEUqipSPPII48E4OV8DJZ2AAAHQUlEQVSXX6732SWXXAI0bzU7Ka3kE2xvxxo3bhwQR5ZNTbDhk3f46+TJkxs9xp/carXL2uIDWbbccksA3n333ZzPfQLxaqBIU0QkhZqINN955x2g4eFXHn2eeOKJ5cySNJM/Lfc2R+/D2dAEHc7bQdfmT+WTT88VYdam6dOnA/DTn/4UgHnz5gGw7777AnD88cdXJmMNUKQpIpKCKk0RkRSq+vZ80aJFAAwcOBCAL774Aoi7F0HcxciHZUpt8Nto7+Tur162Da046d2SNOtRbVm5ciUQP6T95z//CcBll12WTePX8ZdffgnEZey/F9W0rpAiTRGRFKo60vShVWt3P0jOxp6MOqX2+YqkDT0Y8u5K3uXIo1WtLlpd/I7QJ+N5++23AfjHP/4BxBPzJK9rHxrpE/D4A8NqijCdIk0RkRSqMtKcOXMmAD/60Y9y9vsM7ocffnjZ8yTl4W1YSR5ZOm/vyjeRh5Te6tWrgdyhzRdddBEQT+n4xhtvALBixQogXrUhOZHLDTfcAMRTw1UzRZoiIilUVaTpT87+8z//E4Bly5blfL7ZZpsB0LFjx/JmTMouGXE2FH1KdfDJgq+55pp6n/lw57Zt2wLQr18/IH5qfthhh5Uji0WnSFNEJIWqijQnTJgA1J9SygfxP/roo0DuNFEiUjl77703kNuv9r333gPgZz/7GRCvd+7PJGqdIk0RkRSqKtL0UT2+BIaPADn55JMB6Nq1a2UyJiINGjRoUM7rukCRpohICqo0RURSqKrbc59Lz19FRKqNIk0RkRRUaYqIpKBKU0QkBQshNP9gsyXAe8XLTk3oHkKonqXxSkxl3PqpjNNpUaUpIrKu0e25iEgKqjRFRFJostI0s83NbG70s9jMFiXetytVpszsbDN7JfoZVUD6kWa2JMrXq2Z2Ur5j8pxvopkNzpPm/MT/xStmtsrMOrXkeyuhgmXc2czuNbPXojLrlyd92cs4kXZPM1tdaPpqU8EyXmhmL0ff80IB6StxHR9jZvOi73zRzPbKe+IQQkE/wKXAuQ3sN2C9Qs9TwPf0AV4COgBtgaeBbfMcMxIYF21vCSwFuqyVpk2KPEwEBqdIfzTwRLH+Dyr1U64yjs55B3BitN0O6FSNZUxmAMjTwGNpfieq9afMZbwQ2DRF+rKXMdCR+NnO7sD8fOdt1u25mW1vZgvM7A7gFWAbM1uW+Hy4md0SbX87iihmmdlMMxvQ2HkjvYHnQwhfhRC+Af5CplIqSAhhMfAu0M3MrjSz28zsWeBPZtbGzMZE+ZhnZiOjPK5nZjdEUc9UoEuK/w6A44BJKY+paqUsYzPrDPQPIfwJIISwMoTwWaF5K3MZjwbuJHMBtyolvo5bpFxlHEJYHqIaE9gIyPtkvCVtmr2AsSGEHYFFTaS7Drg6hNAXGAp4IfQ3s5saSP8ysH90+7YR8H1gm0IzZWbbA92BdxL5HBhCGAGcAnwUQugH7AGcaWbdgCHAtsCOwE+AvRLn+6WZNbookZl1BA4C7i00jzWkVGX8/4Al0YUwx8wmmNmGhWaqXGUcHXcEcHOheatBpSpjyFRAT5nZbDP7tzSZKud1bGZDzOx1YAqZaLdJLRl7/nYIYVYB6Q4CepqZv9/MzDqEEF4A6rVzhBDmm9kYYBqwHJgDrC7ge443swOAFcDIEMKy6DvvDyF8HaU5BOhtZsOj952AHsB+wKQQwhpgoZk9k8jPhXm+dxAwPU2kVENKUsZkfu/6AqOA2cDvgZ8Dl+X5nnKX8TjgvBDCmsS/rbUpVRkDDAghLDKzLYGpZvZqCOG5PN9T9us4hDAZmGxmBwJXROdvVEsqzS8T22vItIm49oltA/qFEFYWeuIQwgRgAoCZXQ28VcBhd4QQRufJpwFnhBCeTCYws4Jv/xswHLi9BcdXs1KV8ULgfb9YzeweMrfB+ZS7jPsCd0cXbRfgEDNbHUJ4sBnnqlalvI4XRa+Lzex+oB+Qr9Ks1HVMCOFpM7vVzDYNISxrLF1RuhxFNfunZtbDzNYjtw1yGnCmvzGzPvnOZ2bfil7rgKPItClhZj8zs9NakNXHgTPMrE10vp5m1oFMu+mwqE1kK2D/Qk5mZpuRuQVoTRdRg4pZxiGEhcCH0S0YwEBgQXRs1ZRxCKFbCKEuhFBH5tbtlFZWYeYoZhmbWceo6Yqome1gYH70vmrKOGrXtWi7L5mHQo1WmFDcfpr/QeYf8xyZSMKdCewdNdguAE6OMthUW8iUKO0U4LQQwufR/t7Axy3I43jgTWCumc0HbiQTbU8G3idz4f4RmOEH5GnTPBZ4NITwVQvyVEuKWcajgLvMbB6wE3BVtL/aynhdU6wy7go8a2YvATOB+0II06LPqqmMhwLzzWwumXbbYfm+vKaGUZrZw8CgEMKqSudFSkNl3PrVehnXVKUpIlJpGkYpIpKCKk0RkRRUaYqIpKBKU0QkBVWaIiIpqNIUEUlBlaaISAr/H3PnUrgAF34rAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8947975550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print_test_accuracy(show_example_errors=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1000次优化迭代后的性能\n",
    "\n",
    "1000次优化迭代之后，模型在测试集上的准确度超过了90%。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization Iteration:    101, Training Accuracy:  70.3%\n",
      "Optimization Iteration:    201, Training Accuracy:  79.7%\n",
      "Optimization Iteration:    301, Training Accuracy:  81.2%\n",
      "Optimization Iteration:    401, Training Accuracy:  82.8%\n",
      "Optimization Iteration:    501, Training Accuracy:  96.9%\n",
      "Optimization Iteration:    601, Training Accuracy:  90.6%\n",
      "Optimization Iteration:    701, Training Accuracy:  92.2%\n",
      "Optimization Iteration:    801, Training Accuracy:  89.1%\n",
      "Optimization Iteration:    901, Training Accuracy:  87.5%\n",
      "Time usage: 0:00:02\n"
     ]
    }
   ],
   "source": [
    "optimize(num_iterations=900) # We performed 100 iterations above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on Test-Set: 93.0% (9298 / 10000)\n",
      "Example errors:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe4VMX9x/H3F0EpShEs6A+4VoRoggYRiYpBBaMxIkHEFhuIigoauxEbaMQoROwaUQRLFAU7ig0xKl26hRiwEUBBwQTBML8/9szu2dt2z91++byeh+eePTvnnOHO3dnvzJkzY845REQkPXUKnQERkVKiSlNEJAJVmiIiEajSFBGJQJWmiEgEqjRFRCJQpSkiEoEqTRGRCFRpiohEUDeTg1u0aOHKysqylJXSMHPmzFXOue0KnY98URnXfirjaDKqNMvKypgxY0Ympyg5Zra00HnIJ5Vx7acyjkbNcxGRCFRpiohEoEpTRCQCVZoiIhGo0hQRiUCVpohIBKo0RUQiyGicZq6sWbMGgMaNGwNQp47q9trm3HPPBeC+++4D4IQTTgDgoYceAqBBgwaFyZhICqqNREQiKMpI8/jjjwegUaNGAPTr1w+A3/72tzm53ooVKwDYdtttAahbtyh/LbXC119/DcArr7wCgJkB8Pe//x2A/v37A9CtW7cC5E4kNUWaIiIRFGVItd9++wEwfPhwALp27ZrT640cORKAjRs3AnDrrbfm9Hqbs5YtWwKw4447ArBs2bKk92+++WYA9t9///i+bbbZJk+5k1yaN28eAKNGjQJg2rRp8fcWL14MQLNmzQBYvnx50rGXXnppfNvXC4WiSFNEJIKijDRbtWqVl+u89tprANx+++0A/Pjjj4AizXzo0qULkBxtALzxxhsAPP/88/F9J510Uv4yJlnno8jTTjsNgNmzZ1eZtnyE6b3wwgvx7YEDBwLQpk2bbGUxEkWaIiIRFGWkeffdd+flOm+++SaQiDB9X6rkXo8ePQC46667gER/shee31GRZmlavXo1AH369AESfZrV8SNYvv3226T9ixYtim+PGTMGgGuuuSYr+YxKkaaISASqNEVEIiiq5vn8+fMB+Oqrr/JyvcmTJye9vvbaa/NyXYHu3bsDcMABBwAwderUpPefeOKJ+PaAAQMAaNu2bZ5yJ9nw7LPPAlU3y325AgwePBhIPDp94403AnDvvfdWOG7BggVZzWdUijRFRCIoqkjzvffeA+C7775L2u8fp8wWf+PH33zwk0MccsghWb2OpPanP/0JgCOPPDJp/7///e/4du/evYH0biRI8SjfkvP8gwvnn39+fN9ee+0FwH/+8x8A3n///SrP+/HHH2crizWiSFNEJIKCR5rr1q2Lb992221J7x133HEAnH322Vm95sSJEwGYM2dO0vmbNm2a1etIar/61a+AqoeaQKLl8f333wOJfi8pbhdddBEATz75JACbNm0CEoPdP/roo3jasWPHAon7Gv6zWRlfLxSKIk0RkQgKHmn6byNI/uaB3N3N9hPdSuE1bNgQgIsvvhhI9HGGffHFFwC8/fbbABxzzDF5yp1kwvdd+gl3/MMka9euBRJ91VEV+t6DIk0RkQgKFmk+99xzADz11FMV3isrKwOyPy7P942F78xKcfjjH/8IwEsvvQTAP/7xjwpphg4dCiQm+2jevHmecieZ8BPjTJ8+HUiMz5w7d27a5+jUqVN8W5GmiEgJyXuk6e+A+hH/5cdkQuJJgvr162f12p999hlQ8c7cWWedldXrSHRbbrll0k/nXPw9v+0jFb9khiLN0rDFFlsA0LlzZwCuvPJKAC655JJ4mi+//LLSY/2IlgsuuCC+zy+RUiiKNEVEIlClKSISQd6b537lx/B8iZ4ftLrPPvvkNU9q5hUPPxt3ZU0wv8/P6r733nvnL2OSNb5LbtWqVSnT+hvGBx98cE7zFIUiTRGRCPIWaU6aNAmAq666Kmn/HnvsEd/2s3j7jmN/A+CHH36o8rz16tUDKs787Sf5qK7T2Ee2u+66a+r/gORFz549AXjkkUeqTOP/lvwKhVqnvjT4RyUHDRoEJCbOCfOf1759+wKJx2yLiSJNEZEI8vYV7SfJmDVrVtL+8LfNsGHDkt773//+B1Q+Eam37777AhVXuBs3bhyQ/Mjdq6++mpTGD2co9BAGSTjqqKMAaNeuXXzfwoULk9K88847QGK9+vDQFSk+Dz74IJBY06eyCPP4448H4MADDwSSH68uNoo0RUQiyFukOX78+Er3L1u2LL7t+zSj8FPLtW7dGkhMG3byyScD0LFjx3jalStXJh177rnnRr6e5Jbvn+zfv398X1VRh18LW5FmcfL90n7qxfADCwAtW7aMb99///1AaUzPqEhTRCSCvEWavj8j/DgUJMblAeywww5AxeUtunXrBiSmmgrzD/L7afJbtGgBwJQpUwAYNWpUPK0fG9qhQwcA9txzz5r8VyQP0pms5cMPPwRg6dKl8X3hvycpDB9h+mn+ykeY3umnnx7fLoUI01OkKSISgSpNEZEI8tY8P+ecc4DEOtdeuDPYrxPjZ/OOolmzZkmvjzjiCCAx1CnMD2to0qRJ5OtIfoRXp/RdMH6WI8/PmHXHHXfE95VfZ0ryI7xC5NVXXw1UPXOR72bzs/WXGkWaIiIR5C3S9ENJKruZk0vhdYb80JVwdCvF7+ijjwYqRpree++9l8/sSCXCQ8RSRZgvvvgikLhpW2oUaYqIRFDrZzrYbrvtKt2W0uEfVLjuuusqfb9Xr155zI2E+Uk4pk2bVmWabbbZBoDBgwcDpf85VKQpIhJBrY80pfS1atUKgBEjRgCJiV38Aw3+4QfJnyVLlgBw3nnnAbB+/foq0/br1w+Ak046KfcZywNFmiIiESjSlKLnJ5q+8MILk35K4ey2224A7LzzzgAsXry4Qpru3bsDcPnll+cvY3mgSFNEJAJFmiJSY2VlZUBypFm/fn0gMXGHn4intlCkKSISgSpNEZEI1DwXkRp7+eWXC52FvFOkKSISgSpNEZEIVGmKiERgVa3fkdbBZiuBpSkT1i5tnHOlPeNABCrj2k9lHE1GlaaIyOZGzXMRkQhUaYqIRFBtpWlmzc1sTvBvuZl9GXq9ZS4zZmZ1zWyumU1II+3QUN7mmdnRGV57qpl1SCPdiWa20MwWmNmYTK5ZKIUqYzN7xMxWmtmcNNP38+nNbJGZnZnh9ceaWc8UaXoFf4NzzGy6mXXJ5JqFUsAyvjj4bCwwswvSSF+IMr4i9LtYYGY/mVn1Ky4659L6B1wHXFLJfgPqpHueCNe7DHgMmJBG2qHA4GB7b2AlQX9tKE3dCNeeCnRIkWYvYCbQNHi9fbZ/B/n+l88yBroCnYA5aabvB4wMtncEVgEtMijjsUDPFGm2JtHvvx8wv9BlVCplDHQAPgQaAPWAN4Fdiq2My6U/Dng1VboaNc/NbPcgwhoHLABamdma0Pt9zezBYHsHM3vGzGaY2TQz65zG+dsARwCjo+bNOTef2B9As+Cb5h4zmwbcZGZbm9nDQT5mm9kxwfUamtlTwbfbeKB+Gpc6GxjlnFsTXHdF1LwWs1yXsXPubeDbmuTNObcc+BfQOmhljDGzd4GHgxbK7UE+5ppZvyCPdczsbjNbbGavASlX9XLOrXPBpwloBNSqu6Y5LuN2wPvOuf865zYCU4hVSmnJVxmXcyLweKpEmfRp7gWMcM61Bypffi7mDmC4c64j0AfwhXCAmd1bxTEjgUupwR9p0IRa75zzH8iWQGfn3GXAEOAV51wnoBtwm5nVB84HVjvn2hGLWvcNnW+0Vd5U3xNoZ2bvmtl7ZtY9al5LQC7LuMbMbHegDfDPUD4Pc86dQuzLbEVQxvsDA82sNdAb2AVoD5wBdAmdb5iZHVXFtXqb2UfABGKRUG2TqzKeB3Q1s23NrBHwG6BVupnKZxkH728NHA48kypvmTx7vsQ5NyONdIcDbc3Mv25mZg2ccx8AH5RPHPRBfO6cm2Nmh0fIz6VmdjqwFjghtP8p59ymYLs78BszuyJ4XR9oDRwCDAdwzs02swX+YOfcGVVcry6wK7FmZhvgbTNr75z7PkKei11OyjgDJ5vZocCPQD/n3JrgmhOdc369he7Evsz6Bq+bAHsQK+PHg7+FL8zsLX9S59zVVV3QOfc08LSZ/Rq4MTh/bZKTMnbOzTez24HJwDpgNvC/NK6T9zIOHAu87Zz7LlUGM6k0fwhtbyLWJPbCzVsDOjnnNqR53i5ALzP7XXCexmb2iHPutBTH3eqcG5kin0asj2NJOEHoDyGKL4j9kn8ClpjZEmA3Yn8ctUWuyrimxjnnBleyv3wZn+ecez2cwMzSbhpWxjn3psVuXjX1XTK1RM7K2Dl3P3A/gJkNBz5N47BClXFf4NF0EmZlyFFQs682sz3MrA7JfReTgYH+RRVN3fC5LnPO/Z9zrgw4hVjH7GnBscN9P2QNTQLid/HMzDfDpwAnBft+AfwsjXNNAA4NjtmeWIX5WQZ5K2rZLOPqmNkgMzun5jllEnCemdUNztfWzBoQK+MTgn6vnYm1EFLlZXcLvlHNrCOxm0K1qcJMku0yDj4XmFkZ8DvgieB10ZRxcHwzYsHa8+mkz+Y4zcuJ/Wf+QSwK8wYCvwo6bBcC/YOM1qS/6+fA8gzyeD3QyGLDkhYQu5MIcCfQ3MwWAdcQihar6dN8EVgX/J8mAxfV5g9UIGtlbGZPAe8A7c3si6BrBWI3EL7JII/3AZ8Ac8xsPnAPsRbV08AyYCGxG4zvhfJSVX9XH2C+xYZF3UFyt09tlc3P8YQg7QTgnFDXVTGVMcDvgZedc/9N5+Il8xhl8I3/snPuyELnRXLHzF4Ejg26PaQWKvUyLplKU0SkGOgxShGRCFRpiohEoEpTRCQCVZoiIhFktBplixYtnF8sfnMxc+bMVW4zmtVbZVz7qYyjyajSLCsrY8aMdJ7Aqj3MbLNaFkBlXPupjKNR81xEJAJVmiIiEajSFBGJQJWmiEgEqjRFRCJQpSkiEoEqTRGRCFRpiohEoEpTRCQCVZoiIhFk9BiliAjAbbfdFt9+8sknAZg+fXpSmp///OcAjBgxAoBu3brlKXfZpUhTRCSCooo0hw4dCsCsWbMAuPrq2FLFe+yxBwCNGzcGYP369fFjXn31VQDOOCO2PPnkyZMB2HfffZHi98orrwAwYcKEpP2LFi2Kb69YsQKAjz76CAC/REuvXr0AuPfexLpe22232UxOVFAbN24EoH///gBMmjQp/t5ZZ50FwDPPPAPA+++/D8Dll18OwKOPxlbKVaQpIrIZKKpIc8iQIQAES00zceJEANq3bw8koogffkisG19+Sqvbb78dSHybSeGFo8abbroJgMWLFwMwc+ZMIFHmPor0r6vaB4notEePHvF9Z599dlbzLpXzkeYjjzwCJMoToG3btklpe/fuDcCnn34KJPo0S5UiTRGRCFRpiohEUFTN86osXLgw6XV4rfbyTbann34agGuuuQaAPffcM8e5k1R8kxxg7NixQMXmuOdfh2/otG7dOinNunXrgMSNoWeffTb+nprn+bHFFlsA0LJlSyC9G3Ann3wyAFdeeSUAb731Vvy9Qw89NLsZzCFFmiIiERQ80gwPVSjvgQceABJDFt555x0gEWFUZsOGDQD89NNP2cqiZCjcGth+++2BxHAhb6+99gLg4IMPBqBFixbx98pHmv7mUadOnQA47rjjspxjSWWrrbYC4PXXXwegYcOGkc+xdGlpLsWkSFNEJIKCR5pLliyp8r1jjjkGgDPPPBOAb7/9FoDly5dXSOsjlDVr1mQ7i5KhMWPGZPV8w4YNAyr2h0r+tWvXLu20P/74Yw5zkj+KNEVEIih4pLlp06b4dqrIYdttt036GVa3bt2kc/ioVGofP6jd95VGiXakcJ544omk12VlZYXJSIYUaYqIRFDwSLNOnUS97SOH8mMv01H+WD891UEHHZRpFqXA/GOzp556KlBxLKfvz5bi9s033wBw4IEHAtC1a9dCZqfGFGmKiERQ8EhTpDJ+WjFITBXoJ3DxrQlNylIaPv74YyDxNJh/IqhUKdIUEYlAlaaISAQFb57vtNNO8e2dd94ZgK+++qpQ2ZEcmDJlSnx7wIABQMVZ2KPMp+lfH3nkkUDyjSD/SKWfY1PDkQrv/PPPB2DVqlVAopn+wgsvxNM0bdoUgMsuuwyAzp075zOLkSjSFBGJoOCRZs+ePePbfhq3e+65B4AGDRoUJE+SXeFZvX2EWX5YWarX1aXxE7lAYliLv3mU7Uc4JTpfPn4we7NmzYDkh1m+/vprIDEcqVWrVkBiWsitt946L3lNhyJNEZEICh5phvm1gEaNGhX5WP+tpUkcik94kmBfPn4quEaNGiWl9dOF+f4vSEw75vsn/bH+dXgaOT/lXHifFJZfG8hHi02aNKmQxq855O9n3HLLLUDi4RQ/TSTA/vvvn7vMpkGRpohIBEUVaWYik0cwJbfCg9CXLVsGJKLF8pPX+kfrpk6dGt/nI8rp06fnNJ+SG35UTHXq1asHQJs2bQC4++67ARg6dCgAhx9+eDzt3Llzk9LmmyJNEZEIak2kKcUr3L9YVV+jn4zDj+n0y2KA7oBvzv70pz8BMH78+Pg+H4X6fs98U6QpIhKBKk0RkQjUPJeC8rMZlZ+N/aqrroqn0aOQ0qdPn/j29ddfn/Szfv36ec2LIk0RkQgUaUpBrFy5EoDf//73QCLCvOGGGwAYNGhQYTImRal3797xbd8KKdSDLIo0RUQiqDWRZvnHKMPTkUnxufnmm4FEhOkjTj/ERCSsshVoC0WRpohIBLUm0iz/GOX8+fMLmR2pwowZMwAYOXIkkGgZlPq6MZJbzz33XKGzEKdIU0QkgloTaUppKD8e00/c4X+KhG3YsAGA2267Lb7viiuuAGCrrbYqSJ4UaYqIRKBIU/LKL1/g+zJPOukkoOIUcbJ5+/zzzwEYMmQIAEuWLIm/17dvXwDq1ClMzKdIU0QkAlWaIiIR1Jrm+aRJk4DEg/2FXkdEKudXptRgdqmOX41y9OjRST+LgSJNEZEIak2k+Ytf/AJIrKstxcnfCBIpVYo0RUQisEymVzKzlcDS7GWnJLRxzm1X6Ezki8q49lMZR5NRpSkisrlR81xEJAJVmiIiEajSFBGJoNpK08yam9mc4N9yM/sy9HrLXGTIzNqHrjHHzNaa2fkpjulnZiuD9IvM7MwM8zDWzHqmSLOtmT1nZnPN7AMza5/JNQtFZVxtGpVxza/ZyMymBddYaGZD0jhmaChv88zs6AzzMNXMOqRI82szm21mP6X6e4hzzqX1D7gOuKSS/QbUSfc8Uf4B9YAVwP+lSNcPGBls7wisAlqUS1M3wnXHAj1TpBkBXB1s/wx4LRe/g3z+UxmrjLN4nTpAo1AZzwA6pjhmKDA42N4bWElws7qGZTwV6JAizS7APsBjqf4e/L8aNc/NbPfg22McsABoZWZrQu/3NbMHg+0dzOwZM5sRfPN0jnCpI4BFzrkv0j3AObcc+BfQOvjmGmNm7wIPm1ldM7s9yMdcM+sX5LGOmd1tZovN7DWgRRqXag+8EVxzAbCnmTWP8H8raipjQGVc4zJ2zm1yzv0QvNySWMWZ9lAd59x8YhV5s6BVcI+ZTQNuMrOtzezhIB+zzeyYII8NzeypoCUyHki5ILpz7jPn3DxgU7p5y6RPcy9ghHOuPfBlNenuAIY75zoCfQBfCAeY2b0prtEXeDxKpsxsd6AN8M9QPg9zzp0CnA2scM51AvYHBppZa6A3sW+c9sAZQJfQ+YaZ2VGVXOpDoFeQ5kDg/4J/tYnKWGXsRS5jM9vSzOYA/wZecM7NTDdTZtYFWO+c+zbY1RLo7Jy7DBgCvBKUcTfgNjOrD5wPrHbOtSMWte4bOt/oVE31dGXyGOUS59yMNNIdDrS1YKZuYt8cDZxzHwAfVHVQ8Es4Grg4zfycbGaHAj8C/Zxza4JrTnTOrQ/SdAfamVnf4HUTYA/gEOBx59wm4Asze8uf1Dl3dRXXGwbcEfxRfBj8+1+aeS0VKmOVsRe5jJ1zG4AOZtYMeNbM2jnnFqW4zqVmdjqwFjghtP+poOwgVsa/MbMrgtf1gdbEynh4cO3ZZrYglJcz0vg/piWTSvOH0PYmYqG0Fw6LDegU/AKjOBr4wDm3Ks3045xzgyvZH86nAec5514PJzCz4yLmDefcd8BpwfF1iDUXP4t6niKnMlYZezUtY5xzq81sCtADSFVp3uqcG5kin0as/3FJOEGoQs+prAw5Cr4BVpvZHsEfV/gPdDIw0L+IECKfSLlmm5kNMrNzMsjqJOA8M6sbnK+tmTUApgAnBP1eOwNdU53IzJqaWb3g5QBgcqgPp9ZRGauMiVDGZra9mTUJthsSi1QXB6+H+37IGpoEXBC6lm+GTwFOCvb9gtjNu6zL5jjNy4n9Z/4BhDv1BwK/CjrlFwL9IWVfyDbAr4EJ5d5qB3yTQR7vAz4B5pjZfOAeYtH208AyYCEwGngvlJeq+rv2ARaa2UfAYaTfxCxlKuPaL1tlvBPwtpl9CEwDXnTOvRK893NgeQZ5vB5oZLFhSQuIjQgAuBNobmaLgGuA2f6Aqvo0zexAM/uC2BfEg2Y2N9XFS+rZczN7ETjWOfdTofMiuaEyrt0s1oZ+2Tl3ZKHzUlMlVWmKiBSaHqMUEYlAlaaISASqNEVEIshojaAWLVq4srKyLGWlNMycOXOV24xm9VYZ134q42gyqjTLysqYMSOdhwlqDzPbrJYFUBnXfirjaNQ8FxGJQJWmiEgEqjRFRCJQpSkiEoEqTRGRCFRpiohEoEpTRCSCjMZploKffkpMluPHos2ePTvp9UcffQRA27ZtATj//MTCiPvuG58xX0REkaaISBS1LtLcuHEjANOnTwfgL3/5S/y9Z599ttpj3333XQBmzZoV3+ejUhGJ5quvvgLgnnvuAeCxxx4D4J///GeFtGecEVvCp1u3bgCccEJseaB69epVSFtoijRFRCKoNZGm75ccNGgQAJMmTUp5TIsWsaWv99lnn6T9d955Z5ZzJ1UZP348AB9//HHS/tdeey2+/eabbwJw4IEHAnDMMcnLy5x++unx7ZYtW+Yim5LCpk2JZcN9RDls2DAAFi9enPL40aNHJ/30x77+emx9vJ122il7mc2QIk0RkQhKMtIM3xG/5pprALjrrrsAWLt2bVLaJk2axLfPPfdcAE488UQAtt9+ewB23HHH3GV2M+YjjHBkuHLlyqQ069fHliv3fdGV8UuzfvDBB0k/va233jq+fcEFFyD59+CDD8a3BwwYkPRe48aNAfjDH/4AwO67717h+E8++QSA++67D0j87Vx8cWwtu3HjxsXTbrHFFtnKdo0o0hQRiaAkI80rr7wyvh2+Ox7Wo0ePCu/vvffeuc2YJOnVqxdQ+d3SbHrggQfi27179wbUt5kvTzzxBAAjRoyo8J4f9/zyyy8DsMsuu6Q8X9euseXo/b2JJ598EoDhw4fH07Ru3TqDHGdOkaaISASqNEVEIiiJ5rm/8XP11VcDlTfJ/SBY/wikH7LQoEGDfGRRKrFu3bq8XGfBggXx7S5dugDQv39/AE455RSg8E262mrKlClA8rAif2P1pZdeAtJrlnvHH388ANdffz0AX3/9dVbymU2KNEVEIiiJSNNHmOHOYK9NmzYAXHvttUDicSwpvN122w2AL7/8ssJ7/qacH6ry/fffA3D22WcDycNSDj74YCBxc8/zw5fOO++8+L5ly5YBiaForVq1AuDUU0/N5L8iEfjf9a677lrgnOSGIk0RkQiKMtL0fZh+aFH5Pswtt9wyvu2HPHTu3DlPuZN0+cfpfD8jJCJB5xyQiCibNWsGwKuvvgokotTKrFq1CoCrrroKgM8//zz+nh/ofuihhwJwxBFHZPafkMiy8bCIH67m+6v947YAF110Ucbnz4QiTRGRCIoy0hwzZgxQ9cD1yZMnx7cVYRYvP8A8HBn4QdA+gvB90L6sK3vEzlu9ejUAffr0AeDtt9+ukKZ9+/YATJw4MaO8S835RyH9I5A1MX/+/KTXCxcuzChP2aRIU0QkgqKKNKdOnQpU/IbyYzD9ZKYHHXRQfjMmGbnwwgvj2/vttx+QGI/3/PPPA/DGG28A8PDDDwOJPq0wv8+PDfTCfaa+n1MKx08+7MvpkEMOiXyO8lMFFhNFmiIiERQ80vR3USEx+e93332XlKZRo0YA/PjjjwD85z//ib/nn/ipU0f1fynwrYQ5c+YAcNxxxwGJZUVOPvlkIPFEDyTugL///vtJ5/ItkL/+9a/xfT6Slfw488wzgeSlZJYvXw4kns7zP/0EHt5nn30W3w5P/QawdOnSpNf+nAAbNmwAkkfR5JNqGhGRCFRpiohEYOHmcVQdO3Z0fu3wmvrhhx/i2+EZuNPlZ+r2A+FzPY+imc10znXM6UWKSDbKOB1+IHxljzv6v1E/g/svf/lLAK644gqg8ptGmVAZR3fTTTfFt/1jz7niJ/HIZBB9JmWsSFNEJIKC3wjyEyvU1KhRo4DEgHe/cuEOO+yQWcYkr44++mggcSNn5syZ8ffKt4YmTJgAFNcKhZu7P/7xj/HtDh06AHD33XcD8OmnnwKJBxemTZtW4fhOnToBiRuA8+bNAzKvH3JBkaaISAQFjzTD61t722yzDQB/+9vfKj3GD4QGuPfeewFYtGgRkHgE89JLL81qPiW3/Kqhfm3zWbNmVUjj+zSl+Gy11Vbx7aOOOirppx8u5PsgfeQZVv7x2f/+979VXstPeFyoVWQVaYqIRFDwSLMyp59+OpB41K688OSmPtL0wgNmpXT4O6KPPPJIyrS33HILkDyoXYpX+YiwuklZ0uEn7/DT/+WbIk0RkQiKMtL0j01WxS+aJrWHv1vqF2MLr1F/1llnAfD4448D8OijjwIwePBgINrCXZJdK1asAJIn0Tn22GMBGDhwIABlZWVZvWa2x+VGpUhTRCQCVZoiIhEUZfO8vI0bNwJw+eWXA8kzqnj+8UmfRkrDv/71LyDx+KSfucg/FgvQt29fIDGT1aBLsl6aAAAH5klEQVRBg4DEbDdSOM2bNwcSXSiQeLx17dq1Sa+z3UwvFEWaIiIRFDzS7N69e3zbrwviJ2/w8yeuX78+6XVlbr75ZiCxDrqUBv+onV9h0t/c8dFl2EMPPZS/jElatthiCwD69esX3zd69GggsVaQX8vJT67jH3/t2bNnlecNP0YL0K5du/h2w4YNM812RhRpiohEUPBI889//nN8+6233gISj9D5NbLLCw+W9RHmH/7whxzlUHKp/CN1/hHa8MB1H6mUH9SsCTuKh+/bhMTa9b4V6R979EOQfL91dVNB+v5QL7xuWOPGjbOQ45pTpCkiEkHBI03/rQNwzjnnAHD//fcD4CdG9dOF+Z/h6aJat26dl3xKftx4440V9pWfhHifffYBElGpFBf/mfST8QwZMgRIrDTqR8P4deyr4x9y6N27d7azWWOKNEVEIih4pBnmJyANr0Qo4vk+Mj1GWxpatWoFJEY9+Bbk2LFjgeTJdfwddz8ZT/v27QG44YYbAGjatGkecpweRZoiIhEUVaQpm58ePXoAMHHixEr3AxxyyCFA4g5qoda7lprxfdH+/sUZZ5xRIY2PKEuBIk0RkQhUaYqIRKDmuRTUgAEDkn6KFDtFmiIiEajSFBGJQJWmiEgE5h9Rq9HBZiuBpdnLTklo45zbrtCZyBeVce2nMo4mo0pTRGRzo+a5iEgEqjRFRCKottI0s+ZmNif4t9zMvgy9ztmzbGa2rZk9Y2aLzWyRmXVKkb6fma0M8rXIzM7M8PpjzazqufhjaQ43s+9Cv4+rM7lmoRSwjC8xswVmNt/MxpnZVinSDw3lbZ6ZHZ3h9aeaWYcUae4I/S4+MbNVmVyzUApYxhcHZbzAzC5II30hPsfNzOxFM/swyGfK2cyrHdzunPsG6BCc/DpgnXPuL+UuasT6RjelulgEo4DnnHO9gkJtkMYx45xzg81sR2C+mT3nnIv/kZtZXefcT1nMI8CbzrlqC6XYFaKMzawNcA6wN/Aj8DRwPDA2xaG3OudGmtnewJtmtr0Ldcpnu4ydcxeGzn0R0K6a5EWrQGXcATgN6Aj8BLxqZi845z6r/si8f44vAOY45442sx2AxWb2WHXXqFHz3Mx2N7OFZjYOWAC0MrM1off7mtmDwfYOQdQ4w8ymmVnnFOfeFjjAOfcwgHNug3Puu3Tz5pxbDvwLaB1EJ2PM7F3gYTOra2a3B/mYa2b9gmvWMbO7g8j2NaBFpF9ILZTLMg7UA+oT++JuCHyVbt6cc/MBA5oF0cQ9ZjYNuMnMtjazh4N8zDazY4I8NjSzp4IIZnxw7ShOBB6PeExRy3EZtwPed8791zm3EZgCHJdu3vL4OXaAn816a2AV8L/qDsikT3MvYIRzrj3wZTXp7gCGO+c6An0AXwgHmNm9laTfFVgZ/JJmm9n9Zpb28nNmtjvQBvhnKJ+HOedOAc4GVjjnOgH7AwPNrDXQG9gFaA+cAXQJnW+YmR1VxeUOCsL6l8ysfbp5LCE5KWPn3FLgr8DnwNfEyuSNdDNlZl2A9c65b4NdLYHOzrnLgCHAK0EZdwNuM7P6wPnAaudcO2AosG/ofKOtmqa6me0G7Ay8nW4eS0iuPsfzgK4W62prBPwGaJVupvL4Of4r0MHMvgI+BC4It14qk8mz50ucczPSSHc40NaC6aGIRQcNnHMfAB9UkaeOxMLmmcSa6pcC16e4zslmdiix5l4/59ya4JoTnXPrgzTdgXZm5teHbQLsARwCPB40Tb4ws7f8SZ1zVfVVTgfKnHPrgmjmGWIFW5vkpIzNrDnwW2J/4N8D482sr3PuiRTXudTMTgfWAieE9j8ValZ2B35jZlcEr+sDrYmV8XAA59xsM1vgD3bOVZyrLFlf4O9Z7oIqFjkpY+fcfDO7HZgMrANmkyKCC+T7c3wUMA3oCuwJvGJm+zjn1lWVwUwqzR9C25uINZe8cNPHgE7OuQ1pnvcLYJkvyKApNTiN48Y55ypLF86nAec5514PJzCztJsNXrjLwDn3fNBEbOqcW1PdcSUmV2XcHfjE91WZ2bPEooJUleatzrmRKfJpQE/n3JJwgtCHvSb6AmdlcoIilqsyxjl3P3A/gJkNBz6t/gggz59jYhHpdUF0+ZGZfU6s8pxV1QFZGXIU1OyrzWwPM6tDct/FZGCgf1FdMyg41xfAv4PwHOAwYGFw7CAzOyeDrE4CzjOzusH52ppZA2L9LScEfSI7E/vWqZbFOqr9dmfgp1pWYSbJZhkDy4ADzayBxWqzw4BFwbHDfT9kDU0i1krxefHN8CnAScG+XwA/S+dkFrvp1MA5Ny2DPJWELJcxZrZ98LMM+B3Bl2IxfY6J/S0eFpynJbA7UO3NqmyO07yc2H/mH8SiRW8g8Kugw3Yh0D/IYFV9IRD7o3/SzOYS++P2i6O3A77JII/3AZ8Ac8xsPnAPsWj7aWK/vIXAaOA9f0A1fSF9LTZEYQ4wguTmYm2VlTJ2zr0LPEesyTaP2N3VvwVv/xxYnkEerwcaWWxY0gLgumD/nUBzM1sEXBNcmyCf1fVp9iV1BFybZPNzPCFIOwE4xzn3fbC/mD7H1xHre50LvAZc4pyrdpnMknqM0sxeBI7NwdAhKQJB1Pmyc+7IQudFcqfUP8clVWmKiBSaHqMUEYlAlaaISASqNEVEIlClKSISgSpNEZEIVGmKiESgSlNEJIL/B9DnwzedlQ5TAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8947d4b5f8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print_test_accuracy(show_example_errors=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 10,000次优化迭代后的性能\n",
    "\n",
    "经过10,000次优化迭代后，测试集上的分类准确率高达99%。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization Iteration:   1001, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   1101, Training Accuracy:  92.2%\n",
      "Optimization Iteration:   1201, Training Accuracy:  92.2%\n",
      "Optimization Iteration:   1301, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   1401, Training Accuracy:  92.2%\n",
      "Optimization Iteration:   1501, Training Accuracy:  95.3%\n",
      "Optimization Iteration:   1601, Training Accuracy:  95.3%\n",
      "Optimization Iteration:   1701, Training Accuracy:  93.8%\n",
      "Optimization Iteration:   1801, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   1901, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   2001, Training Accuracy:  95.3%\n",
      "Optimization Iteration:   2101, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   2201, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   2301, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   2401, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   2501, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   2601, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   2701, Training Accuracy:  93.8%\n",
      "Optimization Iteration:   2801, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   2901, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   3001, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   3101, Training Accuracy:  95.3%\n",
      "Optimization Iteration:   3201, Training Accuracy:  95.3%\n",
      "Optimization Iteration:   3301, Training Accuracy:  93.8%\n",
      "Optimization Iteration:   3401, Training Accuracy:  95.3%\n",
      "Optimization Iteration:   3501, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   3601, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   3701, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   3801, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   3901, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   4001, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   4101, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   4201, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   4301, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   4401, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   4501, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   4601, Training Accuracy:  93.8%\n",
      "Optimization Iteration:   4701, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   4801, Training Accuracy:  93.8%\n",
      "Optimization Iteration:   4901, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   5001, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   5101, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   5201, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   5301, Training Accuracy:  95.3%\n",
      "Optimization Iteration:   5401, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   5501, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   5601, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   5701, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   5801, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   5901, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   6001, Training Accuracy:  95.3%\n",
      "Optimization Iteration:   6101, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   6201, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   6301, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   6401, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   6501, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   6601, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   6701, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   6801, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   6901, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   7001, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   7101, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   7201, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   7301, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   7401, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   7501, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   7601, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   7701, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   7801, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   7901, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   8001, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   8101, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   8201, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   8301, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   8401, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   8501, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   8601, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   8701, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   8801, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   8901, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   9001, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   9101, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   9201, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   9301, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   9401, Training Accuracy:  96.9%\n",
      "Optimization Iteration:   9501, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   9601, Training Accuracy:  98.4%\n",
      "Optimization Iteration:   9701, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   9801, Training Accuracy: 100.0%\n",
      "Optimization Iteration:   9901, Training Accuracy:  96.9%\n",
      "Time usage: 0:00:24\n"
     ]
    }
   ],
   "source": [
    "optimize(num_iterations=9000) # We performed 1000 iterations above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on Test-Set: 98.7% (9868 / 10000)\n",
      "Example errors:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XncndO5//HPFfmRCScENSUPh5AYimqQltTQmCMRJIcgKuiJKYaqGs7hGKqGCj8lWj1Iqd95ETFUBRFDDUVCREZKCXFSCUmFoCXX7499r33f+xn23vez5yff9+uV17OHe7ierGevfa11r3stc3dERKQ4nWodgIhII1GlKSKSgipNEZEUVGmKiKSgSlNEJAVVmiIiKajSFBFJQZWmiEgKqjRFRFLoXMrOvXr18qampjKF0hhmzJix1N03qHUc1aIy7vhUxumUVGk2NTUxffr0Ug7RcMzsvVrHUE0q445PZZyOmuciIimo0hQRSaGk5rlINaxcuRKAkSNHArDlllsCMH78+JrFJKsvZZoiIimo0hQRSUHNc6l7H3zwAQAPP/wwAF27dgXgP//zPwHo2bNnbQKTsnnzzTcBOOWUUwA4+uijATjppJNqFlNblGmKiKTQ8JnmhAkTAPj3f/93ACZNmgTA4YcfXrOYpLI22mgjANZcc80aRyKlCNklwMEHHwzAO++8A8C7774LKNMUEWl4DZlp3nzzzdnHp512Ws57a6+9drXDkSo78MADAejevXuNI5H2uOGGG4DcIWMLFy7M2aZPnz5VjSkNZZoiIik0VKb5wgsvAHDmmWdmX1trrbUAuOuuuwD44Q9/WP3ApKJuueUWIC7rcePG1TIcaaevv/4agLlz5wLw3nvx7d9mBkDfvn2B+PNcj5Rpioik0BCZZvhmCrfRJV199dUADB8+vKoxSWUl+7juuOMOALp16wbE2Yg0ljDS5bbbbmtzm169egGw2WabVSWm9lCmKSKSQl1nmmGs1v777w/Ahx9+CMD111+f3eb000+velxSeVOnTs0+Xr58OQBXXXVVrcKREoTP7W9/+1sA3D3nZ9I111xTvcDaSZmmiEgKqjRFRFKoy+Z5GJpw9tlnA7Bo0SIAzjrrLADOOOOMNvf95ptvAOjUKfN9EIYySGP46KOPgPgCH8C3vvUtAEaPHl2LkKREYWjRrFmzgNY/k0OGDAFgl112qV5g7aRMU0QkhbrMNMPtVZMnTwbioUbXXXddm/usWrUqZ9tw8WjMmDEVi1PK79FHHwVgwYIF2deOPPJIIJ6o44svvgDiFoluna1voXzCcKKlS5e22ObFF18E4kk8tt9++ypFl54yTRGRFOoq0wx9HzfeeCMAO+ywAxBPNptPmKj2vvvuA2DevHkAjBo1KrtNly5dyheslNXnn38OwMSJE1u8d9555wFxZhlaE3/7298A+OMf/5jddr311qtonJJeyBqHDh0KtD64PWSfYTKe5KQ89UaZpohICnWVaYbBy++//z4A55xzDgDbbrttm/v885//BODCCy/MeT30fym7bAzhhoVp06YBsPfee2ff23XXXQF4/PHHAXjooYdy9g1/L6BMs55ddNFFQP7bKMOSJmHZi29/+9uVDywlZZoiIinUPNP8y1/+kn18++23A/HU9/nGYwYhy2g+lVToP5H6Nnv2bAB+/etf57z+ox/9KPs49Hc1v2V24403BuJxnFLfNt98cyCe2jF5O3QQxmSHcZvJ6ePqhTJNEZEUap5phj4MgK+++gqIx1wW495772319TC2T+pL6IOeMmUKEC+IFzKMILkw3mOPPQbkLsQF0Llz5s83XFWH+G8oTFgs9SeMhgl91RD3Ya5cuRKAxYsXA3FrM9ny2GmnnaoSZ1uUaYqIpKBKU0QkhZo3z8PtjhA3t8KtdGE29gsuuACI0/m33noru8/ll1+ec7xw2+SGG25YoYglrb///e/Zx8OGDQPgqaeeyrtPMStNhouAyVm+e/fuDcTDWrRmVP1Zd911ATj66KOzr4VutqeffhqATz/9FIBf/epXQHzTCsDrr78OwAYbbFDxWFujTFNEJIWaZ5r9+/fPPg4dxBdffDEQT9gRspIBAwYA8aqUAJ999hkQTwV32WWX5TyX2gkZ5rnnnpt9rXmG2aNHj5xt1llnHQDuueee7DavvPJK0ecMrZVXX30VUKbZKMJn/dZbbwVg7NixOe+HC0MA//jHP6oXWCtUs4iIpFDzTDMp3Ga11VZbAfFEDaHvKtxG15q99toL0EDnehCGFYXsMd9tc5deeikQTzj95ZdfAnGLISlMXhturdtnn30AOPTQQ7PbhElsQ8YqjWXHHXesdQgFKdMUEUmhrjLNIEz9FQY4hyUs5syZA8B3v/vd7LbhKmtYG1tqL4xuyJdhHnvssUB8S13wP//zPwAsW7asxT4HHHAAkDsVnNSHZ555psVrgwYNKnr/3/zmNwBceeWVQMuVKltbubJWlGmKiKRQl5lmsOaaa+Y8D1fKk8KkDX369KlKTFJYclG05rbYYgsg7rNcY401ct5fsmRJi32OO+44IJ7QRepHWNP8sMMOy74Wri+ERfKaC1P7JbPTMKF0uCU29F+HWyaT0wHW+rqFMk0RkRRUaYqIpFDXzfPmWmv2JW/DlNr6+OOPgZYD2JMzDoVB6211p4TZjpIz7o8YMQLQDQv1KFykXbFiRfa1P/zhDwA88sgjefdNXtwJzfEwVOwXv/gFEA8nC91w9UB/hSIiKTREpjlz5kwApk6dWuNIJJ8wqD0MUA+SGcduu+2W9xjhhoZw8Qdg5513LleIUmbhQl7yZoLkBC35hJncIS7jMAQtuUZUvVGmKSKSQkNkmmGoUchkwiQPAMccc0xNYpKWwlCQMHykPULfVT31YUnbNtlkEwAeeOCB7GuvvfZazjY33ngjAD/4wQ+A+FbJcePGVSHC8lOmKSKSQkNkmmFC4a5duwLwne98J/veHnvsUZOYRCSWvGWy+e2TjZpRtkWZpohICg2Rafbt2xeIV6oTEakVZZoiIimo0hQRSUGVpohICqo0RURSUKUpIpKCKk0RkRRUaYqIpKBKU0QkBStllTczWwK8V75wGkIfd9+g1kFUi8q441MZp1NSpSkisrpR81xEJAVVmiIiKeStNM1sfTObGf1bbGaLEs/XzLdvqcyss5nNMrMHitj28kRsb5jZwSWe+zkz26nANk1mNi2K8Skz26SUc9aKyjjvNueZ2Twze93MnjCzzfNtX69qVcZm9kFUVjPN7KUith9jZkui7eeZ2Y9KPP9dZja0wDaHR3+DM83sFTMbWOi4eWc5cvePgZ2ig18CfObu1zY7qZHpG11V6GQpnQ3MBroVuf017j7ezLYHnjKzDT3RYWtmnd396zLGdz3wW3e/28wGA1cAJ5Tx+FWhMs5rOvB/3f0LMzsduApouKUCalzGe7r78hTb3+3u48zsW8BsM3vI3Zcm4ix3GT8OTHZ3N7NdgInA9vl2aFfz3My2MrO5ZnY3MAfY3MyWJ94faWa3RY83MrP7zWy6mb1sZrsXcfw+wA+B29PG5u6zAQN6Rt80t5jZy8CVZtbDzO6I4njNzA6NztfNzO6Nvt0mAV3ynSPSH5gWPX4SODxtrPVMZQzuPs3dv4ie/hnYLG2s9azSZVwKd18MvAv0jloZE83seeCOqIXyyyiOWWY2Joqxk5ndbGbzzewJoFcR5/ks8cXbHSh4ZbyUPs1tgevdvT+wKM92NwJXu/uuwFFAKITdzGxCG/uMB35CEb9Ac1F6/aW7fxK9tDGwu7ufB/wHMMXdBwD7ANeZWRfgNGCZu/cDLgd2Thzv9jaaca8TV5TDgXXMbN208da51b2Mk04EHk0bawOoZBk7MM3MZpjZiWmCMrOtgD7AO4k493X3UcDJwEdRGX8XONXMegNHAFuQSWhOAAYmjneFmR3UxrmOMLMFwAPAmEKxlTIJ8dvuPr2I7fYDtrFoMXgy2UFXd38JaNHPEfVBvO/uM81svxTx/MTMRgMrgBGJ1+9NNDkGAwea2fnR8y5Ab2Av4GoAd3/NzOaEnd29rSb3WcBN0R/DM8Bi4JsU8TaC1b2MQ7yjgR2AM1LE2igqUsaR3d19UdTUfsLM5rn7CwXOc4yZ/QD4Chjj7sujcz7o7mFt6MFAPzMbGT1fF9iaTBnfE/0tfGBmT4eDuvuFbZ3Q3e8D7jOzvYHLouO3qZRK8/PE41VkmktBsuljwAB3/0eRxx0IHG5mQ6LjrGNmd7r78QX2u8bdxxeI04Ch7v52coPEH0LR3H0RMCzafx1guLt/lvpA9W21LuNovwPIZMSDUvx+jaRSZRw+I7j7YjN7EBgAFKo073b31hYVal7GY939yeQGZjas2NjaiPcpM7vTzP4lXz9sWYYcRTX7MjPb2sw6EVUmkanAqeFJoWaQu5/n7pu5exMwCng8fJjM7OrQR9VOjwGnJ2IJTbRngaOj174NbFfoQGbWy+JP4gVEzZWOajUt412BXwFDkhcjOqpylnHUt9wjetydTP/17Oj5mWb24xJCfQwYa2ado+NtY2ZdyZTxiKhvc1NgUL6DRPtuFT7HUXlboQtX5Ryn+VMyv8wLwAeJ108Fvhd12M4FTooCzNcX0pYdyTSD2+tSoLtlhkHMAS6JXr8JWN/M5gEXA9mFm/P0d+0LLDCzN4H1yFxZ7ehWtzK+lszFgUmWGZIyuYS4GkW5ynhj4Hkzex14mcwV6qnRe/2Aj0uI8VbgLWCmmc0GbiHTar4PWAjMJXOB8cWwQ54+zaPIXKWfSabfdkQr2+RomNsoo2+DR939gFrHIpWhMl49mNkjwGFlHjpUNQ1TaYqI1APdRikikoIqTRGRFFRpioikoEpTRCSFUga306tXL29qaipTKI1hxowZS1enWb1Vxh2fyjidkirNpqYmpk8v5g6sjsPMVqtlAVTGHZ/KOB01z0VEUlClKSKSgipNEZEUVGmKiKSgSlNEJAVVmiIiKZQ05KhWkhPKHn54ZsWJMPHIdttlpkm87LLLqh+YyGrqrrvuyj4+/vjcuaR/97vfAXD00UdXNaZKUaYpIpJCw2eaDzyQWTI7ZJoPPvggADvvnJmwO2SiUl8+/zyzesH8+fMB+M1vfpPz/kcffZR9HMr45JNPztkmZC577bVXxeKU4iSzyzXWWCPnvdGjRwOwYsUKAPr37w/AnnvuWZ3gykyZpohICg2ZaU6Y0HJ2/YsuugiApUszS7n8/Oc/B5Rp1oPLL788+zi0BEKmuWDBAiBuKYRWRHJy7PDar3/965znM2bMAODRR+OVdXv1KrjUtdTI2LFjgfi6w80335x97/vf/35NYmoPZZoiIik0ZKbZvG8L4NVXXwVa9o1J9YUrqeeccw6Q2z/ZPJPs168fAH369AFg2LCWq7A2HyExYMAAgOwkEwsXLsxuq0yzNsIVcoj7MNsS+rHDT1CmKSLSYTVkpplPyEYa9cpcRxD6lcPPH/+45RLXJ510EgDbbrstAN26dSt43Hnz5uUcNzmKQmqrb9++2cfffPNNq9usWrUq5/kpp5ySfRzKvxHGcirTFBFJQZWmiEgKHaZ5PnnyZCBusrV2QUGqY9y4cTk/y2XlypVAPFwpXPTRxZ/a22CDeOWIQYMGAfDcc8+1um3zwe8QXzxS81xEpIPpMJlmyCzDAGhdCOp4mrcmdCGofoQhYxAPWg+D2dvKOBuVMk0RkRQaMtNcsmRJ9nG4XTJkIWEyAOl45s6dC8TDynr37p3zszXvvZdZdDAMUwrbJvvgpLzCMLJtttkGUKYpIrJaa4hMM2QLITtITng6fvx4IB4c+8wzz1Q5OqmWMEVc6MsM/dahHzsptDzC7bXNM81f/vKX2W010qIybrnlFiC+3bWYtdXDbbVTpkwBcvtK64UyTRGRFBoi0wwTNFx33XUAXHXVVdn3QtZxwQUXAHF/inQcYWq55HRxANdffz2QexW9+UQgm222GQD7778/AD/72c9y3pfKCxOuhKy/tXGawZtvvgnAL37xCyB3+rh6oUxTRCSFus4077//fiCeWuzKK6/MeQ5xxhAyTekYjj322Ozj5n2Z4WdY5iKZNbZnIhCprJDdh4nCG50yTRGRFFRpioikUFfN8zBf4qRJk4C4Mzg0x4444ggA5syZk90nNN3CxYKO0gRYXYQumAsvvBBouWYQxOUfmtphlnANFeq4wpCxwYMHAzB06NBahpNDmaaISAo1zzTDwHWIs43wLROmmHr33XeBeNqoMDUYxLdNXnzxxQA0NTUBMGrUqMoFLe0WMstQxqGlEMp0+PDhQNzagDjTDENXlGE2puYzt+fbJlzsDTcl1BNlmiIiKdQ80zzuuOOyj8ON/RtuuCEQ3+oWbn0Lk82GyWghHm4Sso8rrrgCiPu/tO557YSJVUIrAOIMM2SWoXyaDzpvbdq3RlqxUGKhddGpUyZHyze4PQjbhDoh2adZ60mnlWmKiKRQs0wzZCHPPvts9rXQh/n000/n3be1Acu77LILEPeFhWw19HEmt5HKCqMgDjroICC33zr0QU+YMAFo2T8Z9k1mms37NKWxJG97Tuv3v/89AGeeeWb2NWWaIiINpGaZZvOlC6C8V0XDWL4wcS0o06yWkBGGDDNklcn32soWwq2yyXGaYVRFrTMMaZ+JEycCsN1229U4kvJQpikikoIqTRGRFGrWPG9tzepbb70VgM033xxoX8d/GN4QBkknm/8a8F4d4VbI8H+fLOO2mtih3JrPaAQazN7owoxTffv2BeI5M1tTzAD4WlOmKSKSQs0yzZBFLly4MPvabbfdBsDxxx8PwPz584Hi5soME3Y0n+RDE3hUXxgyFrLGcCEH4vII5R/mvwzrPoVB7+PGjcvuowt4HUMo4z322KPgtmFwe7ihoZ4uAirTFBFJwZqvu5LGrrvu6sWsMFesMLD5wAMPBOLV6/J9y4QZvkNWGm7wD+sJlXtAtJnNcPddy3rQOlaOMk7e9hrKKfQvf/LJJ0A8QUNoIfztb3/L7lPtLENlXBnhs3nuuecC8cD1pG+++QaIM82wRtCYMWPKGkspZaxMU0QkhZpP2JEUJmsIA9PbEjJSiPvNwoQPJ598MlBffSCru+Rtr6F/Mtx0cPbZZwPxypJh3R+VX8cTynTgwIFA65lmI1CmKSKSQl1lmsGee+6Z9/3k6oMrVqyodDhSQWEMX+jL1JjMji+0BsPPRqNMU0QkhbrMNGX10ehZh6x+lGmKiKSgSlNEJAVVmiIiKajSFBFJQZWmiEgKqjRFRFIoacIOM1sCvFdww46lj7tvUOsgqkVl3PGpjNMpqdIUEVndqHkuIpKCKk0RkRRUaYqIpJC30jSz9c1sZvRvsZktSjxfs1JBmdm5ZjbHzGab2d1mtlaB7S9PxPaGmR1c4vmfM7OdCmxznpnNM7PXzewJM9u8lHPWSi3K2Mz6mNnTZjY3KufTithnjJktieKaZ2Y/KjGGu8xsaIFtzk/8X8wxs6/NbN1SzlsLNfwc3xnKrMjta1HGPc3skehzPMfMjit4YHcv6h9wCXBuK68b0KnY4xRxnj7AX4Au0bEnAaMK7HM5MC56vD2whOgiV2KbzilieA7YqcA2+wBdo8enA3eX6/+gVv+qWMabhP9fYB3gbaBvgX3GAOOjx98ClgK9Sijju4ChKbYfBjxe6zJqlDKOjjkIGADMLHL7qpcx8B/AFdHjjYBlhc7Rrua5mW0VZQl3A3OAzc1seeL9kWZ2W/R4IzO738ymm9nLZrZ7Eaf4P2Qqzc5AN+DDYmNz99lk/gB6Rt80t5jZy8CVZtbDzO6I4njNzA6NYuxmZvdG326TonMXOs80d/8ievpnYLNiY2wElSxjd//Q3WdGjz8F5gObFhubuy8G3gV6R62MiWb2PHCHmXU2s19GccwyszFRjJ3M7GYzm29mTwBpp4b/N+CelPvUtUp/jt39GeCT9sRWxTJ2YO3ocQ8yFfU3+XYoZWq4bYHj3H26meU7zo3A1e7+ZzNrAv4AbG9muwEnuPuPkxu7+3tmdgPwPvAV8Ii7Tys2KDMbCHzp7p9YZmLbjYHd3X2VmV0NTHH30WbWE3gp+s89DVjm7v3MbGdgeuJ4twM3hA95G04EHi02xgZSkTJOMrMtybQOXik2KDPbikyL5J1EnHu5+5dmNhb4yN0HWKZb589m9jiwO7AF0J9MpjsXmBAd7wrgeXf/Yxvn6wHsB5xUbIwNpOJl3B5VLOMbgD+Y2YdkWj1HeJR2tqWUSvNtdy9mCbv9gG2iCgwyGWBXd38JeKn5xma2PnAImV/+U2CSmY109/9X4Dw/MbPRwApgROL1e919VfR4MHCgmZ0fPe8C9Ab2Aq4GcPfXzGxO2NndT8h30uicOwBnFIivEVWkjAMzW4dM98vp7v5ZEec5xsx+QObLdIy7L4/O+aC7fxltMxjoZ2Yjo+frAluTKeN7or+FD8zs6XBQd48XZm/dYcAz7v73ImJsNBUt43aodhkfBLxMpiuhLzDFzHbI9/dYSqX5eeLxKjJN4iDZvDVggLv/o8jjDgbecvelAGY2GRgIFKo0r3H38QXiNDJ9HG8nN0j8IaRiZgcAPwEGpfj9GkmlyhjLXIC4H7jd3R8qcre73X1cgTgNGOvuTzY7XynraIwE8q/217gqVsbtVO0yPgG4JMouF5jZ+2Qqz1fb2qEsQ46imn2ZmW1tZp3IdJoHU4FTwxMrcFUaWAjsYWZdLVOb7QvMi/a9OvRDttNjZC7ahFh2jh4+CxwdvfZtYLtCBzKzXYFfAUNCBd+RlbOMo3K9g8wFghubvXemmZXS1HsMGBuamma2jZl1JVPGI6J+r03JZBYFRd04A4GHS4ipIZT5c9ymOivjhWTqGMxsY2Ar4K/5dijnOM2fkvllXgA+SLx+KvC9qMN2LlG/kJntZmYTmh/E3Z8HHgJeA94AvgZ+G729I7C4hBgvBbpbZljSHDJXEgFuAtY3s3nAxdG5ieK8vY0/kGuB7mS6D2ZGGXFHV5YyJvPH/G/ADy0e+rJ/9F4/4OMSYrwVeAuYaWazgVvItKjuI/MBmQvcDrwYdjCzK8zsoDaONxx4NHHRr6MrVxljZvcCfwL6m9kHUVcW1FcZXwIMMrNZwBNkRhYsy3fyhrn3PMpOHnX3A2odi1SOmT0CHObuX9c6FqmMRi/jhqk0RUTqgW6jFBFJQZWmiEgKqjRFRFIoZZwmvXr18qampjKF0hhmzJix1FejWb1Vxh2fyjidkirNpqYmpk8v5maCjsPMVqtlAVTGHZ/KOB01z0VEUlClKSKSgipNEZEUVGmKiKSgSlNEJAVVmiIiKajSFBFJoaRxmiL5PPvsswCccsop2dcWLFgAwJ577glAv379cvaZN29ezr4QTxIdJpf53e8y8wGPGjWqEmGL5KVMU0QkhYplmv/85z8B+PjjzFyjc+fOBWDp0swk56+8kllH69FH4/XIPv88M6P9kUcemXOss88+G4B1180sOd21a9dKhS1lNH/+fCDOLiHOGv/0pz8B8NxzzwFxFhneTy5BMnz4cAC23XZbAA4//PBKhi2SlzJNEZEUypppfvhhvDz5jTdmln655pprWt22eWaRdN111+U8v/baawH4/ve/D8Cll16afW/vvfcuIWKppNBv2dpE1xMmtLpCQot9oWW/pzS2v/89s6jnIYcc0uK90aNHA3DiiSdWM6RUlGmKiKRQ1kxz/Ph4Bd2QLfbq1QuAXXbZJWfbkH189lm8vPCLL75IPs8//zwAP/3pT7OvPflkZhXPtddeu71hS4WEDDHZmgiPw9+F+ic7vhUrVgDxZ3WnnTLrFIbPM8T1wcsvvwzAHnvsAUD//v2rFmexlGmKiKRQ1kzzrLPOyj4+9thjAejRowcAW2yxRav7fPFFvDLq1KlTgbgfNPlNlDRjxozs40ceeQSAkSNHtjdsqbBk/2S4av7zn/8cUKbZkc2ePRuAIUOGAPE1j5BN7r777tltQyszjLqZNWsWoExTRKThqdIUEUmhrM3zjTfeuNXH+SQHqh966KEA7LfffgAcddRRQNwEb82YMWOAeOD7gQcemCJiqYYLLrgg+/iggw4C4oHv4WcYuC6Nb9myZQAcc8wxAPz1r38F4gtAO+64IwBDhw7N7tP8IvCkSZOA+ux2U6YpIpJCXU7YEbLPhx9+GIizkylTprTYduXKlQAcfPDBQDy8oXv37hWPU4qz//77Zx+HoWdhIa/33susb6VMs7G9+eab2cennnoqEF/M2WGHHQB44YUXcvY54ogjso/DMMIw9GjRokWVC7ZEyjRFRFKoy0yzuf/+7/8GYNNNNy24begXnTZtWkVjkvYJA97DsLEw9CiZjUrjeOmllwA444wzsq+FyXjCjQzDhg0DoHPn3Opmyy23bHG8sE8YrhQy2L59+5Yz7JIo0xQRSaEhMs2ePXsCsO+++wLx7VitCRlMmIquHgfHrs5+9rOfAfFEwmGy4fBzr732anPf0Kf9wAMPAHDrrbcCubdphtszw5SD3/nOd8oWu7QUbpcO2WVr/uu//guAp556CohHxYQbYFoTbq8Ofd7KNEVEGlRDZJprrbUWAPvssw+QP9Ps1CnzPdClS5fKByapNZ/EI/wM4zU32GADIB6nB/Dggw8CcSui+b7JTDNMch1GU4TsRtPLVUbylubm/vVf/xWAgQMHArB8+XIg7v+86qqrKhxdZSjTFBFJoSEyzeCcc84B4jsOIJ6gOAgTnB533HFAvJyC1JcwiUeYwCMsvtZ8EbXWXgtXY8MdJ8ksMkwAEpbYGDRoEBCPC+3du3e5f5XVWii/ZOsvXEdoqz85ZKcHHHBAi/dam7C63ijTFBFJQZWmiEgKDdE8X7x4MRAPIwkTAEDLdD48D830r776KvteuKAktRcm8Qi3yAahKR4uCEHc5D7ppJOA+JbLbt26tThuaPqF4SyTJ08G4rlekxeYpHSbbLIJkH/4UHOh2Z689XK99dYDWl8zrN4o0xQRSaGuMs133nkHiG/NCoOZw0Do1r6F2vpmCoPbw22VEA9xaL5ekVRfKJ/QMgg/L7zwQgAuu+yydh03ZJ8howx/HyHjXLLgDjuKAAAItElEQVRkCZCbyUpthJtWALbffnsA5syZU6twiqZMU0QkhZpnmmEqN4ARI0YA8Oqrr5bt+MmhEIcddhgQT1mV/KaT6gpDgpr3YYZ+y3JpPgA+ZJwnn3xyWc8jpQmtv+aZZmh91hNlmiIiKdQ800wOVA9X0JoLV9u22267Fu+FiYrDLVrNhat7EE90usYaa7QvWCmbwYMHA/DYY48BcOKJJwLlG3weJgBpPrpCGWZ9CuucT5w4Mef1MHFxuPmhHijTFBFJoeaZZjKzeOihh4B4jGUQlq5obQmLMD3Y2LFjWz1+uCoHcNppp5UWrLTLvHnzso/D7a1hKrEwFVyY0q1cwuTGoS9TUwTWt+9973tAvNTNl19+CdTnSBdlmiIiKajSFBFJoebN86Rwm+OGG25Y9D4hre/RowcAn376ac77YZ5GgP/93/8Fil+TXcojuYpouM2xmJna2+Oiiy4C4gtM4ULQ+eefX9bzSHmFz2ZolgfJW6brhTJNEZEU6irTbI9woaepqQmAN954A4gvACSHNIV1R6S6wkB2iMslzI3a1mD25DroobUQXku2HiAesA5xhtl8JcQw6YfUpzCsMNxwEj635bzRpVyUaYqIpNDwmWYQpqY677zzcl5PrmIXZnHfeuutqxeY5AwnCn2MYSb10McZXi9m5va2nkM8YUeYei78lPq2/vrrA7DmmmsCcdmGmeHriTJNEZEUOkymefzxxwNw5513AvHUY8nV8s4991wAhgwZAsTfblJZyWwvZIVh1cggOQAectf9Ce999NFHQHw7bWsD4s8880wgt09UGk89T0asTFNEJIUOk2mGqcXC7XP3338/ACtXrsxuEzIVLXtRXcllKdo7ubCsHo488kgAbrrpphpH0jZlmiIiKXSYTDM45JBDcn6KSOO44oorANh0000BuO2222oZTquUaYqIpKBKU0QkhQ7XPBeRxrX22msD8SoL4Wc9UaYpIpKCKk0RkRRUaYqIpGDNV+tLtbPZEuC98oXTEPq4+wa1DqJaVMYdn8o4nZIqTRGR1Y2a5yIiKajSFBFJIW+laWbrm9nM6N9iM1uUeL5mJQIys/6Jc8w0sxVmlnfBcjMbY2ZLou3nmdmPSozhLjMbWmCbw81sVnTOV8xsYCnnrJValHF03g/M7I3oPC8VsX0tyrinmT1iZq+b2RwzO66Uc9ZKrco4Onfn6HPyQBHbXp6I7Q0zO7jEcz9nZjsV2KbJzKZFMT5lZpsUOm7ewe3u/jGwU3TwS4DP3P3aZic1Mn2jqwqdrBjuPjdxzv8DLAIK/ocDd7v7ODP7FjDbzB5y9+ykjWbW2d2/LkeMkceBye7uZrYLMBHYvozHr4palHHCnu6+PMX21S7j04GZ7n6wmW0EzDez35f5HBVX4zI+G5gNdCu0YeQadx9vZtsDT5nZhp648FKBMr4e+K27321mg4ErgBPy7dCu5rmZbWVmc83sbmAOsLmZLU+8P9LMboseb2Rm95vZdDN72cx2T3GqHwLz3P2DYndw98XAu0Dv6Jtropk9D9wRfev9MopjlpmNiWLsZGY3m9l8M3sCaDm7bcvzfJYozO5Ah7qiVsUyTq1aZUymTNeOHvcAlgLflP83qo1Kl7GZ9SHzGb49bWzuPhswoGfUKrjFzF4GrjSzHmZ2RxTHa2Z2aHS+bmZ2b9QSmQR0KeJU/YFp0eMngYIr8JXSp7ktcL279yeTDbblRuBqd98VOAoIhbCbmU0ocI6RwD1pgjKzrYA+wDuJOPd191HAycBH7j4A+C5wqpn1Bo4AtiDzH3gCMDBxvCvM7KA2znWEmS0gkwmPSRNng6hkGTswzcxmmNmJaYKqYhnfAOxkZh8CrwOnJ7OeDqKSZTwe+AntSCgs0931pbt/Er20MbC7u58H/AcwJSrjfYDrzKwLcBqwzN37AZcDOyeOd3sbTfXXiSvK4cA6ZrZuvthKuff8bXefXsR2+wHbWDx9fU8z6+ruLwFt9mVF/wkHk0nvi3GMmf0A+AoY4+7Lo3M+6O5hBfrBQD8zGxk9XxfYGtgLuCdqmnxgZk+Hg7r7hW2d0N3vA+4zs72By6LjdySVLOPd3X1R1NR+wszmufsLBc5T7TI+CHgZGAT0BaaY2Q7u3pHWgq5IGVumv/h9d59pZvuliOcnZjYaWAGMSLx+b6LrYDBwoJmdHz3vAvQmU8ZXA7j7a2Y2J+zs7m01uc8Cboq+uJ8BFlOgNVFKpfl54vEqMql0kEyLDRjg7v9IefyDgZeSfVYF3O3u41p5PRmnAWPd/cnkBmY2LGVsOdz9KTO708z+JWUfXb2rWBm7+6Lo52IzexAYABSqNKtdxicAl0TZ5QIze59M5Vl/i3G3X6XKeCBwuJkNiY6zjpnd6e7HF9jvGncfXyBOA4a6+9vJDawd6wpFf4fDov3XAYYX+lIsy5Cj6BtgmZltbWadQhCRqcCp4UkbKXJr/o1mTXMzO9PMflxCqI8BY82sc3S8bcysK/AsMCLq99qUTGaRV9QfZNHjXcl0onekCjNHOcs46pPqET3uTqbfa3b0vG7KGFgI7BsdZ2NgK+CvJcRW18pZxu5+nrtv5u5NwCjg8VBhmtnVoR+ynR4jc5EuxBKa4c8CR0evfRvYrtCBzKxX+BwDFxB1O+RTznGaPyXzy7wAJC/cnAp8L+qUnwucFAXbZl+Ima0N7E3Lq+b9gI9LiPFW4C1gppnNBm4hk23fR+YDMpdMp/WLiVja6u86iswV3Jlk+ntGtLJNR1OuMt4YeN7MXifT/J3s7lOj9+qpjC8BBpnZLOAJ4Fx3X1ZCbI2gbJ/jPHYk0wxur0uB7pYZljSHTDkB3ASsb2bzgIuB18IOefo09yXTingTWA+4qtDJG+o2SjN7BDis0YZ8SPFUxh1blNU96u4H1DqW9mqoSlNEpNZ0G6WISAqqNEVEUlClKSKSgipNEZEUVGmKiKSgSlNEJAVVmiIiKfx/XUGar1bBMx4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8947975748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion Matrix:\n",
      "[[ 972    0    1    0    0    0    3    1    3    0]\n",
      " [   0 1128    3    1    0    0    2    0    1    0]\n",
      " [   1    0 1021    2    1    0    0    2    5    0]\n",
      " [   0    0    0 1007    0    1    0    0    2    0]\n",
      " [   0    0    2    0  969    0    2    0    0    9]\n",
      " [   1    0    1   13    0  871    3    0    1    2]\n",
      " [   1    2    0    0    2    3  948    0    2    0]\n",
      " [   0    2   11    6    0    0    0 1003    1    5]\n",
      " [   1    0    4    4    0    2    0    4  954    5]\n",
      " [   0    3    0    4    2    4    0    1    0  995]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARIAAAD3CAYAAADRydumAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAGrFJREFUeJzt3XmwXnWd5/H3hwQIm2xBBhIcmIZBM0wLmMEoLaVEaEUGsMe2YFpFh2pmemgFtUexe6qY7tl02nGb7qGaBhRaxCVAmVKbRVxopyQNhMgWkIgCiYGwRERcILmf+eP8rjzE5Obc55xzn+V+XlWn8izn+f1+T27uN7/tfI9sExHRxA6DbkBEjL4EkohoLIEkIhpLIImIxhJIIqKxBJKIaCyBJCIaSyCJiMYSSCKisQSSiGhs7qAbEDGb/e7rdvMTT26ude5td/zqOttv6LhJfUkgiRigx5/czIrrFtY6d8cDfjC/4+b0LYEkYqDMZk8MuhGNJZBEDJCBCUb/CvwEkogBMuY515sjGWYjs2oj6Q2S7pO0RtL5fZZxqaQNku5q2JaDJH1T0j2S7pZ0bp/lzJP0j5K+V8r58wZtmiPpdklfaVDGjyTdKWmVpFsblLOXpGWS7pW0WtKr+ijj8NKOyeOnks7rsz3vLX+/d0m6UtK8Pss5t5Rxd79t2ZoJXOsYZiMRSCTNAf4aeCOwCDhD0qI+ivoM0Mas9ybg/bYXAUuAc/psz6+A422/HDgSeIOkJX226VxgdZ+f7fU620faXtygjE8C19p+KfDyftpl+77SjiOBVwA/B66ZbjmSFgDvARbbPgKYA5zeRzlHAH8IHEP1nU6WdOh0y9mSgc241jHMRiKQUP3w1th+wPazwOeBU6dbiO2bgCebNsb2etsry+OnqX5RFvRRjm3/rDzdsRzT/hcjaSHwJuDi6X62bZL2BI4DLgGw/aztnzQsdinwA9sP9vn5ucAukuYCuwI/7qOMlwErbP/c9ibg28Dv9dmeF0iPZOYsAB7ueb6WPn5xuyDpYOAoYEWfn58jaRWwAbjBdj/lfAL4ANB0+t/A9ZJuk3R2n2UcAjwGfLoMtS6WtFvDdp0OXNnPB22vAz4KPASsB56yfX0fRd0FvEbSvpJ2BU4CDuqnTS9oH7DZrnUMs1EJJENJ0u7AVcB5tn/aTxm2N5fu+0LgmNKFnk4bTgY22L6tn/q38Du2j6YaQp4j6bg+ypgLHA1caPso4BmgrzktAEk7AacAX+rz83tT9V4PAQ4EdpP0tumWY3s18BHgeuBaYBXQyizpRM1jmI1KIFnHC6P/wvLawEjakSqIXGH76qblle7/N5n+HM6xwCmSfkQ15Dte0mf7bMO68ucGqvmIY/ooZi2wtqdntYwqsPTrjcBK24/2+fnXAz+0/Zjt54CrgVf3U5DtS2y/wvZxwEbg+3226fkya86PZI6kHbcAh0k6pPwPdTqwfFCNkSSqOYDVtj/WoJz9JO1VHu8CnADcO50ybH/I9kLbB1P9vXzD9rT/x5W0m6Q9Jh8DJ1J156fF9iPAw5IOLy8tBe6Zbjk9zqDPYU3xELBE0q7l57aUPielJb24/PkSqvmRzzVoFwA2PFfzGGYjsY/E9iZJfwxcRzXrfqntu6dbjqQrgdcC8yWtBS6wfUkfTToWeDtwZ5nfAPhT21+bZjkHAJeVVakdgC/a7nv5tqH9gWuq3zXmAp+zfW2fZb0buKIE/QeAd/VTSAloJwD/vs92YHuFpGXASqrVttuBi/os7ipJ+wLPAee0MIkMiM2oeTEDptzXJmJwjvjtnXzVV+tdQvPSl6y/reGyfGdGokcSMc7GoUeSQBIxQNWGtASSiGhowgkkEdFAeiQR0ZgRz3nOoJvR2KjsI/m1Blu3Wy0j5cxMOcPUljbLmTTZI6lzDLORCyRAGz/Itv4xpJzuyxmmtrRZTiE2e4daxzAb7tZFjLkqQ9oOtY46tpZzR9I+km6QdH/5c+/yuiR9quT4uUPS0T2fObOcf7+kM7dX71DNkeyxz46ev2DnKc/Z98CdOORf7j7lLron7tppyjLmsSsv0j6Nd+KlnO7LGaa21C3nlzzDs/5V7bFIy8OWzwB/BVze89r5wI22P6wqKdj5wAeprmM6rByvBC4EXilpH+ACYDFVrLtN0nLbG7dV6VAFkvkLduYvrp7Wxa9bdfnhja/ujujbCt9Y+1xbrQ5bbN9UUlv0OpXq0hCAy4BvUQWSU4HLXW1vv7lktjugnHuD7ScBJN1AdTHpNq95GqpAEjEbTXQ/kbq/7fXl8SNU11XBtvP8TDv/TwJJxAAZ8axr/xrO3yKX7kW2p3UBom1Lav0CuwSSiAGanGyt6fE+L9p7VNIBtteXocuG8vq28vys4/mh0OTr35qqgk5XbdRC5veIcbfZqnU0sByYXHk5E/hyz+vvKKs3S6jSUK6nStdxoqS9ywrPieW1beqsR9KT+f0EqjHWLWXmt0mSm4ixYsTmFv8/31rOHeDDwBclnQU8CLy1nP41qtyza6iy9L8LwPaTkv4rVUIxgL+YnHjdli6HNr/O/A4gaTLzewJJRI+JdldtztjGW0u3cq6Bc7ZRzqXApXXr7TKQbG3m95Ud1hcxcqot8qO/L3Tgk63l2oWzodpsFjGbjMtFe10GklqZ38vy1UXAdnesRowbm6G/jqaOLr/BUGV+jxhOYqLmMcw665G0lfk9YpxVd9ob/R5Jp3Mk5fYM071FQ8SsksnWiGjEKDlbI6K59EgiopEs/3bgibt2aiWXyHU/XrX9k2r43QOPbKWciG0x7e5sHZShCiQRs9GwJ3auI4EkYoBspUcSEc1lH0lENFIlNsrQJiIaaTf586B0mdjoUuBkYIPt5qnhI8aQYSyWf7sMhZ+hSmEfEdswubO1zjHMurxob2v314iILUwj+fPQyhxJxABV+UiGu7dRx8ADSW+GtHnsOuDWRMy8YR+21DHwQNKbIa2Ne7NGjJJqjiRDm4hoaBy2yHcWCsv9Nb4LHC5pbbmnRkT0MGLTxJxaxzDrctVmW/fXiIge2dkaEY1k1SYiWpHJ1ohoJDlbh1hbmc3+7IF2Mq3993+WTGudU0u/jJ75HQiZI4mIRqpUiwkkEdGENfRLu3UkkEQM0LgkNhr96eKIEddmGgFJ75V0t6S7JF0paV65//YKSWskfaHcixtJO5fna8r7B/f7HRJIIgZoco6kjUAiaQHwHmBxSSY2Bzgd+AjwcduHAhuByV3mZwEby+sfL+f1pcst8gdJ+qake0qEPLeruiJGWcuJjeYCu0iaC+wKrAeOB5aV9y8DTiuPTy3PKe8vlfpb/uqyR7IJeL/tRcAS4BxJizqsL2LktJkhzfY64KPAQ1QB5CngNuAntjeV09YCC8rjBcDD5bObyvn79vM9OgskttfbXlkePw2s5vkvEBEAhk3eodYBzJd0a89xdm9Rkvam6mUcAhwI7MYMpTudkVWbMolzFLBiJuqLGBXT3EfyuO3FU7z/euCHth8DkHQ1cCywl6S5pdexEFhXzl8HHASsLUOhPYEnpv8tZmCyVdLuwFXAebZ/upX3z56MsM/xq66bEzF0WpwjeQhYImnXMtexFLgH+CbwlnLOmcCXy+Pl5Tnl/W/Y/W3t7bRHImlHqiByhe2rt3ZOMqTFbNbmtTa2V0haBqykmqO8nep366vA5yX9t/LaJeUjlwB/J2kN8CTVCk9furyvjagautr2x7qqJ2LUucUt8rYvAC7Y4uUHgGO2cu4vgd9vo94uhzbHAm8Hjpe0qhwndVhfxEiaQLWOYdZlhrTvwJB/+4gBs3PRXkQ0JjZPjP4G8wSSiAFrc45kUBJIIgYo+UhmgbYym71/zd2tlPO/D/0XrZQzlgaQ2awVHt2m90ogiRiwYV+RqSOBJGKATOZIIqKxZJGPiBZMTCSQREQDdoY2U5I0D7gJ2LnUs6xcBxARPTK0mdqvgONt/6xcBfwdSX9v++YO64wYOVn+nULJa/Cz8nTHcozBX1lEu8ZhaNPpJn9JcyStAjYAN9j+jQxpSWwUs5kRdr1jmHUaSGxvtn0kVXq3YyQdsZVzLrK92PbiHdm5y+ZEDCXXPIbZjFx2aPsnVOneZiQRbcTIMHhCtY5h1uV9bfaTtFd5vAtwAnBvV/VFjKpxGNp0uWpzAHCZpDlUAeuLtr/SYX0RIymrNlOwfQfVLSgiYhtyrU1ENGcggSQimsrQJiKaSyCJOtrKbPYH965tXMYVL13YQkvG2A5zmpexeTonD//Sbh0JJBGDlKt/I6IVGdpERHPpkUREU2PQI+n8WptyBfDtkrKrNWJrxuCqvZnokZwLrAZeNAN1RYyWctHeqOs6H8lC4E3AxV3WEzHSWuyRSNpL0jJJ90paLelVkvaRdIOk+8ufe5dzJelTktZIukPS0f1+hdqBRFI/yUI+AXwAmOjjsxGzg1XvqOeTwLW2Xwq8nGo0cD5wo+3DgBvLc4A3AoeV42zgwn6/wnYDiaRjJN0J3F+ev1zS/6nxuZOBDbZv2855yZAWs5pc79huOdKewHHAJQC2ny25gE4FLiunXQacVh6fClzuys3AXpIO6Oc71OmRfAo4GXiiNO57wOtqfO5Y4BRJPwI+Dxwv6bNbnpQMaTGr1R3W1BvaHAI8Bny6LHBcLGk3YH/b68s5jwD7l8cLgId7Pr+2vDZtdQLJDrYf3OK17W4Ctv0h2wttHwycDnzD9tv6aGPEGKs5rKmGNvMne+/lOHuLwuYCRwMX2j4KeIbnhzHAr5Oyt74GVGfV5mFJxwAuSYreDXy/7YZEzFr1f60ft714ivfXAmt7kqwvowokj0o6wPb6MnTZUN5fBxzU8/mF5bVpq9Mj+SPgfcBLgEeBJeW12mx/y/bJ029exCwwUfPYDtuPUP3Hf3h5aSlwD7AcOLO8dibw5fJ4OfCOsnqzBHiqZwg0LdvtkdjeQDU0iYi2tZ/Y6N3AFZJ2Ah4A3kVJdSrpLOBB4K3l3K8BJwFrgJ+Xc/uy3UAi6W/ZSufL9pbjs4joQ50VmbpsrwK2NvxZupVzDZzTRr115ki+3vN4HvBmXjjTGxFNDPn29zrqDG2+0Ptc0t8B3+msReOojWQ5tJOU6LBb2lliv/9fjemen4lpZSWKop9rbQ7h+XXoiGiozaHNoNSZI9nI852vHYAn2WJtOiIaGPcMaZJEtV9/cm15okzQREQbzFhciTblPpISNL5Wbga+OUEkon1tXWszSHU2pK2SlDvmRXRlnBMbSZprexPVbTdvkfQDqr37ouqsbDd3Qblg72mqa3M2bWd7b8TsNORBoo6p5kj+keoCoFMa1vE62483LCNiLI3CsKWOqQKJAGz/YIbaEjE7jfmqzX6S3retN21/rEb5Bq6XZOBvbF803QZGjL0x75HMAXan2U03fsf2OkkvBm6QdK/tm3pPKDkVzgaYx64NqooYTRqD5d+pAsl623/RpHDb68qfGyRdAxwD3LTFORcBFwG8SPuMQWyOmIYxmSOZavm30cBN0m6S9ph8DJwI3NWkzIixNM7Lv2zlsuNp2h+4ptocy1zgc7avbVhmxPgZ8iBRxzYDie0nmxRs+wGq7fURMYVxH9pERNSSm4hHDNoY9EgSSCIGyeO//BttGaKsW21lNnvzPY+1Us41i/ZrpZyRlh5JRDQhxmOyNYEkYtASSCKikTHZ2ZpAEjFoCSQR0dQ4rNp0uiFN0l6Slkm6V9JqSa/qsr6IkTTm19q04ZPAtbbfUu5FmjwBEb1GIEjU0VkgkbQncBzwTgDbzwLPdlVfxKgah8nWLoc2hwCPAZ+WdLuki0s6gYjoNQZDmy4DyVyq5NEX2j6KKgP9b9yhT9LZkm6VdOtzjOn9ZCOmMFvua9OvtcBa2yvK82VUgeUFbF9ke7HtxTvSzg2uI0ZKeiTbZvsR4GFJh5eXlgL3dFVfxCiq2xuZTo9E0pwynfCV8vwQSSskrZH0hbLwgaSdy/M15f2D+/0eXecjeTdwhaQ7gCOB/9FxfRGjp/0eybnA6p7nHwE+bvtQYCNwVnn9LGBjef3j5by+dBpIbK8qw5bftn2a7Y1d1hcxitrskUhaCLwJuLg8F3A81dQCwGXAaeXxqeU55f2l5fxpS4a0iEFrt0fyCeADwOR+2X2Bn5Tb70I1d7mgPF4APAxQ3n+qnD9tCSQRg1Y/kMyfXOEsx9m9xUg6Gdhg+7YZbD2Qa20iBmt6E6mP2148xfvHAqdIOgmYB7yIanf5XpLmll7HQmBdOX8dcBCwVtJcYE/giel/iQSSqe0wp51y3NJVWR6eNcC2Mpu9d83q7Z9Uw8cPfVkr5dDfFMELTffH1NKP1faHgA8BSHot8Ce2/0DSl4C3AJ8HzgS+XD6yvDz/bnn/G3Z//8gytIkYME3UOxr4IPA+SWuo5kAuKa9fAuxbXn8fW9kwWld6JBED1sWuVdvfAr5VHj9AdbvcLc/5JfD7bdSXQBIxSCOwa7WOBJKIQUsgiYgmxiWLfGeTrZIOl7Sq5/ippPO6qi9iZI3BRXud9Uhs30d1fQ2S5lCtWV/TVX0Ro0pDtKzfr5ka2iwFfmD7wRmqL2I05Jad03I6cOUM1RUxWka/Q9L9hrSS++AU4EvbeD8Z0mJWS4a0et4IrLT96NbeTIa0mPUy2VrLGWRYE7F1I9DbqKPrG2TtBpwAXN1lPREjLT2Sqdl+hj4TpUTMBuOyIS07WyMGTBOjH0kSSCIGaQSGLXUkkEQMWDakjbuJzYNuwdhrK7PZv1m9oZVyrlq0fyvlTEt6JBHRVCZbI6IZM1S5ePuVQBIxYJkjiYhGso8kIpqzx2Jo0/UW+fdKulvSXZKulDSvy/oiRlGu/p2CpAXAe4DFto8A5lDlJYmIXrnWplb5u0h6DtgV+HHH9UWMnGHvbdTRWY/E9jrgo8BDwHrgKdvXd1VfxEgyMOF6xxDrcmizN3AqcAhwILCbpLdt5bxkSItZbQZu2dm5LidbXw/80PZjtp+jykny6i1PSoa0mPUmV262dwyxLudIHgKWSNoV+AVVJvlbO6wvYiRljmQKtlcAy4CVwJ2lrou6qi9iJNVdsRnyYNN1hrQLgAu6rCNilFU7W4c8StQwE1nkI2IqEzWP7ZB0kKRvSrqnbAQ9t7y+j6QbJN1f/ty7vC5Jn5K0RtIdko7u9yskkEQMmOxaRw2bgPfbXgQsAc6RtAg4H7jR9mHAjeU5VLeKOawcZwMX9vsdEkgiBsk195DU2Edie73tleXx08BqYAHVNozLymmXAaeVx6cCl7tyM7CXpAP6+Rq5aC/GwlUve3Er5Zxz/32Ny/jhab+c1vldrNpIOhg4ClgB7G97fXnrEWAyDdwC4OGej60tr61nmhJIIgat/mTrfEm9Wygusv0bK6GSdgeuAs6z/VNJPVXZUvuhK4EkYpA8rV2rj9tePNUJknakCiJX2J68Md2jkg6wvb4MXSYT3K4DDur5+MLy2rRljiRi0Fra2aqq63EJsNr2x3reWg6cWR6fCXy55/V3lNWbJVTXw017WAPpkUQMXnsDjWOBtwN3SlpVXvtT4MPAFyWdBTwIvLW89zXgJGAN8HPgXf1WnEASMWBtbUiz/R2qPW5bs3Qr5xs4p426u86Qdm7Jjna3pPO6rCtiJBnY7HrHEOsyjcARwB8CxwAvB06WdGhX9UWMIlFvM9qwb6PvskfyMmCF7Z/b3gR8G/i9DuuLGE1jkEagy0ByF/AaSfuWVAIn8cKlpoiAsQgknU222l4t6SPA9cAzwCrgN26mK+lsqn3+zGPXrpoTMZxMrQvyhl2nk622L7H9CtvHARuB72/lnGRIi1ltHOZIOl3+lfRi2xskvYRqfmRJl/VFjKQhDxJ1dL2P5CpJ+wLPAefY/knH9UWMFhsmRn9s03WGtNd0WX7EWBj9OJKdrRGDNuzzH3UkkEQMWgJJRDQyeae9ETdUgeRpNj7+dS97cDunzQceb1hVG2WknJkpZ0bb8vXtX8RRp5x/Wq9JAMO/2ayOoQoktvfb3jmSbt1ecpeZKCPlzEw5w9SWNst5gQSSiGjEwObRX7ZJIIkYKIMTSAahjdt+tnXr0JTTfTnD1JY2y3neGAxt5DH4EuNG0maq+yXPpbo3yZm2f95nWa8F/sT2yZJOARbZ/vA2zt0L+Le2/+806/gvwM9sf7SfNs5me+60v1/9T86ode61D3/yttbnZ1qS5M/D6Re2j7R9BPAs8B963yzJeqf9s7O9fFtBpNgL+I/TLTcaGoM0Agkkw+8fgEMlHSzpPkmXU+V6OUjSiZK+K2mlpC+V+5kg6Q2S7pW0kp5kUpLeKemvyuP9JV0j6XvleDVVkuDfkrRK0l+W8/6TpFvKvWH/vKesP5P0fUnfAQ6fsb+NcTQGgWQU50hmDUlzqe7Pem156TCqYc7NkuYD/xl4ve1nJH0QeJ+k/wX8LXA8VXbwL2yj+E8B37b9ZklzgN2p7gl7hO0jS/0nljqPoUoqvFzScVT5ZU4HjqT6N7QSuK3dbz9L2LD5N9L0jJwEkuG0S8/tBP6B6l4lBwIPlnu0QpWSYRHw/8qd1HYCvgu8FPih7fsBJH2WkjhqC8cD7wCwvRl4avIu9T1OLMft5fnuVIFlD+CayXkbScsbfdvZbsh7G3UkkAynX0z2CiaVYPFM70vADbbP2OK8F3yuIQH/0/bfbFFH7gjQpjEIJJkjGV03A8dOZuaXtJukfw7cCxws6bfKedtaErgR+KPy2TmS9gSepuptTLoO+Hc9cy8LJL0YuAk4TdIukvYA/nXL320WcXWtTZ1jiCWQjCjbjwHvBK6UdAdlWGP7l1RDma+WydYN2yjiXOB1ku6kmt9YZPsJqqHSXZL+0vb1wOeA75bzlgF72F5JNffyPeDvgVs6+6LjzmBP1DqGWfaRRAzQnnP386tedFqtc6/bePHQ7iPJHEnEoI3Bf+YJJBGDlOXfiGiDk/w5IpoZ/l2rdSSQRAzSmKRazPJvxKB5ot5RQ7nO6j5JaySd33HLfy09kogBMuCWeiTlmqm/Bk4A1gK3SFpu+55WKphCeiQRg2S32SM5Blhj+wHbzwKfB07ttP1FeiQRA+b2ln8XAA/3PF8LvLKtwqeSQBIxQE+z8bqve9n8mqfPk3Rrz/OLbLef+rEPCSQRA2T7DS0Wtw44qOf5wvJa5zJHEjE+bgEOk3SIpJ2okk/NSK6Y9EgixoTtTZL+mCr9wxzgUtt3z0Tdufo3IhrL0CYiGksgiYjGEkgiorEEkohoLIEkIhpLIImIxhJIIqKxBJKIaOz/A2Sw99TALCZMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f88a807ee48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print_test_accuracy(show_example_errors=True,\n",
    "                    show_confusion_matrix=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 权重和层的可视化\n",
    "\n",
    "为了理解为什么卷积神经网络可以识别手写数字，我们将会对卷积滤波和输出图像进行可视化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制卷积权重的帮助函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_conv_weights(weights, input_channel=0):\n",
    "    # Assume weights are TensorFlow ops for 4-dim variables\n",
    "    # e.g. weights_conv1 or weights_conv2.\n",
    "    \n",
    "    # Retrieve the values of the weight-variables from TensorFlow.\n",
    "    # A feed-dict is not necessary because nothing is calculated.\n",
    "    w = session.run(weights)\n",
    "\n",
    "    # Get the lowest and highest values for the weights.\n",
    "    # This is used to correct the colour intensity across\n",
    "    # the images so they can be compared with each other.\n",
    "    w_min = np.min(w)\n",
    "    w_max = np.max(w)\n",
    "\n",
    "    # Number of filters used in the conv. layer.\n",
    "    num_filters = w.shape[3]\n",
    "\n",
    "    # Number of grids to plot.\n",
    "    # Rounded-up, square-root of the number of filters.\n",
    "    num_grids = math.ceil(math.sqrt(num_filters))\n",
    "    \n",
    "    # Create figure with a grid of sub-plots.\n",
    "    fig, axes = plt.subplots(num_grids, num_grids)\n",
    "\n",
    "    # Plot all the filter-weights.\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Only plot the valid filter-weights.\n",
    "        if i<num_filters:\n",
    "            # Get the weights for the i'th filter of the input channel.\n",
    "            # See new_conv_layer() for details on the format\n",
    "            # of this 4-dim tensor.\n",
    "            img = w[:, :, input_channel, i]\n",
    "\n",
    "            # Plot image.\n",
    "            ax.imshow(img, vmin=w_min, vmax=w_max,\n",
    "                      interpolation='nearest', cmap='seismic')\n",
    "        \n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "    \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制卷积层输出的帮助函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_conv_layer(layer, image):\n",
    "    # Assume layer is a TensorFlow op that outputs a 4-dim tensor\n",
    "    # which is the output of a convolutional layer,\n",
    "    # e.g. layer_conv1 or layer_conv2.\n",
    "\n",
    "    # Create a feed-dict containing just one image.\n",
    "    # Note that we don't need to feed y_true because it is\n",
    "    # not used in this calculation.\n",
    "    feed_dict = {x: [image]}\n",
    "\n",
    "    # Calculate and retrieve the output values of the layer\n",
    "    # when inputting that image.\n",
    "    values = session.run(layer, feed_dict=feed_dict)\n",
    "\n",
    "    # Number of filters used in the conv. layer.\n",
    "    num_filters = values.shape[3]\n",
    "\n",
    "    # Number of grids to plot.\n",
    "    # Rounded-up, square-root of the number of filters.\n",
    "    num_grids = math.ceil(math.sqrt(num_filters))\n",
    "    \n",
    "    # Create figure with a grid of sub-plots.\n",
    "    fig, axes = plt.subplots(num_grids, num_grids)\n",
    "\n",
    "    # Plot the output images of all the filters.\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Only plot the images for valid filters.\n",
    "        if i<num_filters:\n",
    "            # Get the output image of using the i'th filter.\n",
    "            # See new_conv_layer() for details on the format\n",
    "            # of this 4-dim tensor.\n",
    "            img = values[0, :, :, i]\n",
    "\n",
    "            # Plot image.\n",
    "            ax.imshow(img, interpolation='nearest', cmap='binary')\n",
    "        \n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "    \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 输入图像"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "绘制图像的帮助函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_image(image):\n",
    "    plt.imshow(image.reshape(img_shape),\n",
    "               interpolation='nearest',\n",
    "               cmap='binary')\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如下所示，绘制一张测试集中的图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAADV5JREFUeJzt3X+oXPWZx/HPUzeNYKrmNtMYbextc0UJwabLEFYra1dtuAmB6D+SICUFaQoqrlB0xaKr+E9YbYqgVG80NC6tbTGVBAmubqhooJaMJv6Ku+uvG5twzZ0YoSkIadJn/5iTcqv3fGecc2bO3DzvF1xm5jznzHlyyOeemfmeO19zdwGI5wtVNwCgGoQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQ/9DPnc2bN8+Hh4f7uUsglPHxcR0+fNg6WbdQ+M1sVNIDkk6T9Ki7b0itPzw8rEajUWSXABLq9XrH63b9st/MTpP0kKQVkhZLWmtmi7t9PgD9VeQ9/zJJ77j7e+5+TNKvJK0upy0AvVYk/OdJ+uOUxweyZX/HzNabWcPMGs1ms8DuAJSp55/2u/uYu9fdvV6r1Xq9OwAdKhL+g5IWTnn81WwZgBmgSPh3S7rAzL5uZl+UtEbS9nLaAtBrXQ/1uftxM7tJ0n+pNdS32d3fLK0zAD1VaJzf3XdI2lFSLwD6iMt7gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCKrQLL1mNi7pqKQTko67e72MpgD0XqHwZ/7F3Q+X8DwA+oiX/UBQRcPvkp41s5fNbH0ZDQHoj6Iv+y9z94Nm9hVJz5nZ/7j7C1NXyH4prJek888/v+DuAJSl0Jnf3Q9mt5OSnpK0bJp1xty97u71Wq1WZHcAStR1+M3sDDP70sn7kpZLeqOsxgD0VpGX/fMlPWVmJ5/nl+7+TCldAei5rsPv7u9J+maJvQDoI4b6gKAIPxAU4QeCIvxAUIQfCIrwA0GV8Vd9ITz55JO5tU2bNiW3Pffcc5P1008/PVm/7rrrkvVzzjkntzYyMpLcFnFx5geCIvxAUIQfCIrwA0ERfiAowg8ERfiBoBjn79Ctt96aWxsfH+/pvh9++OFk/cwzz8ytLV68uOx2ZoyFCxfm1m677bbktvX6qf8t9Jz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvk79Oijj+bWXn311eS27cba9+3bl6zv2bMnWX/++edzay+99FJy23ZTqH3wwQfJehGzZs1K1ufNm5esT0xMJOupf3vqGgCJcX4ApzDCDwRF+IGgCD8QFOEHgiL8QFCEHwiq7Ti/mW2WtErSpLsvyZYNSfq1pGFJ45KudfePe9dm9a688squap0YHR0ttP3HH+cf+nbXCLQbz969e3dXPXVi9uzZyfqFF16YrF900UXJ+pEjR3JrixYtSm4bQSdn/p9L+vT/ztsl7XT3CyTtzB4DmEHaht/dX5D06V+hqyVtye5vkXR1yX0B6LFu3/PPd/eT11Z+KGl+Sf0A6JPCH/i5u0vyvLqZrTezhpk1ms1m0d0BKEm34T9kZgskKbudzFvR3cfcve7u9Vqt1uXuAJSt2/Bvl7Quu79O0rZy2gHQL23Db2ZPSPq9pAvN7ICZXS9pg6Tvmtnbkq7KHgOYQdqO87v72pxSscFtlGbu3Lm5tSuuuKLQcxe9hqGIrVu3Juup6xsk6eKLL86trVmzpqueTiVc4QcERfiBoAg/EBThB4Ii/EBQhB8Iiq/uRmUmJ3MvDJUk3XDDDcl668ryfHfddVdubWhoKLltBJz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvlRmYceeihZb3cdwNlnn52st/vq7+g48wNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIzzo6d27dqVW9uwodh0D9u2peeKWbJkSaHnP9Vx5geCIvxAUIQfCIrwA0ERfiAowg8ERfiBoNqO85vZZkmrJE26+5Js2d2SfiCpma12h7vv6FWTmLl27Mj/b3Hs2LHktldddVWyfskll3TVE1o6OfP/XNLoNMt/6u5Lsx+CD8wwbcPv7i9IOtKHXgD0UZH3/DeZ2WtmttnM5pbWEYC+6Db8P5O0SNJSSROSfpK3opmtN7OGmTWazWbeagD6rKvwu/shdz/h7n+VtEnSssS6Y+5ed/d6rVbrtk8AJesq/Ga2YMrDayS9UU47APqlk6G+JyR9R9I8Mzsg6d8lfcfMlkpySeOSftjDHgH0QNvwu/vaaRY/1oNeMAN98sknyfozzzyTW5s9e3Zy23vuuSdZnzVrVrKONK7wA4Ii/EBQhB8IivADQRF+ICjCDwTFV3ejkPvuuy9Z37NnT25txYoVyW0vvfTSrnpCZzjzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQjPMj6emnn07W77333mT9rLPOyq3deeedXfWEcnDmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgGOcP7qOPPkrWb7755mT9+PHjyfrKlStza0yxXS3O/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QVNtxfjNbKOlxSfMluaQxd3/AzIYk/VrSsKRxSde6+8e9axXdOHHiRLI+OjqarL///vvJ+sjISLLe7u/9UZ1OzvzHJf3I3RdL+idJN5rZYkm3S9rp7hdI2pk9BjBDtA2/u0+4+yvZ/aOS3pJ0nqTVkrZkq22RdHWvmgRQvs/1nt/MhiV9S9IfJM1394ms9KFabwsAzBAdh9/M5kjaKukWd//T1Jq7u1qfB0y33Xoza5hZo9lsFmoWQHk6Cr+ZzVIr+L9w999miw+Z2YKsvkDS5HTbuvuYu9fdvV6r1croGUAJ2obfzEzSY5LecveNU0rbJa3L7q+TtK389gD0Sid/0vttSd+T9LqZ7c2W3SFpg6TfmNn1kvZLurY3LaKId999N1lvNBqFnn/jxo3J+qJFiwo9P3qnbfjdfZckyylfWW47APqFK/yAoAg/EBThB4Ii/EBQhB8IivADQfHV3aeA/fv359aWL19e6Lnvv//+ZH3VqlWFnh/V4cwPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Exzn8KeOSRR3JrqWsAOnH55Zcn663vesFMxJkfCIrwA0ERfiAowg8ERfiBoAg/EBThB4JinH8GePHFF5P1Bx98sE+d4FTCmR8IivADQRF+ICjCDwRF+IGgCD8QFOEHgmo7zm9mCyU9Lmm+JJc05u4PmNndkn4gqZmteoe77+hVo5Ht2rUrWT969GjXzz0yMpKsz5kzp+vnxmDr5CKf45J+5O6vmNmXJL1sZs9ltZ+6e3pWBwADqW343X1C0kR2/6iZvSXpvF43BqC3Ptd7fjMblvQtSX/IFt1kZq+Z2WYzm5uzzXoza5hZo9lsTrcKgAp0HH4zmyNpq6Rb3P1Pkn4maZGkpWq9MvjJdNu5+5i71929XqvVSmgZQBk6Cr+ZzVIr+L9w999KkrsfcvcT7v5XSZskLetdmwDK1jb81vp61sckveXuG6csXzBltWskvVF+ewB6pZNP+78t6XuSXjezvdmyOyStNbOlag3/jUv6YU86RCFLly5N1nfu3JmsDw0NldkOBkgnn/bvkjTdl7Mzpg/MYFzhBwRF+IGgCD8QFOEHgiL8QFCEHwjK3L1vO6vX695oNPq2PyCaer2uRqPR0bzpnPmBoAg/EBThB4Ii/EBQhB8IivADQRF+IKi+jvObWVPS/imL5kk63LcGPp9B7W1Q+5LorVtl9vY1d+/o+/L6Gv7P7Nys4e71yhpIGNTeBrUvid66VVVvvOwHgiL8QFBVh3+s4v2nDGpvg9qXRG/dqqS3St/zA6hO1Wd+ABWpJPxmNmpm/2tm75jZ7VX0kMfMxs3sdTPba2aV/v1xNg3apJm9MWXZkJk9Z2ZvZ7fTTpNWUW93m9nB7NjtNbOVFfW20Mx+Z2b7zOxNM/vXbHmlxy7RVyXHre8v+83sNEn/J+m7kg5I2i1prbvv62sjOcxsXFLd3SsfEzazf5b0Z0mPu/uSbNl/SDri7huyX5xz3f3fBqS3uyX9ueqZm7MJZRZMnVla0tWSvq8Kj12ir2tVwXGr4sy/TNI77v6eux+T9CtJqyvoY+C5+wuSjnxq8WpJW7L7W9T6z9N3Ob0NBHefcPdXsvtHJZ2cWbrSY5foqxJVhP88SX+c8viABmvKb5f0rJm9bGbrq25mGvOzadMl6UNJ86tsZhptZ27up0/NLD0wx66bGa/Lxgd+n3WZu/+jpBWSbsxe3g4kb71nG6Thmo5mbu6XaWaW/psqj123M16XrYrwH5S0cMrjr2bLBoK7H8xuJyU9pcGbffjQyUlSs9vJivv5m0GauXm6maU1AMdukGa8riL8uyVdYGZfN7MvSlojaXsFfXyGmZ2RfRAjMztD0nIN3uzD2yWty+6vk7Stwl7+zqDM3Jw3s7QqPnYDN+O1u/f9R9JKtT7xf1fSj6voIaevb0h6Nft5s+reJD2h1svAv6j12cj1kr4saaektyX9t6ShAertPyW9Luk1tYK2oKLeLlPrJf1rkvZmPyurPnaJvio5blzhBwTFB35AUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4L6f6yMEem39pFEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8947e53518>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "image1 = data.x_test[0]\n",
    "plot_image(image1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "绘制测试集里的另一张图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAADihJREFUeJzt3X+I3PWdx/HXW00RbJBoxmWx0a1FDpbgpTIsBxHN0WuxWo1BDI0QIkq2YgIWI55EyCVGZDWXFsGzuD2XZo9qKzZiFGPrxSNSPWImJpfEev442dqENdnVhFr8o2rf98d+U7Zm5zPjzHfmO5P38wHLznzf8/1+3/kmr3xnvp+Z+Zi7C0A8pxXdAIBiEH4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Gd0c6dzZ071/v6+tq5SyCUsbExTU5OWj2PbSr8ZnalpIcknS7p3919KPX4vr4+VSqVZnYJIKFcLtf92Iaf9pvZ6ZL+TdJ3JfVLWmZm/Y1uD0B7NfOaf0DSu+7+nrv/WdIvJC3Opy0ArdZM+M+X9Idp9w9ly/6GmQ2aWcXMKhMTE03sDkCeWn61392H3b3s7uVSqdTq3QGoUzPhPyxp3rT7X8uWAegCzYR/t6SLzezrZvYVSd+XtC2ftgC0WsNDfe7+mZmtlvRrTQ31jbj7G7l1BqClmhrnd/fnJT2fUy8A2oi39wJBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxBUU7P0mtmYpI8lfS7pM3cv59EU8nPs2LFkfe/evcn6Cy+8kKxv2rQpWTezqrUbbrghue6FF16YrK9ZsyZZ7+npSdajayr8mX9098kctgOgjXjaDwTVbPhd0m/MbI+ZDebREID2aPZp/2XuftjMzpP0opn9r7u/PP0B2X8Kg5J0wQUXNLk7AHlp6szv7oez30clPS1pYIbHDLt72d3LpVKpmd0ByFHD4Tezs8xs9onbkr4j6WBejQForWae9vdIejobyjlD0uPunh4XAtAxGg6/u78n6e9z7AVVfPrpp8n65s2bq9Yefvjh5Lrj4+MN9XRCahy/Vv2pp55qat+Tk+kR5pGRkaa2f6pjqA8IivADQRF+ICjCDwRF+IGgCD8QVB6f6kOLPfroo8n6Pffc06ZOTrZo0aJkfefOnS3b95YtW5J1hvrSOPMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCM83eAgwfT34GycePGNnVysgceeCBZv/3225P1devWVa09+OCDDfWEfHDmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgGOdvg1rj+GvXrk3WJyYmkvXU12PXmuZ627ZtyXp/f3+yftpp6fPHvffeW7W2ZMmS5LrXXnttsl7ruFxyySVVa/v370+uGwFnfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IquY4v5mNSPqepKPuPj9bdo6kX0rqkzQmaam7H2tdm91t7969yfpzzz2XrLt7sj5r1qyqtVWrViXXnT9/frLerFRvAwMDyXVvuummZD01NbkkHThwoGptcHAwue7w8HCyfiqo58z/M0lXfmHZ3ZJ2uPvFknZk9wF0kZrhd/eXJX30hcWLJZ2YLmWLpOty7gtAizX6mr/H3cez2x9I6smpHwBt0vQFP596QVr1RamZDZpZxcwqtd6LDaB9Gg3/ETPrlaTs99FqD3T3YXcvu3u5VCo1uDsAeWs0/Nskrchur5D0TD7tAGiXmuE3syck/bekvzOzQ2Z2i6QhSd82s3ck/VN2H0AXqTnO7+7LqpS+lXMvp6zt27cn66nP49dj0aJFVWtr1qxpattFGhpKn1NqHdfUOP/u3bsb6ulUwjv8gKAIPxAU4QeCIvxAUIQfCIrwA0Hx1d05+PDDD5P1Xbt2tXT/y5cvb+n2O1WtP/ddd93Vpk66E2d+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiKcf4c7NmzJ1kfGxtravuXX355sn711Vc3tf2Ijh8/nqyPj48n6729vXm2UwjO/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOP8OahUKi3d/oYNG5L1OXPmtHT/p6L3338/WT948GCyzjg/gK5F+IGgCD8QFOEHgiL8QFCEHwiK8ANB1RznN7MRSd+TdNTd52fL1ktaKWkie9had3++VU12uk8++SRZd/emtn/FFVc0tX5UzR73U109Z/6fSbpyhuU/dvcF2U/Y4APdqmb43f1lSR+1oRcAbdTMa/7VZrbfzEbMjPeXAl2m0fD/RNI3JC2QNC5pc7UHmtmgmVXMrDIxMVHtYQDarKHwu/sRd//c3f8i6aeSBhKPHXb3sruXS6VSo30CyFlD4Tez6R9pWiIp/REoAB2nnqG+JyQtkjTXzA5J+hdJi8xsgSSXNCbpBy3sEUAL1Ay/uy+bYfFjLeila9X6PL+ZtakTTJc67vyd8A4/ICzCDwRF+IGgCD8QFOEHgiL8QFB8dTdCmj17drJ+7rnntqmT4nDmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgGOdH1xodHW143fXr1yfrl156acPb7hac+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMb5czA0NJSs79u3L1mvNY3ZzTffnKyPjIwk66eqWsftvPPOq1q79dZb826n63DmB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgao7zm9k8SaOSeiS5pGF3f8jMzpH0S0l9ksYkLXX3Y61rtXMtWLAgWd+0aVOyvmLFimT9ySefTNZXr15dtdbNn0tfuXJlsn7kyJFkfenSpVVrZ555ZkM9nUrqOfN/JmmNu/dL+gdJq8ysX9Ldkna4+8WSdmT3AXSJmuF393F3fz27/bGkNyWdL2mxpC3Zw7ZIuq5VTQLI35d6zW9mfZK+KWmXpB53H89KH2jqZQGALlF3+M3sq5J+JemH7v7H6TV3d01dD5hpvUEzq5hZpdZ7sQG0T13hN7NZmgr+z919a7b4iJn1ZvVeSUdnWtfdh9297O7lUqmUR88AclAz/GZmkh6T9Ka7/2haaZukE5epV0h6Jv/2ALRKPR/pXShpuaQDZnbis6lrJQ1JetLMbpH0e0nVx1WCW7hwYbJ+4403JuuPP/54sr5z586qtU4e6nvppZeS9a1btybrPT3py0zr1q370j1FUjP87v5bSVal/K182wHQLrzDDwiK8ANBEX4gKMIPBEX4gaAIPxAUX93dBhdddFGyft999yXrr7zySrK+YcOGqrVab6m+//77k/Va3n777WT9tddeq1q74447kuseP348Wb/zzjuT9f7+/mQ9Os78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4/wdoK+vL1l/9dVXk/XUdNOPPPJIct3t27c3vG2p9mfmJycnk/WUa665JlkfHBxseNvgzA+ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHO3wV6e3uT9dHR0aq1t956K7nuxo0bk/XbbrstWa/1mfqU66+/PlmvNefAGWfwz7cZnPmBoAg/EBThB4Ii/EBQhB8IivADQRF+IKiaA6VmNk/SqKQeSS5p2N0fMrP1klZKOvHF8Gvd/flWNYrqzj777Kq1gYGB5LrPPvts3u2gS9TzLonPJK1x99fNbLakPWb2Ylb7sbv/a+vaA9AqNcPv7uOSxrPbH5vZm5LOb3VjAFrrS73mN7M+Sd+UtCtbtNrM9pvZiJnNqbLOoJlVzKxSa+ooAO1Td/jN7KuSfiXph+7+R0k/kfQNSQs09cxg80zrufuwu5fdvVwqlXJoGUAe6gq/mc3SVPB/7u5bJcndj7j75+7+F0k/lZS+sgSgo9QMv5mZpMckvenuP5q2fPpHzZZIOph/ewBapZ6r/QslLZd0wMz2ZcvWSlpmZgs0Nfw3JukHLekQQEvUc7X/t5JshhJj+kAX4x1+QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoMzd27czswlJv5+2aK6kybY18OV0am+d2pdEb43Ks7cL3b2u78tra/hP2rlZxd3LhTWQ0Km9dWpfEr01qqjeeNoPBEX4gaCKDv9wwftP6dTeOrUvid4aVUhvhb7mB1Ccos/8AApSSPjN7Eoze8vM3jWzu4vooRozGzOzA2a2z8wqBfcyYmZHzezgtGXnmNmLZvZO9nvGadIK6m29mR3Ojt0+M7uqoN7mmdl/mdnvzOwNM7s9W17osUv0Vchxa/vTfjM7XdLbkr4t6ZCk3ZKWufvv2tpIFWY2Jqns7oWPCZvZ5ZL+JGnU3ednyx6U9JG7D2X/cc5x93/ukN7WS/pT0TM3ZxPK9E6fWVrSdZJuUoHHLtHXUhVw3Io48w9Ietfd33P3P0v6haTFBfTR8dz9ZUkffWHxYklbsttbNPWPp+2q9NYR3H3c3V/Pbn8s6cTM0oUeu0RfhSgi/OdL+sO0+4fUWVN+u6TfmNkeMxssupkZ9GTTpkvSB5J6imxmBjVnbm6nL8ws3THHrpEZr/PGBb+TXebul0r6rqRV2dPbjuRTr9k6abimrpmb22WGmaX/qshj1+iM13krIvyHJc2bdv9r2bKO4O6Hs99HJT2tzpt9+MiJSVKz30cL7uevOmnm5plmllYHHLtOmvG6iPDvlnSxmX3dzL4i6fuSthXQx0nM7KzsQozM7CxJ31HnzT68TdKK7PYKSc8U2Mvf6JSZm6vNLK2Cj13HzXjt7m3/kXSVpq74/5+ke4rooUpfF0n6n+znjaJ7k/SEpp4GfqqpayO3SDpX0g5J70j6T0nndFBv/yHpgKT9mgpab0G9Xaapp/T7Je3Lfq4q+tgl+irkuPEOPyAoLvgBQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwjq/wHi31d/HSnFFwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8947e1fba8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "image2 = data.x_test[13]\n",
    "plot_image(image2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 卷积层 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在绘制第一个卷积层的滤波权重。\n",
    "\n",
    "其中正值权重是红色的，负值为蓝色。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUoAAADuCAYAAABf005JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAD9ZJREFUeJzt3W1sVOeZxvH72B6/4beYwRjXmClxgoUIcZpJoGmTkDRbYLupkGhI2W1JU7lSg7qLaLObSKVtdskqqxJ10bZSNi8C0dImaYKIsqhF7GZxAopoMiQUEKBg0gEbB+yJIbZrjD2esx/ycRVd9+wyORPl//t86bmH4/HFsfSc8wRhGBoA4KOVRP0BAKDYUZQAIFCUACBQlAAgUJQAIFCUACBQlAAgUJQAIFCUACCU5RO+6qp42NKSkLmqqdH/6+f53yordcbxdFG6t9cyQ0PBFfhEBRGfNi1MNDTIXG7mLL3Y2wddMz0XI6iqkpn0xIRlstnivbYVFWGiuloHJyZ0ZmzMN7SuTkbSsWtcS73//sFMGIYzfIM/fvEgCNscuSv5BRlyZGKOzHkz+yAM5UfLqyhbWhL27LMpmVs4+rpeLJv1De3ouCJrJZcv982LSKKhwVJr18rc2Pofykxumu8r6fnhV7a3y0yyp8c1LyqJ6mpL3XGHDvb1yUjuzTddM0u+8AWZ+Xbz71xrbd0anHYFI9JmZq85cnmVjfCcI9PsyPydcx5/egOAQFECgEBRAoBAUQKAQFECgEBRAoBAUQKAkNfWpqpY1hY2D+hg+zIZOTMy4pqpt2Cb1T3/vA6NXsFN8IVQVmY2Q+8pri7Tm6LfcY686Mi0HzkiM84dsZEZv3jRju3cKXOefXeNr7zimvlO650ys6U951pr61ZXLDIlHR1Ws22bzB1btEhm5t9/v2vmtxYskJmX278vM7nvJ13zuKMEAIGiBACBogQAgaIEAIGiBACBogQAgaIEAIGiBAAhv3dpTky4Xm5qn/mMjBw9ccI1styRueuhh3Sov981LzIlJa63uR+tqJCZGufIm2+9VYd6e2WkrMiv7ZSZDTtynruGRs/338zGvqRfntztWqn4HT8zzW7+3s0y1/AX+iSC/9yqH3AwM7vpputkxvPO7wsXXOO4owQAhaIEAIGiBACBogQAgaIEAIGiBACBogQAgaIEAIGiBAAhvydzRkbMurt17sknZSR3++2ukfrgAzP75jd15plnXPMik8m4PmObY6nfOUcmNm/WoUce0Rnv4w0RmRaL2eJmfdDDmOMpJNu1yzWzxZHpvOYa11p28qQvF5F43KyrS+ceflhn7rlHP3FjZvbCC8dk5o27tstMsvysax53lAAgUJQAIFCUACBQlAAgUJQAIFCUACBQlAAgUJQAIOS14Xyqr8+Gf/ADmdMHGpjd5pw56gklEjpT7jlUIjoTo6N2Zt8+nXOs9fXVq10z//rxz8nMbzZs0Avdd59rXmRaW802bpSx6qkpmfny9jWukXv+8Z9k5tvpH7vWspP6WIkoVVWZLVigcwcO6My13U/5hnY7vpdxxw73Ml8FckcJAAJFCQACRQkAAkUJAAJFCQACRQkAAkUJAAJFCQACRQkAQhCGoT8cBINmdrpwH6eg5oRhOCPqD/FRuLaF8wm/tmZc30JyXdu8ihIAPo340xsABIoSAASKEgAEihIABIoSAASKEgAEihIABIoSAIS8zswJgumhWZvMNTaWysysWb6ZlcfflpnJXE5mes1sKAyL9vCR+iAImx05faqLmfd0IP1T8v1PWuzXNt7YGCZaW3VwbExn+vtdMwcuXZKZptpa11oHR0YyxfxkTjwIwkSJ/qZMLrhBZmJB1jf01CmdcfwM0rmcZXI5+d3Nqyg/LMlXZWrp0jqZ8ZxZZWY2f7Feq39kRGaW+8ZFptnMnnDkhh0ZRyWYmVmjI1PtyCx1zotKorXVUrt26WAqpTOOQ8rMzH5x6JDMfC+ZdK0V7N1b1I8HJkpKLDVtmsyd36Ov78zYkG/oihU6c+SIjCQd3WHGn94AIFGUACBQlAAgUJQAIFCUACBQlAAgUJQAIOS5j7LEzKpk6jcrfquXOjDqmtjt2Of0jmOdi65p0aktKbE7HXvRfuvc9+XhuW5LHJk8v0Qfv2PHzK6/XucSCRnZ7Ngfaea7Jq/v3etaq+jNm2e2fbuMzdz2U5lJPfSQa2Ry9WqZmdi3T2a85ztwRwkAAkUJAAJFCQACRQkAAkUJAAJFCQACRQkAAkUJAEJee4VvbDlnqbV606g9d1BGtuzc6Zo535G5zZH5N9e0CFVUmF19tYyt2rZNZo6VLXSNXJY9rEPd3TISPP64a15kOjst94Z+aezhUv2S9i87Rx5wZDLOtYre6KjZ/v0ylnNsJk92drpG9jz7rMy0x2IyE2R9b1TnjhIABIoSAASKEgAEihIABIoSAASKEgAEihIABIoSAASKEgCEvJ7Mudzfbz0bNshc2rGW79kRs2FHZvGtt8pM5dtvOydGJB436+qSsfMz9ZV75G99I196Sa81OdnsWGmLb2BEpqY+fHhE6ZwxQ4ccPyMzs/meJ8/uuce1lm3c6MtFZKK3186sWydzbbW1erH1610z6+67T2bOTU7KjE58iDtKABAoSgAQKEoAEChKABAoSgAQKEoAEChKABAoSgAQ8tpwbuZr1rsSCZkZSKdd85oefVSHHPPsRz9yzYtMTY3Z5z8vY55/6vj43c6h/yoT3/1uu8zs2JH31+hjVVJiVlnpCKb0cRF/eK/NNXPRuXM65Hh4w8yKfsN5+dy51vbYYzL3VvsqmTl61Ddzzd0v6pCjY2I9Pa553FECgEBRAoBAUQKAQFECgEBRAoBAUQKAQFECgEBRAoBAUQKAEIRh6A8HwaCZnS7cxymoOWEYOt71Hw2ubeF8wq+tGde3kFzXNq+iBIBPI/70BgCBogQAgaIEAIGiBACBogQAgaIEAIGiBAAhr3f4B8H00Gy2IzniyFx2zYzFmmVmYeuQzKQHBy0zMhK4hkagvDweVlYmZG58/MrNXNiS0aHz52UkPTlpmWy2aK9tvKoqTNTXy1zo+Ld6dx3nHJmy2Z7fJbODvb2ZYt5wXlcXD5uaEjL37rt6rTA845pZWqqP5OgM/igz6akpy+Ry8rub52Ens83sFUduryPzJ9fEePzvZSa18dcykyzyM3MqKxOWTOozW5xHfLikfrxFhzZtkpGk8/yjqCTq6y21Zo3MTTj+rRPOmZ7/z+IPPuhaK1i3rqifemlqStjPfqa/u1/7ml5rcvIB18y6uidkJlUxS2aSGcfNgvGnNwBIFCUACBQlAAgUJQAIFCUACBQlAAgUJQAIFCUACHltOK+rK7Nbbpkuc7t398lMR4feSG5mdvy6VTKT+cYLMpN1TYvO3LlmL76oc9On62u7enWrb+j+/Trz/vs6ky3yq1tfb7ZsmYyVOzacly9e7Bo5euCAzAyvW+daq9i9957Zo4/q3OTkdsdqa10zhzL62aczpedkxvsAAXeUACBQlAAgUJQAIFCUACBQlAAgUJQAIFCUACBQlAAg5HkUhFks5kl+RyaOf/YvXTP7X/i9zLQ06+MiypxvMo5K2cSYNabfkrnwzx16seY618z/HtFHdgw71rnomhadC9lae37wTpm7d8cOvdju3a6Zrzk2nK9avty1lv1e/w5EqbzcrNXxjMObb14nM/ffrzNmZrlSffKIfjSDDecAcMVQlAAgUJQAIFCUACBQlAAgUJQAIFCUACBQlAAgUJQAIOT1ZE77+FF7+cS1OrjjX2Tk5ZW+pw0aHZmWykodKinu/xMmjh+39I03ypx+3sPMd1iBWcqR0QcomFU450XlqvH37N6ef9bBO+6QkcNPP+2a6fneFv0RGk6xmNnMmTp3993Xy8yWr+/xDU3pJ3j2HDkiM54nz8y4owQAiaIEAIGiBACBogQAgaIEAIGiBACBogQAgaIEACGvDeejly/b/pMnZS65cqXMeA9m0C/wN3s3nZaZy855URk1s9cdOccb993+ypFxHDxhVf/fD1Jok5NmZ8/q3MMPy0jNqdA1Mne1PqrABgddaxW7tpI+e6L2H2Qu8x+b9GLnbnLNHHNsJvccNvO8axp3lAAgUZQAIFCUACBQlAAgUJQAIFCUACBQlAAgUJQAIFCUACAEYeh70sDMLAiCQTM7XbiPU1BzwjCcEfWH+Chc28L5hF9bM65vIbmubV5FCQCfRvzpDQACRQkAAkUJAAJFCQACRQkAAkUJAAJFCQBCXkdBTJ8eD9vaEjI3MKDXmpjwzZxTOyQz49WNMnP2bNouXMg43s8fjYYgCJsduWHPWs6Znv8lKxyZ02aWCcOivbbxqqowUVeng7Nmychbh32/MtdeqzNTU66lrKfnYKaYN5zHGxrCREuLzI2XVMtMZXjJN/TiRZ3J6ANn0tmsZaam5Hc3r6Jsa0vYq6+mZG7zZr1WX59v5lO3/1pmjt3wNzKzalXSNzAizWb2jCP3X46M5ywcMzNHddhcR2aRc15UEnV1lrr3Xh3csEFGylubXDO3bNEZz++6mdlXvhIU9VMviZYWS23fLnPHKj8nM/Ozh31DX3pJZ7ZulZFkf79rHH96A4BAUQKAQFECgEBRAoBAUQKAQFECgEBRAoCQ1z7K0vE/W92JN2Sus/Nmmenqcg7dqTebjYzoZbybe6NSM2+efdGx+e6Ljv1j2U2bXDPLHnhAhxx7C4Ply13zopIdGLCBn/9c5pqWLJGZieecQ8cd2/57jjoXK3KlpWYN+t+bPqGX6k4vdI1c+40aHVqxQmdWr3bN444SAASKEgAEihIABIoSAASKEgAEihIABIoSAASKEgCEvDacD584YXsW6de0fjWRkJlfptOumXc5Mou+80eZmZYp6nef2p8Ga2zNv98ic+fO6Ux37KeumTWOzdNdjn29fedjrnlRKauttabFi2UutXKlXss5s92Rcb7kv/hdumR26JCMLVl5tcw432Vstm2bznR06Ewu5xrHHSUACBQlAAgUJQAIFCUACBQlAAgUJQAIFCUACBQlAAgUJQAIeT2ZU2Fmcx25bsdTN63Oma85Ml99+mmZ8e2/j87Q0LD96ld7HMmdjozv6sbjP5SZMsc3JAhc46JTX2+2bJmMJfft02vFfE8h7XacT+L9HSh22VOnbMjxVJPnSaQ+58yWBx+UmfTgoMx4n47ijhIABIoSAASKEgAEihIABIoSAASKEgAEihIABIoSAIS8NpyXm2+TrOd17p5X5ZuZNcyerUNdXTJS8uSTzonRuLGj1FLbGnTw0A0689hjvqFLHMdj7NSbsPdcTPvmReWDD8x27dK59et1xrHR2cxsWSolMwNLl7rWKnZl8+ZZ41NPydw7zbfJzNg839MLux2byZc5joso/8lPXPO4owQAgaIEAIGiBACBogQAgaIEAIGiBACBogQAgaIEAIGiBAAhCMPQHw6CQTNzPM5RlOaEYTgj6g/xUbi2hfMJv7ZmXN9Ccl3bvIoSAD6N+NMbAASKEgAEihIABIoSAASKEgAEihIABIoSAASKEgAEihIAhP8BYXKFph14rU4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f89e12f7f28>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_weights(weights=weights_conv1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将这些卷积滤波添加到第一张输入图像，得到以下输出，它们也作为第二个卷积层的输入。注意这些图像被降采样到14 x 14像素，即原始输入图分辨率的一半。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUoAAADuCAYAAABf005JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAG85JREFUeJzt3XmMldX5wPFn2GZYB5jLvtwhCGVX1kJBBYJYGrGKQqKttW1KbNLW1jQ2dvEPmxrUWE01tZqYWLsqjhilYoGgWBBklc0FCjIDwyIzw75v8/uj+T0853jvPfcOd5l75/v567l9XobjOy9P33PuWYrq6+sFABBfs1w3AAAaOwolAARQKAEggEIJAAEUSgAIoFACQACFEgACKJQAEEChBICAFqlcHIlE6svLyzPUlMyqrKyU2traoly3Ix7ubeZEIpH6aDSa62Y02MaNG2vr6+u75Lod8ZSVldX37t07181okOrqaqmrqws+uykVyvLyclm/fn3DW5VDY8aMyXUTEuLeZk40GpXVq1fnuhkNVlxcXJXrNiTSu3dvWbJkSa6b0SDTp09P6jq63gAQQKEEgAAKJQAEUCgBICClL3MSOXXqlMbV1dVO7vTp0xp37NjRyfXo0UPjkpKSdDWnYF28eNH5bO91y5YtnVxZWZnG3Nurc/ny5bi5Zs1434jn6NGjGp88edLJde7cWeM2bdpkrU0NwW8YAAIolAAQkLau99q1azV+5ZVXnNyJEyc0Li0tdXLNmzfXONnuzZAhQ5ycnQt1zTXXJNni/FFTU6NxRUWFkzt06JDG3bp1c3KtWrXS2HbL/e57ixYtYl4n4t7rESNGpNLsvLBw4UKNN2zY4OR27typsf9cjR49WmM7xOE/w/a59YedOnTooHG+TtgOOXPmjMb+82n/m8+ePevkDhw4oPGFCxfi/vyioitzxf2ff/78eY0vXbqUZItj440SAAIolAAQQKEEgIC0jVHasQJ/TMEeiXvkyBEnZ8fE7DQi/xhdu5Z03LhxTm7q1KkNaHH+2LJli8aVlZVOzt4nO64j4o4N29+JP0Zpf3d2OoeISK9evVJvcB6xY4r2fomIvP766xr7U4Dat2+vsR139++tfab9NfETJ07U+KGHHkql2XnDPlv++LqdEnTnnXc6uX79+qW1HV988cVV/XneKAEggEIJAAFp63pPmjRJ48GDBzs5273ZtWuXk7Oz9e3qns2bNzvX2W2yiouLnZyd4V+IunbtqvGUKVOc3L59+zQ+duyYk7PDGrbr7Q+NvPjiixr7XR77dxei2bNna+w/tzNmzNDYH5Kwz7HtXm7bts25zj7Hu3fvdnLDhg1rQIvzy9KlSzX2u947duzQeN68eU7ODmHYGuH/Hn7zm99ofNdddzk5W0/69u2bSrO/hDdKAAigUAJAAIUSAALSNkZpl8H5S4ksu1uQ77PPPtP4ySefdHL2Z956661OrtDH0YYPHx4z9tklWyLusi2b++Uvf+lcZ8eD7FizSOEurft/9r4MHDjQydnP/rQfOy3LTj1Zvny5c11V1ZVTHGpra53ctGnTUm9wnrnnnntixiLu8+mPPdpluwcPHtTYP/to/PjxGq9YscLJ2bHjq8UbJQAEUCgBICBtXe+Gsq/fixcv1tjfhcUe5Tpz5syMtysf2d2CfFu3btV40aJFTs5u6nvfffc5udatW6epdfnNDi35Pv/8c4393Zfq6uo0tjsOiYhMmDAhTa3LT3bnMLsDk0j8nZX69+/vXLdy5UqN/VV/6Zx+xRslAARQKAEgIOdd702bNmlsZ+77mxDMmTNH40L/JjZd7Aqc559/XmP/2/Gf/OQnGvfp08fJcR5MbPbbbPuN7UsvveRcZ7vs999/v5Nr7OfEZJO/CY7tRidaeWc3Bo5EIk6ubdu2aWodb5QAEEShBIAACiUABGR9jNKfgf/73/9eYzuV4tprr3Wuszu5IDb/gCY75vvaa69p3LNnT+c6u2KC879j81d5rFmzRuONGzdqbFeUiIjceOONGo8cOTJDrct//qol+yzbzaM//PBD5zo7HSuT313wRgkAARRKAAjIStfbbigwf/58J/fRRx9pbLt9c+fOda7r1KlThlqX3+y0ir179zq5X//61xrb38HDDz/sXGenVaRzI4FC8sEHHzif7aoSe7aOPUtHROQHP/hBzD+DxOdujx07VmO7GbKdliXibsjrb+idTrxRAkAAhRIAAiiUABCQlTFKexDT008/7eTsZr0PPvigxnZaBeKzBy/593bPnj0a33zzzRp/4xvfcK5jSlBs9uA2f9rPCy+8oLFdbudvijx06NAMtS7/2XPU7e5gvv3792vsP6tXe2hYsnijBIAACiUABGSl623PvPBn4NtXaXuGCFMpkmNXhaxatcrJ2Z1rZs2apXG7du0y37AC8PHHH2v8/vvvOzn72Z7n5A9rsPtSfKWlpTFjEZFz585pbDf19c/jSrShcjrxWwSAAAolAARQKAEgICsd/C5dumj885//3MnZ3Yub+mFLDWGXgY0aNcrJ2fPP7Q7x2RrXyXdDhgzR2N8V3h7WNnHiRI07duyY+YYVCHsw2KOPPurk7JJbuzO8PwacLbxRAkAAhRIAAor8Q30SXlxUVCMiVcELG6dofX19l/BlucG9zZw8v7ci3N9MSureplQoAaApousNAAEUSgAIoFACQACFEgACKJQAEEChBIAACiUABFAoASAgpd0RIpFIfTQazVRbMqqqqkpqa2sb7aHVZWVleX1v6+rqGu29jUQi9YnOZGnsNmzYUNuYV+bk8/2trKxMqi6kVCij0aisXr264a3Koca+M1E0Gv3SLtr5orEfBFdeXi7r16/PdTMarKioqFEvD8zn+ztmzJikrqPrDQABFEoACKBQAkAAhRIAAtJ2JoDduv3w4cNO7uTJkxq3bt3aydmjKFu1apWu5gBJsdsM+lsOctTs1bO1oKjI/XK5U6dO2W5Og/EkAEAAhRIAAtLW9bbzqCoqKpxcTU2Nxv4pdSUlJRrbro7fDbInB/bu3dvJTZo0SeNhw4al0uy80L59+5z93cePH9fY7zoVgo8++kjjzZs3O7mWLVtqPGDAACdnn8FkT7Xs0KGD89kfhipEixYt0ti/vzt27NC4e/fuTm7gwIEat2vXTuNEwyGTJ092Ptv73a1bt+QaHAdvlAAQQKEEgAAKJQAEpG2Mcv/+/Rr744t2CpCvqurKMtazZ89qfOnSpbh/xk43EhEZPHhw0u3MR6+88orGf/nLX5zc9u3bNfbH0caOHatxJBLR2B9r/P73v6+xHU8WETl27JjG/fv3T6XZeeHjjz/WeMGCBU7uyJEjGp8/fz7uz7DPuz+GduLECY3nzJnj5H70ox9p3KVLo93z4qrYe7Nr1y4nd+DAAY3tvRYR2bJlS8yfV1lZ6Xw+ePCgxo899piTGzdunMaMUQJAhlEoASAgbV3vfv36aXzNNdc4ua5du2p89OhRJ2dX9Jw+fVpjO3VARGTx4sUaDxo0yMlNnTpV40Rd9nxlp0f40yhs1/s///mPk7P3zHa3+/Tp41xnu5X+dll26lUhdr3tc3v77bc7uT179sSMRdwu37lz5zS2Q1AiItXV1Rr7XU/77BeqoUOHauzvWWn/jdtuuIi7oufChQtxr+vcubPG/v0dP3586g2OgzdKAAigUAJAAIUSAALSNkY5cuTIuDk7RcCOKYiING/eXGP71f8TTzzhXGenDvnTLC5fvpxSW/ONPWrBP3bB3lt7j0TcqT32Ors8TERk3rx5Grdp08bJ+ctFC40dg7WxiHvPzpw54+TsWLi9zw888IBznV1GZ6drieR2aWq2jBo1Km7O3m87Dini3t/33ntP49raWuc6Oz7cq1cvJ+d/V3I1eKMEgAAKJQAEpK3rnYidmuJvzmu7zfPnz9fYThUScbv2d9xxh5NLtGqi0Nl76+9GYz/bbt6mTZuc6+xnOxVJxF3d0NTYe+sPSVj//Oc/NbbTgURESktLNZ4xY4aTawq7ByWSqC7Yf///+te/NPan/9mhDX/3ILsa7WrxRgkAARRKAAjIStc7kZ07d2psuzB2Q18RkYceekjjptzVTkW8M4jeeecd53Pbtm01vuuuu5xcU/hmtiH27t2rsV0R5d+vu+++W+OePXs6OTvjo6nzN9JZunSpxnY2h39/hwwZorG/OU5xcXHa2scbJQAEUCgBIIBCCQABWR+jtIdViYj87ne/09juyDJ9+nTnugkTJmjMGGVs/oa8dozGjkvacWERd3en6667LkOty2/+M/e3v/1NY7uzkL86ZNq0aRrbsWC4tm3b5ny2z6tdseevFLM7h2XynHDeKAEggEIJAAFZ6Xrb2fR29Y2Ie+6v7Zo8++yzznV0t8P8VTV2dcMLL7ygsT8t5d5779U4nVMqCsmSJUucz/Z8HXuGkz0HRyTxeVFNnT1PqKKiwsnZFU526tD111/vXGc3obbnsKcbb5QAEEChBIAACiUABGRljHL37t0aP/74407OjlP84he/0Ng/h5cxytgSjcts3bpVY3ve9A033OBcN3z48PQ3rADYZYovvfSSk7MbyM6ePVtjOx1IhB2CElmxYoXGb731lpOzmyHb3cK+/vWvO9dlckqQxRslAARQKAEgICtd7w8++EBj/3xkuyrkW9/6lsZ0tZNjd1b54x//6OTefvttje2ZyjNnznSua9Ei55tINUrLly/X2E4HEnHPV58yZYrGHTt2zHi7CoVdjWNX5Ym4m+7ae+3vHpStHZh4owSAAAolAARQKAEgICuDU3YMzD/32J7R3a9fP40Zo0zOsmXLYsYiIjU1NRrbKUD+UkfEZs/y9qdQ2WfVnpPu7+CE+Hr06KGxP61q2LBhGt92220a5+rZ5Y0SAAIolAAQUOQf6pPw4qKiGhGpylxzMipaX1/fJdeNiId7mzl5fm9FuL+ZlNS9TalQAkBTRNcbAAIolAAQQKEEgAAKJQAEUCgBIIBCCQABFEoACEhprXdZWVl93759M9WWjNqzZ4/U1dU12oW4ZWVl9dFoNNfNaJCqqqpGfW8jkUi93Y8z32zYsKG2MU84Lysrq7fHxuaTvXv3JvXsplQo+/btK++++27DW5VDU6dOzXUTEopGo/L+++/nuhkNcuONN+a6CQmVl5fL+vXrc92MBisqKmrUq1769OkjS5cuzXUzGuSmm25K6jq63gAQQKEEgAAKJQAEUCgBIIBCCQABaTsKwh43uWvXLid34cIFjbt16+bk7LSN1q1bp6s5BaWurk7j6upqJ3f69GmN/aNS7VSutm3bZqh1+c3e23Pnzjm5y5cva+wfQWA/c9xvfFVVV76w94+kbdbsynuaPRZCxD3GulWrVhlqXfJ4owSAAAolAASkrc+wbt06jd955x0nZ1+5i4uLndyePXs0vnjxYtyf37x5c40nT57s5O68806NR40alVyD88iiRYs0fvPNN53cvn37NLb3SETk5MmTGttuju1SirhdxxtuuMHJffOb39S4sU8sbwj73G7cuNHJ2WGNDh06OLl4py36/7s9QWDChAlO7itf+YrGXbo02oU3V+X111/X+MSJE07O/jf7K3s+++wzjc+ePauxf3/tsztz5kwnZ4f1rnboiTdKAAigUAJAAIUSAALSNkZpv8KfOHGik7v11ls13rRpk5Ozn2trazW242v+5169ejm50tLSBrQ4f9jxlenTpzs5O6XFjuuIiHz66acxr/Pv7eHDh+Pm/HGlQmOnrtkxSRGR48ePa3z06FEnF4lEYv68Q4cOOZ937typsT/FaPjw4ak1Ng916tRJ47Fjxzq5WbNmaeyPm3/yyScanzlzRuMjR44419nP/ncX9vd56tSpFFr9ZbxRAkAAhRIAAtLW9bbTSnr27Bn3On/jX9sttw4cOOB8XrFihcZz5syJ+/P9V/NCYKeV2BULIiJt2rTR2E5FEfnyVKz/9/TTTzuft2zZovG3v/1tJ/e1r31N40TTt/LVgAEDND527JiTs1PXvvjiCydnu9i227hq1Srnuq1bt2rs/7uwXc9CHT6aMWOGxomGGuz0NRGRYcOGxbzutddecz77w3BWSUmJxnS9ASDDKJQAEEChBICAtI1R2p1/GjpOuGzZMo0ffvhhJzdixAiN/TFKf2yp0Pg7q1h2edelS5ec3Pnz5zX+97//rfGzzz7rXGfHxx599FEn5//MQjNo0KCYsc/fWch+ttOwPv/8c+c6O7XLjveKxB9DLiTdu3fXuKamxsnZMXV/epBdjvvUU09pvGDBAue6xx9/PO7fbae9XS3eKAEggEIJAAE533HUrgSxO434u7W8+uqrcX+G/9relNjdVPwNZO2KhoqKCo3Lysqc677zne9o7E9hKfSVOclK1E1+++23NU40zcXf2ap9+/Zpal1+ss+uv/OV7Ta/8cYbGvube992221xf74/Xe5q8EYJAAEUSgAIyHrX238dfvfddzXetm2bxqNHj477M/wNCvA//r21G/6uXbtWY3+T1Llz52pMVzs2/97aFThr1qyJe51dseZ/q+53N5syf/jsT3/6k8Z245IXX3wx7s/wv1VPJ94oASCAQgkAARRKAAjI+hilv7nsvHnzNLYbpf75z3+O+zPS+bV/Idm+fbvz+Q9/+IPGdgXPHXfc4Vxnd1mx40G4wj9PPd7z6e+QM2bMGI05Wz0+O8VKRGThwoUa22lvkyZNcq6zq88yiTdKAAigUAJAQFa63rZL/dxzzzk5u6Hmr371q7g/oxA35E0He//8e2unS9hNRX7605861zElKDY7XDF//nwnZ8+qt6tx/HPR7VSseGeBN1V2M+RnnnnGydnhNX9IycrWhji8UQJAAIUSAAIolAAQkJUxynXr1mn897//3cmVl5dr/OMf/1hjOz6E+Oyha/6UFbvjjd0ImXubnM2bN2v88ssvOzk77v7d735XYzsWLMKUoERsLbD3WkTkwQcfjPlncvVdBW+UABBAoQSAgKx0ve35yP7mpY899ljMP2M3nUV89owWf1PT6dOna3zddddpzOqb5Njn1p+GYs9THzdunMb+ufVMCYrPTvvp1q2bk3viiSdi/plcnS3PGyUABFAoASCAQgkAAVkZo7QHLD3//PNObuzYsdloQsHq37+/xnZXaBGRe++9N+afYclicux55zfffLOTs4ew2XFJxiSTN2PGDI0feeSRuNflalzS4o0SAAIolAAQUJTKJrhFRUU1IlKVueZkVLS+vr5LrhsRD/c2c/L83opwfzMpqXubUqEEgKaIrjcABFAoASCAQgkAARRKAAigUAJAAIUSAAIolAAQkNJa70gkUh+NRjPVloyqqqqS2traRrsQt6ysrN4ebZpP9u7dK3V1dY323ubzcysisnHjxtrGPOE8EonU2yNd8kllZWVSdSGlQhmNRmXNmjUNb1UOffWrX811ExLq06ePLFmyJNfNaBC7QXBjFI1GZdWqVbluRoOVlJQ06lUv5eXlsn79+lw3o0HGjBmT1HV0vQEggEIJAAEUSgAIoFACQEDadjg/ffq0xvv27XNyJ0+e1LhDhw5OrlevXhqXlJSkqzkF5dChQxrX1tY6ueLiYo27du3q5Nq1a6cxO2/HZu+nf4/szlotW7Z0cvYURj+HKw4fPqzxrl27nJw9NbRfv35Ozt7fxvDs8kYJAAEUSgAISFvXe+XKlRo/88wzTs7OvezYsaOTs5/btm2r8aVLl5zrbPdm5MiRTm727Nkajx8/PpVm5wXbDRk+fLiTKysr03j//v1ObsuWLRqfO3cu7s9v3ry5xj169HBy3bt317hZs8L7/9U33nhD4zfffNPJHTx4UGN7mJiI21W0wx8+e2/9A8rswXD5utggxM6vfO655+Lm7H0ScZ87+/z7z2CrVq00njBhgpP74Q9/qLE/LJWqwnvyASDNKJQAEEChBICAtI1R9u7dW2N/LMaO4ezYscPJ7d69W+PLly9rfOHCBee6s2fPamzHjkRErr32Wo0LcYzy1KlTGldUVDi5M2fOaDxr1iwnd9NNN6W1HXaaUqGw09P8cUj7rFZXVzu5zZs3a2zHz48cOeJcZ3939s+IiNx3330aF+oY5bBhwzS+++67ndyoUaM0Xr16tZOrq6vT2P57b9HCLVnbtm3T2E5RFHGff8YoASDDKJQAEJC2rrd9xR4xYoSTe+CBB1L+eR9++KHz+eWXX9bYzugXERk0aFDKPz+ffPrppxr7Xe9169Zp/Nvf/tbJ2VVQx48f19hOtxAR+dnPfqbx7bff7uSOHTum8dChQ1Npdl6YOXOmxv5/u3X+/Hnnc2VlZczc/PnznesOHDig8cWLF51cU1jRY4fTrr/+eic3efJkjefOnevk7L9x+zOefPJJ5zr7b/+WW25xcv5Ut6vBGyUABFAoASCAQgkAAWkbo7TjCDZOhV1y5y93qqmp0XjOnDlObvTo0Q36+/KFHcuxsYh7r+0uTSLu+JjdxcU/38ROr1qwYIGTs0vECpFdKusvm03EnsHz1ltvaewvFe3cubPG9j6LiAwYMCDpvy9f2WmDPvvs+uO3lv2+4pNPPnFydpzXn97Vt2/fpNsZwhslAARQKAEgIG1d74ayr9z2tdquxBFxX6P9lT/+ziNNid1Nxd8U2a5iGDhwoMbdunVzrrOrIuwQh4jI1KlT09LOQmOnW23fvl1j/1m0XU+7I45I4a7GSZZ9dv0hHlsXli9frrE/NdAOZ/jTj9K52xVvlAAQQKEEgICcd73t5p0LFy7U2O962xUUV7vAvVD5sw1s98Xvllt2wwe7SYSISGlpaZpal9/8e7ts2TKN7TfdXbp0ca6zwxz+t7KcEXWFPZ9IxK0FdjMR//5OmzZN40QbKF8t3igBIIBCCQABFEoACMj6GKV/AJZdCbJ3716N7XQWEXeTT8Rmp6yIuOd622kVa9euda6zY5l2FyhcsWHDBuezXS1idw/yx9AikYjG6VwpUmh27tzpfP7rX/+qsR0f9u/hkCFDNM7k+d+8UQJAAIUSAAKy0vW2XZP33nvPyW3dulVje8b3/fff71zHlKDY7Jk5/hQWOx3lv//9r8Z+N9Ju8OBv6tuU2Y1EFi9e7OTscIWd5uNvdGGnWzXlFWSx2GlVr776atzr7BSr2bNnOzl/pU6m8EYJAAEUSgAIoFACQEBWxijt7ir/+Mc/nJw9l9ceMDR48ODMN6wA2KVfiQ5ZsweU+RvU+ofB4X9WrlypsT3ETcRd2ml3rfF3ZvKXhOIKe0/9umDP6H7qqac09jedzuSUIIs3SgAIoFACQEBWut52xYg9J1rE7cLY1Th+97ApnIHcEHZKVSK2Czhy5EgnV+jn4jSU3VXJ3+nHfj569KjG/ma86dw8ttCcOnVKY9vVFnFX3NgVZv4uQ9nCbxEAAiiUABBAoQSAgKyMUdpzt+2uICLuGJvdeSXROb+4wh4M9sgjjzg5e1DYPffco/H3vve9zDesANxyyy0aT5kyxcnZM9TtztqZ3GW70Nhpaf558vYgNn9n+FzgjRIAAiiUABBQlMrX7UVFRTUiUpW55mRUtL6+vkv4stzg3mZOnt9bEe5vJiV1b1MqlADQFNH1BoAACiUABFAoASCAQgkAARRKAAigUAJAAIUSAAIolAAQQKEEgID/A/3gr7BVI9CHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f88e80d2208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_layer(layer=layer_conv1, image=image1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是将卷积滤波添加到第二张图像的结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUoAAADuCAYAAABf005JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJztnWmQldXxxntAWWRnhp2ZuYDs+6YIQdZAAMXCBBJFNKZiYhKrjGb7YJUpTaUqlb9mIzGUUUPFhAQpMZGdIGCBCIERkF22WdgZhh3Z5//J9umWO2eWe2fmvvP8Pj2vfeZy5tx32vf026c7rbi4WAghhMSnVlVPgBBCqjt0lIQQEoCOkhBCAtBREkJIADpKQggJQEdJCCEB6CgJISQAHSUhhASgoySEkAC3lWVwRkZGcSwWS9JUkktubq4UFhamVfU84sG1TR6pvLYiIjk5OYXFxcUtqnoe8WjevHlxZmZmVU+jXBQUFEhRUVHw3i2To4zFYrJx48byz6oKGTx4cFVPoURisZhs2rSpqqdRLgYNGlTVUyiRWCwm//vf/6p6GuWmdu3aeVU9h5LIzMyUZcuWVfU0ysX48eNLNa5MjjIR3Lx501wXFhaq3r9/v+qrV6+acRkZGaqzs7ONrWHDhomcYiS5dOmS6sOHDxtbWtrn/0Nt3bq1sXFtK8b169dV4zqLiNSuXbuyp1NtwXUSETlx4oTqixcvqm7Tpo0ZV1n3J2OUhBASgI6SEEIC0FESQkiASolRnj59WvVbb71lbO+8847qffv2qb799tvNuKFDh6p+5plnjK1Xr14JmWeqk5+fb65xrfPyPn8fcOPGDTOuQ4cOqv1Lr5EjRyZwhqnLsWPHzPU///lP1Xv27FHt67t27txZ9ZAhQ4ztS1/6UiKnmHIcOXJE9YULF4ytXr16qocPH16qzzt//ry5xthmReETJSGEBKCjJISQAEnZel++fNlcz5kzR/Vzzz1nbOfOnVPdqFGjuJ+B28UpU6YYGya7NmnSpBwzTl0wXPH973/f2DZs2KC6cePGqnv27GnG4dr69evXr5/qpk2bVmyyKca6detU+zzBxYsXq96yZYtqn/7WtWtX1cePHzc23IrfdlulZ+pVOn5rXKdOHdX33ntv3J/D+9OnWNWq9fmzHvoPEW69CSGkUqGjJISQAHSUhBASICmBkZ07d5rr119/XTXGJEVsDOdnP/uZap9GlJubq9rHi+68807VUY9R+vV7+eWXVf/3v/+N+3OTJ09W7dODkDNnzpjr7du3q456Ogum+YjYmOLmzZuNDdOAxo4dq9rH1ouKilRv27bN2NasWaN61KhR5ZhxauHjtyWl9X3zm99U/eGHH6pu0cLWBvnqV7+q2qcN1q9fX/Wnn35aprl6+ERJCCEB6CgJISRAwrbe+FiNaSkitipQgwYNjO35559X/fDDD6vu1KmTGffUU0+pxrQN/3P+8R7TB6KAD2ssX75cNaZbiIg8+eSTqn/84x+rnj9/vhmH2208RSViT/v4Uyc+VSPVOXv2rLlev369akzDErH38bPPPqvap/nMmjVLtU9Xefvtt1VHdeuNYZ66devGHefXF0NyuIX29zj+vWOFLBEbhuPWmxBCkgwdJSGEBKCjJISQAAmLUWJaxNatW40NqxD7Cirjxo1TjTGw/v37m3GYArRr1y5jwyrp165dM7aS4iKpAsZX/Npi3AvjOiI2rtuuXTvVo0ePNuPmzp2rGtNZRGzc7sqVK8aGFV5SFfydfAwRbT4+i1WVRowYodrH0DD29v777xsbViTyMTSMy6UyJcUo0Xbq1CljwypWDz74oGo8Lup/zsc5+/TpU44Z3xo+URJCSAA6SkIICZCwrTduUwoKCowNH7l95R9sGobbG7/Vwc/wpx/8aZWogek7Bw4cMDZsBoZhDBFbVQnTpFq2bGnGYUqLP5mDKRf+O4kC+Dv59JKTJ0+q9oWkp0+frhq32/7UE1Zc8jYMa/h7OCpb75LWF0Nmfm2w8Rqm+QwcONCMw613IrfaHj5REkJIADpKQggJkLCtNz46+60xZs/74q9YzBPfjvs+v/hW0L9txUfzKLzl9uCWxW9fYrGY6m7duhmbX8N4/x0/E7ebIvY7iWJxWXzT7U8lYR8XDBGJiPTu3fuWn+dPguG1P/kT7/uJEugXfEZKs2bNVGNPLBFbBAfx2QElnbjBU2X+3i3riT0+URJCSAA6SkIICUBHSQghARIWdMLYoI+V4Sv8HTt2GBs2usKKLB999JEZh+lHPj7kT6REDUyV8Gkq2Df66tWrxoZxHiyS6lMxsGCtjy/ffffdt5xHVMA0HB/HwtQWn1KFBY379u2r2sfWWrVqpdrHxfA+xuZvUQLvp5ycHGPDvt7+tBj6Aoxt+vszKytLtS+8fPjwYdXYu16k7OlXfKIkhJAAdJSEEBIgYVtv7Kk7Y8YMYzt48KBqX3gWt+JYKPW1116L+29NmDDBXOP2M4pgn5AuXboYG55u8GlThw4dUn3HHXeonjlzphmHvWH8FgjXNmpFkEXsuvhi0Zh65X/3Dz74QDV+Jz60hP1emjdvbmzYxygqJ3E8mBLl068wbRBTeUTslhoLwWC6mohIx44dVa9evdrYMG2wousbvTufEEISDB0lIYQEoKMkhJAASTmT5it8PP7446p94U2MWebl5alOT08347ASDhbyrAlgqsT48eONbcmSJap9HA3jl5s2bVLt48R4lOzRRx81tigeCY0HpkKJiHz88ceqMdYoIrJgwQLV2Gd+2LBhZhzGiX2M0v97UQTTco4ePWpsJ06cUL17925jwxglpl955s2bF9fm17si8ImSEEIC0FESQkiApGy9/ekR3CrjI7WIzabHNBhMNxIRGTBggOr27dsbWxQLysYDU1ZEbO8W30sIwW25375jT2lMtxCJZsWgePie8E888YRq7NkkYkNImGLkT0d1795d9f33329s/h6POoMGDTLX2Nd85cqVxvbiiy+qxlQs/7c/duxY1ZhuJfLF01QVgU+UhBASgI6SEEIC0FESQkiASglAYdUZnxJRnhSJmhST9KSlpZnrHj16qPbxX6xWjn2SfaUavI7iMcXygjFL7N0tYtcdj+b5mC42f8NYZk3E9zyfNm2aakzFErEVgzC26SuFDRkyRHWbNm0SMs9bwb8KQggJQEdJCCEB0sqyjU1LSzspInnBgdWT7OLi4hbhYVUD1zZ5pPjainB9k0mp1rZMjpIQQmoi3HoTQkgAOkpCCAlAR0kIIQHoKAkhJAAdJSGEBKCjJISQAHSUhBASgI6SEEIClKkoRkZGRnF2dnay5pJU8vLypLCwMC08smrIyMgo9kUtUoX8/Pxqv7a+4HEqkZOTU1idT+ak8vrm5uaW6t4tk6PMzs6W9evXl39WVQhWGamOZGVlyZo1a6p6GuVi+PDhVT2FEonFYrJx48aqnka5qVWrVrU+HhiLxUzzulTCV12PR6XX+fdHJj/99FPVWBYM/7uISL169VT7Eu81vXwVqXzwPkbNMnXl4/r16+b63LlztxxXUmdF71t8ScKKwG+VEEIC0FESQkgAOkpCCAlQKTFKjDdi60kRkblz56o+ePCgamwfISLSt29f1Q888ICxDRs2LCHzTHUaNGhQ4c84c+aMufath2sSGDc7fvy4sW3evFl1QUGBar9e2OqkV69expbIGFoqkpf3+TuqAwcOGBv6CVxrHwOeOnWqamxpLfLFFsMVgU+UhBASgI6SEEICJGXr7V/1v/vuu6qfe+45Yzt69KjqJk2aqO7UqZMZd+nSpbifj6kEvsNg1Clpu/273/1O9dKlS1W3bdvWjPvJT36iGr+DW11HGZ9e8v7776v+zW9+Y2zYNfDw4cOq/dZ7woQJqn/9618bW5cuXco/2RTEh3WWLVumev78+ca2Z88e1bm5uar9+p44cUL1888/b2wdOnRQ7UN5ZYVPlIQQEoCOkhBCAtBREkJIgKTEKPfu3Wuuf/vb36rOz883tm7duqn+xS9+oXrHjh1mHMbVMF4pIrJ9+3bVQ4cOLceMU4eSjmviOvtrTGHp2bOnGffee++pnj59urFhPPi22yr9xGul8sknn5jrP/7xj6p9WhuuBdYR8LUQMNa2detWY+vatavqmtANde3ateYa313gOonY2PiUKVNUr1q1yoxDX+B9y/79+1VXNB7MJ0pCCAlAR0kIIQEStpe6efOmanztL2Jf4bdq1crYZs+erRoz68+ePWvG7du3T3XdunWNrUWLz0v14TxEolfNxZ/mwNSodevWGRvWt/zhD3+o+urVq2ZcnTp1VONpCRF7qqpPnz7lmHHqsHz5cnO9cuVK1T4t5bvf/a7qb33rW6pfeuklMw7Luy1evNjY+vfvrzqRp0iqE9euXVON6yliTztlZGQY29ixY1VjOGjBggVmHG7F8WSfiPUL3HoTQkiSoaMkhJAAdJSEEBIgYTFKfE3v0yAaNWqk+sEHHzQ2rK6C8QxMnRCxx8l8jA1jZ5cvXza2qFc/x7iaj1+OHDlSNcZ5ioqKzDhcM/wOREROnTql+saNG8ZW0WNh1YGLFy+qzsnJMTY8HnrXXXcZ23e+8x3V2EfqkUceMeOw8g2upf+3owp2LfAVgtAv+BgiVgXC+K3/HjBG6e9dTG3zfgE7JpQGPlESQkgAOkpCCAmQsK03PmIXFhYaW3p6ump8pBb5YiWgz2jdurW5jteETMQ+cte0Yqi4frjOIiJjxoxR3bBhQ9W+4hCmVPlTVbhd9A3f8DNTFfz9StoaTpo0ydh8Ostn+HbOmHrl71sMgUQ1re3IkSOqvV/A03b+tBi2v8Vtsk8vvHLliurTp08bG6bO+bBRWYnGt0EIIUmEjpIQQgIkbOt94cIF1f7tU0kFNPHNH76h9p+Bj9j+lAQeoK9fv35Zpp0S4Ft+3+8YtyI9evQwNuwzhPjwBH4n/lQVbk3xtERUwPX0W7dmzZqp7tevn7HFK5hcUm9pb8PtYFSLYuDbZn/aDjNb2rVrZ2zxCnD74r/4PfgMF+9DKgKfKAkhJAAdJSGEBKCjJISQAAmLUWI6g483tG/fXrWPMWAcAVMEfBFVjOH4FAHM6k9kXKK6gFVWNm3aZGwdO3ZUjSdxROL35PZxXIwvY6UnERu/xFSXqIAxRL9eaPOpLfHwKUYYa/PrhykwUQXfH/gYIqZH+bXHhm3oP3bt2mXG4akdrJYlYhsUVvTe5RMlIYQEoKMkhJAACdt643bb963BFAGfIY9bGuyp8fvf/96MO3/+vOoZM2YYWxS32wimQPh+QViAZPDgwXF/DvGnPlasWKHab4/wu4xigRE8STNixAhj27Ztm2rfC6d79+6qmzdvrtr3hcHvYNSoUcbWpk0b1VEoMHIrMjMzVX/jG98wti1btqj2aWkYHkIf4QtLY0jOF7rAflzxwlClhU+UhBASgI6SEEIC0FESQkiAhMUomzZtqnrYsGHGtnPnTtW+6C4eG1u6dKlqX1wW42/Tpk0ztqjHKDEG5o8lbtiwQfWiRYuMDdcaUydmzZplxmHKFlbMERHp3bt3OWacOmAFpG9/+9vG9vLLL6v2aW3/+Mc/VOMxSIxr+s/3sfsoHrf1oF+YOHFi3HE+LQ3j5mjzFZjw832MPpHVrfhESQghAegoCSEkQMK23ojvf4HFZX2fEHx0xvSJcePGmXGYWlDRIpypDKaliNi0qSVLlhjbzJkzVWOIw/dFx4K/DzzwgLHFK1AbRfzaPvHEE6pXr15tbFhVCYvT+oLT9913n2qsoiUSneK8paVz587m+stf/rLqo0ePGhv2L8LUHkz5EbHri4WARbj1JoSQSoWOkhBCAtBREkJIgKTEKH3sBasCYaUaEduwyleRRqKeAlRa/FG3IUOGqPYxIKzAhDE1jAuL2LhPixYtEjLPVMTft7i2vnr8oUOHVOPROb9+mG5V0xrfeUryC3jU0dtKOjqLa1/WXt1lgU+UhBASgI6SEEICpJWlqVFaWtpJEckLDqyeZBcXF1fbfSXXNnmk+NqKcH2TSanWtkyOkhBCaiLcehNCSAA6SkIICUBHSQghAegoCSEkAB0lIYQEoKMkhJAAdJSEEBKgTGe909PTi7GlQCqRn58vp06dqraHbTMyMlJ6bQsLC6v12sZisaqeRrnJyckprM4J5+np6cX+rHaqUFBQUCq/UCZHmZWVJatWrSr/rKoQ31O5upGVlSVr1qyp6mmUi+HDh1f1FEokFovJpk2bqnoa5SYtLa1an3rJzMyU5cuXV/U0yoUvEB4Pbr0JISRAUsqslcSVK1fM9d69e1Xv2rUr7jjcOvXq1cvYfNmwmoovRYfl9Q8ePKgaS9uJiLRs2VJ1p06djK1JkyaJnGLK4te2sLBQNbbjuO02+yeFbTZ8uTBscVDTuXTpkrnevn27aryPfRsY9AXZ2dnG5lueVAQ+URJCSAA6SkIICUBHSQghASolRoktCWbNmmVsb7/9tupz586p7tixoxmHb6caNGhgbP3790/IPFORvLzPX4jOnTvX2P785z+rPnnypGrfxnPMmDGqv/71rxvb5MmTEzLPVKSgoED12rVrje3DDz9Uffz4cdXNmjUz4zDbwmdeYGy4JvLee++pfvPNN40N45L4DmLw4MFmXN++fVVfvXrV2BijJISQSoSOkhBCAiRl643bPBGRP/zhD6rfffddY8NUlbFjx6rGDngiIu3bt1ftO92dOHFCddS3M2fOnDHXr776qurXXnvN2C5evKi6devWqrt06WLG1a9fX/X169eNDb/LqHdoPHv2rLn+97//rXrOnDnGhp0BMGXFJ7bv2LFD9V133WVsmAKXyG1idWXfvn3m+uc//7nq/Px8Y0Nf8NRTT6keOnSoGde4ceO4/x5+ZkU7NPKJkhBCAtBREkJIADpKQggJkJQYJaZOiIisWLFCda1a1jdjakXz5s1L9fnbtm0z14cOHVId9RilL5yBcTTPM888o/pXv/qV6gULFphxGKPEtAwRkY0bN6qeOHFi2SabYuTk5Jjrv//976qPHTtmbJgGNGDAANU+RonX77zzjrHNmDFDddTjvyIif/nLX8w1Hl8eOXKksS1cuPCWn+Fj9CWRyA6zfKIkhJAAdJSEEBIgYVtvTHXAjHsRm1n/wgsvGBtutzGdZf/+/WYcVmsZPXq0seHJn5s3bxqb3+qnIli5ZvXq1cZWu3Zt1XjCRkTkl7/85S0/w5/M2b17t2pfgQVPO/jKLfhvpyqYDrVs2TJjy83NVe0r/+C2GU8v+dDPG2+8oXrRokXGhj8X1a03VlZasmSJsWHIp6Q6txie89/R6dOnVT/99NPGhvcyfoaISFpa2epMp74XIYSQJENHSQghAegoCSEkQMJilBgrwKorIrYSkK9Og2AczVc4x54c/tgSHsnDOKeIjYOkKqdOnVKNlcpFbGzrySefNDb83XFdfPUlTIvxMd2BAweq9pXRfawzFblw4YLqrVu3GhtWd8d1ELHVrPD+fuihh8y4f/3rX6rxexQROXz4sOoOHToYWxTivyIie/bsUX358mVj+8EPfhD35/BdA8YTfSX0//znP6p9jBIpa0zSwydKQggJQEdJCCEBErb1xgo+nvvuuy+uDYv1In4LiNsWX51o0KBBqn3xziiAv69vctWnTx/V/fr1Mza/TfmMRo0amWvcfmIIxVPR7Ut1pKioSDVuhUXs7+uL7mKzO9wmZ2RkmHGYGufDSRgOiUIa263AU3Q+der//u//4v6cD1N8hq+y5ENtySKa3w4hhCQQOkpCCAmQsK03bgl9n+177rnnluNE7CM2Fhrwh9/xLVhJb1t9X+UogFs2//vh21JfuAGLAuAbXDwt4fG9pnGL6d+WRwE8meO3cXXq1FHt30rHKwTrwx2YeeC311EMZXgwrFOW8ALe8/g9+KyPeKG7RMMnSkIICUBHSQghAegoCSEkQMICeunp6aqxMZCIbWzlGzhh6gvGwHyBWjzt4xuPRR2ML/bo0cPY2rVrpxqr3YjY1BdMofKFlZFOnTqZ67Zt26qurFSMygTvObxPRex6YvUqEXtKCWNvvnAvpmJhKpeIPVXli8xGJX7ZtWtX1cOGDSv1z2EsHk/0LF261Izzp8WSBZ8oCSEkAB0lIYQESNjWG7do48ePNzZM7fEnF/72t7+pnj17tuo333zTjMNTJz6FxW/nowZur33RYtyW+OKyWOQBt9uvv/66GZeVlaUat+gi0dxuI3g/Tp061djwHnzrrbeMrVu3bqoxfOT71uPW++677za27t27q47qyRwsJuKLE2M/psGDBxvb9u3bVb/yyiuqsci0iMgjjzwS99/2J/gqQjS/HUIISSB0lIQQEoCOkhBCAiQsRonHjHyFj5LiL3hscfPmzap9kyvfVAhJZP/e6gjGHnv27GlsJfVFX7t2rWpMt/LH7x599FHVWIRWJPoxSrxXv/a1rxkbxrh8ehA2w8I18kdMMd2qf//+xoZ/M1EF0wb9vVSSX8Cfw+pM2JBNxMYvPezrTQghlQgdJSGEBEhKqR1f3QfTg3z1oMcff/yWuiSwIklNwxc/xe2L31L/9Kc/Vf3YY4+pxnQjEVvhprJOOlRHMjMzzTWGJHw/dbwHcT392rZp0ybu5/s0t6jTvn17c42Vf3xoA8MU8+bNK9Xn+89IJHyiJISQAHSUhBASgI6SEEICVEo5cIyjRT3dJNn4lAo8FuYbq2GVaIyVYcxYhN9JPPCYok9Xy8vLU43rjpWeRERatWqlOl5V9JqCv3dxrfy9W1KzwqqAT5SEEBKAjpIQQgKklSV7PS0t7aSI5AUHVk+yi4uLW4SHVQ1c2+SR4msrwvVNJqVa2zI5SkIIqYlw600IIQHoKAkhJAAdJSGEBKCjJISQAHSUhBASgI6SEEIC0FESQkiAMp31zsjIKI7FYkmaSnLJzc2VwsLCtKqeRzyaN29e7OsVpgoFBQVSVFRUbdc2le9bEZGcnJzC6pxwnpGRUezPwqcKeXl5pfILZXKUsVhMNmzYUP5ZVSG+p3J1IzMzs8S+QNUZ38e9uhGLxUwP6VSjVq1a1frUS3Z2tqxfv76qp1EuhgwZUqpxlVI9CPGVa06fPq06NzdXta+0jY2z/P+9GjRokMAZpi5YLUjEric2IfMNsO68807Vvkl9Wlq1fVCsVHx1G7xvb9y4EffnsHo8ahFWE0KuX79urg8dOqR63759qn11Jmze5pvrJRLGKAkhJAAdJSGEBKCjJISQAJUSo8TuaPPnzze2v/71r6o3b96s2scs+vXrp/pHP/qRsT300EMJmWcqUlRUpBrjkCK2cfy0adNK9Xm+k53/HmoSGBtbt26dsS1cuFD1li1bVPv1wu6XU6ZMMbY+ffokZJ6pysGDB1XPnj3b2NauXat6x44dqvF+FxG59957Vf/pT38ytq5duyZimiLCJ0pCCAlCR0kIIQGSsvX2qT1LlixRPXPmTGPLz89Xfccdd6j2jYiOHTumGrdEIiLnz59X3ahRo3LMOHU4e/asub799ttVjxo1qlSfce3atbifkZGRYWxHjx5VHfVUIR92WLBggWofMsKxeD/6+xZzY336CqZl4b0fVc6dO2eucU3RR4jY+3zgwIGqt23bZsbh335OTo6xde7cWbX/XsoKnygJISQAHSUhhASgoySEkABJiVHu2bPHXC9evFi1P8Y1adIk1S+88ILq3bt3m3EY68G0AhEbS3r44YfLMePUwR/XbN++fdyxTz/9tOo33nhDdevWrc24733ve6qfffbZik4xZcGUFBGRrVu3qm7VqpWxYe2Al156SfW8efPMuJMnT97y80RsbLMmpAph+p+IXSsfN0e/MHXqVNV+DfE7W7RokbFhbLOiqUJ8oiSEkAB0lIQQEiBhW2+soFJQUGBs+Ao/PT3d2GbMmKEaH4+zsrLMOEwj2rlzp7Fhtr6vTlTRtIDqAG5LfOgCOXLkiLnGlJP7779fta97iZVwfNgEvxNM0YoKly9fVu3vW0yH8hWrhg4dekuNJ8hERF588UXVFy5cMLYPPvhAde/eveP+26kM+oWPPvrI2PBv1Yc2Jk+erBrvQR96Onz4sGq8j0VsChe33oQQkmToKAkhJAAdJSGEBEhYjPLSpUuqDxw4YGwYf2jWrJmxDRgwQDXGLHyMBmONjRs3NjaM4eE8REQaNmwYnHt1B6sA+QrPyK5du8x13759VWPalI/lYAqLj3MmsgJLdeTEiROqT506ZWwYJ/f3HMYs8disP77bq1cv1RcvXjQ2PNLn02Pq1KkTnHsqgO8nfMofrttXvvIVY8NYb9OmTVX77wjfT/gK//hve1vdunWDc0f4REkIIQHoKAkhJEDCtt64bfZpELj9xWx5kS+mBdzq80Ts1sQ/RmPjJ7+FiQIYdiipkZXf9uEaYtMwXyEIt9fxvo+ocubMGdV+bbHFLYY/ROw6YbM2/xn4t+DDSZiahFokmltvH/LBcMaIESOMDbfb8X7G49cefUFxcXF4siXAJ0pCCAlAR0kIIQGSUhTDPwJjHxH/+I1vqfEkid9GIr4vNZ40KevbrFTDbyGw6O64ceOMzZ80+Qy/tvgZHizc4MdFYa1LyrTA4rHe5sMXn+GL/+IW0r/1xvWMykkcD/oCHxbDv3fMPhCx9zmujQ/r4XXLli2NDbfpLNxLCCFJho6SEEIC0FESQkiAhMUo8cRI9+7djQ3jA9isSkRk/fr1qkeOHKl67969ZhzGMNq2bWts99xzj+ooNmnCFBZfuBTjhLh+IrbSCqZQ+d7TWMg3Ly/P2LDiS//+/eP+26kKnrDxp8bwXvXpQWhr166dat/4rl69eqrbtGljbPg3E5V0IA+evhk7dqyxffzxx6pXrVplbD169FCNxamx4pKIjQFjszYRezKtouvLJ0pCCAlAR0kIIQGSkh7kD7hjCoY/GL9mzRrVx48fV71ixQozDreO+Egt8sW0gKiBv7vv641s377dXOP2BbfsfguI4QrcDonYU1VR7JmO223fb+mVV15R7dN+cKuIxai3bNlixmEvb9/fCK+jEMa4FbjlHT58uLHh7+z7db/66quq8X71fXfwM3zm7u9jAAAB70lEQVQBl0Ter3yiJISQAHSUhBASgI6SEEICJCVG6St8YNqKP964bt061div24/r0qWL6jFjxhibbzgUNTAGi0V2RWyxXmy0JGKLxvrUHgRjcVjtReSLKUdRxvc7f+yxx1Rjz3kR26seY2E+DQXjxD42XFJP9qiAa+N//wkTJqj2RzgXLlyoGt9d+KpiWBXL3+M+pasi8ImSEEIC0FESQkiApGy9fYUb3NJgH28RW/EGT9/4V/tYRNU/fle0KGcq4VMg8ITIypUrjQ1TX/B0jy+KOmTIENUTJ040Nt/POsr4+6hTp06qZ8+ebWxz5sxRjVWBfAFaTB3yJ6JK6tEeRXAtROzfO/adF7Fhj08++US19wsdO3ZUncyi3XyiJISQAHSUhBASgI6SEEICJCVG6cHYj0/7wUrR8apG3+rnaiq+uvvo0aNVY0xNxFZ+x6OJvsrKpEmTVEe9j3dZiFdlW0Rk+vTppfoMjEv6766m4auMY4M2n5aG/bsxDonVmEQqr5kgnygJISQAHSUhhARIK0tqTVpa2kkRyQsOrJ5kFxcXtwgPqxq4tskjxddWhOubTEq1tmVylIQQUhPh1psQQgLQURJCSAA6SkIICUBHSQghAegoCSEkAB0lIYQEoKMkhJAAdJSEEBKAjpIQQgL8P+cy8aCGvxU0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f88a0c82400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_layer(layer=layer_conv1, image=image2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从这些图像很难看出卷积滤波的作用是什么。显然，它们生成了输入图像的一些变体，就像光线从不同角度打到图像上并产生阴影一样。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 卷积层 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在绘制第二个卷积层的滤波权重。\n",
    "\n",
    "第一个卷积层有16个输出通道，代表着第二个卷基层有16个输入。第二个卷积层的每个输入通道也有一些权重滤波。我们先绘制第一个通道的权重滤波。\n",
    "\n",
    "同样的，正值是红色，负值是蓝色。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuUVPWVL/Bd/X530100TQPNEQkyiIyRvkETo4bLICphKXEJXnoIQ4ivEHQyqIQgQVZE5BLCEKKEy0UukYBKhCDoZCQECYlCCnkEEQli8bDppqtf9Kuafpz7BzRrVvbep7pgV93l9fv583vYnN+hujbVdX7n9/O5rksAAHB1Ev5fDwAA4P8HaKYAAAbQTAEADKCZAgAYQDMFADCAZgoAYADNFADAAJopAIABNFMAAANJ0fzh1FS/m5XlsLypyatGzvv312va2nh29myQ6upCPq/xWfBnZ7tOQQEfU65frTl0qEPMh39ZH27L/v0sKyeiWteN+TX26OF3i4sdlkv/7l1ywufEvD2/UK2pqpLz8vJ9Idd1e3qN0YI/Lc11MjP5+VOvUWs65JeS+hU0qzX7PsoQ0iC5bux/XgsK/G5JicPyAwf0F3P48GQx114vIqL8fJ6dOhWk6urYXyMRkd/vd52SEpa3d+qfB5MqPxPzzt591JqEuhqWBauqKNTQEPE6o2qmWVkO3XlngOUBHl127bVy/stf6jVnz/JsypTSCKOz4RQUUGDuXJZX3jNVrSkqqhXzwG7lfxIiOiS8yR/sxvgsFBc7tH49f9EqKvSa0UeXiXlN2Qy1ZvlyOf/xj30nPQdoxMnMpMA997B8rrNWrWlslPMlZR+oNb7h/yikIyINz0RJiUPvvstfy9xc4U10yd69vcV81Sr9PBMn8uz22+PzniQickpKKLB7N8trwtJ/ZBflL54t5s1zFqg1GZvWsaz0mWe6MUL8mg8AYALNFADAAJopAICBqL4zLSkhWrGC5zkVx9SaI+2DxPzwYf08d6ftYFlmZ0PE8ZloaxO/tP3pT/WSdet6yAfqytWaIUKWFmFoVtKT22lYEb+hNOyPr+tFP/uZGK9p178z/fGPj0Q9NlNFRUQzZ7J4fmC1XvPzn8t5UL9p5Q7+iGWlwWCk0ZlIbAtTzhn+73zwoPQTdtH998v5lCn6eSZM4NmJExEGZ6migmjhQhbn33GHWtL8/PNinqLkRCR/0d/eHml0RIRPpgAAJtBMAQAMoJkCABhAMwUAMIBmCgBgAM0UAMBAVFOjEmuqKGfDyqhOMCTtfTlPTNSLbrmFZykpUZ33imVnE3396yxeeKteok0p8fuL1ZrRlZUs840eHWl0Nk6fFqcM0QiPRyAPHhTjmXlGY4qBsC+djqUNY3ned6THPy8q/Phj+YDXXL7jx3m2TH781tyJE0STJrF46IEDasm0bfKOxHdvmKzWDPwpfwT3gQe6MT4jp9qL6dGq+Sz/hM+Wuuw/9+2TD7z9tl5UXc0zbcGGv4NPpgAABtBMAQAMoJkCABhAMwUAMIBmCgBgIKq7+ZSbSzRmDM+XLlVLTkxfIuYDDryhn2frVp7V10canY2KCqLFi1mcEAqpJWunKavwP/20fp5t26IdmZ2EBKI0YVmVk/qazQuW54j5rFn6afx+ebGNf/s3z9GZSWuuoUF/4Yv9dnbId7OJiMqVBbJXHJAX7CEiml8nLPbitW2BpeJiImHx4vC3vqWW3B1SFsf2WDRk0PV8oXl96XN7xcVEzz3H8/yN+uyilYGHxPyhrd/TTzR9Os+07UL+Dj6ZAgAYQDMFADCAZgoAYADNFADAAJopAIABNFMAAAPRTY1qaCDauZPF44Py9CciojJlvYUBHvuqVE7gU03a/re+17klt76e2t98k+VJeR4repSVifGx//OeWpJUx7PWjuhejiumTXEbOlQtmT1rvJgvuVWf4vaDsfLeYPGaGnWmOZ+eOsgXAVl0wyG1pj2PL4xCRNTY6HGipDi9bpJz58R9izJ+8xu9JhCQ8zlz1JJ9109lWfOU0ojDs5JE7ZRPNSwft1We/kQkb+dERESZwvSnLp99xrMLFyKM7iJ8MgUAMIBmCgBgAM0UAMAAmikAgAE0UwAAAz7X1Rd9YH/Y56siIn01jNjq77puz1ifBNcYF1+E68Q1Gvo8XGdUzRQAAGT4NR8AwACaKQCAATRTAAADaKYAAAaieqjYn5TkOikp/MDgwWrNgUNyv77hBv08iTVVLAtWV1OosZHvnWDMn5fnOsXFLG9LzlBrkmsq5QPS1iCXnD6fy7Lz54PU0hKK/TVmZrqOsNZAQ1ZvtSY7o0PMW9sT1ZrUkPCcMxHtq6gIxeMusD811XWyslge7n2NWpMWFhZNICKqrdVPJNzEDTY1Uai1NeavZXa23/X7HZY3N3vVyHne6b/qRf36sSh47hyFzp+P+TUSESUn+93UVIflgzNOqTVNBSVi3tqqn0dqbxUVQaqri/y+jKqZOikpFBg4kB/YvVutye8rN6F339XPk7OB7+tSKm0AEwNOcTEFXnmF5ZV9blJreq1TFnq57jq1Zsbv7mHZq6/GZ+EIJy+PAo89xvIdt/xIrRlZel7MT4TkvaGIiAasmi3mvuefj8sUFycriwJ33snyI3N+rdYMOb5FPrBxo36icJhFpdu3RxyfBb/foWef5QuXaGuZEBGNGiXn4x7X/5OhRYtYVPrUU5GGZyY11aFhw/hF/flG/nPcZc+3XxTzTz7Rz9O3L88eeqh770v8mg8AYADNFADAAJopAIABm1VtJ05UD40YIX8HtXSp/tfN3fQSD6v4TalYaHIzaE8b/350xA/54riX3XuvnD/7rFpSOp1/ZyqsSR0TTTm9ac8o4fvRJo8i5QUbMGuWXvP++9ENzFpWlrgXvPS92GVHlUXLhe9FL5N+/j/4wHNoVrKyiG69lednz+o140j5XnjbNr1o0yaetbR4D87Q4KTj9Gf/OJbX/ES5FiIa8cgDYv77G19Ta8aO5Vl6euTxEeGTKQCACTRTAAADaKYAAAbQTAEADKCZAgAYQDMFADAQ1dQot6WF2v/Kn99Nev55teaHynPAtx1frZ9IeA6YPv000vBMZLbV0YjP+F7wu6bo4xWePiUiopX6NuQ0eeNkli07H59rPHmS6NFHef7B+/r+4A+smCvm03bq5xm9eLF8YPhwj9HZqU/pSW/15fuqb56p1/zxj+PF/Lnn5JyIaPzgIzz0WJfBUn090datPH/6ll1qzaE8PsWIiGjYihn6iaT5V3G6RiKixqKBtGsmnwZ12yvL1JpHC+QpUC/N1H/OR4/lD+d7PX76X+GTKQCAATRTAAADaKYAAAbQTAEADKCZAgAYiOpufuv1w+nYa3yBVo+F9um29evkA9ItyEt2PMHv2jV8HJ+Fkzuy8+j8KH7ndvM8vUa60UlERI6j1tT86lcsU5bYMDfkS20U2FrODyxVpiUQ0euvf0/MFy7MVGsGjNIX1I6Hmhp5poXXTehvflPOd+7Ua8af/QMPz8uLaVsr9HfSjGl8Wf3bxtym1uxavFc+sHChWvPGf/BF3ut8fMHoWGluJjpwgOdne+kzEFY8/nvlyH9Xa554gmePPx5hcJfgkykAgAE0UwAAA2imAAAG0EwBAAygmQIAGEAzBQAwENXUqI4OooYGnieMGqkXlZXJuceeOtIePcnJEQZnJLGhjnK284VOlkwZqBdpe6of1q8x/8knWZa0dm3E8ZmoqCCSFiGR5oVc4vpflQ98TdhL6pKjJ+WNiFJTPUdn5preYfr1HGEREmk/o0vOf1++npxVS9SaE3f9gGWtK16OPEALypyhYPCres3u3XI+b55aMv7mm1m2oMVjoyljPh9RktCtJoR+odZM+Pif5ANjB+kn2pfHornlH0UaHhHhkykAgAk0UwAAA2imAAAG0EwBAAygmQIAGPC5rtv9P+zzVRHRydgNx1N/13V7xvokuMa4+CJcJ67R0OfhOqNqpgAAIMOv+QAABtBMAQAMoJkCABhAMwUAMBDVs/m5uX63sNDh+WfC889d8vPl3Oth+4ICFgWDQQqFQr4IQ7xq6el+NyfHiaqmX2i/fOCaa/SiUIhFweZmCl24EPNr9Ofnu06/fiwPd+ivSVpDlXygp8dNznZ5I5Z9Bw+G4nEX2J+Z6Tp5/Flrt1zYsuUS303yViudrv6yJNQIr2V1NYUaGmL+Wvbo4XeLix2We1wi9e4t5xnN/Dq6HKvxsywcDtKFC7F/TxJdvM4+fRyWp1UE9aLcXDnPytJrhL7U3d4TVTMtLHTo3/+d7wF194++rBc9+KCcFxWpJZ1lk1n2la/EZw+onByHJk3i16j0BSIiWrYmRz6wyGOPnDVrWFS6a1eE0dlw+vWjwNtvs/xIXbFaM2T3SvnAtGn6iYT/MIiIfL16xWWKi5OXR4HHHmP5hTlz1JqU994T8+b2FLUmY8NqlpXOn9+NEV694mKH1q/nP68ea5aQdvk3HeDX0WX0hqkse//9+LwniYj69HHojTf4dQ5ayMd12Zgxcq5u2kZiXyr9ylciDY+I8Gs+AIAJNFMAAANopgAABqL6zjQ1lWigsEby3G8qN2CIaP6YQ/KBVavUmgTpJI2NkYZnol9+Ey2ZKOwr7rFxerO0YjYRbQ6PV2sODObHzuyN03dQ5eXil2pDpk/Xa6SVeYmI1q/Xazo6ohuXtaYmoj17WJxSXa2WvLhK/m40wL+uu2zgQP693dm2FyOPz8BHHxGNGMHzL3vcxrjpsLIIueOoNcJ9PPVHIhbSas/SoNef4wc8FjR/9aNhYr5HWBe9i/RvWVsbaXQX4ZMpAIABNFMAAANopgAABtBMAQAMoJkCABhAMwUAMBDV5Ia05A4aVHSe5fPnpOlFQeXY4cN6jbSne2VlhNEZSUggShPGLE3XuiTjT38S80lf26bW/PM/38OytrbIw7PQ3qc/1Szkj4fmz+GPXl6mPBpKs2bpNUuXRjkyYwMHEm3ezOK1r+ifIbQnDfv21U9z5gzP4jVt6KZr6ymwVPg5+/BDtebEt58W8wEe77HXVvH3fentcZz6lp5OdMMNUZVM8L0m5sm3PqDWjD+zjGX/s+1ct86HT6YAAAbQTAEADKCZAgAYQDMFADCAZgoAYCCqe46t7Yl0IsQXQr722uNqjXswLB945RX9RNId4oQ49f3OTqKwMOZgUC0Z/4q8oMlvf6ufRrprfEhZE8ZaUmMd5e/ewvIlA/XFOZIGy/mMrR6LIGsLR//qV17DM9PaSnQiyH9uvBZaHxYWFrkhomFb9YV56Dj/+V9deyzi+ExcuED02Wc891i0e0BZmXxAmpZwybEsvgNBuC0x4vCstGXmUXnpOJZ7LUCzfbe80Mmyn/CZCZfdKSzcU1MTaXhEhE+mAAAm0EwBAAygmQIAGEAzBQAwgGYKAGAAzRQAwEBUU6M6O+WtmNzqfLXmhf8lLxDSo4d+njXH+R41R1vjsz9SS0ImHUrj+2QPm6ivdPHG67eI+a48eQ92InlKR3Nz5PGZqK0l2riR5zfyqSddZoyRp/rc9fhctea/tUc9MlPKVle09q8eGyQ9+6yce+2PVVER3Z+3lJNDNGoUz9/Tf/bUaVMei6NsWM6z7u6NZCH5zKdUPGsyy4Olyn5WRLTskSPygTu/o59om7BozMiRkYZHRPhkCgBgAs0UAMAAmikAgAE0UwAAA2imAAAGfK7rdv8P+3xVRHQydsPx1N913Z6xPgmuMS6+CNeJazT0ebjOqJopAADI8Gs+AIABNFMAAANopgAABtBMAQAMoJkCABiIaqETv9/vOo7DD3R2qjWffCr361699PNkZfC/L3jqFIVCIV+kMV4tf3a26/j9LP/gZIFac+ONcp6wf59+ImEjomA4TKG2tthfY1qa62Rns9wt6a/WVFcrf1eex2omyr5d+/bvD8VjSo0/N9d1iopYHmrl1365pvZvYt7mfEmtSQ43sCxYUUGh+vrYv5bp6a6Tm8sP5OWpNW2p8iZYra36ebLS+OscPH2aQtXVMb9GIv21JJ9++qoW+TpTUvTzJAkd8ezZINXVRe49UTVTx3Fo716+3FFCWF/uaHxZhpg/8YR+nttK+d9XKu1AFwOO308BYeWg9If5ijVddu+W84xMj3//L/OVi0r37484PgtOdjYF7ruP5ReWr1RrtP0Pp449p59I2bnOl5kZl/mCTlERBV56ieWrg/oqQFM33i3m5aveUmuKj+5gWemjj3ZjhFfPyc2lwGThZ3PsWLWmfOBtYi7sC3jZbUP5pnKl3VxNyYL2Word75KVR+Xr7KsvAEcFwmemKVO6t2Idfs0HADCAZgoAYADNFADAQFTfmVJTEyUE9rL4Bxv4yvRd3lihfKe2cKF+noff5lkwGGFwNloyCujQjfw7qHBYuQNDRBlj+PePRKR/0Ugk3yDw+iLZUkYGUSn/HsjrK9u0NDmvSSpUa/IpXlsHyMobsmnuTv693vzr1qk1q++XvxtN2q6fZ3JQ+NJc2pIiBiqT+9KSokUsn3SdXlPcdELM6/wD9KIDB3jW0hJpeHaysoik+yavv66WaD/PXjsECLcSKDExwtguwSdTAAADaKYAAAbQTAEADKCZAgAYQDMFADCAZgoAYCC6qVEffyxOT1iyaZNe0/9+OZ8zRy0p//1HLGu7q3uPdF2t5GT5cbPf/U5/Nv8D/y4xv2n5VP1EU6bwrLtzMK5SbVJPejX3IZb/SZ8xRMsWKtOcXn5ZL/qHf4hyZLba2ogqKnj+P45PivrvmjjR46D0s7xlS9TnuBK9MhvpBzf/mR/4p0f0olWr5DzLY2qU9Fx8crL34Ayd+NRHD5Txh+rHjtVfyw8/lPMf/lA/T0lfvi6INi3w7+GTKQCAATRTAAADaKYAAAbQTAEADKCZAgAYiO5ufu/eRI89xvN9HivKCwstExHRI/rdxonCurYn5LUZzCUmymsajy7li+NetmaNnM+bp9dsF1bOCIe9hmamR3Y7TfgGX4Bmwrvz9KKvvSfn2i1TIv2ucZxod/N/9CO9pqNDzr9Kwh3zS/b85assa2qKNDobDW4W7Qjz8yctP6TW5Cl3p4e16zVrDwxjWXVTN29zGxhQ2EivTRdmzQwcqNZMrtso5juOz1BrSjYs5qH0QyTAJ1MAAANopgAABtBMAQAMoJkCABhAMwUAMIBmCgBgIKqpUVVJvWllTz6vZOdOvWbUKDmfOkefnrDr4REsK33GY2qSIV+4hVKOClNEHn5YLyorE+Nmf4laklTGF0Fxf/FixPGZCIflTdK1F4uIaOlSMd6xmy8+0UXf5uvb+nkMDbxwhLYE+ZSeBb/XpwDNvv+YfOCwPj1mxK18mllmanvkARpobCTaLWxB5bGdPNXVyfmi20+rNZPLhrJs2bJIozPU0kJ09CjPvfZZkxYTIqKRh1fqNdI/jjZf7u/gkykAgAE0UwAAA2imAAAG0EwBAAygmQIAGPC5rtv9P+zzVRHRydgNx1N/13V7xvokuMa4+CJcJ67R0OfhOqNqpgAAIMOv+QAABtBMAQAMoJkCABhAMwUAMBDVs/n+rCzXKSjgB7Kz9aKWFjlvaFBLKlL4M+21tUFqagr5Io3xavmzslwnP58fCIX0oqIiOW9tVUvOdPRmWX19kFpa4nCNqamuI+zNcrzjGrUmM1POe7vl+ona5efT91VVheJxFzgnx+8WFjosz8vp1IsqK8U43IO/Xl3SKoIsCzY2Uigcjv1rmZ0tvydrPNayKCyU88REvUa4UR2srKRQfX3Mr5GIyJ+b6zrSuDs9XsvGRjnvrb+WFzp5SzxzJkg1NZHfl1E1U6eggALSBjq33qoXHT4s5x6ro7zQny/48fOfl0YYnQ0nP58CTz/ND3jtZzRzppwHg2rJU/X833Ht2jhdY1YWBe68k+XjGn+t1tx8s5zPDs/VT6T8B+R76aW4THEpLHRo0aIAy8ePadaLFgt7ABHRsYn6dQ5ayBetKd2yJfIADTgFBRSYK4xtwwa9aPp0OZc2P+si/MdYqv09MeAUFlJAWmzHa7MtaQUYIqI5c9SSU2HesMeO7d77Er/mAwAYQDMFADCAZgoAYCCq70wpL49orLCp/R/+oNdoK9F6uOEGnqWnR/3XXJm6OqLf/IbnL7yglhzpO1rMh3xpr1ozSvhn2bw54uhMHGu7hkaH+Pej77yzXq05c+ZBMZ+9/Qn9RNu3y/lLL3mOz0pee4jG163mB2by71Ev69NHjNO8togfyhdOpnfe8R6cldxcojFjeP697+k1ysW0v/mmWpL05JM8PH8+0ujMhFNz6diX7mG5tF50l6x7HxDzwR7rdpcs5ovWp1Tqi2b/V/hkCgBgAM0UAMAAmikAgAE0UwAAA2imAAAG0EwBAAxENzWqtpZo0yaeHzyo1yiP550I5aglYxye5eZGGJuVzEz52ckKfd/0IW3b5AMej7qNTuN/X05n9NPIrkRysrycwNe/Lk9/IiLatUF5Bv/wcf1EXvNW4iElhahvX56vWKHXKNO5SrweJ/7+96McmKGODvkZ9HXr1JI9fcaL+bYb9dPM7yvsNZ+aGml0ZtJaamnQgddYvt+Vpz8REY1bOlI+UOrxeOgdd/DsrbcijO4ifDIFADCAZgoAYADNFADAAJopAIABNFMAAAPR3c3v2ZPou9/l+U9+opYcOSPftR+SdkKtOXR4AMu0BfvNaXfzvRbAdhwxnj1dXwhiwUxhJXSvxXkNXdOjjtbe+wY/4LUoTXCwnHvd5ZbupMdRsCaHpm7gi9AEwvLCNEREhzYfEfNmZ4hak3G9sEPBuXORB2igsT2N/hwaxPK+pTzr8v375fwvf9Fnnxy+7yGWfXJeuMMfIw1JPWiHn9+5P6ispUNElP7EDjEfN1P/t9kxZhE/d+KCyAMkfDIFADCBZgoAYADNFADAAJopAIABNFMAAANopgAABqKbGtXZSRQO89xj4YAhgzvlA1+bpNZsvus9ll3BVlJXpC0zjypHjGN5vbwFPBER+YPyFKjSnXrN7MX5LPusMrqX44qdOkX0hLB30/Lleo3fL+cTJug1/frJ+fPP6zWG+veX1zTxmuVWeIc8BUrbgp2IaNCDwgIxL78cYXQ2strr6KsVwjS3oaPUmr1tt8sH9vxSP9HDD7OotPXjSMMz095OVFXF88OH9ZoFFVPlA9LUx0tGlvL3cnZGR6ThERE+mQIAmEAzBQAwgGYKAGAAzRQAwACaKQCAAZ/rut3/wz5fFRGdjN1wPPV3XbdnrE+Ca4yLL8J14hoNfR6uM6pmCgAAMvyaDwBgAM0UAMAAmikAgAE0UwAAA1E9DO7v0cN1iov5gYYGvShB7tcXcpRnvZWS06eDVF0d8kUa49UqKPC7/fo5LK+v96hJVBYO8Lq5l53NouDp0xSqro75NfozMlwnN5cf6Olxw1J5jdtyCtSSZF+7mO87eDAUj7vA6el+NyfHYbnXFjgD+I45RESU9Onf9KJOvv5EMBymUFtb7F9Lv98tKXFY3tqq16TXlisH0tWatqweLDtzJj7vSSKipCS/m5zssLxPH70m7/wp+YDUwy6pquUtsbo6SI2Nka8zqmbqFBdTYP16fmDnTr1I2dfo1ChlEQKlZORIfTEVS/36ObRjR4DlW7fqNZOzhIUmiC6uzqC54w4WlY7W9yay5OTmUuBf/oUfeOQRvUh5jctHTVZLipPkfZB8vXrFZYpLTo5Dkybx19JrcYwNG+Q8v+xuvaixkUWl+/dHGp6JkhKHdu/m13j8uF4zbONc+cDQoWpN5e18/6XRo+PzniQiSk52yHH4dT73nF4zfvtj8oF589SaFzcWsuyFF7p3nfg1HwDAAJopAIABNFMAAAPRrUbsuuL3gMfGzFBLBq2ZLeYlm/kCzJcJCw4nnRf2mY+BigqihQt5Xlam13T+47fEPOGuu9SaUzfz76AudMZncegLPfvQqUf4XuBTpug1Z87I343+7dtn1Rq3o1e0QzPVr2eYlkw7wg94fZ85fI6c33efWvJCryUsqzgRn+8TE+pqKGPTOpYP87qZKC0MTuT5ZXKvX/DvWZOrlBtZMdCrF9GTT/J8zRq9pmjWi2J+r/7VMJ2buYhlqzsqIozuInwyBQAwgGYKAGAAzRQAwACaKQCAATRTAAADaKYAAAaimovTlpxB5UU3sXzQ8V1qzegAn4JDRPSfE1frJ/rGN3gmPMseCwkJRGlpPPfaUn7lv/6rmHcu5lNmutQJs1A6urc991Xr7BSfgKRR+lbrdO+9cv7mm731oo2vRTcwY50ffkiN11/P8qzqar1I+tkjoi0B/Xnu3wpT6eqU5RrMJSTI740VK/SavDwxnl2kvycXlIZ4mJERaXRm/FlhmnqzMM2Nhqg13/mOnJ/bqPerCzc/xTL31e79HOOTKQCAATRTAAADaKYAAAbQTAEADKCZAgAYiOpu/qlTRNOn83zx4tvUGm292Req9MWhvyncCQ13JEcanoniwnaaP11Y1HjmTL3oP3aKccLgwWrJtFUPsSwYjDA4IzU18iLI87P4Ig9djiXxu5xERN/9rn6et97ni7lcxBeyiYXm64bTodV8QeHtHjMzjh6V82nT9Jo5wtoojz8eYXBWGhqItm/nuddiLtu2ifGCvufVkkefHs+yU+flmTrxFA7rxz56cL6Yn3KUxbGJaKfwvqjp5hpL+GQKAGAAzRQAwACaKQCAATRTAAADaKYAAAbQTAEADEQ1Nera4hZ6Y94hfmDFK2rNkj+8Ix945hm1ZleIT8Pw2oLeUkNLEu04zPfOHum1ckVBgRgfu4NPf+oyUVho5Gc/izg8E+3tRFVVwoGdW9WaQRs3ygdmzVJr7o7XXC9FSwvRgQM8HzhQr9HWB/n1xC1qza48vp9ZYmKk0RnJzCQaMYLn0nytCGbMyVGPSYvwuG7Up7hi5TVpNHcDX9Sk1GurrVr5RSgZNUgtmSzMGVyW3hxxfET4ZAoAYALNFADAAJopAIABNFMAAANopgAABnxuFLfkfD5fFRGdjN1wPPV3XbdnrE+Ca4yLL8J14hoNfR6uM6pmCgAAMvyaDwBgAM0UAMAAmikAgAE0UwAAA2imAAA7Z0e7AAAAHElEQVQG0EwBAAygmQIAGEAzBQAwgGYKAGDg/wIenpzuqVsHswAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8947e92e80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_weights(weights=weights_conv2, input_channel=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二个卷积层共有16个输入通道，我们可以同样地画出其他图像。这里我们画出第二个通道的图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X90VeW5J/DnkN/JIQnJIQkhwBYRUqWUatpSoBQZhjqYodZB4CLj4rKyqHpZDKWMUuEyLJYyqAiMZY2UakoZquJQigzKj5t6KSIiBkoRhatcPEAIgZz8Dvkd9vwhYXn7PM8+nOE5p8vx+/nze3jY7+acPJzs/e739bmuSwAAcGt6/a0HAADw/wM0UwAAA2imAAAG0EwBAAygmQIAGEAzBQAwgGYKAGAAzRQAwACaKQCAgfhI/rDfH3Czsx2W9+3Tpda0d8uHiIvTj9Ml/HUXLwapri7kCzfGWxXw+10nO5vlbqCvWtPSIuednfpxkpN5FqtzTE4OuGlpDstvu82j6NIlOU9Li/j4Rz/7LOS6rv4PaiQlJeCmpzss93ror08fOW9o0Gv8fp5duRKkxsbov5epqQE3M9NheX7GVb1I+gEjotruDLVE+ow3NASptTX650hEFEhKch3hs9Y5YLBa090t58lt9fqBEhNZFKyspFB9fdjzjKiZZmc7tGRJOcvnTq1Va87WZ4l5ZqZ+nKoqnk2bVhR2fBac7GwqX7KE5R2z56o1f/6znF++rB9nyBCexeoc09Icuv9+/j5u2qTX9Prvz8gvfOc7ER/f96MfnYu46P9BerpDDz/Mz7O1Va956CE537NHrxk7lmcLF8bmvczMdKikhJ/jiuIjelF1tRi/2nC/WnL8OM82b47NORIROWlpVD5xIssv//INtaamRs7vPLNTP1BBAYuKZs0KOz4i/JoPAGACzRQAwACaKQCAgYiumfbN7KS5xZUsnzA1X63553+Wb1zccUc/tebdd3nmdcPKVHU10Usvsbh8uH7NdMYMOT9/pkOt6UpKYhlPoqOggGjVKp732v2WWlO5dKmY53/zm2rN/PEnIh6bpcRE8RIY5eXpNROKGuW8vkwvGsmvHa5I1d97S716yfcA52z4rlojXeMl8r4uLH3Gt28PMzhLOTlECxaw+IMP9JIp2x6RX1A+y0REHc5QlrkpqWGHR4RvpgAAJtBMAQAMoJkCABhAMwUAMIBmCgBgAM0UAMBARFOjyOcjiucl7xSv0WteVx7FWqpPNaLf8mctE2qEZ0yjISuL6O/+jsXSI649zr+kTCkq2arWlP6KPyAeeiY2j+clNNdR/kH+GN6hgmlqzehRo+QXPKaZnHwh4qGZyqk4So//nD9SnfzYY3rRcnkKVO3hT9WSrJeER221ZxmN5fVppycfOstfqKjQiwoLxTgQyImoRFpfIlouNflpRdlolv/0p3rN2/GbxXxyvPDvdV3i//4dy3x1+uPyX4ZvpgAABtBMAQAMoJkCABhAMwUAMIBmCgBgILK7+XFx8qrOXiskKCvxbh67US155Gm+2ACFQuFGZyMtjaiI31V/sEK+M0hERKcuyLnHorJzX+Z3zjc26XcZY6FojMdi4sqy5TNn6f8fnz59qyO6NXW33UPbn+YLJ898fYpepNyibmvTS56N54uJV/n+EHZ8JioqiBYt4vmpU3rNHXeI8RRtBRQiapn4BMuERemjpl9iDS1z+M/g20eVxUyIaLLzifxCQFj9psdPfsKzF25uWgq+mQIAGEAzBQAwgGYKAGAAzRQAwACaKQCAATRTAAADEU2Nar7qo0PlfD7E6OJitWZf4Xz57zrjcaBt23gmLD4SFfX1RLt28fw3v/GukfzjP6olpffxhUZCR2Kz0ElLUh86NoRPzbr7B+vVmp275P93V6/Wj5PfLC8O4hvmPT4raWlE0vosjcX6vunpyfLeTfnFk9SavwT2sUyZEWiuu76eGv/Ap2GlnzunFwl7KRGR5/yv1D18w6dejcrnPgquJmfTB8P4NKjD+rZlNNlRXli+XC+aOpVnLS1eQ7sB30wBAAygmQIAGEAzBQAwgGYKAGAAzRQAwIDPdfn2Geof9vmqicjjNmFUDXJdt2+0D4JzjImvw3niHA19Fc4zomYKAAAy/JoPAGAAzRQAwACaKQCAATRTAAADET2bH0hJcZ30dJaHUgaoNVlZcn7xon6cAWm1LAtWV1OoqcljXw0bSUkBNzXVYfnt2fpzyN29ha1ciCju/Of6gQYNYlHw/HkKhUJRP8dA796uk53N8rq4gFrTIT+yTrlNHossCMcgIjp69mwoFneBA5mZrpOXx/Jm8qs1/mr5PWvrd5tak9zVzLJgVRWF6uuj/l7GxQXc+HiH5QH9raT2djl34pTtd4iIBvCf8WAwGJPPKxGRzxdwifjPzD15lWpNc0Z/MfenyFvwEJG4PU+wooJCtbVhzzOiZuqkp1P5ww+zvHT4GrVmxgw5X7pUP86ae37HsiKPRUMspaY6dO+9fN+g7bP1xTEax8t7CqXP0/enoQ0bWFTksQePJSc7m8qXLWP51rQ5ao32n9/C/R77KSl7YPmmT4/JFBcnL4/KS0tZfohGqzWjN8jv2adL9T3AhlYdYFnR3Lk3McJbFx/vUP/+/PM6e7ZeEwzKeWnmQr1IWNGm6Lvf9R6cqUFE9B5Ly/9+hVpx4L6VYj5uZKN+GGHRoiKPhZy+DL/mAwAYQDMFADCAZgoAYCCia6ZUW0v02mssnuO8r5ZkLZJf27JFP0zb/fxaW6ye07q9fxttX8UXNT7WrF8bvHs1v/5IRLT1fv062/R64cK5sje9uUCArs3m10enF0/Wa1atkvPD8j7zRER0330RDszWpSY/rdzPr4828/tFN4xW7twMXeRxbbhIWNS7qSnc8Ex88442Kn+D7w+/ufxOtSYUUl4YOVI/UFxchCOzdc+wq1ReeozlJ/zydVEionH/5xn5hdA39AOVlfGsqirc8IgI30wBAEygmQIAGEAzBQAwgGYKAGAAzRQAwACaKQCAgcimRt1+O9HGjSyuHDJOLVm+Tc4nl+uPgUnzpnwxepyUiIi6uli0TTkPIqK7F8wT8+mPPqgXLf4zzyr154wttbYSnTzJ8+G73lZrtMeChwx5Q61Z2XYl0qGZSkggEh7Nl54YvGHSHvnR6H2z9GluJ0byR1Bbt+qPH5tKTCRyHBYff1kvKedPn36hWPjHuo7/RMRuuqKXERNz1NcmjZQ/f6v+g/73HSzkP7NXko/c1FjwzRQAwACaKQCAATRTAAADaKYAAAbQTAEADER2N1+RH1CWYSeikpJE+YUHDup/YbKweIa21LuxK43J9GIZXyRi5dPX1JrL1fIdxT9O3a7WzCzmixbTCo8ZDoZSfG00Ip4vjjHpPn1xDO1u9v9s1hfA3nlYv9MaC9qsBWktix7Ll8v5Zo/zLC7gWUKC99jMXLsmrtzS1ZWqlgjrkn9horyYNxFR/Mcfs8w3bVrY4Vmp7fDTq0G+aM3M06fVmtl75Pxuh+/k0aOtjW8Nkqr/U/4b+GYKAGAAzRQAwACaKQCAATRTAAADaKYAAAbQTAEADEQ0Narxmp/2tfFFTQLC9JMed6+Sp0/sW7RPrVm6lGen6vQFNSzlJNbTfIcvUvHs8/oeQE8Of0vMZy74e/1Aw4fzrFafsmEqFCLatInFY8c+p9fskeeZPH7XBbXkxa4lkY7MVN++RI8+ynNtOysiosOH5fzB8R7vzfjxLIo/c8Z7cFa6usSVW8aM0aeljfjp9+UXPFaAqczk0+Y64zz2/zKmLVqzcgOfytTjqQf49D8iomNBfQrg6LZ3WOZ3b24/L3wzBQAwgGYKAGAAzRQAwACaKQCAATRTAAADPte9+c0HfD5fNRGdi95wPA1yXbdvtA+Cc4yJr8N54hwNfRXOM6JmCgAAMvyaDwBgAM0UAMAAmikAgAE0UwAAAxE9mx8IBNxBgxyW19XpNVl+ZbuR6mq9KDeXRcELFyhUU+MLM8Rb1qdPwM3Pd1iekqzfqOs4dkzMvXau8N1xB8uCly9TqKEh6ueYlRVwBwxwWJ4Q/Czyv0w4jx4VFXJ++fLRUCzuAgf69HGd/v35C42NelFcnBifa85WSwb1bWFZsLKSQvX1UX8ve/cOuNnZDsszM/Wa+A4+XiIStz/xei149SqF2tujfo5ERNnZAbegwGF5QpW+NgRlZIhxS3y6WiItj9HQEKTW1lDY84yomQ4a5ND775ezfNs2vWbm2PPyC+pGNES0YAGLiiZNCjc8E/n5Dr32Gj/HEYX6HlTnk5Lkv8vjOPHr17OsaN68sOOzMGCAQ7t383PML5kc8d91bdfb6muLF8v588/7YjLFxenfn8q3C/tw/dM/6UW9e4vx3IP6HlAbH+X/mRbN0vdTspSd7dCyZfy9fOABvSYrKP/nT++/rxf96U8sKvLaTMtYQYFD+/bx88x9Zr5eVFwsxscCei95/XWebd5cFHZ8RPg1HwDABJopAIABNFMAAAMRXTP1XW2mxPJDLJ+5Y51e5JevHZUOWamWzFk8h4fa3QxjKQ1VNGKPsEjyR8KNjOsGJiuL5J7TLw0+tY4v3nuxSb8wbimhupLyNyxjee0W/fpnVrx808brPqJw6ZuIiJ5/3nN4dpKSiByH53/8o1oysFy4xkre1yDp1CmetbV5j81IV5f8HpSU6DXbH1BWc9+7Vy+aOpVnH37oPThDCadOUO53BvIXpJXkr6stkq+N3l0lLxpNRHT3r8ew7J0mLA4NABAzaKYAAAbQTAEADKCZAgAYQDMFADCAZgoAYCCiqVGhNj+Vnh7N8jlT9WlLBzLl/ebnFH6qH+iB1TxTnn83l5JCdNddPE/weNL+vffEeOYCfe9yaRt6r0fGTeXkEAmPrqrrKBDRs2vlaVvf/rZ+mG99K+KR2aqvJ9qxg+erVqklgRlyXljocZyf/IRnL7zgPTYj1dVEv/oVz8+c8SjKWyTnHtOMdmbyx2nr414MMzo7zYNH0KFS/jhpKKTXTNlRKr8gTfO6busGvtBI3VN4nBQAIGbQTAEADKCZAgAYQDMFADCAZgoAYCCylfbTWmlO0QmWT1k6Ta3ZtEnOS3cMVWvmjBIWIujqCjc8G11dRDU1PF+7Vi355Hd/FvNXVykLYxMRJS9nUdHOYJjBGenq+uJO91/Zd1yfffDkf64U863v6ktg56bEanqCrOZaH9rcxj+bZ7boNcdcZXqC67FyyNMXeXbpUpjR2cjLI1ok3JzvlemxaM7TT4vxExX6QsvP3cdn36xIjs1iLkREVVXyJIyd6/WfsZVbhAWTiMhrCfbpKTtZ9ryP/6xI8M0UAMAAmikAgAE0UwAAA2imAAAG0EwBAAygmQIAGIhoatSVphR6cf8IlmvTn4iIsurPivmcXGHfnOuOtd3PspZryj5Lxtr92XR2LF/UYfDw4WrNnZuekF/wWFCB7rmHZ7Hah1zZG2lSvPxeERHNWTpYzL225pr+HyP6eJlLSPhi6tBfy8zUazqWytPcEpcq7zERkfTZ0PYFM5aQQFRQwPPtm/RpaQ/694n5c03PqDXbTy5hWX1bbM6RiGhIaiXtHMn3LXti/Qq15rnlLWK+ZkOqWrNwhrCoSVpa+AESvpkCAJhAMwUAMIBmCgBgAM0UAMAAmikAgAGf67o3/4d9vmoiOhe94Xga5Lpu32gfBOcYE1+H88Q5GvoqnGdEzRQAAGT4NR8AwACaKQCAATRTAAADaKYAAAbQTAEADES2B1R8vOskJrK884471Rpt66aUuA61JljJj9HcHKS2tpAv/ChvTSAQcB1hERBqaNCLWuQFFdSciCg3l0XBqioK1ddH/xz9ftfJyuIv+P16kbaiSVycWnI+4XYxr64+GorFlJrs7IBbUOCwPOGqx54+ra1i3F0p74FFRHSxL1+0pqkpSK2t0f+8ZmQE3Nxch+XpTcK+VD2U99lNz1BLgkGeNTcHqb09+udIRBTw+VxHyDtHCAsGXZcQkvfhcvP6qTXt7Ty7eDFIdXXhzzOiZuokJlL5sGEsr3yrXK0JheR8RKa+Edac5QNZtnOnsJpLFDiOQ0eO8PPptfstvej4cTk/elSvEXZBK5ojbwBmzcnKovInn+QvfP/7epG0axuR5xJMj+VuF/MNG3wxmS9YUODQvn38vcz9gG+adsNHH4lx/dKlaskv/hM/xu9/H5vPa26uQ+vX8+NP2v+UXjRmjBh3/Hu+WluP2bN5tndvbM6RiMghoiNCXrVb7z35L8srSnUs5qtP9ThzhmfTpt3ceeLXfAAAA2imAAAG0EwBAAxEthT6sGFEf/oTi7s8rudXVcn5iHmz1JrS5ctZVvReU7jRmWhoINqzh+fr/od+PWnfjnvlFxYvVmsu3z6aZZ1JHjeALLW2Ev3lLzz3usY7Y4acz9Lfx5faasV8wwavwdnp1YsoJYXnTxycotaMHSu/NqX9v6o1Pwvy7N13w43ORvqZYzTpx/wkd26Vb6QREb3+Ozl/9Rv6Tgtr1/KdFiZNCj8+M1lZ1OtHP2Jxfps+5mVd8rVR/eo30V13dQrpzT1yj2+mAAAG0EwBAAygmQIAGEAzBQAwgGYKAGAAzRQAwEBkU6Ncl6itjcUDTx9WSwZWV8svlJTox2lu5ll3d7jRmbhwgWjBAp7Pm+dRNHKkGLd89plasn/MiyyTTjsajoYKyPfrZ1n+0EN99KIyOR6vrL1ARPR4SWQfL2t1dUTbtvG8uFivGTeyUczfLktXa+4RHg/3WLLAVkYG0b18at6UIZ+oJVO0uUEH9Uczc/P4c5YJrfK/VVQMGkT08sssHjw8VS3RngBeu1Y/zO9/n8CyJ564ueUH8M0UAMAAmikAgAE0UwAAA2imAAAG0EwBAAzY3G5dv15/TVnsY0UZX+ijh3TnvMsvL/RqLTdXvpv/eMjj+Ifl2QwVIWE1++s+EBb7iNXd/JSUOCos5HfuV6/WawaSvJj39nK+kPcN8X/bu/m9ehElJ/P8hz+8qtY0NMh37SeXLdQPdDKPRQk1ygo/xhoCt9PbJXwR7gKPWRbKx5W2bNF3zHj+eZ5djdNnOJg7d06cAXT8+KtqSfomPmOGiKjxsfl6TT3/nK9M1XcF+TJ8MwUAMIBmCgBgAM0UAMAAmikAgAE0UwAAA2imAAAGIpu7UllJJOzPtLNE34d8/HA5X1Yo7w9ERES7drEovqEm3OhM9K39F3r89XH8BY+9jqTFX4iIhn72llqyJvs4yw7EXwo7Pgt3dn9E5XW38Rfq39SLhstv5IPKlKkvFEQ2MGNZ/g6aOZaPL++P+nSugwflfPLu3fqB9u7l2ZYt4YZnor6eaMcOnm+cd0KtKWseIeYHSjarNQfaH2GZe3NbI9kYOFDcPOzUKb3kewXy50+a/tTjRD3/bLR2J4YfH+GbKQCACTRTAAADaKYAAAbQTAEADKCZAgAY8LkR3JLz+XzVRHQuesPxNMh13b7RPgjOMSa+DueJczT0VTjPiJopAADI8Gs+AIABNFMAAANopgAABtBMAQAMRPRsfkpKwM3IcFje1+M+V6LyWKuvU98KoLqBF9XUBKm5OeQLN8ZbFcjMdJ1+/Vhe35mm1mRkyPlVfXcMcYuSurogXb0ag3NMSXEdYdCX4vRn6WuVpRT699ePk9l0QcyPXrkSisVd4MzMgJuX57DcH9eqF/nkf/6OXsL+J9clXq1jWbC6mkKNjdF/LzMyXCePb5vyeai3WtOH71hDRERdHludBCjEsmBNDYWamqJ+jkRE6ekBNyfHYXlcnF5TWSnnw4Z5HKipiUXBqioKNTSEPc+ImmlGhkOPPFLO8kcf1WuUtQYosUpfbGDjHr7YwDPPFIUdnwWnXz8q/+1vWb694rtqTXGxnGt77RARvf8+z375yxidY0YGlT/CF65Y4X9OrXntNTl/9ln9OFP2y/sm+daujckUl7w8hzZu5J/XcZn6IiDiplFEdD55qFoy8PAbLCv6xS/CD9CAk5dH5S+9xPKZL09Qa6ZOlfP6ev04c6iUZUUrYrMvGxFRTo5Da9bw99Lv12uENZmIiOjA/mt60f79LCp67DHvwV2HX/MBAAygmQIAGEAzBQAwENE100BA3Lpau8xERESnT8v5iLHKqtFENHfRIpZt7IrNwsltcWn0aSa/Pnqcr1d9w4OhjWI+tmSuWjNqFM/e4JfeoiM7W1zsuqhCLynnl6uIiGjKLI+90/fskfO1az0GZ8f/r3+hcdP5zUT6xjfUmmVj3xHzFcVH9AMFgzxrbw8zOiNxcUSZmSx+dfY+vUZaTZrI+25idzfPOjvDDM5Oba283vYbq/V7LxMe2Ca/sE4/TmMJv87fnarfzPsyfDMFADCAZgoAYADNFADAAJopAIABNFMAAANopgAABiKaGpXcVE1D9/NpQC2z9ClA+SeVKRra84lERO+9x7NrHo+AGUr+149p6I/51JkVr7yiF9XJU0p6ha6oJYkffsgyX2ND+AEauNyYQmvK+N7pP/+5Pv3M3fuR/Hf9ulGt2bo18rFZupTzLVr5D3xO14wZes0oZSrftSL9ceKyev5aY2qM5rkpU6OosFCvkeblEelTpoiIhOf/xblKUTJ4QCe9sY4/bN+SyR897/HRGPlxZsfRj/Pscp5pz/j/NXwzBQAwgGYKAGAAzRQAwACaKQCAATRTAAADEd3Np0CAaPZsFqfu8Lhzqa2CIt2B7CEcg3bu9Byamfh4ceuA8wWj1ZKBdW/JLwwZoh+nrEw+dgzkVp+khRv4YseZr3yqFxXw1eSJiBo8JiDMD8p3U/+L5+js9OvdTE+NP8Tyljz9vRxcJi9aQ/v193JSG982If2ax0rLhhrbk2jfmcEsP+2xMM+CBXJeUsIXDO+xdCnPOpI9FrmxVlMjzh5Y1fyEWiKNmYgosatFrVlTsIFlBxIvhx8f4ZspAIAJNFMAAANopgAABtBMAQAMoJkCABhAMwUAMBDRXJzOLh9VhhJZHiqcptZoiwqkk75AxjW/MOXCa6MpQ6HsYVQ6+wDLnx6v15w8eb+Yp2p7IBHRxuN8cYzqlrSw4zOhbOY15/VJes3El8V4aDFfMOWGv/EeUOdq/DR3E58GNVzffozmVwXFfCPpi/nM/W/CPlOhULjhmejoIKoQ9u6aP1zey4qI6NRPJ4j5D36gH2fgej4FKfGKx6Zhxi505tHCKj4GYbu4GxK1KZteUxCFvdFo8+Ywo/sCvpkCABhAMwUAMIBmCgBgAM0UAMAAmikAgAGf67o3/4d9vmoiOhe94Xga5LouX4HEGM4xJr4O54lzNPRVOM+ImikAAMjwaz4AgAE0UwAAA2imAAAG0EwBAAxE9Gx+nz4BNz/fYXmdvKMFERHl9+2UX+jo0IuSklgUvHCBQjU1vjBDvGV+f8DNznZY7rXLSgLJ59jhJqg1if/yEcuCXV0U6u6O+jkGEhJcR1rrYMAAvej0aTnv31+vSZDP/+jnn4dicRc4KyvgDhjgsDyh7opaU5eQI+Z9qk7pB+rk73+wuzs272V6uuvkCGO+dk2t6eydJeYJrsfP5MWLLAo2N1OovT3q50hElJkZcPPyHJZ3d+s1Gcnt8gtnz+pF6XxdkGBDA4VaWsKeZ0TNND/foddeK2f5tm16zYpHK+UXpNUZegh7JxVNkBdnsJad7dCSJfwci4v1mnySz/F8V75aM/CHt7GsqFL5tzLmJCdT+be/zV9Yt04vGjVKzn/2M70mN1eMfbNmxWSKy4ABDu3ezd/L/G0vqjVbc+eL+fQX+MI0N1RVsahIyKLBycmh8uee4y+0tqo1lyc+LOa57ef1Ay1ezKKivXvDjs9KXp5DpaX8vaz32GprcqHSNGfM0IsmTmRR0W9+E254RIRf8wEATKCZAgAYQDMFADAQ0TXTlE+O0ohv8euwI154QS9aelKMa1eXqiVZB3fysJnvTR4NcXFEfj/PPdZ5pjmrxot5wWef6UXS9aZ587wHZyUjQ74I7LEA94Ey+ebEwYP6YZ46ujDSkZlKuFxB+euEfdU9rmdOPy1fG23Zf0StSR05lIe9YvM9pTUpk04MeZDlI04rCyMTUe5728X8g/787+nxvdWrefjxx+EHaKSzk+jCBZ4r9zg95QT197JMuJza+ua+m/p78c0UAMAAmikAgAE0UwAAA2imAAAG0EwBAAygmQIAGIhoalTzsHvowEb+SNe4XcL0kxtF8pSm48f1klETp7DsWvqKsOOzkJV5jWY+0MJf8Jq2VFQkxp7/Ux0+zLOrVz3HZuXz1jyaeZy/Z68u0J/NrlDer7w8jwNl3B7hyIx1dsqPLXtttq48api6S59qVHv4U5Z1TZA/E9ZSOhtpRJUwdae6Wi96800x/l7yJr1myxaexWj6F9EXp/PKKzxfvlyveXHXYDHXnowmIhpRtoZlKY2Xw4zuC/hmCgBgAM0UAMAAmikAgAE0UwAAA2imAAAGIrqb709zadwofsf3vCMsTnuddrd3QoXHatfEi3qRvnK4pbPBXjRtdirL3/jxv1Nr3uknL7Y7ISAvNExE1LF4GcvcN4UFXqLA51PWNJFmGFw3c2RAfuHzz9WaZ0/+g/JKjBZ0GTCAaP16nnusKHy3n9+ZJyJa5zFrYdyuzSyLb6gJOzwTaWni7emzQyapJYO3bhXzY6v0BT32CP+Ml67EhR+fkX79xPWpqbBQr1mwQM4PfeixaP73nuaZx64FX4ZvpgAABtBMAQAMoJkCABhAMwUAMIBmCgBgAM0UAMBARFOjTnzko4FDElnutQ9QKCTnyZnyIgRERFknhT1aPPYBt6TMNCHq21etmZD3iZgfm63vzz4mg2ft7eFGZyMQICopEV5oblNrtp++U8yrquSciOjJe+W9doQZLlHR3BZPh05nsXz1ap71WLVKzsdlntAPNGQIz5KSwg3PhusSdXWx2GObKwouPyDmhR7Tv6QZZtrPdjT0piaaQO+w/MUtE9SaIwXanlbfUWsO3buEZc3/6w9hx0eEb6YAACbQTAEADKCZAgAYQDMFADCAZgoAYMDnuu7N/2Gfr5qIzkVvOJ4Gua6r31I3gnOMia/DeeIcDX0VzjOiZgoAADL8mg8w7OY1AAAAMklEQVQAYADNFADAAJopAIABNFMAAANopgAABtBMAQAMoJkCABhAMwUAMIBmCgBg4P8CQryJtjd6YwwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f8947e11ef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_weights(weights=weights_conv2, input_channel=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于这些滤波是高维度的，很难理解它们是如何应用的。\n",
    "\n",
    "给第一个卷积层的输出加上这些滤波，得到下面的图像。\n",
    "\n",
    "这些图像被降采样至7 x 7的像素，即上一个卷积层输出的一半。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmQlNX1/k/PsMw4LLM0YVhmpllkj6iIC4q4oEbFJSmXinGJcamUJQlqViopNWWiCcZKKZUqt1CWUQo3NBFEUYkCFUF2EBQVZoad6RnZBhiGmff3R+pX5XnOlX7fnttv69fn899zuXf63n7f99DvueeekwiCQAghhHSMgnxPgBBC/i9AY0oIIR6gMSWEEA/QmBJCiAdoTAkhxAM0poQQ4gEaU0II8QCNKSGEeIDGlBBCPNApSueKioqgqqpKteEJqk6dIv3J0NTV1Uk6nU7k5I9/iWQyGaRSKdXW3t6udHNzsxmXSOipFRTY/6fwu+rcubPSW7dulaamprysETl69GjGv+NaI35X+L2IiKxatSodBEGvjB/QQZLJZFBTU5OTv51pnfX19bHcr65nsrW1VWmcq4i9dq5rmalPfX29NDY25nyNIiLl5eUZ1+laQxi6du2qND6nYa9lJMtXVVUl77zzjmrDBSWTSTMOL6br4mbizDPPjDwmG1KplCxbtky1HThwQOmlS5eacV26dFG6qKjI9Glra1O6d+/eSl9++eWR5potqVRKlixZcsw+X3zxRca/gzehiMjhw4eVxv8wRETKysrqMv5xD9TU1MgHH3yg2vBBcRn7MP8hHDp0SGn8ETF+/PhIc82Wqqoqefvtt1Xbjh07lD548KAZh/dn9+7dTZ/i4mKljzvuOKXPOeecKFPtEFVVVTJ37lzVtnPnTqW7detmxoW53pl+PIW1PXzNJ4QQD9CYEkKIByK95h85ckRqa2tV22uvvab0iy++aMbhT+3S0lLT55577lF6zJgx5rPjoLW1VbZv367a1q9fr/S//vUvMw59jNu2bTN98NXq5ptvVhpfkeOksLBQadcrE74iodtCROTdd99VetiwYR5mlx0tLS3y6aefqrZZs2YpjS4cEZEBAwYoPXLkSNOnVy/t8h08eLDSrtfJXHD06FFJp9OqDdf43HPPmXH4HLvcMRMmTFD6/vvvVxpdHbkkCALzjOF3PHToUDMOx7jsiMtd9WXC+mL5y5QQQjxAY0oIIR6gMSWEEA9E8pkWFBQYX9qll16qtCukZvr06UoPHDjQ9Nm8ebPSPXv2VLqlpSXKVLPm8OHDsmHDBtWGPpNHH33UjFu3bp3SixYtMn1OPvlkpTEGEkNRcgn6SJcvX660K3wNw97+/e9/mz5z5sxR+oEHHsh2ih3m6NGj0tTUpNowTOixxx4z4zDc76STTjJ9TjzxRKXvvvtupePyfxcVFRm/NH7nrmuwdu1apefNm2f67Nu3T2n0q8blFxb5n39+7969qg398+jTFbH7G664Y/Qf79mzx3x2GPjLlBBCPEBjSgghHqAxJYQQD9CYEkKIByJtQHXu3FkqKytVW1lZmdLXXnutGTdp0iSlL7rooigfKyIiPXr0iDwmG4qLi83mQkVFRcZxl112mdJXXHGF6YNB+o2NjVnMsOMEQWCCl/FstiuIG4PZH3zwQdPnhBNOOKaOk5KSEjn11FNVGx4GmTZtmhmHZ9vxTLqI3ZzBwG/X95cL2trazFzwIMLGjRvNODxIc/bZZ5s+o0aNUhoPnZSUlESaa0coLi42hyeGDBmi9MSJE804zH/huh8zbVJyA4oQQmKExpQQQjxAY0oIIR6IHLSPfpP9+/cr3dDQYMZhUDH6pERs8pNMyQdyRadOnaS8vFy1oX/pjjvuMOMw8Hfq1Kmmz+7du5VGn1OcQdAI+sJd3//vfvc7pTGfpIjIs88+q3RYf1OuwO8U14VJQkT+l6T7y6BvTsTer+hzzlWSdKSgoMCsCYPOXb5NPFSAhzhE7CESPMyAz0UuCYLA3EsYxL9gwQIzDv2+d955p+mD9givpeu7ccFfpoQQ4gEaU0II8QCNKSGEeIDGlBBCPNBhLzlmWEKnsIh1VLsqX36dnN1IfX290i+88ILpgxmhXKADHTNwhXV054JMGzUiIn/84x+Vdm3MYPB3mCqncYIbpB999JHpU11drXSYzaR83p8IBu27Mq7hmlzXMq7qFtmC19I131tuuUVp1+Y3Xjssjhl2Y5i/TAkhxAM0poQQ4gEaU0II8UCHfaboT3Bl0ccAYVfSEvSR5hP0obz66qtKX3jhhWbMzJkzlcYKpyI26QL6SOMM2sc1YqDyn//8ZzMGqz/Onj3b9HFl6P86gRnmMUBfxCZ0wUQiIvZa5vPABVaCwEMyYatrIvhM5nONiUTCPC+4Lky6JCKyePFipV17G3369FEa72H6TAkhJEZoTAkhxAM0poQQ4gEaU0II8UAiSrBxIpFoEJG63E3nmNQEQdAr1x/CNcbCt2GdXKNHvgnrjGRMCSGEuOFrPiGEeIDGlBBCPEBjSgghHqAxJYQQD0Q6TppMJoNUKqXa8Kio6zghHvtypWXLlAJuy5Yt0tjYmPPzbOXl5UFVVZVqwzVmezwPa6mjrqurk3Q6nZc14tFBVzozTCHoOmaH6ctcqfzWrFmTjmMXuLy8POjXr59qww1XV317vL6uTdpM90BtbW0s19L1TOK1c60Rr12Y5xaJa40iIqWlpQHWKcP69q77EdflWhPWyMI6d/X19aHWGcmYplIpWbp0qWrbuHGj0rhAEZu3EwvLiWTOr3jeeedFmWrWVFVVyZtvvqnaMOclrkfEXiRXIbm+ffsqjWeCTz/99EhzzZaqqiqZN2+eatu2bZvSW7ZsMeOweKLrIUUjffzxx5s+lZWVsYS49OvXz+RVQEODxlbE/gfgMjSYfxc55ZRTwk6zQ6RSKVm2bJlqw/y7aIRE7H96zc3Nps9xxx2nNH4Pp512WqS5doTKykr5xz/+odpw3bgmEftDyPWf+xlnnKH04MGDlcYcvV8FX/MJIcQDNKaEEOKBSK/5Bw8elJUrV6o2TGHmelXFsgmuOt6YAg5r18dVh/zIkSNSW1ur2jZt2qS06zUf5+96zcfvBn1xcaU469y5s3ExoA7zmuoqUYPrjuu6uWhvbzclPHCdrnSQ3yRaWlrk888/V22o0fXiYufOnaZt0KBBSuezrE5xcbEMHz5ctY0bNy7jOHzNd60B3VV4X4c92MRfpoQQ4gEaU0II8QCNKSGEeCCSQ+vIkSMmhAbDJ9C/KCIybdo0pdHvKiKyatUqpV9//XWlXf65XNC1a1cTlvXxxx8rjaUQREReeuklpXft2mX6YFlsLH3h8rPmgr1798obb7yh2h555BGl3377bTMOfcX33Xef6XPttdcqXVpamuUsO87evXtNCBiGvaXTaTMOS5usXr3a9DnhhBM8zLDjtLa2mvLFGP716aefmnEzZsxQ2hWu+PTTT3uYoR+am5vlww8/VG0PP/yw0gsXLjTjMFwKy5WLWJ8olq1Bv/tXwV+mhBDiARpTQgjxAI0pIYR4gMaUEEI8EGkDqkePHjJx4kTVhhtQLvbs2aP0z372M9Nn1qxZ5rO+TFybM4WFheazL7jgAqVdm0sY5D5p0iTTB4On8buLKyjaFbR/4YUXKu1yuuMGlOt8Oh54yOcGVHFxsYwcOVK14Ubm7Nmzzbjq6mqlw9zj+H25zvPngi5dupj7qn///kq7zua/++67Sk+ZMsX/5DxSWFgoZWVlqg2D9keNGmXG4UEZ12GUv/3tb0r/6Ec/Utp1yMgFf5kSQogHaEwJIcQDNKaEEOKBSD7TRCJhkgJgUmFX/krMr4j+GhGR6dOnK33++ecrHVdCikQiYRKOHDp0SGnXoQPMVYp+KxHre8HvLq5KscXFxca/hH7CO+64w4zD+Tc0NJg+eGjjO9/5TrbT7DDdunWTs846S7X16qVzUuO/i1g/GybdERFpampSGhO6xHUtXf5vfEZHjBhhxmFikx/+8If+J+eRoqIiGTp0qGrDJCau/Lp4LV15kfEewL8TNhk8f5kSQogHaEwJIcQDNKaEEOIBGlNCCPFAh9Ogv/fee0rX1dlaaXPmzFH6iSeeMH1wowI3nPKZ5RsLd7kc0lh0y1VsLK6DB9mAgelYodEFBuiL2GB1DLTON3iABDekRNwbTghW2O3Zs6fS+bxf16xZozRmPRMReeaZZyL/3XxtmH4VuOmLlQFERH75y18qjZvJIiJTp05VGje2mGmfEEJihMaUEEI8QGNKCCEe6LDPFP2JruBf9Mft37/f9Lnxxhv1xCAIOq7KnSLWJzpmzBilMeO3iMi6deuUdn0PmYJ/41qj62BCpkoH/3/clxk9erTp40rwki8SiYTxXa5fv17p//73v2bcAw88oHRFRYXpgz5l9KvF6U/Ez8Jn8qKLLjJj8HlzgRn780lBQYHJmt/c3Kz0bbfdZsY99dRTSmPVDxG7l4H3TNjnkr9MCSHEAzSmhBDiARpTQgjxAI0pIYR4IBHFUZ5IJBpExEblx0NNEAQ2wtozXGMsfBvWyTV65JuwzkjGlBBCiBu+5hNCiAdoTAkhxAM0poQQ4gEaU0II8UCk46TJZDJIpVLH7OPa0MLjWK6a4jgOx9TV1UljY2POz1tWVFQEWIccU4+5joWGOXKGtWVwTG1traTT6ZyvMZlMBjU1Nbn+mK9kxYoV6Th2gcvKygJM09bY2Ki0614sLi7O+LexDx6Zrq+vj+1aZnomswW/G7zv47pfRURKS0uDyspK1YbHXV3PIK7B1QePBuOx1S1btoSyPZGMaSqVMud+ETQ8IvasK56pFbH5IfFs/oQJE8JOs0NUVVWZgn9bt25VGh8cEZGuXbsq7bpovXv3VhqN6ymnnBJprtlSU1NjzqTjg+L6TxHPMIcpNPYVxiqWEJe+ffvKzJkzVduzzz6rNOauFBEZOXKk0q51Dhs2TOmxY8cqfeaZZ0aaa7aEeSZd1wDvT9f1bmlpURqNjCtnb66orKw0eZC3bdumNNoMEZGDBw8q7cozi9cSi0teeOGFoebI13xCCPEAjSkhhHgg0mt+W1ubKfuwevVqpV11yPE1qXv37hk/C90FcaWnC4LAvPph3fHdu3ebcfg60a1bN9PHVdc7H7S0tMjnn3+u2vbu3au0yw+HPquvOwcOHJAPPvhAtb388stKu177pk+fnvFv//Wvf1Uay9aErbWeC/D110W/fv2Urq+vN32ampqUPumkkzo2sQ5w+PBh2bhxo2pDm+Byk+F97fKH4zh8/sOWoOEvU0II8QCNKSGEeIDGlBBCPBC5bAn6gnbt2qU0lnwQseWfZ8yYYfpgaAaGZcSVkKVz586mhCz6ajB0QsSGZbhKHD///PNKX3DBBUpjeFiuOHz4sHz66aeq7b777lN65cqVZhz6TM8991zT55JLLlF63LhxWc6y45SVlclVV12l2m6//faM47Asjavs9Xe/+12lly5dqrQr/C8u8NkZOHBgxjEbNmwwbRjul09KSkrk1FNPVW2bNm1SGu9hEZFXX31V6ffff9/0QX8x+lXDlmjnL1NCCPEAjSkhhHiAxpQQQjxAY0oIIR6ItAFVUFBgnLN4Phc3b0REfvKTnyjtCv7GeuYlJSVKx7U509raKjt37lRtuMmGAfoiNsj9uuuuM30uvvhipXv10rk+XGeLc0Fpaalceumlqg03k1xB59iGgf8uXBtxcYKbBwsWLFAacxSIiKxatUrphx9+2PS59957lcbnIK771UWYxCfbt29XesqUKabPxx9/7GtKHaZz587Gbrg2BpHy8nKlx4wZY/q89957SqMNc+UbccFfpoQQ4gEaU0II8QCNKSGEeCCykw6TC2AAOwbSith8gH/5y19MH/ThoD8krkQnLt8M+kwOHTpkxmHO0yVLlpg+Tz75pNL5qgzb3t5uEutirkpXwDZe2y+++ML0wQQv+fSZFhQUmNyzmLQCD5SIiAwaNEjp119/3fRZu3at0r///e+V7tGjR6S5+iRMkhVMdBImiUk+KxknEgmTKGjOnDlK79u3z4ybO3eu0ni4QsQecujfv7/SYRMU8ZcpIYR4gMaUEEI8QGNKCCEeoDElhBAPdDhKHDeGcJNFROT8889XGjebROzGAB4OyGfmcgymHzJkiOmDGy+jRo0yfUpLS5XOp0MfSafTSmOmLBHriHc55ocOHao0bnTlGzyA0bNnT9PnwQcfVNqVHWvy5MlKu4rWfV1wZXLDYPYVK1bENR1vYHYvXJOIPUyBBRZd47K1PfxlSgghHqAxJYQQD9CYEkKIByL7TDFxBFbyc1XlxOBu9M+JiAwfPjzqVGKjT58+Sl9xxRWmD2ZWx6qYLjAZRj59qJhZf9asWaYPXtsbb7zR9IkrWUsYEomEOYxw9dVXH1OLWN+2y/995ZVXKv119pliIg8RkV/96leR/87XLWj/pptuUnrw4MFm3COPPKK06zDF9773PaXRRxr2wBB/mRJCiAdoTAkhxAM0poQQ4gEaU0II8UAiilM5kUg0iIhNsxMPNUEQ9MrcrWNwjbHwbVgn1+iRb8I6IxlTQgghbviaTwghHqAxJYQQD9CYEkKIB2hMCSHEAzSmhBDigUgHqSsqKgIsoJfNOdZsIgjq6uoknU7nvKpeMpkMUqnUMfu4cnTiul05EPH8Nvapr6+XxsbGnK/RdR3xmhw4cMCMy2aNeDZeRGT9+vXpOEJqysvLg6qqKtV28OBBpbMt1JipAGFc96vrWiJh8nGGeSbxu4prjSLua9nU1KQ0XlsRu3bMmyxirx3muN26das0NTVlXGckY1pdXS0LFixQbZhI1fXw4AOGyVJc4IU7/fTTw06zQ6RSKVPBEC9IbW2tGYdJGLAqpohNCoMVWM8777woU82a6upqk/wCKzQuXLjQjMNrW1JSYvpg5Vas9CgiMnr06FjiBauqquSNN95QbZgEGa+BiDUsrvu1pqZGaUyyEdf96rqWiOuZxHsak+64QEM0bty4EDP0Q1VVlbz55puq7bnnnlMak/WI2HvUlQx8wIABSl988cVKX3755aHmyNd8QgjxAI0pIYR4INJrfltbm+zdu1e1Pf3000q7fG39+vVT2uWTnDBhgtLZ+rI6ShAExie6aNEipZPJpBmH7o7NmzebPvv371ca6yWFedXyQVNTk/zzn/9UbcuXL1c6TH5OvBdErP/JlfM0LlpaWmTTpk2qDdeF9clErCvAdU8PGzZMabxf47p/29raZN++faoNfYeuPJ/4mu9yBWQizmf0yJEjsmXLFtW2ePFipTFvsojI+vXrld69e7fpc9ZZZyl94oknKt3a2hpqjvxlSgghHqAxJYQQD9CYEkKIByL5TFtaWuSzzz5TbR999JHSc+fONeOwVjmGIoiI3H///UpjaElc9dddvhn0QaFPxcXatWtN21VXXXXMMehvzBXdu3c3teCvueYapV1+4Z07dyrtCsm59tprlQ7rb8oFxx13nJx00kmqDX1o6DsWEXniiSeUvuuuu0yffPn0kebmZlmyZIlqe/zxx5WeP3++Gde9e3elp06davrcfvvtSmPYW5zXtqSkRMaMGaPaXnnllZx8FtoaV5ijC/4yJYQQD9CYEkKIB2hMCSHEAzSmhBDigUgbUN26dZPx48erNgx4dW2ioKMaz7GLiHz88cdKh0nOkAs6d+5sDhmUlpYqvWfPHjNuypQpSs+aNcv0wQ0odOiHCZT3QVFRkQwfPly14cEE19l83KTo1cvmKsGNLdy0ihs8Z9/Y2Kg05poQsd/FSy+9lPFztm3bpnRcmzM9evSQiRMnqjZ8Rl05FFxtSF2dTqHgymMQF62trbJjxw7VhkH6b731lhm3ceNGpc844wzTBzdf8QBLc3NzqDnylykhhHiAxpQQQjxAY0oIIR6I5DMNgsDkdkSfFCZsdfVx+VWxDYPGXX7WXFBQUGCSlmDQ7vbt2804DCCeNGmS6YP5TFHHVXa7vb3dHETAxB2uwxeTJ09W+qGHHjJ9MCjelT8yLhKJhLlv0B/umt9//vMfpXv37m364MEO9CfGGdSPn4VzCeMfdd3TmEAFv7tOnSKZjw5RWFhorhVe227duplxuKdz/fXXmz54qAGvtyuhtAv+MiWEEA/QmBJCiAdoTAkhxAM0poQQ4oEOe5DT6bTSa9asMX2GDBmitMshjkH6uAmUzyw9uFF0ww03mD6YRf/FF180fTAzfVlZmdJhHd254Pnnn1e6oaHB9MGMUPX19aYPHjzo06ePh9n5Y926dUq7NlGw6oMrAB83YnHDNM7NGQQ3c13zx80bvH9FbGYpzKYU14bpV4G2x7WZiPcsHg5yMWjQIKXDZnPjL1NCCPEAjSkhhHiAxpQQQjzQYccOVvvDQF8RkU8++URpDBAXEamsrFQ6X/4Y18EE9LO5fJv4PbgqJebTJ/plEomEmQtWsMSEJSIi5eXlSmPiCRGR6upqpbOpeumLRCJhfJd4mOL73/++Gbds2TKlXcHgWGE3Xz79RCJhPvudd95ResOGDWYcJqnp37+/6TN27Fil40rE4yKRSJh7acSIEcfULtDP6hqHh3TCJl3iL1NCCPEAjSkhhHiAxpQQQjxAY0oIIR5IRNnoSSQSDSJSl7FjbqgJgsCmdvcM1xgL34Z1co0e+SasM5IxJYQQ4oav+YQQ4gEaU0II8QCNKSGEeIDGlBBCPBDpOGkymQzwGF1c1NbWSjqdzvmZvWQyGdTU1Ki2bI4KhtnYw78b1xorKioCPPZ56NAhpV1p23C+ruOxePTOlb5s9erV6Th2gb8N96vrWuK1cx0DzVTLTcSm6cPaUnV1dbGsUcR9LXENYY5rh3kusU99fX2odUYypqlUypxbzhX4RZ122mmxfG5NTY0sXrxYteFN5AJv2KNHj2Ycg+fGTz311BAz7DjV1dXy3nvvqTbMQ7tt2zYzDnPMuvJH4rlm/I9JRKR3796xhLikUilZunSpagt7zrqjnHLKKbF8TnV1tSkAiMXxmpubzTjMoeG6X/v27as05m8488wzo0y1Q7iuJeYH7tGjhxmHPwBcPxIQ/C7Gjx8fao58zSeEEA/QmBJCiAcivea3t7ebV4bZs2cr7Uo9h+m9sJa1iE0Jlq90dQ0NDfL444+rtj/96U9KY7o9EZtC8Be/+IXpc88993iYYcdpbW2VnTt3qrYPPvhAaXR1iNjXKpcvefjw4UrfdNNN2U6zwxw8eFBWrFih2rA8C7qTRGx6vcsuu8z0wVfefOGqJ+9yv+SCONMOtre3y8GDB1XbQw89pPTMmTPNOHTRYQkiEZEpU6YoffPNNysd9mATf5kSQogHaEwJIcQDNKaEEOKBSD7TIAhMaAH6E9C/KCLGPzdw4EDT55ZbblH6kksuURr9JbmiZ8+e5rPRfztr1iwzDkNu0L8oYv2SmeIDc8WRI0dk8+bNqm306NFKu3y+e/bsUdrlHx8wYIDS+Uyk07VrV+PbxPk98sgjZlxtba3Sjz32mOmDfnUMAXP55nJBOp2WGTNmqLZnnnlGaQyDE7Hl1u+++27T56677lK6paVF6bjuV5H/PV84Z/RlY6y0iMjq1auVfv/9902frVu3Ko3PBq77K+cYqhchhJBjQmNKCCEeoDElhBAP0JgSQogHIm1AFRYWSmlpqWq74YYblL7uuuvMOHQMu2qp19Ud+7h2XEH8Xbp0MZsJd9555zF1WFauXKk01qHHs/q5olOnTtK7d2/VhglJ1q5da8ZhAL4rIP/nP/+50q5z4XHhul8nT56s9O23327G4XVw3XuYiwE3KVz3eC4oKSkxeQAqKiqUvvLKK804PMc+duxY0wfzM+A94kqgkiva29uNHcFNU9dhIHzmMN+AiJhnAdeF+Sa+Cv4yJYQQD9CYEkKIB2hMCSHEAx120mGiCFfgLPqTDhw4YPpgAPCgQYOUjssH5YslS5aYNswHikkY4kocUVxcLCNGjFBteMjg1ltvNePQ//T666+bPpgnM87A7jDgvee6F9FH5jp4gPcjXtu48qYWFRXJ8ccfr9rQ5z9p0iQzDufX0NBg+uzYsUPpPn36KI3Jo3NJIpEwvmv0bb/22mtmXGNjo9Ku640+U7RXYX3D/GVKCCEeoDElhBAP0JgSQogHaEwJIcQDHd6Aqq+vV3r58uWmDwbKYrC6iM0OHpcDPwyYJcq1UdS9e3elXVU54yqYlw0ffvih0q7sOkOHDlXaFQCdTqeV7tatm4fZ+QMLB27YsMH0wUJxrnXihtPXCdzYDPMs4WaT6+/ka8P0q8ANqHnz5pk+uJl0/fXXZ/y72R5G+PpYLEII+QZDY0oIIR6gMSWEEA9E9plikP7Ro0eVxqz6IiKrVq1S2lXZ8eqrrz7m58QJ+oKqqqqUxuBlEZH58+crPWrUqIx/N5/gXDC7/FVXXWXGPPnkk0q7AvLRr+byHccJ+r/KysqU3rVrlxkzZ84cpa+44grTB33keL/ms8LAU089pfSyZctMH3wGzz777Ix98rmmIAjMtUR98sknm3GYNd91+OfIkSPms7KBv0wJIcQDNKaEEOIBGlNCCPEAjSkhhHggEcXZmkgkGkTk2Cnxc0dNEAS9cv0hXGMsfBvWyTV65JuwzkjGlBBCiBu+5hNCiAdoTAkhxAM0poQQ4gEaU0II8UCk46TJZDKorq6O/CFhjtrh8Uas91JXVyeNjY05P4+ZTCaDVCqVk7+Nx99wzXV1dZJOp2NZI15HPFLnSkOGKc9cR35xTa7jeytXrkzHsQtcVlYW9OvXT7XhvYfHX0XCpWDDtHb4d+vr62O5lhUVFRmfyTAp+LLZiI7rmRQRKS8vD/r376/ampublcb709XmWifWssLvq76+PtQ6IxnT6upqWbhwoWrLZCBEbBEr15lufOiwqNmECROiTDVrUqmU8yzzl3EZETT+LvDi47n10047LcQMO051dbXJV7pt2zalca4iIr16afuHRfhE7HXEB0BEpKSkJJYQl379+skrr7yi2g4fPqz0kCFDzDjs47qn8drhPTF+/PhIc82W6upqWbBggWpDY+DKj4BGBXNsuMDW9+/LAAAE70lEQVQxcT2TIv+7j+bOnavaFi9erDQWxhMRSSaTSrtsD44rKSlR+pxzzgk1R77mE0KIB2hMCSHEA5Fe89va2mTPnj2qDX1OFRUVZlw2pSvwlSKfZUyw9rZrjYjrNfmLL75QGl+B40rRd+jQIVm3bp1qw7m5XlPxOmIaOhHrf0J3TZx06dLFpEt0zRlx+VEz4brecVBYWCilpaXH7IPpFUWsO2bLli2mD5aywTR9WBIklzQ3N5vXekyv57IRn3/+udJbt241fW677Tal0c8a9rnkL1NCCPEAjSkhhHiAxpQQQjwQyWfa2toqu3fvVm0zZsxQGkNRRGzYzciRI02f3/zmN0pjCQIMV8kVLS0tsmnTJtWGvprjjz/ejDv99NOV/vWvf2363HrrrUq7wobioEuXLlJTU6PacP5hyOSrE4nvurlobm6WFStWqLZnnnlG6UWLFplx6D8eO3as6XPllVcqPWjQIPPZcbB792557LHHVNvMmTOVdpUSGj16tNKuZ3L79u3HHBNnaaGioiIZPny4arvggguUdpWQx7WvXbvW9MFS3hg+RZ8pIYTECI0pIYR4gMaUEEI8QGNKCCEeiLQBVVxcbBzVU6dOVfrGG2804zBQ1hX0jueHwyTeyAWFhYXSs2dP1Xbeeecp7TqEMG3aNKWx9rqIyPTp0z3MsOO0tbWZc/WPPvqo0p988okZN3v2bKUx2FlE5A9/+IPSuJkXJwUFBea+wo2jH/zgB2YcbrTgho6ISI8ePZTGZCOuBC+5oLS01Kxp0qRJSuMGsIjI4MGDlb744otNn3PPPVfpAQMGKB3XGkX+twmEwfSYX8L1zD311FNKh0nognk2uAFFCCExQmNKCCEeoDElhBAPRPKZBkFgAnUxKQTmvBQRGTZsmNKYfMAFBkFnk3wiG1w+0zCJTf7+978rPXHixIxjMDg4rkqxXbp0EUya/Nvf/lbpWbNmmXGXXXaZ0jfffLPp89ZbbyntyjEZF0VFRTJixAjVhtfS5ffDBBquoPdx48YpjQlVMOFLrigsLDT+W0xActZZZ5lx2Oa6Tj/+8Y+VLi4uVjrO5EOdOnWSyspK1Yb+8J/+9KdmXJjnEMHvL+x+DX+ZEkKIB2hMCSHEAzSmhBDiARpTQgjxQKQNKBeYed9VWG7fvn3H1CJ2YyDbbNdxcMkll5g2zGb+5JNPxjQbP6xZs0ZpV8XLiy66SOnly5dn/LsY6J1v8MCFa3MRDx64iu4NHDhQaSxIF9dmogusbnDvvfeaPpgJ7dlnnzV9MAsTbqrl+5nEwxS4QSUiMn/+/Ix/x2WPvkzYa8lfpoQQ4gEaU0II8QCNKSGEeCCSzzSRSBg/yf79+5V+4YUXzLh0Oq30NddcY/qgDyqf/hj0f2F1AdehAwx6d5GvIH0X6JNGfxj6R0VEduzYoTRWbRWxPtJ8Vid13a94LbHipojIZ599pvTLL79s+pSUlHiYYW7ApDWuBDtYWQGz2IvYIH38LuN8RhOJhNmPwTm7Ep2EAZ/DbH3D/GVKCCEeoDElhBAP0JgSQogHaEwJIcQDiSibIIlEokFE6nI3nWNSEwSBTUnlGa4xFr4N6+QaPfJNWGckY0oIIcQNX/MJIcQDNKaEEOIBGlNCCPEAjSkhhHiAxpQQQjxAY0oIIR6gMSWEEA/QmBJCiAdoTAkhxAP/D0POzyBiD0ddAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f887be39d30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_layer(layer=layer_conv2, image=image1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这是给第二张图像加上滤波权重的结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmUVMX1x2/PAjODDMs0hHV6AEFFBFFZFEUF4wIuUVHBuAWj0biG6FGM0bjkxKORxSVH45aouAbxGPejKAIuIIKgAqIwM4AIzAAKM2wz/X5/GM/xfm9J9+uufoM/v5//vjVV3VX9Xt/pd+vWvbEgCIQQQkh25DX1BAgh5P8DNKaEEOIBGlNCCPEAjSkhhHiAxpQQQjxAY0oIIR6gMSWEEA/QmBJCiAdoTAkhxAMFYTrH4/EgkUiotu3btyudTCbNuOLiYqVjsVjK98KTWVVVVVJTU5N6YJbE4/GgoqJil322bdtm2iorK5XeuXOn6dOqVSulO3TooPSqVatkw4YNkawRryN+3nl5qf/P1tfXmza8/njtRUQWLFhQEwRBu3Tmmg2udSKZ3Isidp34eUV5v+IaGxoalP7mm2/MuNraWqVdn0M8Hlca79/q6mqpra3N+Rr/NxezTvweoi0SsdelpKTE9Cko0GYwU9sTypgmEgmZPXu2eaMf4vqC9e7dW+nmzZunfK8dO3YoffDBB6c7zayoqKiQOXPmqDa8IJ9//rkZN3bsWKVXrlxp+pxwwglKX3XVVUqfeOKJoeaaKYlEQj744APVhsbfZQSRjz76yLRt2bJF6f3228/0adu2bZVpzAGu+xWvZbNmzVK+DhonEXuf4+sMGTIk3WlmhWuNaChfffVVM+7xxx9X2vXP8ze/+Y3SI0eOVHrYsGGh5poNrnUuW7Zsl1pEpLS0VOl+/fqZPu3a6f/raJQPOeSQtObIx3xCCPEAjSkhhHgg1GN+Y2Ojebx59tlnld5jjz3MuIEDByrtemy65pprlMZH4MbGxjBTzQp85Jk1a5bSp556qhnzi1/8QukzzzzT9EFf7Jdffqm0y+eTC4IgMG6UdHj++eeVxkd6EevHat++fej38Ul+fr7SdXV1Srsexz/88EOle/XqZfqcd955Sv/+979X2rV3kAuSyaT5zBctWqT0G2+8Yca9/fbbShcVFZk+ffr0URpdNpncQ5kSBIH5TKdPn660ay9j8ODBSs+YMcP0GTVqlNLosknHry7CX6aEEOIFGlNCCPEAjSkhhHgglM90+/btsnTpUtWGMVlHH320GXfllVemfG30S1ZXVysdpX8Gufrqq5Vu3bq16TNt2jSlu3fvbvpMmTJFafQBRVX1IBaLGb8w+sw6depkxuG6H330UdPns88+U3rDhg2ZTtML6O/65JNPlMaQNhGRiRMnKu0Kc0O/5OLFi5V2+e9ygctnir75v/71r2bck08+mfK1n376aaU3b96sdJT7GMlk0vjo586dq/See+5pxt17771K4x6PiA2pcr1OOvCXKSGEeIDGlBBCPEBjSgghHqAxJYQQD4QO2v/2229VGyZDwHP4IiLr1q1T2nUO+NBDD1V6+fLlSmPwda7Ytm2bLFmyRLV169ZN6RdffNGMw80l/JxERE477TSlv/rqK6XTDQ7OllgsJoWFhbvss2bNGtN24YUXKn3QQQeZPlOnTlW6Z8+eGczQH7iph4HnGNQtYjdW8N509cFNoKiC9vPz86Vly5aqDTcKXddy06ZNSru+k5hYBHNqpJMMxxeNjY0mYUubNm2UPumkk8y4k08+WemamhrTB9eBB5PSvZb8ZUoIIR6gMSWEEA/QmBJCiAdC+Uzz8vKkRYsWqg1z/WGSCBGb89SVKATBhClR+Uzr6+tlwYIFqg39fmVlZWYcBu1jIgkRkeuvv15pTI7iSlybK/DzxEMR48ePN2NGjx6t9AsvvGD6YM7Jc845J9MpZk0ymZStW7eqNvSP4bUWsclZysvLTR+8dphgOCr/d15ensk9i7lpcf9BxB5ecOWdPfDAA5VG32E6eYl9EYvFUiYgwYM+ItaOuPzfnTt3VhqT4dBnSgghEUJjSgghHqAxJYQQD9CYEkKIB0JtQLVo0UIGDBig2tAJjdnjRURuv/12pV2Fr4477jilsfAVOvhzRX5+vqnCuPfee6cc9/XXXyt9+OGHpxyTaXBwLsD3xkzyIjbj/Lhx40wfdPg3ZdB+fn6+mQ9uQM2fP9+MmzBhgtJYBULEBoMj6RQk9AUeTMDvpCuD2Xvvvaf0qlWrTB/8rjfVprCISGFhodn0w/efN2+eGYdVPlwbopgNCzfw0s3mxl+mhBDiARpTQgjxAI0pIYR4ILQjEv0HmPChR48eZswzzzwT9m2ajJYtW8rQoUNV24oVK5R2+VDefPNNpV2Z6jGxSSp/Xq4IgsBUiMX3bteunRn37rvvKj1o0CDT5+yzz1baVYk2KoIgMPcnrvOCCy4w48aMGaO0q+Iurgt9+lH6ExG8PzFRj4g9QJIOTenTdyXnwQrG//73v804zMaPCV5EbMIeDOJP91rylykhhHiAxpQQQjxAY0oIIR6gMSWEEA/EwpQXjsVi60WkKmXH3JAIgsDuiniGa4yEn8M6uUaP/BTWGcqYEkIIccPHfEII8QCNKSGEeIDGlBBCPEBjSgghHgh1nDQejwdYSxvrsKxbt86M2759u9Ku43lYWwqPjlVVVUlNTU3OC+vE4/GgoqJCtWEdofXr15tx+DlgTW8R97p/SGVlZWRrxOuINaBQi9ja8K7jr5h6rqioyPSZP39+TRS7wGVlZQHWb8La63hvitg1YB36dIjyWuL9iuCaRez3y5UyMFUdq6jWKPLdOvFaYuo81z2Lx4ld30Fsw2Oz1dXVaa0zlDFNJBImDyIWubrnnnvMOMxfikX4RESGDBmidIcOHZR2nQPPBRUVFaYoIBYfu++++8w4/BxGjRpl+uAa0Bhh/shc4bqOWPTQVZzsiy++UNr1Bezdu7fSe+21l+lTWloaSYhLeXm5vPXWW6rt1VdfVRrXJCLSt29fpU888cTQ743nvXNFRUWFzJkzR7XhffXKK6+YcVg0sE+fPqZPqoJ5Ua1R5LtrOXPmTNU2ffp0pV05WfEs/mGHHWb6oD3CHw2uInwu+JhPCCEeoDElhBAPhE7Bh36U3/3ud0r/85//NGPuvPNOpffff3/TBx8r27Ztq3RUhwsaGxuNj+mWW25RGn01IiIdO3ZU+qyzzjJ9sM48lkOJao3JZNI8ymCpluHDh5txrrZUfPvtt6HH+GLr1q3GRbNy5UqlL7/8cjOutLQ0p/PyiSvN4B133KH0jBkzzLgHHnhA6TVr1pg+qXyxUdLQ0CC1tbWqDcuYnHDCCSlf5/XXXzdtn3/+udK47lS+4+/hL1NCCPEAjSkhhHiAxpQQQjwQymfqKnfx4IMPKu0qkbDvvvsq/dxzz5k+8XhcafRBYvnVXLFt2zZZsmSJasPwCle4D4Z/YakTkdRlStL1zWRLXl6eCWtCP+G0adPMuFNOOUVpVyztO++8o7Qr5CYqtm7dKp9++qlqQ7+0q4T42LFjlUafmojIrFmzPMwwe2KxmCmrce211yo9ePBgMw7L0rz//vumz+7kM3Xds1jG+YknnjDjfv3rXyvtKsGO4XIYr5puuRb+MiWEEA/QmBJCiAdoTAkhxAM0poQQ4oFQG1B1dXXmTPc+++yjNAZJi4iMHz9e6S1btpg+48aNUzqdWvW5IJlMmsQmuLmE589FRK677jqlP/vsM9MHzw5jspGocNUgRye7q1b4yJEjlXZtbODGYVPiCtpHSkpKTNvXX3+ttKvW+u5CfX29LFiwQLVhcpkpU6aYcZhLAoP4RUSGDh3qYYZ+yMvLM8mQEMxRIGLzKjz77LOmD34WuBHsSujjnGNavQghhOwSGlNCCPEAjSkhhHgglM+0oaHBJEbGnIZr164141577TXzOgjmTnz77beVdiUXyQUFBQUmGB19Zq7kHUuXLlW6R48epg/6fDA4OCq/8I4dO6SyslK14TU54ogjzDj0HWEymu9f+4fg5xI1mARk9erVKcdgXtl0EmjU1dUpnW6gd7Y0Njaa+/OXv/yl0t27dzfjpk6dqnRTX6d0wM+0pqZGaVd+VTyA4bou6EfFwH68h34M/jIlhBAP0JgSQogHaEwJIcQDNKaEEOKBUBtQhYWF0rVrV9V2xhlnKL1x40YzrmXLlkpj5noRkb///e9Kl5WVKR1VRqXi4mLp16+fasNif65M7DjfI4880vTBDSYMnI9qjdu3bzcbUBjc3rNnTzPu+OOPV9q1YYZZ3VMFWueSZDJpqo9+9dVXSruKzd10002h3wuzmkW1megKZsdAddeGKW44uioO7E7s2LHDXLvFixcr7dpow+KII0aMMH2WL1+uNGaNc2UWc8FfpoQQ4gEaU0II8QCNKSGEeCCUz7SkpET69++v2rDKpcu/gElMXAkUsPIg9pk8eXKYqXoF3xt9NyLfJZz4IZgsRcRWHIjKR4q4ruPHH3+s9MMPP2zGjR49Wmmsripik6Gcd955Gc4ye9q3by+XXXaZarvxxhuVxuq6IraC6dlnn2364Oti0pqofKbFxcXmvsLv4IYNG8y4Qw45RGncC3ER1ZpcFBYWmmqkWAFh4cKFZtyjjz6qNCaFEbH2Cf3s6a6bv0wJIcQDNKaEEOIBGlNCCPEAjSkhhHggFsapHIvF1otIVe6ms0sSQRC0S90tO7jGSPg5rJNr9MhPYZ2hjCkhhBA3fMwnhBAP0JgSQogHaEwJIcQDNKaEEOKBUMdJ4/F4UFFRkaOpaHBjrKqqSmpqanJ+/tK1RjxehunWROzRUKzFLWJT7iGVlZWRrbG8vFy14fxzedR13rx5NVHsAsfj8QCPeWKtJkxFKGKvd/v27U0fPNqI1zvKa5nJdxJT8LnqsuGasAZYVGsUESkrKwvwyOuaNWuUdtWJ22OPPXapRb47Xv1D8HtaXV0ttbW1KdcZyphWVFTInDlzVBt+wOngKmqFr4M3NJ4lzhUVFRXy4YcfqrYVK1Yo7SrKhueh0ViJiHTq1GmX7+0qCJYLysvLZebMmaoNvzipDH+6uIqRFRQURBLikkgk5N1331VteP/+9re/NePwzDfm7BWx57nxeg8cODDUXDPFdb+mA57Xx+J0IiKdO3dWuqioSOlBgwaFft9M6dq1q7z55puq7dZbb1Ua/y5ic3wMHjzY9MFrhf8oXcUlXfAxnxBCPEBjSgghHgj1mO/im2++Ufq+++4zffBRatGiRaYP/pTGx8OoDhc0NjaaNWE6vUMPPTTl60yaNMm04ePDmDFjMpihH9Ctgo/1mJJPROS2225TumPHjqbPhAkTlM7Pz890ilnT2NhoSnZgWZXevXubcVjKxFUO46dQZ/57sCSQiHVLHXXUUaZPU5acQbZs2WKu3VNPPaU0lhsSEbnnnnuUdvmG8bPYtGlTRnPkL1NCCPEAjSkhhHiAxpQQQjwQ2meKvraLLrpIaVcc19NPP600hpWIiDz44INKn3XWWbt831wRBIHx16JfzRVesWXLFqWnTJli+nTo0EFpV4hYVOB7Y6nnN954w4xBH1U8Hjd9Lr30UqVd/saoCILA+MjQN3/99debcY888ojSWPpX5LvSwz+kqcqWpAOGEIlYnzjOf3djw4YNxo5gON+rr75qxmG8NIZ7idhwOYwrTtf28JcpIYR4gMaUEEI8QGNKCCEeoDElhBAPZB20/+WXXyp9+umnmz6jRo1S+pJLLjF91q5dm+1UvFBQUCBt2rTZZZ8PPvjAtN11111KuwKIZ82apTQG8UdFY2OjCUzGgwkDBgww495//32lmzdvbvps27ZN6abcZIvFYiYgu107nV9l+vTpZtz8+fOV/vOf/2z6tGrVSummOmTi4txzz1UaD6GI2A2n6667zvTBXBK4uRglDQ0Nsn79etU2ZMgQpV0HTRBX/gu8jzEvSLrXkr9MCSHEAzSmhBDiARpTQgjxQNY+U/SruBLVYhCsK6kC5h1ct26d0q4EBbkCA32XLVumNM5VxPqp0O8iYoOK00mYkgt27txpEuu+9tprSvfo0cOMw8Qs77zzjumDvte2bdtmOs2syc/PN/5v9JnefffdZhwm/3YFbVdXVyuN+UHxc4gS9FPff//9KcfggQwRkQsvvNDbnLKlqKjIHJ45/vjjlX7uuefMOEzW4spN+9FHHymN9yzuA/wY/GVKCCEeoDElhBAP0JgSQogHaEwJIcQD3jegXJsSmEnq4osvNn2Ki4uVxgzprsJsUYEZ5THDlYhIaWmp0j179jR90i3MlWsKCgrMRgwGRGMmHRGbCct16ACLrLk266IimUxKfX29asPrdOKJJ5px+NmUlZWZPpjBfcSIEZlO0zvnn3++0ocddljKMfvvv79pu/baa73NKVtat25trhXaDDygIWKrQ2Cgv4jIf/7zH6Xxs0jX9vCXKSGEeIDGlBBCPEBjSgghHgjlMw2CwGQYv+GGG5RG/4OIrUaKSSJErP/jgAMOUDrKTPu4Rgzibt26tRn3+uuvK33ggQeaPg888IDSeBAhquQYhYWFxg+Mhw4effRRM27evHlKY6ZzEZFu3bopXVRUlOk0syYvL88kscDPuFevXmYcVmpF/6iIyIoVK5TGBBpNWdkT5+Ly+WHV2GeeeSbl6zZl8paSkhJjEzCYHu9PVx9XYP/GjRuV7tu3r9Jom34M/jIlhBAP0JgSQogHaEwJIcQDNKaEEOKBWBinciwWWy8iVbmbzi5JBEHQLnW37OAaI+HnsE6u0SM/hXWGMqaEEELc8DGfEEI8QGNKCCEeoDElhBAP0JgSQogHaEwJIcQDoc7ml5WVBeXl5bvsg8XoRDI7V49RBlVVVVJTU2Nf3DPxeDxwFQVMBRZUcxUAxHPqmOe1uro6kjWmcx2/+eYb04ZFDl35IzFvgSsPw8KFC2uiCKmJx+NmnZs3b1a6qspG25SUlCiN1+l/r600fhaVlZWRXcsuXbqoNvwOur6TeDY/HZrqOyny3bVMJBKqDXNmYJFDEXsfu3JFYJ4KvGerq6ultrY25TpDGdPy8nKZMWOGasNKiK4vmOtmTAV+UIMHDw79GplQUVFhEiOnc3NidceamhrTZ88991Qaq5NGlUg5neuIlVRFRO666y6lMYmyiMivfvUrpY855hjTp3PnzpHEC5aXl8vMmTNV2/Tp05V2JSrH5MCHH3646YOJYdC4Dhw4MNRcM6VLly4myQ4aSkzcIuL+J5cKTAB08MEHh36NTEkkEjJ79mzVht+xSy65xIx74YUXlHb9UPrTn/6k9MiRI5UeNmxYWnPkYz4hhHiAxpQQQjwQ6jE/FosZ/+eWLVuUvv322824yZMnp3xtfBzDR2DXo3WuQN8QrtlVEwj9Vnfffbfpg4/xmGsxyjUiU6dOVfqWW24xfdC/uGzZMtMH60K5fK9R0dDQYHzZV155pdJYa0xE5MUXX8zpvHzi+k6iy8HFzTffrDT6iUVErrrqKqUxf21T3q8iIrNmzVJ606ZNps+NN96o9F/+8peUr1tbW6t0uqdE+cuUEEI8QGNKCCEeoDElhBAPhPKZJpNJ2b59u2pbsGCB0q7Qg5deeknpL774wvSpq6tTGv2JUWW3CoLAhGX98Y9/VBr9cCIiDz/8sNI9e/Y0fdAXg77XqHxQDQ0NZi5YJ+jOO+804zDOzxU+NWDAAKX79OmT6TSzpr6+XubOnavajjjiCKUfeughM27s2LFKP/LII6bP7pRtLZM4boytxO/17kYQBCZ2G2tCYbifi4ULF5o2DOfs3bv3Lv/+Y/CXKSGEeIDGlBBCPEBjSgghHqAxJYQQD4TagAqCwGwMde/eXekRI0aYcRi47TqvjedsMSAc3zdXNDQ0yMaNG1UbbrLh2W0RmzvgpptuMn1OPfVUpYcPH57pNLOisbHRBDij0x0PTYjYzRrcxBKx58DTdd7ngmQyac6T77PPPinHYXB6cXGx13n5JpPNsNNOO01pTFCzuxGLxaR58+aqrVevXinH4dl814GGvn37Zje5/8FfpoQQ4gEaU0II8QCNKSGEeCC0Qwt9YOhfev/9980YDPZ25VLE18UEKphvM1e4/IkY4OwK9Mag58cee8z0Ofroo5XGIOQoA8Hx80S/oOvz3rp1q9KuJCb4Ongdo6R58+bSo0cP1YY5ZV2g3zedPLN4z+AhiFxRX18v8+fPV21t27ZV2nVwIh0fKR6kwb0E9EfnklgslpH/HROau75jmG955cqVSqe7Tv4yJYQQD9CYEkKIB2hMCSHEAzSmhBDigVAe3YKCArN5hJnVMUhWROSCCy5QGjP3iIhceumlSrds2VLpTDLjZEJ+fr6UlpaqNiwc5nKEL1myROljjz3W9MFic7iBE9UmW3FxsdmUWLp0qdKujFDICSecYNo6dOigNG4AREmLFi1MZqH//ve/SmMguIi7SgKCmxK4Tsw8lisKCwvNBinSv39/07Z48WKlXQUr8fqefvrpSke5YerKGuUqFIhg4cN0xuABIlelYRf8ZUoIIR6gMSWEEA/QmBJCiAe8Z6HAZB4iIuXl5UqnkwylqKhIaTwckCsKCgqMH7hfv35Ko09FxGbWf/bZZ00frISJgd1R+aBcAdCYAMKVEKJbt25Ko59YROSTTz5R2lV5IUqwegFWN1i9erUZM3HiRKXxsIKIyBlnnKH0gQceqHRUPv5mzZqZyrgI+jpFxAT6H3XUUaYP3gOdO3dWOh3/o0/wWuL3Jz8/34xZs2aN0q5M+3itMBmO63Vd8JcpIYR4gMaUEEI8QGNKCCEeoDElhBAPxMJsesRisfUiUpW76eySRBAE7XL9JlxjJPwc1sk1euSnsM5QxpQQQogbPuYTQogHaEwJIcQDNKaEEOIBGlNCCPFAqOOk8Xg8iOp4IG6MVVVVSU1NTexHunsjHo8HePw1qqOBlZWVka0R63JhmjGs9yNij9G6PhesP9SmTRvTZ+HChTVR7AK77le8r/CIoi92p2vpqmGEfVxpJUtKSpTGzyqqNYq414nzcR37TacWF9aAytT2hDKmFRUV8uGHH4YZkjF4sQcNGhTJ+5aXl8s777yj2rBIXLpndVOBF23AgAFeXjcViURC3nvvPdWGxvPpp5824x5++GGl8csmIjJ69Gil8Qy7iEiHDh0iCXGpqKiQuXPnqjY0LK58pj446KCDcvK6SCKRkHfffVe1YXG/FStWmHHr169XGvPQiticFGhwBw4cGGqu2ZBIJEyxTvxn/umnn5pxmA/D9QPgkEMOUXrbtm1KDxkyJK058jGfEEI8QGNKCCEeCJ2CD0trYI3pyspKM+aBBx5QesqUKabPHXfcofS4ceOUzpVvC4nFYia1GK7pySefNONuueUWpV2+ZSyZ0atXr8wmmSWuEhBPPfWU0lOnTjXj8HMZM2aM6YOPRNu3b890mlmTTCbNI9tnn32mtOsx/6KLLlJ69uzZps/XX3+tNKZtjIpkMmk+Yyz54ypJ8lMjFouZR3R0VblK7eD1njZtmulzzTXXKH3bbbeZ904H/jIlhBAP0JgSQogHaEwJIcQDoXymO3fuNCEVy5YtM32QoUOHKu0KYdiwYUOYqeSMhoYGE0+JvsLrr7/ejDvrrLOUdpW8Xr58udJN5TPdsWOH8QNPmjQp5ThXiE0qsBxNlDQ0NMjatWtVG5ax6N69uxmH5aFdYHgchlxFWYIG54Ilfp544gkzDvcoDj/8cNPnhhtuUBpjiKMGfaZffPHFLv8uInLfffcp3bdvX9MHw8LQB53uteQvU0II8QCNKSGEeIDGlBBCPEBjSgghHgi1AZVMJqWurk614QaDy6GPjuv777/f9MHz75i0AA8L5IogCMx7YeKTpUuXmnF77bWX0pdffrnp88orryg9b948pevr60PNNVPq6urMmXXcZEPHvYjIPffcozSeCRexmx2uBBpR0dDQILW1taoNz627DldgEPeqVatMH6wpj68b1f0ai8XMwYMvv/xS6d69e5txl1xyidL9+/c3ffB7m07SkFyCwfPV1dVKt2/f3ozBNtcGFH4+uInODShCCIkQGlNCCPEAjSkhhHgglEMrFouZgGAM2kc/oIhNWoJ+VxEbaIyHAzAxRy5BH8lbb72lNCYsERGZMGFCytfdsmWL0i1atFA6Kj9bMpk0/tmuXbsqPWzYMDNu/PjxSs+ZM8f02Z18pi4+/vhjpfE+ExH5wx/+oDQmD3aB92dUQfubN2829yfeV5ivU8T6BV25afEewYMemESmqUG/tYhdg+taYtJpPATBRCeEEBIhNKaEEOIBGlNCCPEAjSkhhHgg1O5AYWGhCVY+6qijlHYVYjv55JOV7ty5s+kzfPhwpY899tgwU/NGs2bNpEuXLqoNi3LdfffdZtzEiROVxuJzIvaAA665qKgo1FwzpaCgwARkH3nkkSnHYZardDZmXJUxo6KkpMRkgFq4cKHSuIEqInL++ecr7crGj4UCMetSVBtvrsxYuBHjqnaAlQHWrVtn+uBngwdyotpk+zHwcAVeWxGRhx56SGnXJhWua88991SaG1CEEBIhNKaEEOIBGlNCCPFA1kH7++67r9LoxxARmTFjhtKYiEFEpGPHjkqjTy8qH1QQBCaguU+fPkq7qgkgWL1SxK6hqda4xx57GB9ft27dlHb5lh577LGUr43+OVf28yjBgxDnnXee0i5/2KJFi5R23a+YMCOqAxdIaWmpHHPMMart5ZdfVhqz6ovYpDuuBCBXXHGF0uhXd/mSc4Wroi6+/4ABA8w4bHNVi8B7P9MDGPxlSgghHqAxJYQQD9CYEkKIB2hMCSHEA7EwgbexWGy9iFTlbjq7JBEEQbtcvwnXGAk/h3VyjR75KawzlDElhBDiho/5hBDiARpTQgjxAI0pIYR4gMaUEEI8EOr8YjweD7CGPB6jc21o4ZHCTI4YVlVVSU1NTXq5sLIgHo8HrlrqYXEdL8R142cV1RrLysoCrPmExypdtZHwqKirbhCmaHTdD/Pnz6+JYhfYdb+60tEheKy3sLAw5Ri83tXV1ZFdS6xhhJ+56/tSIdWAAAAF20lEQVSG19tVYw1fBz+XqO5XEfc9i8e6sUaViL127drZ265Vq1ZKZ3otQxnT8vJymT17tmrDglWuc+vFxcVKY8EvEXvhcEGuomC5oKKiwhSKy8T4b9682bS1bNlSabyBBw0aFPp9MqFr167yxhtvqDbMufCPf/zDjMObtX///qbPueeeq7TLmJaUlEQS4lJeXi4zZ85UbcuXL1fadTYfcyZ06tQp5Xvh9+DQQw9Nd5pZkUgkTO6LxsZGpV3fNzQyGzZsMH3wu4xGZ8iQIaHmmg1du3aV6dOnq7aVK1cqfeGFF5pxmLf14osvNn2OO+44pTO9lnzMJ4QQD9CYEkKIB0LnfMPHb3wUcPkk8FEL68eLiBx//PG7fN3dCSxrISIyePBgpTHVmwv0QaVbHsEHeB0ffPBBpW+99VYzZv78+Ur37NnT/8Q8kkwmU9ZNx/RrIiJjxoxResGCBabPpEmTlMb0dFFdyyAIjCuldevWSr/55ptmHJbzOPjgg00fdG9giaKmSjv4PTifmpoa0+fee+9Vev/990/5uujyYtkSQgiJEBpTQgjxAI0pIYR4IJTPtLGx0fg70U+xdOlSM+7UU09V2hWGgX5VDEeI0p+IoVAYRuQq44ylf1evXm36YKnnfv36ZTpF7+Tn5yvt8msjo0ePNm1PPfWUtzllS15enimfjT79U045xYybNm2a0q5yGPvss4/S6JuNyp8Yi8VMmBOW1z7zzDPNOCzt7PKRV1XpCLavvvpK6Sj3Nerr62Xu3LmqDUO+0O8vInLSSScpncvETvxlSgghHqAxJYQQD9CYEkKIB2hMCSHEA6E2oJLJpDlzvm3bNqXx/LmIDRB2bW5g8HRdXZ1576biiSeeULq0tNT0Ofroo5UeN26c6YMO86bagNq6dat88sknqg2Dzl3st99+SuNriOxeG1Ai9mDE888/r/QBBxxgxpx22mkp+2CSl40bNyrtShySC2KxmFnjokWLlL7sssvMuFGjRindvn170+fjjz9WGt8nqjWKfLfZhZtmRxxxhNK42SRiN9FuvPFG0+df//qX0pkepuEvU0II8QCNKSGEeIDGlBBCPBDaZ4o+0sWLFyvtyv2IvrZNmzaZPuhHRY2ByFGCCYb/9re/pRzz0UcfmbaRI0d6m1M2NDY2mgMEw4YNSznu5ptvVhr9US4w12aUBEFg7huc8957723GYdIdzA8qYv3FeOjBNSYX7NixQ1atWqXaysrKlEZ/voj1d77wwgumDybS7tKli9JRH6RBP/XJJ5+cctzkyZOVxvveJ/xlSgghHqAxJYQQD9CYEkKIB2hMCSHEA6E2oFxO4LVr1yrt2pTATFKuoHfc2Bo6dGiYqXkjCALjeB8+fLjSWKRLxG5IjBgxwvS5+uqrPcwwewoKCswaMJgdNw1F0nP4L1myROlly5ZlMEM/7Ny509yfGJzuCla/9tprlXYVF7ziiit2qTHwO1ckk0n59ttvVRtuLmHmfRG7xhUrVpg+WHyuY8eOSqdTtdUXRUVF0qtXL9VWW1ubchweTkinMGemmaX4y5QQQjxAY0oIIR6gMSWEEA+Ecuw0a9ZMunbtqtrQN4hJQUREpk6dqrQrQ/c555yjNPpnsGJgLsFgZMzOjlnVRb5LHvJDjjvuuJTvE1VgN1JQUCBt2rRRbcuXL1f68ccfN+Nmz56t9FtvvWX6HHvssUpfeumlmU4za/Lz86VVq1aqDQPPMXGPiMhLL72ktOsgytixY5Vu27at0lH5TJs3b26qxOIexcsvv2zGoR91/Pjxps8xxxyjNB5MiGqNIt/5Zzt06KDaMADfdUBkyJAhod8Lfc7p+lD5y5QQQjxAY0oIIR6gMSWEEA/QmBJCiAdiYQJUY7HYehFJnSooNySCIGiXult2cI2R8HNYJ9fokZ/COkMZU0IIIW74mE8IIR6gMSWEEA/QmBJCiAdoTAkhxAM0poQQ4gEaU0II8QCNKSGEeIDGlBBCPEBjSgghHvg/jmnurp2bQzAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f892c044dd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_conv_layer(layer=layer_conv2, image=image2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从这些图像来看，似乎第二个卷积层会检测输入图像中的线段和模式，这对输入图中的局部变化不那么敏感。\n",
    "\n",
    "这些图像被铺平并输入到全连接层，但是这里没有展示出来。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 关闭TensorFlow会话"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们已经用TensorFlow完成了任务，关闭session，释放资源。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This has been commented out in case you want to modify and experiment\n",
    "# with the Notebook without having to restart it.\n",
    "# session.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结\n",
    "\n",
    "我们看到卷积神经网络在识别手写数字上的表现要比教程#01中简单线性模型要好得多。卷积神经网络可能达到99%的分类准确率，如果你做一些调整，还可能表现得更好，而简单线性模型只有91%的正确率。\n",
    "\n",
    "然而，卷积神经网络实现起来更复杂，并且光看权重滤波也不好理解为什么它能奏效或者失败。\n",
    "\n",
    "因此我们需要一个更简单的实现卷积神经网络的方式，同时也要寻找一种更好的方法来对它们内部工作原理进行可视化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 练习\n",
    "\n",
    "下面使一些可能会让你提升TensorFlow技能的一些建议练习。为了学习如何更合适地使用TensorFlow，实践经验是很重要的。\n",
    "\n",
    "在你对这个Notebook进行修改之前，可能需要先备份一下。\n",
    "\n",
    "* 如果你不改变任何参数，多次运行Notebook，会得到完成一样的结果吗？随机性的来源是什么？\n",
    "\n",
    "* 再进行10,000次优化。结果有变好么？\n",
    "\n",
    "* 改变优化器的学习率。\n",
    "\n",
    "* 改变层次的属性，比如卷积滤波器数量、滤波器的大小、全连接层中的神经元数量等等。\n",
    "\n",
    "* 在全连接层之后添加一个drop-out层。在计算分类准确率的时候，drop-out层可能为0，因此你需要一个placeholder变量。\n",
    "\n",
    "* 改变ReLU和max-pooling的顺序。它的计算结果相同么？最快的计算方法是什么？节省了多少计算量？这也适用于Sigmoid-function和average-pooling吗？\n",
    "\n",
    "* 添加一个或多个卷积层和全连接层。这对性能有帮助吗？\n",
    "\n",
    "* 能得到良好结果的最小可能配置是什么？\n",
    "\n",
    "* 试着在最后一个全连接层中使用ReLU。性能有变化吗？为什么？\n",
    "\n",
    "* 卷积层里不用pooling。这对分类准确率和训练时间有影响吗？\n",
    "\n",
    "* 在卷积层里用2x2的stride代替max-pooling？有什么变化吗？\n",
    "\n",
    "* 不看源码，自己重写程序。\n",
    "\n",
    "* 向朋友解释程序如何工作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## License (MIT)\n",
    "\n",
    "Copyright (c) 2016 by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "\n",
    "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n",
    "\n",
    "The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n",
    "\n",
    "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}