Practical_1_Deep_Feedforward_Networks.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Practical 1: Deep Feedforward Networks",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [
        "tHy4bjbTVCzV",
        "SIUsSjNwVk_n",
        "ot-FjimzV6op"
      ]
    },
    "kernelspec": {
      "name": "python2",
      "display_name": "Python 2"
    }
  },
  "cells": [
    {
      "metadata": {
        "id": "dYUH0qf8Dqkm",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Practical 1: Deep Feed-forward Networks"
      ]
    },
    {
      "metadata": {
        "id": "6hH4wd81A_P8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Introduction\n",
        "In this practical, we implement and train a feed-forward neural network (also known as an \"MLP\" for \"multi-layer perceptron\") on a dataset called \"Fashion MNIST\", consisting of small greyscale images of items of fashion. We consider the practical issues around generalisation to out-of-sample data and introduce some important techniques for addressing this. \n",
        "\n",
        "## Learning Objectives \n",
        "* Understand how to use **Tensorflow Eager** and **Keras Layers** to build a neural network architecture\n",
        "* Understand how a model is trained and evaluated\n",
        "* Be able to explain why there is a difference between **in-sample** and **out-of-sample** model performance\n",
        "* Understand the concept of **train/validation/test split** and why it's useful\n",
        "* Research at least 1 technique that can be used to improve model **generalisation**"
      ]
    },
    {
      "metadata": {
        "id": "NxSD3yPDfEIj",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "6c6beb36-83c9-4658-c4d0-08dbdbbc6a0f"
      },
      "cell_type": "code",
      "source": [
        "#@title Imports (RUN ME!) { display-mode: \"form\" }\n",
        "from __future__ import absolute_import, division, print_function\n",
        "\n",
        "import tensorflow as tf\n",
        "import tensorflow.contrib.eager as tfe\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "try:\n",
        "  tf.enable_eager_execution()\n",
        "  print('Running in Eager mode.')\n",
        "except ValueError:\n",
        "  print('Already running Eagerly')"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Running in Eager mode.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "sU33KeDUDomX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## The Data"
      ]
    },
    {
      "metadata": {
        "id": "nL0dFs-DD6sM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "In this practical, we use the Fashion MNIST dataset consisting of 70,000 greyscale images and their labels. The dataset is divided\n",
        " into 60,000 training images and 10,000 test images. The idea is to train a **classifier** to identify the class value (what type of fashion item it is) given the image. We train and *tune* a model on the 60,000 training images and then evaluate how well it classifies the 10,000 test images that the model did not see during training. This task is an example of a **supervised learning** problem, where we are given both input and labels (targets) to learn from. This is in contrast to **unsupervised learning** where we only have inputs from which to learn patterns or **reinforcement learning** where an agent learns how to maximise a reward signal through interaction with its environment. "
      ]
    },
    {
      "metadata": {
        "id": "3lEM3aBa2N63",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###Aside: Train/Validation/Test Split\n",
        "\n",
        "When we build machine learning models, the goal is to build a model that will perform well on *future* data that we have not seen yet. We say that we want our models to be able to **generalise** well from whatever training data we can collect and do have available, to whatever data we will be applying them to in future. To do this, we split whatever data we have available into a **training set, a validation set and a test set**. The idea is that we train our model and use the performance on the validation set to make any adjustments to the model and its hyperparameters, but then we report the final accuracy on the test set. The test set (which we never train on), therefore acts as a proxy for our future data."
      ]
    },
    {
      "metadata": {
        "id": "9VRxcYWHfZJ5",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Tensorflow has convenient modules for loading a number of standard datasets\n",
        "fashion_mnist = tf.keras.datasets.fashion_mnist\n",
        "(train_and_validation_images, train_and_validation_labels), (test_images, test_labels) = fashion_mnist.load_data()\n",
        "\n",
        "text_labels = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1Z_Ks31qEfUM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Construct a validation set\n",
        "Next, we remove 10,000 images and labels from the training set to use as a validation set. We will come back to the validation set later! "
      ]
    },
    {
      "metadata": {
        "id": "dZKEjjGM8I4t",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Construct a validation set from the last 10000 images and labels from \n",
        "# train_and_validation_images and train_and_validation_labels\n",
        "validation_images = train_and_validation_images[-10000:, :, :]\n",
        "validation_labels = train_and_validation_labels[-10000:]\n",
        "\n",
        "# Construct a training set from the first 50000 images and labels.\n",
        "train_images = train_and_validation_images[:50000, :, :]\n",
        "train_labels = train_and_validation_labels[:50000]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "fbWk7qJOEkl6",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### What does the data look like?\n",
        "Each image in the dataset consists of a 28 x 28 matrix of greyscale pixels. The values are between 0 and 255 where 0 represents black, 255 represents white and there are many shades of grey in-between. Each image is assigned a corresponding numerical label, so the image in ```train_images[i]``` has its corresponding label stored in ```train_labels[i]```. We also have a lookup array called ```text_labels``` to associate a text description with each of the numerical labels. For example, the label 1 is associated with the text description \"Trouser\".\n",
        "\n",
        "The 2 cells below demonstrate how to visualise the input data. Make sure you understand what's happening, particularly how the indices correspond to individual items in the dataset. "
      ]
    },
    {
      "metadata": {
        "id": "PgfvNeX2gcAL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 280
        },
        "outputId": "4b853a99-0389-4dc8-d163-0714ad00507b"
      },
      "cell_type": "code",
      "source": [
        "# We use the Matplotlib plotting library to visualise an image selected at random from the training set \n",
        "plt.figure()\n",
        "random_index = np.random.randint(0, len(train_images))\n",
        "plt.imshow(train_images[random_index], cmap='gray')\n",
        "plt.colorbar()\n",
        "numerical_label = train_labels[random_index]\n",
        "text_description = text_labels[numerical_label]\n",
        "plt.title('True Class: {} (\"{}\")'.format(numerical_label, text_description))\n",
        "\n",
        "plt.gca().grid(False)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS0AAAEHCAYAAAD71BC+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAGbhJREFUeJzt3X+QHGWdx/H3JoQkbEISiCQYQxHU\n+2KIZR25GJGKBkSCiqQ0IOXl0INYcpbhREq9UHgI1JVQiR5qpDgjGAR/HJAcGAIFAURBTjEgPzX5\nBsVIkR+3S0JifmySTXbvj+5dh2X66clMz84+k8+raouZ/nb3fKfZ/ebpp59+uqW7uxsRkVgManQC\nIiIHQ0VLRKKioiUiUVHREpGoqGiJSFRUtEQkKoc1OoGBxMxuBE5L374V2Ah0pO+nufuOOn3uBcBl\nwHDgcODXwJfdfaOZXQW8xd0/U4/PzshnBPA94Hx3D/6OmNnngPcAjwAzgVuAq4CvA99JVxtL8ru2\nOX3/dXe/tei8q5EeX4D1JPn/N/BvwAfcvasxWUmIilYJd/9cz2szWw/8k7v/qp6fmf7RXwac4+5r\nzGwI8FXgUTObUs/PDvhfYGXeSmZ2PHA5MAX4eGnM3VcBJ6brXUU/F95qufv9ZvZJ4BLg243OR95I\nResgmNkvgMdJ/kDnkbQmbnL3H5XEb3L3H5nZqcC3gDHAq8A/uvtLffY3CPga8Cl3XwPg7p3A18zs\naaC7z/oG3AwcDQwB/t3df5rG/gM4D2gBXiEpuBsDy28F7nT3e8p81YuBTcBXcg7JV4Cl7v5XM9ue\nbrMbeDlnO8zsn4FzgFHAU+7+FTP7V+BfSLotHPiMu7eXHtd02973ge9X9vj3/dySXHvyB7gOeNjM\nbnT3fXnfRfqXitbBmwqc5O5dSQ15IzMbCdxDcnr1YPov9x3AP/RZ9USSP6oH++7D3e9O91W6+BvA\nSne/zszeB9xvZsuAvwM+kebVaWaXAGeY2VPllgO3uvunsr6gu/86bUXlmQPMSre5C7grXZ657z7O\nBP7e3V80s/cAXwamunubmS0GrgUyW2dmdhLlv/ddhI9/7+f22eVd6XdZkxbhGcDDFX4X6SfqiD94\n91XQ1zEDeMXdHwRIW0NvM7Pj+qx3FNDu7pXeSzUbWJS+/hUwDDgW2Aa8CZhrZmPcfXHaZ5S1vGZp\nURsFPFfDbtaVFI6PAMvcvS19fxNJcQnJ+n55x39dmYLV1xPAKQf5faQfqGgdvK0VrDMaeKuZre35\nAfaS/IGVehUYZ2aVtnhnkfR1rQP+QHJKNMjdN5Ccsp4HvGxm95rZxKzlFX5WnmOALTV2VpceyzcB\nr5W8fy39jEyB75d3/Cv5f9iW9/nSGDo9rM0BYHDJ+zHpfzcCa9y97+lgX+tI/jjOAf6nNGBmVwI3\nlrwfAtwJfMLd7zOzofztyibu/gjwiJm1kpxGXgfMzVp+sF+0jJYC9lHq/0j66nocnS6D7OOc9b2/\nT8bxN7N3Fpy39DO1tGqzCXgXgJmdQtK3BMmpxbFmNj2NnWBmt5nZ6/7Q01bKV4HvmNm0dN0haefy\nx4C/lqzemv48mb7/ArAPGGFmZ5rZDWY2yN13Ac8C3VnLC/rubcDR6cWEItwLfNzMegrXxekyyDjO\nge9X0fHP8SagvdYvJcVT0arNfwIfMbM1JJ3PqwDcvQM4F1icxu4iuVL3hoLh7ktJOqC/n572PU8y\nrul0d99bst42YCHwdHpl8U/A3SRDEx4HjgDWmdnvgfOBK4FHM5ZjZrea2Uf75mNmJ6enUw8Dg0tO\nr/paT1JUC2m5uPtvSVpJj6WfNxq4Ig2XPc5Z3+9gjn/AdJLxcjLAtGg+LamWmf0XsNHdr2l0LkVK\nh5b8Aji+9B8OGRjU0pJaLATmpSPom8lXgG+oYA1MKlpStXSw7EJgcaNzKYqZnQm8jWRgqgxAOj0U\nkaiopSUiUemXcVotLS1qztXB8uXLM2N//vOfg9uOHDkyGG9rawvGd+7cGYy/5S1vyYxt3rw5MwbQ\n1RUer3rttdcG44eq7u7uIsbOHczfatFj9SpSddEys+tJpiTpBr7g7qsLy0pEomdmC0luqTqM5D7S\nc0ju3d2SrrLI3e81s7nApUAXsMTdbw7tt6qiZWbvB97u7qeY2TuAH6D7tESidzB93C0t2Q0tMzsN\nmJLWiKOBp4GfA5e7+8qS9VpJxg6+m2Sw9Gozu8vdM2+1qrZP6wMkAxtJp1QZY2ZHVrkvERkgurq6\nKv7J8SjJPaGQ3NjeyutvxeoxHVjt7tvTQcGPA6eGdlzt6eF4krmIerSny/5afnURiUFRownc/QCw\nK307D7iP5B7S+WZ2GcltYPNJ6kbp7VJtJDOXZCrq6mFDOuREpFjd3d0V/1TCzGaTFK35wG3AAnc/\nHXiGZFruvnJrSbUtrY0kFbLHm/nbrI8iEqkix22a2SyS+0fPcvftvH5CxRUks5gs4/W1ZALwm9B+\nq21prSK5IRUzO5nk/rO6PPRBRPpPUS0tMxtFMmHl2T2d6ma23MxOSFeZCbxAMiPHNDMbnd4Odirw\nWGjfVY+IN7PrgPeRXKb8vLs/m/khGqdVlRNPPDEYX7NmTWbsD3/4Q3DbSZMmBeNDhw4NxnfsCP8b\nNXz48MzY+vXrq94W4F3velcw/tprrwXjzaqIcVp79uyp+G912LBhmZ9nZp8lOf1bV7J4Kclp4m5g\nJ3BhOrX2uSQznXQDi939x6HP7ZfbeFS0qqOiVZ6KVnlFFK3du3dX/Ld6xBFHxDW4VESaTwz3Iqto\niUgvFS0RiYqKlohERUVLRKJSwe05DaeiJSK91NKSmpx99tnB+KuvvpoZC92BD7Bt27ZgfMiQIcH4\nK6+8EoyPGzcuM9bR0ZEZAzjyyPC999OmTQvGV61aFYxLNhUtEYmKipaIREVFS0SioqIlIlHR1UMR\niYpaWiISFRUtqcns2bOD8dCwhkGDwlOl7du3Lxjfs2dPML5ly5Zg/JhjjsmM5Q1pGDFiRDD+zne+\nMxjXkIfqqWiJSFRUtEQkKuqIF5GoqKUlIlFR0RKRqKhoiUhUVLREJCoqWlKT9773vcF46Kk2e/fu\nDW6bN3XNsGHDgvGJEycG46FxYJs3bw5uO3r06GC8tbU1GJfq6eqhiERFLS0RiYqKlohERUVLRKKi\noiUiUVHREpGo6OqhiERFLS2pSd5YqgMHDmTGhg4dGtx2+PDhNX12Xjz0L/aoUaOC2+7fvz8Yz3sE\nmVSvaYuWmc0E7gR+ny563t0vKSopEWmMpi1aqV+6+7mFZSIiDdfsRUtEmkyzd8RPNrMVwFHA1e7+\nYEE5iUiDNHNL60XgauAO4ATgETN7m7uHn5YgIgNakUXLzBYCM0jqzLXAauA2YDCwCbjA3fea2Vzg\nUqALWOLuN4f2G35kSwZ33+Dut7t7t7v/CdgMTKhmXyIycHR3d1f8E2JmpwFT3P0U4CzgW8A1wA3u\nPgP4I3CRmbUCVwJnADOBL5rZUaF9V1W0zGyumX0pfT0eGAdsqGZfIjJwFFW0gEeB89LX24BWkqK0\nIl12D0mhmg6sdvft7t4BPA6cGtpxtaeHK4CfmNls4HDgczo1PHgjR46safvQvFR5c1KNGTMmGN+6\ndWswnjeWauzYsZmxHTt2BLfNe6Zi3jMbpXpFnR66+wFgV/p2HnAfMMvdeyZ6awOOBcYD7SWb9izP\nVFXRcvcdwEer2VZEBq6irx6mDZt5wJkkfeE9skYnh0ctU+XpoYg0pwJPDzGzWcAVwIfcfTuw08x6\nbsWYAGxMf8aXbNazPJOKloj0KrAjfhSwCDjb3Xv6Gh4C5qSv5wD3A08A08xstJmNIOnPeiy0bw0u\nFZFeBQ55OB8YC9xhZj3LPg3cZGYXA38BfujunWa2AHgA6CYZ87k9tGMVLRHpVWBH/BJgSZnQB8us\nuwxYVum+VbREpFczj4iXApx88snBeN4vUGdnZ2bssMPC/2tD09oAjBs3LhjPyy30CLMhQ4YEt82z\nYYOGBNZLs997KCJNRi0tEYmKipaIREVFS0SioqIlIlFRR7yIREUtLRGJioqWBE2ePLmm7Q8//PDM\nWN44rbzpXXbt2hWMT5gQnvMxNBZr27ZtwW1D3wvyp66R6qloiUhUVLREJCoqWiISFV09FJGoqKUl\nIlFR0RKRqKhoiUhUVLQkqLW1NRjP+wUKdZqG5rMCmDRpUjD+sY99LBj/9re/HYyHxnHlfa+8ub42\nbdoUjEv1VLREJCq6eigiUVFLS0SioqIlIlFR0RKRqKhoiUhU1BEvIlFRS0uC9uzZE4zXMk4r79mC\n+/fvD8ZXrlwZjP/0pz8NxkPfLfS8Rsgfp7V27dpgXKrXNEXLzKYAPwOud/fvmtlE4DZgMLAJuMDd\nw6MZRWTAi6FoDcpbwcxagcXAwyWLrwFucPcZwB+Bi+qTnoj0p+7u7op/GiW3aAF7gQ8DG0uWzQRW\npK/vAc4oNi0RaYQYilbu6aG77wf2m1np4taS08E24Ng65CYi/exQuXrYUsA+RGQAaIo+rQw7zWx4\n+noCrz91FJFIxXB6WG3RegiYk76eA9xfTDoi0kgxFK3c00Mzmwp8Ezge6DSzc4G5wC1mdjHwF+CH\n9UyyWXV0dATjeeOVQvNx5T2X8Pnnnw/G84wYMSIYDz2bcNCg8L+VGzeq4d4oMZweVtIR/xTJ1cK+\nPlh4NiLSUEV2xJcZ33kLMBXo+Rdtkbvfa2ZzgUuBLmCJu98c2q9GxItIr6JaWhnjOwEud/eVfda7\nEng3sA9YbWZ3ufvWrH1X26clIk2owD6tcuM7y5kOrHb37e7eATwOnBraQC0tEelVVEsrY3wnwHwz\nu4xkfOd8YDzQXhLPHfeplpaI9Krz1cPbgAXufjrwDHBVmXVyx32qpSUivep59dDdS/u3VgA3AstI\nWls9JgC/Ce1HRauB2tvbg/HBgwcH46ErPcOHD8+MASxdujQYr9WuXbsyY3l/GK+88krR6UiF6nkb\nj5ktB77s7i+RjEh4AXgCuMnMRgP7SfqzLg3tR0VLRHoVePWw3PjOxcDtZrYb2Alc6O4dZrYAeADo\nBq529+2hfatoiUivAjvis8Z3Li+z7jKS08SKqGiJSK+mGBEvIocOFS0RiYqKlohE5VCZBFBEmoRa\nWhKUNzVNS0t4cHBoHFfetr/73e+C8VqFHlGW9wixl19+ueh0pEIqWiISFRUtEYmKipaIREUd8SIS\nFbW0RCQqKloiEhUVLRGJioqWBO3evTsYz3uE2LBhw6r+7NWrV1e9bSWGDBmSGcsbn/bAAw8UnY5U\nSEVLRKKiq4ciEhW1tEQkKipaIhIVFS0RiYqKlohERUVLRKKiq4cSlDcOK09oLFSeev9yjho1KjOW\n90zGtWvXFp2OVKhpWlpmNgX4GXC9u3/XzG4BpgJb0lUWufu99UlRRPpLUxQtM2slecjiw31Cl7v7\nyrpkJSINEUPRGlTBOnuBDwMb65yLiDRYd3d3xT+NktvScvf9wH4z6xuab2aXAW3AfHd/tQ75iUg/\niqEjvpKWVjm3AQvc/XTgGeCqwjISkYZpipZWOe5e2r+1ArixmHREpJGapU/rDcxsuZmdkL6dCbxQ\nWEYi0jBN0dIys6nAN4HjgU4zO5fkauLtZrYb2AlcWM8km9Vhh4UP/+bNm4PxoUOHZsbq/Uu1c+fO\nYHzv3r2ZsVDeANOnTw/GV61aFYxL9WJoaVXSEf8USWuqr+WFZyMiDdUURUtEDh0xXD1U0RKRXmpp\niUhUiixaZW7/m0gyXGowsAm4wN33mtlc4FKgC1ji7jeH9lvtOC0RaUJFXT3MuP3vGuAGd58B/BG4\nKF3vSuAMkr7zL5rZUaF9q2iJSK8ChzyUu/1vJsm4ToB7SArVdGC1u2939w7gceDU0I51ethATzzx\nRDCe1ymaN2Sinnbt2hWMt7S0ZMaOPPLI4LZjxoypKiepXVGnhxm3/7W6e89YmDbgWGA80F6yTs/y\nTCpaItKrH68eZv2rlv2vXUqnhyLSq84j4neaWc8MkBNITh03krS26LM8k4qWiPSqc9F6CJiTvp4D\n3A88AUwzs9FmNoKkP+ux0E50eigivYrq08q4/W8ucIuZXQz8Bfihu3ea2QLgAaAbuNrdt4f2raIl\nIr0K7IjPuv3vg2XWXQYsq3TfKloi0ku38YhIVHQbjwR1dnYG44MGha+ThOJ52x533HHB+MsvvxyM\nt7e3B+MTJkzIjA0ePDi4bUdHRzAu9aOiJSJRUdESkaioaIlIVFS0RCQqunooIlFRS0tEoqKiJSJR\nUdGSmtTSv7Bt27ZgfNKkScF43jitAwcOBOOhsVj79+8PbpuXu9SPipaIREUd8SISFbW0RCQqKloi\nEhUVLRGJioqWiERFRUtEotI0Vw/NbCEwI13/WmA1ZR5vXa8kD1UTJ04Mxrdu3ZoZCz13EGp/ZuK+\nffuC8dB8Xrt37w5u++yzz1aVk9QuhpZW7tN4zOw0YIq7nwKcBXyLMo+3rmuWItIv6vw0nkJU8gix\nR4Hz0tfbgFbKP95aRCIXQ9HKPUdw9wNAzzPQ5wH3AbPKPN5aRCIXw+lhxR0bZjabpGidCbxYEsp9\njLWIxCGGolXRE6bNbBZwBfCh9EGK5R5vLSKR6+rqqvinUSrpiB8FLALOdveey1XlHm8tIpFrij4t\n4HxgLHCHmfUs+zRwU+njreuT3qEt7zFdw4cPz4zl/VKtXbu2qpx67N0bHuESmppm+/bgU89z41I/\nMZweVtIRvwRYUib0hsdbi0jcmqJoicihQ0VLRKLSNLfxiMihQS0tEYmKipaIREVFS0SioqIlNVm/\nfn0wPnny5MxY3iO+pk2bFoxv2LAhGA+NwwIYOnRoZqyzszO4rTSOipaIRKWoq4dmNhO4E/h9uuh5\nYCEFzMNX0b2HInJoKPg2nl+6+8z05xIKmodPRUtEetX53sOZFDAPn04PRaRXwX1ak81sBXAUcDXQ\nWsQ8fCpaItKrwKL1IkmhugM4AXiE19ebqufhU9ESkV5FFS133wDcnr79k5ltBqaZ2XB376CGefjU\npyUivYqaBNDM5prZl9LX44FxwFIKmIdPLa0B7LnnngvGTzrppMxY3jitT37yk8H43XffHYyPHDky\nGA/9Uo8ZMya4rTROgaeHK4CfpNO0Hw58DngauLXWefhUtESkV4GnhzuAj5YJ1TwPn4qWiPTSiHgR\niYqKlohERZMAikhU1NISkaioaIlIVFS0pCYvvfRS3fbd1tZW0/ZHHHFEMB4aJ5b3zERpHBUtEYmK\nipaIREVXD0UkKmppiUhUVLREJCoqWiISFRUtEYlK0xQtM1sIzEjXvxY4B5gKbElXWeTu99Ylw0PY\nli1bgvFhw4Zlxjo6OoLbPvTQQ1Xl1CPvl3vfvn2ZsXXr1tX02VI/TXH10MxOA6a4+ylmdjTJRF4/\nBy5395X1TlBE+k+ztLQeBX6bvt4GtJI8bFFEmkxTFC13PwDsSt/OA+4DDgDzzewykkcBzXf3V+uW\npYj0ixiKVsUPtkjnep4HzCd5tPUCdz8deAa4qi7ZiUi/qvPDWgtRaUf8LOAK4Cx33w48XBJeAdxY\nh9xEpJ/F0BGf29Iys1HAIuBsd9+aLltuZiekq8wEXqhbhiLSb5qlpXU+MBa4w8x6li0Fbjez3cBO\n4ML6pHdoe+aZZ6reduzYscH4cccdV/W+ASZOnBiMH3ZY9q9W3lAOaZwY+rQq6YhfAiwpE6rqmWUi\nMnA1RdESkUOHipaIREVFS0SiEsPVQxUtEemllpaIREVFS0SioqIlNXnyySeD8VtvvTUzlveIrx//\n+MdV5dTje9/7XjB+5plnZsba29tr+mypHxUtEYmKipaIREVXD0UkKmppiUhUVLREJCpFFi0zux54\nD9ANfMHdVxex34onARSR5lfU1DRm9n7g7e5+Csnkod8pKkcVLRHp1dXVVfFPjg8AdwO4+xpgjJkd\nWUSOLTGcw4pIXMxsCXCvu/8sff8YMM/da35+nFpaItIfWorakYqWiNTDRmB8yfs3A5uK2LGKlojU\nwyrgXAAzOxnY6O47itix+rREpC7M7DrgfUAX8Hl3f7aI/apoiUhUdHooIlFR0RKRqPT7bTz1Gtpf\nKzObCdwJ/D5d9Ly7X9K4jMDMpgA/A6539++a2UTgNmAwyZWYC9x97wDJ7RZgKtDzUMNF7n5vg3Jb\nCMwg+f2+FljNADhuZfI6hwFyzGLSr0WrdGi/mb0D+AFwSn/mkOOX7n5uo5MAMLNWYDHwcMnia4Ab\n3P1OM/s6cBFw4wDJDeByd1/Z3/mUMrPTgCnp79jRwNMkeTb0uGXk9XMGwDGLTX+fHtZtaH8T2gt8\nmGS8S4+ZwIr09T3AGf2cU49yuQ0UjwLnpa+3Aa0MjONWLq/BDcgjev19ejgeeKrkfXu67K/9nEeW\nyWa2AjgKuNrdH2xUIu6+H9hvZqWLW0tOa9qAY/s9MTJzA5hvZpeR5Dbf3V9tQG4HgF3p23nAfcCs\nRh+3jLwOMACOWWwa3RFf2ND+ArwIXA3MBj4N3Gxmhzc2paCBdOwg6TNa4O6nA88AVzUyGTObTVIc\n5vcJNfS49clrQB2zWPR3S6tuQ/tr5e4bgNvTt38ys83ABODPjcvqDXaa2XB37yDJbcCcnrl7af/W\nChrQ19bDzGYBVwBnuft2MxsQx61vXry+T7Chxywm/d3SqtvQ/lqZ2Vwz+1L6ejwwDtjQ2Kze4CFg\nTvp6DnB/A3N5HTNbbmYnpG9nAi80KI9RwCLgbHffmi5u+HErl9dAOWax6fcR8fUa2l8rMxsJ/AQY\nDRxO0qd1XwPzmQp8Ezge6CQpoHOBW4BhwF+AC929c4DkthhYAOwGdqa5tTUgt8+SnGaVToHyaeAm\nGnjcMvJaSnKa2NBjFhvdxiMiUWl0R7yIyEFR0RKRqKhoiUhUVLREJCoqWiISFRUtEYmKipaIROX/\nAYPHSa3T0HhhAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f67cfc18790>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "-utfMlxhhL3b",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 589
        },
        "outputId": "18781e96-bf98-4a18-8e3e-3149fac0df80"
      },
      "cell_type": "code",
      "source": [
        "# Another view, showing 25 randomly selected images at a time\n",
        "plt.figure(figsize=(10,10))\n",
        "for i in range(25):\n",
        "    plt.subplot(5,5,i+1)\n",
        "    plt.xticks([])\n",
        "    plt.yticks([])\n",
        "    plt.grid('off')\n",
        "    \n",
        "    img_index = np.random.randint(0, 50000)\n",
        "    plt.imshow(train_images[img_index], cmap=plt.cm.gray)\n",
        "    plt.xlabel(text_labels[train_labels[img_index]])"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjkAAAI8CAYAAAATJrreAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsnXm8XdP5/z+pmRAhkUSERIYloxBJ\nRCaRiLHUULT9lk6KGqr9tjooUdVqje1Xixpaww8tNZaigiAkhAghrAySCBnIQMxF/f4456581pO7\nlpObe3Pv3efzfr28PCd7nX322WuvffZ9Ps/Q4rPPPoMQQgghRNH4QmMfgBBCCCFEQ6CHHCGEEEIU\nEj3kCCGEEKKQ6CFHCCGEEIVEDzlCCCGEKCR6yBFCCCFEIVk/t7FFixbKL28EPvvssxYNsd/Gms+v\nfOUr0evx48cH+80336zXz9p8882DPWrUqGjbXXfdVa+fVSkNMZ+NNZfrrx/fMk444YRgX3LJJcn3\nfeELq/6e4rIVuRIWZ555ZnLfK1as+PyDbQCKtjarnSKtzXXJeuutF+wNNtgg2vbhhx8Gm+8P9957\nbzRu3rx5te4PAD799NM1PqbUXMqTI4QQQohCooccIYQQQhSSrFwlRI6ddtop2Ndff320bcsttwx2\ny5Yto23s3vzPf/4T7EceeSQax1LW+++/H+x27dpF43bfffdgt2nTJthWCmGJ44knngj2N7/5TaSo\nDzdqkdh6662j18ccc0yw+Rp46623onH//e9/K9o/y1p77713sKdOnRqNu/vuuyvanxCibvBaBGKp\nmu/buXviwIEDg33ZZZclxzVk5wV5coQQQghRSPSQI4QQQohCooccIYQQQhSSFjktrBpS4ZoijZ2m\nylqsjaXYeOONgz1nzpxa/x0A3n333WB//PHH0baNNtoo2O3btw+2TU+uFNaEX3/99WBzKqM9Ro7d\n8d5H43bdddfkZ3GMTqXxOUVOUz3llFOCvXDhwmDbueQYHY6NGjx4cDRu8eLFwd5iiy2CPWHChGjc\n7Nmz63bAa0ljr8365uSTT45eb7LJJsE+77zz6vWzDjjggGCPHTs22safxWu4oSny2mzRYs2/2oYb\nbhjsjz76KDmOfyN+8pOfRNv69+8fbP79+NnPfhaNUwq5EEIIIcRaoIccIYQQQhQSpZCL1cil+15x\nxRXBZjlhwYIF0bgZM2bUOg6I3agrV64MNrvK7TjGSqzvvfdesFkKY9er5bXXXgs2p8IDwNFHHx3s\n6667LrkPEct+3bt3D3arVq2icZx6PmnSpGBbaXDy5MnB7tq1a7AffvjhtT/YZkKlrvvcON5myzuw\nRMilGYBYShw9enSwZ86cGY3bdNNNg718+fJgv/3229E451ywd9xxx1qPFQBuvfXWYN95553BPvfc\nc5HCrm/eZ+pc1PbZ1QDfM3PngyWq1q1bR+MuvfTSYI8ZMybYXI4DAH71q18FmyuV83sA4Kqrrgp2\nXaS1SpEnRwghhBCFRA85QgghhCgkkqvEGtGlS5dgcwaNrXDL2S/Lli2LtnXq1CnYAwYMCPbSpUuj\ncexiZXemrcS52WabBZultpxbmmUtO+7AAw8MtpWrqtHVnYNljGnTpgXbZtu9/PLLwd5///2D3aNH\nj2gczzPPv72Gikyl11hu3He+851g77nnntE2bpTIDW0BYKuttgo2V7Xt2bNnNI4rTnMm5ZAhQ6Jx\nXPmcrwHeNxBXRR8+fHiw//3vf0fjnnnmmeQ+GCvJVCOp+2fuuvnNb34T7IMPPjjaxtlRtulyCs6o\n+/vf/x5tY7nqk08+ibblMnzXFHlyhBBCCFFI9JAjhBBCiEKihxwhhBBCFBLF5Ig1gjV8Tj/l2BoA\nePDBB4PNVY2BOA6DtVirFXPcDOuyNt2Qt7EObWN3eBunn1o9uHfv3hCVwdVNOU3cnvvddtst2NyF\nvkOHDtG4+fPn17oPG/MlVofX4KGHHhrsp556KhrH8SpPP/10tI3nKccee+wRbC4XwPE5APDKK68E\nm+ea44IAYNCgQbV+zqmnnhq9/va3vx1sG5PDa5q3VWt8TqVxLVwW5KWXXgp27j7IsVtcTgCIy3Nw\ntXubQn722WcH+8wzz4y21WdXcnlyhBBCCFFI9JAjhBBCiEIiuUpksa5IdjmzzGNdx3vvvXew2bUJ\nxG5lTg22bmXb2DMFy1dsWxctfy67cm0jul69elX0udWInaO5c+cGmxu2tm3bNhr34osvBnvgwIHB\ntk1UWbK06c0iz89//vNgs2RgJUE+r5y6DQCLFi0KNjfK7NixYzSO54k/y8IVkLfbbrtg9+3bNxrH\n25599tlg22rkxx13XLC5mm6Oai37wGuV74Xt2rWLxnGlcj73N998czTOVqSvwZaL4Ea9XC7EVsNm\nedsiuUoIIYQQ4nPQQ44QQgghConkKpGFsyiAWL565513gm0zlLgZn5WdWKJgCcm6KFMuS5tdlaqG\nnJO7OMPHNilkCc266dmFX41wlWsAWLx4cbC5uu2SJUuicXzd5Jqysoudq2aL1bEZjcOGDQv2Oeec\nE2wrV3HGk82G4kxItm22HDfl5DVsm2ayjM0yhm3gytlzfA2wzAnEWWN//vOfk58l4vsf3yPt2hw1\nalSt77/sssvW+himTJkS7DfeeCPaxsfBWZpAXD19bZEnRwghhBCFRA85QgghhCgkesgRQgghRCGp\nipgc24W3devWwb799tvX2XHYWBKmPlPm6hMbk8PaPNv2u3HchdXpbVxHDfYcpM5XLiYn9zkcN8Sf\nZdOi+X2cCg8A11xzTa2fVS3YdNzdd9892NxFvl+/ftG4WbNmBZtjM7p27RqNGz9+fLBt+rCI+dWv\nfhW9fuSRR4LdrVu3YNvrm1N5bUwUb/vggw+CbWNo+H28/1x1YY4RsbFAPNec4s6dy4E47objcwDg\nb3/7W/Kzq51UR3Igvt/x+rZxWDbuMgX/3vK6Hzt2bDSOY/imT59e0b7rgjw5QgghhCgkesgRQggh\nRCEplFy17bbbBnuLLbYIdpcuXaJxXKGRqz/a1DomVVUXiF2BOdmpqUpSObbffvvoNVfOZNd0Lv2b\n07WB1SsMp/bBVCr1pZpwArGrm5t/Wvh7STKJsdcDu7S5ai2nlgPxXOy///7BnjRpUjSOJRJ2Z4sS\n3LDS3tduueWWWrfxOQXiObRVaLkCsl0/DK8RljisnMmvWaKyTXu5bAO/x8pfLF99+ctfjrY99thj\nwa72Ug857H2WZUS+V+fuucxVV10Vvebf1C9+8YvB5mraQP664c9e299NeXKEEEIIUUj0kCOEEEKI\nQqKHHCGEEEIUkiYZk5OLeWFsuXJuJeCcC7YtEc3prbvuumuw77333uQxVRp3Y+Hv8qMf/SjY119/\nfTTOxjA0FXbYYYfoNcfkcDyGTdfmuCfbDTyXzlgJlcY92XF8jLzNHjsfL19HAhg0aFD0mlti/PWv\nfw22jbngtOCDDz442LaDNZd34JT0hiz73tThdG0+d9wxGojPObfRsK0ROF6R2zMAq8fv1GBbJvAx\n8XtsV3OOqcm1XXjzzTdr3bftcM1xk3y/B4D//d//DfYPf/jD5GeJNLlWOHx9XXHFFcG+8847o3FX\nX311Rfu29xJGXciFEEIIIT4HPeQIIYQQopBk5apKpQSWLWwqWF3IuapYPrAplD169Ag2p6vZTtJb\nbbVVsDmtsVevXtG4BQsWBJs7blv4PFkJbfTo0cHu3r17sIcPHx6N4/TPpoSVHXhubEVMhiUgmzKe\nex9TFymL922vI06P5PRY60bnyp49e/Zc42MoMjbl+MADDwx2586dg20r5LJs0bt372Bvs8020The\nFzxHF198cd0OuACMGzcu2HxeX3jhhWgcywG77LJLcn/z588Ptq14zPfv3BpheJuVu1i+YknKymR8\nj+Z7qP3cFStWBJur6QKxfMWVuCdPnpw8dhHfM1mqt+Uzjj322GAfdNBBa/w5NkTj5z//ebCt9PjK\nK6+s8f5TyJMjhBBCiEKihxwhhBBCFJKsXJWTJlhKsNkzawtH/wNAnz59gs1uTetqff7554PNbtLN\nNtssGscZHUOGDAm2lWbYHczf0TaX43NhJS8+XpbQvvWtb0XjmqpcxRIEEEtPuarBqcrIQMNWfs5d\nlyyhsW0bz7HL3jaQrHbsGuFMxZ133jnYb731VjSOZYypU6cGm6ujArG0wNLMG2+8Uccjbn7YDKUD\nDjgg2M8880zyfXw/tLJiapytRs4Sfa76Ma9pviZeffXVaBxXV+Z7h5W1WJZiqcI2euXjsPcVPna+\nv0quirFhAHyf5Dl/9NFHo3Enn3xysDkLkiXEHLnfiwkTJkSvbWX1tUGeHCGEEEIUEj3kCCGEEKKQ\n6CFHCCGEEIWk4orH9R13Y+Nk+vbtG2ybtsv6Lds2NoZTDznmwsYTsa773HPPBdvGn4wYMSLY3NXW\naov8Pqtxs2689dZbB9umzlp9ualg4wO4wi1XVbXpnPz9Vq5cGW1LdS+vtNK1hd+Xq7DK1VNZv7el\nCPi6stWQqx2+hoG4tEKu2jTHanDsnK0o7b2v1d5vv/2icdddd92aHHaz4vzzz49e87XK2AqyfK1y\nXIst4fDee+8FO5dCnusGztty6eUM3y9yHc75eO19hd9n7+v8G8X3n7333jsa98ADD1R0vEUld1/l\ne9+RRx4Zbfva174W7LPOOivYNtV8xowZwX7ooYeCbbvGc/XqNm3aRNtGjRoV7Icffjh5vJUgT44Q\nQgghCokecoQQQghRSCr2xXMaNxA3lGT3FzfiA2K5g22bisquVlsNs23btsHmpn2dOnVKHi9LSrbJ\nG8Pp6gsXLkweE7vVrauVU+isrMcuZXbdjRkzJhqX+y5NCZaG2LX50ksvReP4++Tco7y/SsdVWgnZ\njuPrgOfCVsSuS6XlasHeB1h66tatW7CthMFrmu8RVq6aOXNmsNkNzlXKiwifOytlc/r8lltuGWyW\nf4BYOmQ56cMPP4zGpSQpC0tUVq5KSVR2XEqGz8ngLKexDcTymv1efEx8zlhmASRXVYqViVKy0SGH\nHBK9vvLKK4N93HHHBduGLXDpDnvvP/TQQz/3cytFnhwhhBBCFBI95AghhBCikGTlKnbvX3HFFdE2\nziLiar3svgZiKYfdU7ZKIruwrVzAshG7y61Esu2229a6D+sKS2WK2QhvdpVyxHjOdW4rs7ILmSW6\nXXfdNRrXkFWA15RctUl2P+cy7iqVoeryve17+LNy7ne+jvjY2VVux1n4GrPyZlFhiWCPPfaItrFk\nyevbytEsV3HFY86gAuKqyXfeeWewOfuyiFx77bXBthlPs2fPDjZLMjZr6qmnngr22LFjg22zUFmi\nt3JQilx2VW7NsbTPVZjtd+Tmnfwd7eeytMzvse/jbdwYGQB+9rOfJY+3qOSaIld6D06Nu/3226PX\nLKn+6U9/Cratcs3YeZ43b15Fx1QJ8uQIIYQQopDoIUcIIYQQhUQPOUIIIYQoJNmYHE7d3nPPPaNt\nrLsdddRRwT788MOjccuWLQv2448/HmxbrZPjXKzGy1ouV0q2sTGsyXIKudWdOZ6IYzNyqcOc4mlT\nyDmVceTIkdE23j9rw1Yn5vPU2PTu3buicaz12wrWPIc5PbghsTFDHDPC17ZNxeVYG7uNr4Nqiclh\nLf0Pf/hDtI3jQn7xi18E217PXFmc32PXJs8L3yNsmj/Pi42pao5whdff/va30bbddtst2MOHDw/2\ngQceGI2r9Jxw/KONjUxVIq5rRXZecxyHY+OJ+H7B9xU773Pnzg329OnTo21cUZf3z3GRwOpxo9VA\nfXQs4N9Hvqfb32tOFc9VQeffYbvtscceW7uDJeTJEUIIIUQh0UOOEEIIIQpJVq5q1apVsC+//PJo\n29133x3sSy+9NNjsngRiFxenNdoUU3Yvtm7dOnlM7HYbPHhwtC2VosjSGhCnCLNL3LrMWIbi72Fd\nrewafuKJJ6Jt7Or/+9//HuxFixZF42yaZ2Ni5yYFn8ecdJgjl2pelyrH7Fa3blR+zcdu55Mbx02b\nNi3aVt+NapsD3KiWK5EC6eaRdg2zlMIlCmxTXIalQS5TARRDomJ4vfzwhz+Mtg0bNizY3/zmN4N9\nzz33RONYSsxVK2bsGuH7Zm4tVQrfX/n3xMpkXAGZ74021TxVugQArr766mBfcsklwT799NPX9LCb\nJbb0Bf8Wjxs3Ltj7779/NI7Lo/z+978PNkvMQHx/zl0PP/3pT4PNZWjsPZyvBxu+YqXItUGeHCGE\nEEIUEj3kCCGEEKKQZOWqF154IdgHHHBAtO3II48MNlcmtVk23HBv1qxZwbYR71wp2GZmsCuX3WRW\n8uFMDXaH5qoQs9vbNhBryCrE++23X/T6Bz/4QYN91pqSa2jKLtFUtVuLPY8cmV9p1eS6YN/Px8ju\ncevm5WqbXB0WqFzKKxK8pu3aZOmJ5Q2bNbXDDjsEmyuk2vPJVcd79OgR7BNOOCEa989//jPY8+fP\nz3+BZs7EiRNrtS18jnPnhCX0VDYVEEtDtlotzzXvw0pjvI9cJWOWLjgz1t5Xco2MWSKttApzc4fv\npbl7MMvMPXv2jLb1798/2Jwdbfd33nnnBfvcc88N9j/+8Y9oHGcD8v5uvfXWaNwjjzwS7EGDBkXb\nctWR1xR5coQQQghRSPSQI4QQQohCooccIYQQQhSSbExOLnaC06HZttVtDzrooGCzxs46IAAcccQR\nqw4q0wU613E6lbZsNViOMeDUVhuLwu/j72W/I8ecsAZtj3frrbcOtk1b5liEiy66qJZvse6wVX4Z\n/n4cw2Q1VB5nzxe/rrRbOV9/uQrKuTgens+33nor2PY66tWrV/KY6ppK25zh823195NPPjnYfD3Y\nmDveNnny5GA/++yz0bgBAwYE+/777w/2Qw89FI1jDb/oMTmVwuecr1Mb92S7dzO8RnL34VQcjo21\n4X3kYncYjs+xcZJcqsI5F22zHe1TVFreornD52e77bYL9vLly6NxqQ4A9neT09CPPfbYYHfp0iUa\n17dv32BzXK/lmmuuCfYuu+ySHLe2yJMjhBBCiEKihxwhhBBCFJKsXFWX6q72PXfcccca70M0LtzA\nz8KuTXaP24qVXKnUShcMy0GVXm92HMsp7Aa3shbLlFxWwKY7c/kB69q137Ma4KaQu+++e7SNr5VX\nX3012PY8sYx48MEHB9u6urt27RrsDh06BLtSabqIpCoPW8mH5XCWpKw8lWuumPpcS+o4cu/he4Kt\nZMzzyZLU4sWLo3HcwDVHrlpzXZuNNkVy98zjjz8+2ByCYO93qZABu8a4AwBfa9zJAIglKpYrbUo6\nl2/h8IH6Rp4cIYQQQhQSPeQIIYQQopBk5SpRndgqvwy7HxcuXBjsnDxh3Z5c0ZSx8hK7YtmNajOo\neBy7RK17lMexC5sbuwJx5o5tQJmrKlpUOGvFupU5K5KznHbddddoHJ/jhx9+ONiPPfZYNG6vvfaq\nddzAgQOjcbnGns2RujTDtFlTnA3D1cg7duyYHGezlziziY/DZtqw3MTr28pQvF5SxwekKyPbewfL\nV1bqrDS7qjlnSNp7Xy4j9dRTTw12Tg5i+Sp1bwbiRtc333xzsB944IHke3JyGocMsBRW38iTI4QQ\nQohCooccIYQQQhQSPeQIIYQQopAoJkfUGU6vtnE8rHuzlgukUzjXtus4EGvUVl/muCGOFbBxN9zN\nmW2gYbXjpgrHbey0007RNtbcuWI4d3IH4pgLrihtx3FMB+/v0UcfjcaNGDGigiNvPuTiRFLbbHX1\nSjuD57qB8/ty3co5fq7Srua5bbyu+Pg4RgiI7zk2JoepS4xTcyAXgzNt2rTkNht3yFRavZrHff3r\nX88eZyW8/vrrwW7IOZInRwghhBCFRA85QgghhCgkkqtEFlutmF2W7DodOnRoclzOTd1Y8PFyKjyw\nevNUxspX1UC/fv2C3a1bt2gbSwssL1n5kqVCllmGDRsWjWN5ok+fPsG2qai5qtxFIyW92Ea6XNmY\nq3uviVyVkg1sKjfPNUuMVkJjyZhlLdsMmb8Lf5Zt/Mtrk6vuWpqbRGXPGx9/Lg2bJSquFg7EJR34\nXm3vx/yaP7d169bROG7QWSl8fdnvwTJ47p67tsiTI4QQQohCooccIYQQQhQSPeQIIYQQopAoJkdk\nsWndHFvBqcD77bdfNG7IkCHBXrp0aXIfrM3nUmJZU7atFVjD523cggKIuxtPmjQp2HfccUc0jrVj\nm4bO+6gW+Bqw2jmngHNKaO/evaNxs2fPrnXfM2fOjF7vvPPOweZu8C+++GI0rtJu1EXGxlbweuE1\nZruQ83qx+0jFw9g2DEyuMzpfL7k05lQ7ARsLxMdRaZf05oBti5PiggsuiF4754LN6w+I45z43mrP\nG98z+X5s7/1nn312rcdk95dqx2Ph68HOc30iT44QQgghCokecoQQQghRSCRXidXIuXrZrcjp1C+8\n8EI0zr5uynTo0CF6zS52K8/kqqwWFU7Vtam/nD7M5QZsR3ket8022wTbSo8sfy1ZsiTYVvLMucGL\nRiqF3Ep2tit5DfaaZRnKSg08TzzOSk18HGy3bNmy1mMAYqnJ3mNS0pMt2cD77969e7Ttl7/8Za37\nqDRNvjH58Y9/HL0+8cQTg33rrbcG+8ADD4zG8f3JVpbnNcjf2a4dnnM+v/azUuTOZ26d8vVlu9fX\nJ/LkCCGEEKKQ6CFHCCGEEIVEcpVYjSuvvDLYJ510UrSN3Zm5jItcozeOvq+PppwMu0dzmWHMW2+9\nFb3ebrvtgm3lqhtuuGFtD7HZwXLVIYccEm3jxplcrdi6zvncv/HGG8Hu2bNnNI6zslgaffLJJ6Nx\nuWaERSMlBzzwwAPRa86UWrx4cbBt1iJLA7ZydEo2stkv/L5cM1CG7x1WWktVYbbfnSUObtaZoynK\nU5bbbrstes33MV4TtqEtVzXefffdo22VVgVnKZIbb95zzz0VvT9HSjID4qro22+//Vp/VvIYGmzP\nQgghhBCNiB5yhBBCCFFI9JAjhBBCiELSIpfi1aJFi+rJ02xCfPbZZ/UbqFKmLvN59NFHR6/ff//9\nYC9YsCDYNmaiOTFq1KjoNcfk2JTYyy67bI333xDz2Vhrc+TIkdHrgQMHBpvjlzp37hyNe+2112q1\nbaVbjg/glPSJEydG42zq+bqiKa1NsfYUaW02RTgusqHLPqTmUp4cIYQQQhQSPeQIIYQQopBk5Soh\nhBBCiOaKPDlCCCGEKCR6yBFCCCFEIdFDjhBCCCEKSZNq6+Cc2w/AzwB8CmAzAHMBHOe9fyv7xsr3\nfxaA9b33v6iP/Yk0zrnOADyASeV/2gDAfADfS82nc24CgHMAfALgHO/9sIY/UlFXEnP8GICzvffv\np94nmiaaz+ZPQ/yGOufWB/Cx9z6Zbu+c+waAMd77/6nr5zQUTcaT45zbEMD/A3Ck936U934QgHkA\nvt2oBybWhje993uW/xsK4HUAesAsFmGOAYwGsDmAGxv3kMRaoPlspug3tHaakidnE5SePDer+Qfv\n/U8AwDk3D8AfAOwHoAuA4733DzrntgdwKYBNAbQE8HPv/Xjn3E4A/oySR2ALAL/w3t/PH1Z+8jwK\nwBcBDAMwDkALAB8DONZ7P7f8uX8HsKP3/ssN8aWrjEcBHFc+r2O897Odc3si47VxzvUAcDlKD+Tr\nA/gpgBUAbvPeu/KYTgAmA9gewGEATkZpLt8E8B3v/TLn3EoAVwNYz3t/SoN9wyrGe/+hc+5UALOc\nc98DsBeA1gAuAvAESvPYFkArABd67290zo0C8FsA7wPYGMApAJ4FcBUAB+AzAM96709c19+n2tF8\nNjtyv6GHADgNwIco3Ue/7r2fV/aejwewB4AeAMZ5729wzjmUHpjeB/Bwzf6cc+0AXF/eRysAf/De\nX9fwX63uNBlPjvf+bZQeNKY558Y7504vn+gaPvDej0VJzqj5kboMpcW1F4CDAFxVdq21B3CG9350\neeyv+bOcc3uj9HR7GEou2csBHOq9HwngEgAX0PBZesBZe5xz6wE4FCX395pwCYDLyn9ZngDgOu/9\niwA+cM71K485AsBNALYFcDpKD1DDAEwA8PPymJYA/qUHnIbFe/8xgKdR8gD0B7C/9/4elNbtfeW1\nOgLA2c65tgBOBXCR934UgG8A6ACgL4DB3vsh3vs9ULontFr900RDo/lsPnzOb+iWKHt4APwLwEn0\n1pbe+/1R+k08rfxv4wD8pfyb+DyN3RbAH8vzfiBKD7xNmqbkyYH3/nfOuasAjAUwCsCTzrmflTdP\nKP9/PoCtyvYoAJs758aVX38MYBsAiwCc75z7NYANAbShj+kL4LsA+nrv33PODUJpId5Wvh7WQ+mv\njRqeqL9vWHW0Lf+lAJQeqB8DcDFKDyuVMhjAkQDgvZ/unNvCOdcGwA0ADkdpAR6J0pwOQWku7y/P\n5UYoadJAybPz+Np8GVExrVCKCZjqva/p9TAKwEDn3DHl1x+j5JW9EcBvyuvwTu/9Xc65jQEsdc79\nC8A/AdxcvoGLxkHz2UzI/IbOB3Ctc+4LKDkBJtHbJpT/z7+tfQGcW7YforELAZzmnDsNpWti64b4\nHvVJk3rIcc5t6r1fhtJf5Tc5524BcGF5MzerqQmA+gglD8xSs58HANzkvf+Lc64PgLtpczeUJvUk\nAGeU9/Fq2VNQG/+p+zeqet6s7bw65/ghcsPP2YetVtmi/G83AbjPOfdXABt776c553YA8JT3/sDE\nvjSXDYxzblOU/uK/CfH5/giloPOnzVuecs7dj9JN+Uzn3FPe+58DGO6c2xWlvxanOOeGeu8XrYOv\nIAjNZ/Mi8Rv6fwC2A7Cr936Wc+4kALvR22r7bW0B4L9lez3afg5K6sZXnHMtAbyDJk6Tkaucc/sA\nmOSc25z+eUcAszNvm4iSVAHnXBvn3O/L/94OwItl+0iU/qKv4XYA3wRwmHNuJICZANqUH4bgnBvh\nnPvu2n4fkWUlgE5le6/PGTsZwD4A4JzbBcAy7/0y7/1rAJYC+DFK2jEATAEwyDnXvjz+y865g+v7\n4EXtOOc2QOmG+gBW3SBr4LUdXicrAAAgAElEQVS6iXPuUufc+s65X6IUJ3UzgO8DGOKc2805d4z3\nfqr3/mwAz6AULyDWIZrP5kXmN3QRSvM3r+xVOxjxb2JtzEDJMw4AY+jf+bf1qwD+65z7vH01Kk3G\nk+O9v78cZPqgc+59lJ4klwA4EbFrjTkFwBXOua+gNGnnlP/9QgDXlQNcLwJwqHPuQpSfOssy1f8A\nuAXAQAD/A+Bq59yH5ffrIadhuRCl8z0Tny8hnQzgcufc8SjFT32dtt0A4E8oLWR47xc6574P4O7y\nNfQ+gGMgGpIaSXI9lIJS/42Sl/QoM+4slGLmJqK0Vq/w3n/inJsF4AHn3IryPsYBmANgnHPuOJQC\nJedAUuO6QvPZTMn8hn4NwJko/RE4H8D5AK53zuViTc9G6Tf0yyjNVY23548ALnHOfQfAXwA8iJJE\n+c8G+Er1gnpXCSGEEKKQNBm5SgghhBCiPtFDjhBCCCEKiR5yhBBCCFFI9JAjhBBCiEKihxwhhBBC\nFBI95AghhBCikGTr5LRo0aJJ5Jevt96qgouffvppIx7JuuGzzz5LtrRfG+p7Plu0aFGrDQC50gSp\nbbvvvnv0ul27dsF+//33g92yZcto3KJFqwqnTp48OXPEtbMmx14XGmI+62Mu+XvnvvPQoUODfcQR\nR0Tb2rZtG+yvfvWra3tIEaNGjQr2vvvuG2276aabgj1t2rTkPr7whVV/x/33v7ae3ZrTXNZmXeHz\nvMMOOwR7o43iem8ff/xxreN69+4djeP3jR07tt6Os75oqmuzLrRv3z56feKJq3qgvvbaa8HeZptt\nonEbbLBBrfZHH30UjeP1w9u6dOkSjfv1r1e1ilywYEFFx14fpOZSnhwhhBBCFJJsMcCGfCLlJ0Yg\n/svgl7/8ZbRtr71WVf7nJ0j+Kw2I//K79tprg73xxhtH4/gv06lTpwZ7+fLl0Th+cv3rX/8a7IkT\nJ6IhaS5/LVbqCchxyCGHBPvoo4+OtvH5nzdvXrD79u0bjRsyZEiwN998c6wt9fG9mKb612Lqex5+\n+OHRuFNOWdW4/Z134lY1vI8lS5YEmz0tALBw4cJgs1euR4+4uv9XvvKVYG+11VbB/uSTT6JxvKaf\nf35Vk+Sf/OQnaEiay9pk77e9T/K91rK217v1tPNxPPnkk8G2Xlumvr1vOZrq2qwLF198cfT61FNP\nDfayZcuCba+Ht956K9ibbLJJsO1af/vtVT1Ve/XqFWzr5bvuuuuC/a1vfauiY68P5MkRQgghRFWh\nhxwhhBBCFBI95AghhBCikDRaF/KcLtytW7foNWt+7777bq3/DgBPP/10sDt37hzszTbbLBrHEd+s\nM7INxDEG9RHrUTRy+j2fcxtDc9hhhwWbz/krr7wSjevZs2ewOQbDZgf885+rGuCOGTMm2BzTAcSZ\nV3wN2O9RLU1rU9/zoIMOil7zmvvPf/4TbeOYHJ6XM844IxrHsRnMc889F71OZXfYz+Vj2mmnnYJt\nM0wWL15c6+c2V3KZphxrwdvWJCP11VdfDTbHw9g4Ds5w5LlZuXJl8pj4vs6xHwDwwQcf1Pq5lmrL\ntF0TnHPRa/495PO76aabRuOmT58ebL5v2+yqHXfcMdgcy2p/h7fbbrs1OewGR54cIYQQQhQSPeQI\nIYQQopA0mlyVo3v37tFrdm2uv/6qQ54zZ040juUJdrutWLEiGsf74GJmtqgRF1BaunRpRcdeTey2\n227BtpIUn1cri7BUye5tTjEFgEcffTTYLEPtvPPO0bjf/va3wWa3rJWrRo8eHewtttgi2LNnz47G\njR8/PtjWZVtUOJW7Y8eO0bb33nsv2FZ24rnlc2VlIpYbeV7YBQ6svlZrsAUbWdLgY9h+++2jcUWT\nq1ii4fsYsHqafQ3Dhg2LXnOa/YEHHhht4/PKUrLdN88HrzkrV7F8yBLXzJkzo3EsVY8bNy7YEyZM\niMbx92/oIp7NDRtSwWuOt9nwDZaM99tvv2A//vjj0bgOHToEm9e6LQfT1GREeXKEEEIIUUj0kCOE\nEEKIQtJk5CqOyLauUe99sNlF27Vr12gcu9XZ1frGG29E47jXCmcGvP7669E4dr2yvFHNcIVilqts\nthxH31v3Je/jzTffDPZRRx0VjWM36pZbbhnsW2+9NRrHx8GVPXNSE19vW2+9dbSNr7E777wzuY8i\nwRVMc5JUTjZirJTCkgbPOf87ELu++br58MMPo3F8jGzb3klPPfVUrcfXXOFspZQ8BQBXX311sL/2\nta9F27ji9Pz586NtnCnTqlWrYNvqt3yvZVnfZsaybM2Vde31scsuuwT7wQcfDPYtt9wSjeN7hJWn\n6rtSeXMjlyXK0hXPAxD/tvH1ZTPqeK3yPdOuzdx12RjIkyOEEEKIQqKHHCGEEEIUEj3kCCGEEKKQ\nNJmYHE4bt7o/x81wrM2GG24YjeNKjlwR1VZg5DRT7jxu08Q5loc/i7Vqe0xFw6Zh9+vXL9ic9rn3\n3ntH4/ic21R/jglgbZ9jNQDgpJNOCva5554bbBtncfrppwebU6H5GgDiSstc5ZO7WFvatWsXveYq\n2EWCY1lsDBVf+xwbBQCLFi0KdqriLhDHYLBmbyvfpir12orHfBwce8DVj4uIvTcynA7O3Z+5ujcQ\nx1DY+Ctem3xfy8V78P3QxmPwZ+euD4734HvCEUccEY3785//HOyHH3442pbbfzVg45z4WuFYq7lz\n50bjWrduHWyeP7s/jrvk68TG5dmYn8ZGnhwhhBBCFBI95AghhBCikDQZuYpdYVZmYBeabS7GzJgx\nI9j9+/cPtpWr2J3GqXUWlq/Y3WffU2S5auzYscltnGI/ZcqUaFunTp2CzY0xgXgOuRLn3XffHY1j\nSWLevHnB3nbbbaNxV155ZbA5hdVKSxdccEGwWYJhdy0QS3S2+nZR5SpurmndzywDWLmKJUaWMGz6\nKW/jNHErg/A2llWsNM0prHwdNrXmgPVNrnnl0KFDg50rn8DnMtcoOQdLIXxMNtWcJQ++h1r5kdcg\nv8eWGPj6178ebCtXVaNExdhUbpaCeY7stcEVkHlecs14eZuVnOt6TTUU8uQIIYQQopDoIUcIIYQQ\nhaTJyFX77LNPsK1cxXIQu8vZxW7fl6qwCsTyFzeK40wrIHaDswvVNiQrMlae4Cam7L60zTU588pW\nFOb38Xm1UgOfZ55P24SRj5Hn2mai8HHwtWKrWfM4e+0UFZYN7XlLZUYB6YaRueaJLGXlqtamrhN7\nvJzpYe8JRSNXyXevvfYKdipLDUifYzuWP8vOO88Hy1+5itgsk1jZjUMAeJzd3+DBgyFqx0p7LOuz\njGgbeTK8zYZl8PXA14mVpptaM2t5coQQQghRSPSQI4QQQohCooccIYQQQhSSJhOT06dPn2BbDZk1\nWu4yzd3JAWDXXXcNNqcrWj2fKz7yZ9nqnxwTwml2Ta3LakNiNXE+RxwLwd2Ggfh88ZwBsYbbsmXL\nYFtN+YUXXgg2a8U2JopTvjkmw3af5/gaTlnl+CE7zsYT8bHn0nmbGx07dgy2XX+c8m/nefbs2cFm\nzT53bnhcrpIun2u7hnl98zXK15Mdl0urLgJ8jvj823PH6zaX6p+LiUrNjb038v447diWBEhdL/Za\ntGnNYhU2fpDLafC91Vax55jXSy+9NNijRo2KxvFc8Lqya5iroDcF5MkRQgghRCHRQ44QQgghCkmT\nkavYNcruVCB2k+2www7BthUe2Q13wAEHBPuuu+6KxrFLm2UVm7qewrpaiwbPhU0jZJf/brvtFuz7\n778/Gsfpi3Y+2dXN1TJtYzce16tXr2CzRAIAu+++e7C58rKtRM0uVp5D51w0jqUsrrQMAO3btw/2\nwoULURRYhrLzsGLFimDbUgEs57EsaeWNlAySS2HmtTlr1qxoHF+HfE+w6atDhgwJ9oQJE1Bk+P7H\nVWetvMvnjs8xEMtNvP6srJWbw9Q4xspT3JQzlU4OFL9EwNrw8ssvR68HDRoUbD7fVl7ie/D5558f\nbP4NBWKpMFe2YfHixWty2A2OPDlCCCGEKCR6yBFCCCFEIdFDjhBCCCEKSZOJyenQoUOwFyxYEG3j\nFFbW/W03ak45fvrpp4P9yCOPROMGDBgQbNakbcoxfy7HcxS92y1rtu3atYu2cTxT586dg21jJjhe\nqm/fvtG26dOnB5u1fqu/83XA+7Op/hxD0rt371o/B4jnlz+Lrz0gjjOxn5WLP2hucOwRnxu+7gFg\nzpw5wbZxToceemiwcy0w6nLe+DhsXMmNN94Y7HPOOSfYNq6Or72ixeTk1ibHT/C9C4jjMyq9l9n5\nq3Q+eRyvJft+bjvA38PeE7gFC1+/QBw7Vo1wPCIAnHzyycHmFHIbX8Xxc2zbcQzHNNr4O/tb0NjI\nkyOEEEKIQqKHHCGEEEIUkkaTq3IVTDmdEIhdr9yp2rorL7nkklo/61//+lf0ev/99w/2Sy+9FOxW\nrVpF4zhNjl281VTxeMmSJdFrTk3lzvHf+c53onHsBrcVjzkNm/dv3ers3ubPtdfO1KlTg81u2UMO\nOSQax7LW448/HmyWuIBYdrEuW1uVuTnDHeVZDrLz8Pzzzwfbng9O92UJwo7j9cPbrGzBa4vX3447\n7hiNY+mJx9n98XcsGlxWAYjXBcu7Nk2c4XUFpOcmJ10wORkrVToCiO8RuWrZLJNwlW5ActW0adOi\n13yvSlUSB+JrgNdfpZKkvSfaCvKNjTw5QgghhCgkesgRQgghRCFpNLmKJQsgdnnZiP9UJLdtBHbz\nzTfX+lm33HJL9Pq8884LdqrRHxC7fNntbeWSosGSIDfGBGJpiDNZbLNUdjnbytQpN6r9LM6sYFkk\n10CSszT+8Y9/JMctX7482PZa5OwAe01w5gfvozkycODAYLM0aDPKJk2aFOyhQ4dG21KVb+15Yxd5\nLsuG1xbPv80k4muIXeycfQOs3lC0SHBT4xw2W67SCsU5iSolP1p4rnmebNVdHsf3+1y19K5du0bb\nOLu2GrFhAbZyeQ32nPJvoL1XV4K9H9sQh8ZGnhwhhBBCFBI95AghhBCikOghRwghhBCFpNGCS0aP\nHh29Zv3QxgSwZsiaPccK5LA64/z584PNqbP2c1l3Zp24SGnEtcGpmbbSLKf3c8wEd0AGgLlz5wbb\npiQzrOfatFLeB8fr2Ngd3gdfH1tuuWU0jjVr3mbjNvg72/TbIsVjcckEvqZHjhwZjePuxmPHjo22\npdJ9K437yMXkcEqwnSNej3wdrly5Mhpnq5gXCZsez+cklwqcio+qbWwNNoamUlLd520cB2/j+4W9\nvvh4u3fvXqdjqhZ4TfN9zMZQ8Wtec7aUS2rd2nH2dWMjT44QQgghCokecoQQQghRSBrN926ba7Jb\n01bhTDUQq6tbjN3vzrlgW9coV1RmN16uImcR4LmxbmV2babSeIFY8rHbuAEmpzNaaYznmo/DXh+8\njV3dNlWS05DZ7W2bAOZSYm06bnOGSymwbaVHxlbZ5TIOLJdY6tKgk+fSzsmgQYOCve+++67xvouA\nrQLN13SuCSdfw3aNcKkGxt7zKp3PVKVdK3+lpJBcA9EddtihomOoVrhyO5dWsHNp77s12BIZqcbF\nTU2essiTI4QQQohCooccIYQQQhSSRpOrrOSQytoBgG222SbY7PKsq5uM3fGpjAQglmO4kq7NSCga\n7M60rk3OvGJX8osvvhiN23nnnYNtM164UjK7RK37nd2j7MK27lXOoOEMLXt9cJYdN5Hbc889o3Hs\nzrdu9SJlV6XIVXLu169f9JrljpzMx/PH68ee31Tmz5w5c6JxXK154sSJyc8tMrZBJZ/LnKTO6+rt\nt9+OtvG1n6tqzGu/0nnPhSTwfZjXaW7f9n4hYl5//fVgcyZerikuY6VCvvdx02y2myLy5AghhBCi\nkOghRwghhBCFRA85QgghhCgkjRZgYLVWjn+xGh/HYPD7cnotb7P6NOvQHTp0CLZNn+QqzFxlt+hx\nGRyzZPVbTkV88MEHg/3cc89F44YPHx7sxYsXR9s4VoZjaHLVV1kfzqWf8rxbTZlfc6dcjhEC4tRl\nmxbNVYKbO6m03Vw8RyrdFMjH2lTyufZ9uTVsu5JXIzYmJxVbYeNf+D6Xi8/IpXyn5jc3n7mYHI67\n5GOw9wT+bbAVn0UM/37l1lLqd9T+Dqfiq2wMbVNDnhwhhBBCFBI95AghhBCikDSa7mJdZuyutK5o\ndlmyXVfZiBs1tm/fPthc4RgA2rRpE2yWVepSvbU5we5Lm/7NUhbLULNmzYrGcVVpi23EWYOVhlga\nYakp527lVH8rP7LkyMewdOnSaBynsFp5JtdstLmRap6Ywza8TEkflcpQOekxVcEXWL1hZzVi71ec\nzs/XsJUT+Nq31zOvs1xV8JRsmZO1ctcYN5BkmcSuYT4OVTzOwxWPWR6099nUvNh7P88lXzf8OU0R\neXKEEEIIUUj0kCOEEEKIQtJochVLRkDsluzRo0e0jTOv2E2Wy7DIuUZfeumlYLNb1zYNZRdfLpOr\naLAUwA0YgbgiKruO7bnjbbbyMMtGPE/Wdc7zzhlQXLEViOeJXd02g2PBggWojZYtW0avOSvBZmjl\nsoaaM7n1wvOSk4hT2RdAev3Y7BmWpXLHVE3rMYWVjPic83m1TTh5m11LnHmak4iZXGYeb+P1mJOc\neQ3bewJvY4lLrA7LSHwebdZUqmKxDStIZa7a34imhu4UQgghhCgkesgRQgghRCHRQ44QQgghCkmj\nxeRMnz49es3xNTYljbVB1qFzaaS52AnuQM1YbZJjQpxzwea4lGqD01anTZsWbE63B+K4qtdeey3a\nxjECqTgCINbcuQqqTW3kKswcu2M1e477mjFjRrBtnAlrz3wNAMVKIWcqTe+16aepWA27Pz6nvM2e\ne9b6c8dkj6Na6NOnT7Dtuedzx+fVrqtUN2kgXTXZ3k9T+7CxUqn7sE1J59ccJ2lTyHmbPdauXbsG\n23atr0Y49pTjoWycoY1drCF3r+Nravny5XU9xHWCPDlCCCGEKCR6yBFCCCFEIWk0uWrq1KnRa654\nbN1k7AK1rtcUOblq4cKFwWbpyUouXAk3l0JZNNidaV2bnHLKqYO9evWKxuWkBpYcc6mNLEnwPNny\nAyn50B47y1q8D7u/bt26Bdumndt0+KKQmy8+b3ZtsvSYkqTstty4VEVz61K3qc8pcmntzZHtttsu\n2Pb6ZsmHJcZc1e558+ZF27gUBMvCVlZMnVd7f+Z5S5V6AGJZmNdzrnK2lZJZIpdclf79snNpK2fX\nwNKgha8hpZALIYQQQjQCesgRQgghRCFpMg06WUKylYy5ESS7L22VXSbnmmZ5gt3t1hXPlXmZVIPJ\nosDuS66ACsQu8iVLlgR7n332icZx5oN1q7Orms+lzYZiWcNmWaTG5eaG5S/+rBUrVkTjttxyy1rf\nA1RndhVXhLbZM6n3VSpXWXmDr5tcVWO+9nIUTa7q0KFDsG0l41TzysmTJ0fj+JzYat+8jeeC1wRQ\neWVq3sb3AbuOWFrhNde5c+doHN87rJySq4BfjXBWK18rVp6y978abAYczznPw5QpU9bqOBsaeXKE\nEEIIUUj0kCOEEEKIQqKHHCGEEEIUkkaLybGwbjxy5MhoWyoVjnXnNYFjfFjXzVX15NiBosfksBZr\nU3ffeeedYHPqIJcAAOJO3rl4Go75samNPNf8uRYex7E2b7zxRjSOY6xYl+ZjBWK92abf5uIPikqr\nVq2CbWNoUungufgXHmfjbnj/PM6e90rnoQhxOEynTp2CbWNS+DVft4888kg0bujQocG2qfh2LdSQ\nSw2v9BzzXNtYNy7VwNXwTzvttGgcp43bfXC8kojh+6ftFJDq5m7jpvga4HukUsiFEEIIIRoBPeQI\nIYQQopA0GbnqrrvuCrZNR+YKmJzWaF3WHTt2DPbrr79e0efmGjqyO5RdrVZWKRrslrRSE8tLnFZq\n0xJ5nK1uyvvn88qVdYE45ZRd8bbCccpNb0lVX81Vs7ZySjU2huR1ZmULlipyMlRK1srJULkU8krl\nqqKlkLdv3z7YuRRf5vLLL49eH3fcccG2zRX5fsjVlW114dR9OFehmNecnVsuB/Kb3/wm2Fau4v1Z\nuS7XsLna4WvF/n5V+nvGYRosf9lSBk0NeXKEEEIIUUj0kCOEEEKIQqKHHCGEEEIUkiYTXHLfffcF\n26YxshbIsR42PmLPPfcM9g033FDR5/Jn2ZiQlIZfaQfk5gqnDnL6MBDHZHDauO3kzbE8Vn+3MTo1\n2PPNKaGsv3MLECC+DnKp/lzCnr+j1fZ5HJcvsJ9VJHIxLqlSCpZcp+NUW4dc7E6l43IUIQ6H4dYF\n9tpMtaGx8YkcuzJ37txoG8fWcUycXbOp9ia5dh6p+Bwg7iA+ceLEWvcNxNefbUeQSoUWcUyVXRM2\ntqsGe19MrX2Oz2qKyJMjhBBCiEKihxwhhBBCFJImI1ex+9KmK3KqMm+zbra99tor2JXKVez+tR15\nU9VXObWyiOQ6s3PKKY+z8gFLFza9nNMPOSXWulH5s1lKtF3NeW44hTyX/s1zbeUvdvtbd35Kaisy\nfO7tHKVkLjsu1bW60v3VVa4qWgo5V3m3979u3boF23YeZ1heyHWBZznIlmbg13wcVrpIpSfbNZwq\n/eC9j16zhP3iiy9G23r16lXrPkR837bhIKnfs5UrVyb3wffwVJXspoI8OUIIIYQoJHrIEUIIIUQh\naTJyFXPsscdGr/v37x/se++9N9jWxVmXzJfvfve7we7cuXO0bebMmcFm153NaigaLMlYCY8zNbgB\npnUVz5gxo6L9szRkqyvz/OaaorKcye5y27CvZ8+etR77k08+GY3jLA0rTxW9OWttsKRh5Q2Wg3j+\ncnPJ+7BrNiVHW3kjld1TdJ555plgjxkzJtrG0tCQIUOS++BxuSrjXEHenm9+zXNmrw++DliytFKb\nrWJeg614fM011wTbNgWeNGlSrfsQwE477RRsmzHLIRu8rV+/ftE4nsvU/DdF5MkRQgghRCHRQ44Q\nQgghCokecoQQQghRSBotJsemFnIshY2D2G+//YLdvXv3YNsKlxybceKJJwY7pxl26tQp2F/84hej\nbS+99FKwOfbAphX/7W9/S+6/OfLee+8F26YXproAjx8/Pho3b968Wt8DxOeS592mi3IMDccH3Hbb\nbdE4rkDN2r6ND1iyZEmwOf30tddei8ZxSnmbNm2ibXZsNcDlHWxaKaeS8nmz557nhVNR7X2Ax/F1\nYmM4OHaE4wPsuEq7lTcXuIN4XfnWt74V7C5dukTbuBoyx1HZ2B0+z5xCbGN3OIaNq6Lb2LY//vGP\ntR7rXXfdFb3eaqutah0n8px//vnBHj58eLSN1yN3FDj99NOjcfvss0+wed039dIM8uQIIYQQopDo\nIUcIIYQQhaRFU3c1CSGEEELUBXlyhBBCCFFI9JAjhBBCiEKihxwhhBBCFJIm1dbBOXcegEEANgaw\nC4CaOt1Xe++vb7QDEw2Cc64DgPMB9AVQk5N4lvd+fPpdyX19FcDfvPf//dzBol5wznUG4LFqnW4A\n4DEAZ3vv30+9TzR9tDabD/rdzNMkA4/LN8+J3vvae8CLZo9zrgWAyQCu897/qfxvfQE8AGCo937O\nGu5vFoCe3vtPPnewqBfsOnXObQzgQgAdvfdfasxjE3VHa7N5ot/N2mlSnpwUzrmzAHQBsAOA/0Xp\nL4vLUZLb1gfwU+/9ROfcNShN8lXl932G0l+XwwH8FsD7KD3tnuK9n+KcGwVgHIAWAD4GcKz3fq5z\nbh6AvwPY0Xv/5XX0NauN0QA+q7mJAoD3frpzrieAlc65SwAMAPAZgIe892c4576A0rzvBGAjAE96\n709xzv0SQDcADzrnDvHeL1/t00SD473/0Dl3KoBZzrnvAdgLQGsAFwF4AqW5awugFYALvfc3ltdg\ntDYBPAvgKgAOpfl/1nt/ov080WBobRYA/W6WaE4xOV0AjPLePwPgEgCXee/3BHACgOs+572nArjI\nez8KwDcAdHDObYrShB/qvR9Z3ucF9J5ZTWmiCkhvAFPsP3rvVwA4AqX5HgpgBICxzrmRKP1gPu+9\nH+G9H1z+9z7e+3Hlt4/WTbRx8d5/DOBpAJsD6A9gf+/9PQDOAXCf934vlOb0bOdcW9SyNlGSSAZ7\n74d47/cAMM0512r1TxMNhNZmcaj6381m4ckpM9l7X6OtDQZwJBD+wtjCOdcm/VbcCOA3zrlBAO70\n3t9VtjsAuM05BwDrofSXSQ1P1Ps3EMynKJ3z2hgMYHx5vj91zj0GYCCAiQA6OecmAfgIpfnLzbto\nHFqhNL9Tvfc19f9HARjonDum/PpjlG7Ata3NjQEsdc79C8A/AdzsvX8bYl2htVkcqv53szk95HBD\nKxtI1KL8b+HfnXMb1tje+7875+4HMBbAmc65p1Byq71afqr9vM8T9c90AN+x/1jW/lPzexRKN9Th\n3vtPnHNPN/hRijWi/JdefwA3IV5DHwH4nvfeztlTdm16738OYLhzblcABwKY4pwb6r1ftA6+gtDa\nLBJV/7vZnOQqZjKAfQDAObcLgGXe+2UAVgKo6bg5GuXJK+vC63nvbwbwfQBDAMwE0MY516c8ZoRz\n7rvr9FtUMd77RwC845z7ac2/Oed6A7gLwGIAezvnWjjn1gcwEqU5b1d6q//EOTcAJa2/piNgjY4s\nGgnn3AYA/g+lAFWbSTMRJakDzrlNnHOXOufWr21tOud2c84d472f6r0/G8AzAHqsu29S3WhtFpaq\n/N1sTp4c5mQAlzvnjkdp8Xy9/O9/AXCzc24EgH8DqHFxzwLwgHNuBUrutXHe+w+cc/8D4GrnXE07\n7SY9WQXkAAAXOedeALAMwIcouVOfBrAtSj+M6wG4w3v/uHPuVQD/dM49AuBxlLTg/3PO7Q7gPgBP\nO+cOWtPsD7FWtHXOTWMKIDIAACAASURBVEBpnlqjtO5OQukve+YsAFc55yai9ON3RfkHcbW1CWAO\ngHHOueNQuibmoDTfYt2htVk8qvJ3s0mmkAshhBBCrC3NVa4SQgghhMiihxwhhBBCFBI95AghhBCi\nkOghRwghhBCFRA85QgghhCgkesgRQgghRCHJ1slp0aJFk84vb9GiRfQ6lQ5/9NFHR6+/+MUvBvu5\n554L9osvvhiN69y5c7AHDRoU7NatW0fjttlmm2Cfcsop0baJEyfWekw5PvvssxafP2rNWZfzOXTo\n0GDfd9990bYHH3ww2Outt6p6/PLlcWubDz/8MNj/+c+qQpqffBI3M15//VWXMe/v/fffj8aVy5AD\nAO66665gX3nllYlvUT80xHw21trs2rVr9LpXr17BXrJkSbA//fTTaByv1U6dOgV70003jcZtuGEo\nuIotttgi2K1axa2rzjnnnGD/97+27mDDUYS1yfDaAeK1NWDAgGD/5S9/icadddZZwb799tuT++fr\n449//GOwTzrppGjcjBkzKjvgeqZIa7NHj7he5tlnnx3sbbfdNthPPBF3XliwYEGwP/7442B/4Qux\nD6R3797B3mOPPYJ92223ReP++te/BnvhwoUVHXulv+U5UnMpT44QQgghCokecoQQQghRSJprWwcA\nlbu4dt111+g1u+fYxbfPPvtE45599tlgL1q0qjdghw4donHnnntusPv06RNte+mll4K9bNmyYFtX\n4Lp0ua8LunfvHuyPPvoo2savN9988+Q+WHrabLPNgm1lLT6X9rMYfl+3bt2S46oRPoe5a5ElByCW\nkT744INgd+zYMRrHa5Wlx7Zt20bjVqxYEWxec6NGjYrG8XXz4x//OHm8Ik9OFnjvvfeCbSV6lqt+\n97vfBXvp0qXRuJdffjnY06dPD7a9/1U7lf6WsQQFAGeccUaw33777WgbS8abbLJJsIcPH17n46xh\n5cqVwT7ttNOibT/60Y+CzSEHhx9+eDTu8cdXdWppyM4LutKEEEIIUUj0kCOEEEKIQqKHHCGEEEIU\nkmYdk1MpG2ywQfR64403DjZrzXfccUc0juMNOFagf//+yf3tvPPO0TbWpCdMmLAGR9282WGHHYL9\n2muvRdtY63/rrbeCzbEaQByTk4PTHlmHtrElb775ZrA5jVnkGTZsWLBtKve8efOCzengHIsGxHPB\naco2FoFjB3iO7r333micjX1LUWmsUbVizz/D88Sp/UAc+8ZxF7Z0AMdm8X1AMTkxuZiUa6+9Nti2\nHArf02w8Ip9jLqdhyzbw6/nz5wfbxmFxuQFbniNFy5Ytg/3www9H2ziW5/e//320ja/LtY3X0ZUm\nhBBCiEKihxwhhBBCFJJmLVdV6n6++OKLo9ecQvelL30p2FylFwB23HHHYE+ePDnYnB4NxHLVpEmT\nom3WbV9D0V3nfO5s1Ut2b/O5438H4nPEkqN1iTPssrWueHbfNlaF1aZK7nrk0gpW+mV5ieeFJS4g\nTm/lcgA2hZy59dZbg81yMRBXr2a393nnnReNK/o6a0h4bVpYrmBZy8pQfE1svfXWwbbXkUiz7777\nBpvLKljsOWWZh++FVmriMAEur2LnMlXuw1bNZvizOKwAAL7//e8H28pV9ZlSLk+OEEIIIQqJHnKE\nEEIIUUiatVxVKbNnz06+5uyQ7bbbLhrHrkG2f/3rX0fjuGoyV/8EYqmGGxhWE++88070mt2enLVh\npQXOwuJMK5tFsNFGG9VqW3c7V+lUdlUaW5mUK4YvXrw42rblllsGmytK//vf/47GcUYiz+WUKVOi\ncY888kiwOTPDrk2urLvnnnsG28pfqoacJycLsKRrq4zzHPL65vVn989rbubMmWt+sFUEZw9yRmNu\nHix87lk2zMlLnB1s5Sq+7/Ln2muIpTGW0KxcxY2tbVYyN85eW+TJEUIIIUQh0UOOEEIIIQqJHnKE\nEEIIUUiqIiYn1+HVVmGsC/fdd1+wx48fH23jVFe7rchwFU2rI6fSR+2/p1IWbcVOfh/btkorx+TY\ndPVq55JLLgl27969o21cqdbq7/369Qt2mzZtkuOmTZsWbJ4/G4c1ZsyYYLdv3z7Yc+fOjca9+uqr\nweaYkN122y0ax+UjfvCDH0DE5FLseZtdixyTkUont6+XLVsWbBunJ2L69u0b7Fx1aJ6jXPXqSmNo\nOEaVq9bbz6q0Gn3qc4A4Nmjw4MHRNsXkCCGEEEJ8DnrIEUIIIUQhqQq5KpcmyS6z3DhOwTvllFOi\nbZyqzO4+ABg4cGClh1ko2L3NTd+AWFLKVVVllyiff+sqZemJJSmbKsmu80obzBUZTunt1q1bsG2p\nA54jrnAMxHPLMhQ3XgXilNiUhGE/i0sIWPgaYne+rYzcoUOHYOeuh2ql0sqyb7zxRvQ6JSXbauR8\njm0KsUgzaNCgisaxBJQ7v3ztW4mS97HFFlvU+u92HzyvdZGu7PtYnqtv5MkRQgghRCHRQ44QQggh\nCklVyFU56uKyPuyww6LXLJFMnTo12sbZIuw6t43WcpHxzZ0PPvgges1SUapZp32da+jHWVRcldO6\nb/lzc1U/q4URI0YEe6uttgr266+/Ho3jyqTcZBGIZQyWl7jpH5CWsmzl6VdeeaXWY+XKynZ/LJHY\natgsr9kGvFxduVrJZZ5yk9UuXbpE47iRY+vWrYNtsxa5MStLIaNGjYrG1UeWa5HgzCaeEysNVRqK\nweQaF/M918padcmuSjUJtfuzmVz1iTw5QgghhCgkesgRQgghRCHRQ44QQgghCokCEwirM7LWz5WL\nbUXYSZMmBdumxHLXZu6WfNNNN63VsTZ1clVQWbfPxdrkttXlPaw9V5o6W2R69OgRbI6nsZ28OV3Y\ndiHn2JjOnTsH+6mnnorGcXzAm2++GWyOZwNWT1Gv7fiAeG1yHMG7774bjeM1zd3UAcXkAPl1cMIJ\nJwTbxmdwnF0qxs6+ZvuQQw6JxikmJ4Yrd3N8Wy6exsbBpeJmbJp/CvtZ/Jo/114bHCPJx2CvDY6f\nGzBgQEXHVBfkyRFCCCFEIdFDjhBCCCEKieSqCjnmmGOCbdNo58yZE2wrZXFqrq0Cy1STfMLu7VRz\nTSCWSVgufOGFF6Jx3OyPXaJW4mC3aqUu2yLTq1evYLOEZNM5OU2c04WB+Jxy9WNbETfV7JHLKgCx\n3MtzxCnLQLyucqmtvI8dd9wRonI4zdtWCGe5gtP7ly5dGo1LyRVdu3att+MsIlxagdPw7fXNcrEN\nC0iVJcmVDeCyG7YRMr+P32MbiLI0zdKVLenB63bbbbet9VjrA3lyhBBCCFFI9JAjhBBCiEKihxwh\nhBBCFBLF5BC5OA1uz2DbFEyZMiXYNiYn1Um7mjsic8ovnxPbToA1XI7J4Y7ZAPD4448Hm89jbj6X\nL1++BkdcTDiuZdasWcHm1HIgjpWyKd+pbsT2euaYH04Tnz17djSO57xly5bBbtWqVTQuNbe2rQOv\ns4YsHV9EOD7KxstxPEWlpRn4PbZMgUjDqeEc4wLE5z6XXl7p70uurUMqbdzG5HDsVa59zrr6zZMn\nRwghhBCFRA85QgghhCgkVS9XpdLiLLvsskuwX3755Wgbp1dalyG73FkGsF2VOQ29CHBat01FZFmD\nz5dNj1y4cGGwv//97wf7oYceisZxxelUxVyLTXWtBmy69mabbRZslols1eCOHTsG20qKqWrD3HHa\nbuPPtWuOXd/s9uZO6EA8f+zOt9caS8s2rV2sTv/+/YPN85Qrx1DpPZTliVy5CL53VAt1lVJzvz38\n2kpPqXE8L1aGSu3DzjnvL3dt5K6VYcOGBXvixInJcZUgT44QQgghCokecoQQQghRSAolV6Wi/HNu\nMY7+thUZOYuHs0rGjx8fjVu0aFGwrZTF+586dWqwR48eHY175plnksfYHMllL3HWBs8Zu8eBWOqz\nFW9TcNaNzbThStW2kWo1YKuK8vXOEoGVdfjcz5s3L9rWs2fPYHPmRy7Lic+9zb5o06ZNrcdnK+7y\nOJZBclKHzUzhDB/O/qpmeD432mijYNvminxPzclQ/JqvCStNsxxZjXIVh0MAsczK59CeG54XK3lx\nhf1cNlSlc8nYfaRINesE8tmv3ExXcpUQQgghRC3oIUcIIYQQhUQPOUIIIYQoJE0mJqdSja/Sipq8\nP7tv1gJtHA7D+vSll14a7MmTJ0fjuFrs0UcfHW1bvHhxsPfdd99g21gEToMtAty5unPnztE2Piec\ngmxjcl599dVgc2Vcq+VyOj7HRNmKmnzOed/Vgi1bwORSPbnzuI2N4bgpjlvj+CcgXpvbbbddsF96\n6aXkMXFMgI2hSsX/2DRXvgbsGuNOz4rJKcGd6Xm92PsVXy88FzbuKVWB3KY7V3t6v71HcikM/o3i\n+E8gXqu2Ejy/rz4qI6dS0m2sDR8Tv8f+Dud+e3faaafktjVFnhwhhBBCFBI95AghhBCikDQZuSpX\nkbEh98du9Z/+9KfRtqeffjrYXJHYpvFdfPHFwebUViCWstgVaN2/nK5ZBF588cVgDx8+PNrGacjs\n3rbuTNsItYZcOjnPe871ysdXLdiU0JS72Lqz+dq3pQG4kjFvs2tkxYoVweZr3R4Dy40skw0YMCAa\nx3PLqbI21Znd/lbmtE0/BdCvX79g51J8eZuVK1LkQhJYtq5GbHkHhquHL1myJNpmZWGG54jXvv1t\nTFU8zs1rSpIC4rnke4eVMrlBsKVdu3bJbWuKPDlCCCGEKCR6yBFCCCFEIWkychXLN9atWd+ZRxyF\nzo0fn3/++Wgcuwm/+c1vBpsrpQLAE088Eeybb7452sZZA+yaz0WkF4Hnnnsu2FYmYbclZ+tY9yXL\nEIyVTFKVOW22Frtpc5H9RYXlHyBeV1Y+ZVgCsueNM6rYTZ2bI86GsjJtqtKrvSfwMbG73DaSzDUD\nzbn6qxXO8slJUvw6J0NV2qzRXpvVRu/evaPXqdAGW3Gcf1+sDMXnmOfIypCpZqsW3h/Pv90fr2mW\nrW3zZG78a0MLbEPetUGeHCGEEEIUEj3kCCGEEKKQ6CFHCCGEEIWk0WJyjj/++Og1p2936NAh2pbq\nMmz1d47vYNvG9Dz88MPBnjBhQrBHjBgRjdtjjz2CfdVVVwX7sMMOi8Zx2qutKsvaJWuV9tiLpknP\nnz8/2Pa7so7Mc2O1/bfffrvWfdtYHT7HXIGXbUBVbe21mYp1s/EXfN3aNFC+bjlOZtNNN43GsW6f\nq77KcVmcnv6Pf/wjGte+fftg8/ewc87xDLZas1LIV6fSc5KK16lr/KS9XqqN7bffPnrNsWl8v7Sd\nxrlqt73P8jnNlVThWJtU2rkdx9j7Nq9pjr+z9w4eZ2P9bNzr2iBPjhBCCCEKiR5yhBBCCFFIGlyu\n4rSxU089NdjccBGIU7kHDRoUbePqh+wety4tdpuym4ylEwCYMWNGsL/3ve8F28oZAwcODDa77Pk9\nQOxW588FYpcfy27WxVdp1dDmiHVTprbZFENbNbeGhQsXRq9TacK2+m01po0zVhJgFza7qe04Po92\nLnkb29bVzdc3u6lt6mhqjbA8BcTlHVgiscfHr62UkrsuqxWW+1jesyUG+Hrhe6NNJ+Z552siN64a\nsQ1KU/cq29CWm92ydAXE8hWvJXuuUw2r7RrmffB91kph/JqvJ/ubx/uw94H6lC/lyRFCCCFEIdFD\njhBCCCEKSZ3lKnY95SomnnPOOcFeunRpsLlhHxBXP7RVHVnaYhearZaaco1aGYpdaNOnTw82y2kW\nzryy2T0sUdkqu+zm5WwhG6luI+OLRE6aY7eknU++XhgrdQ4bNizY7PbNVfGtRqxLPJUhw1IQEM+L\ndU1zFgiPy1XBZez9IvU+O46Pg9efnfNc5k+1N4WsDZ57vl9ZiYPPP8vM9vpINUi14+x9s9rInV/G\nNvLk8A0rcaWqF9vfHl5zvIbt+/l9bFtZi3/LuIK2heXthpQr5ckRQgghRCHRQ44QQgghCokecoQQ\nQghRSLJBC7musawZtmnTJtgHH3xwNI5jb1iT5TgKIE4btynfrOvycdjYAU555OOzHZEHDBgQbFvl\nOAVXYrXxM7luyanO41bHtCl0RcJ+Vz6XHN9kdWhOzWeefPLJ6PXgwYODnYsFqTROpKjYtEy+5jgm\nwqap8nmzc8nXN98v7FxWen3z+ub92bnj+Bq+r9g4glxVVcXkrE6qW3Uu1omxqeG8v9z6q/a5yFUX\n5mv9oIMOisbx+V65cmW0jecoVa0YiNdILjYm1Xnc7jsVB2evDY6Ntdvs+VgbqvuuL4QQQojCoocc\nIYQQQhSSrFyVc3ExXJHYpmvPmjUr2NyEjCUuAHj++eeT+2fJi6vg2oaDLCM99dRTwT700EOjcaec\nckrys1JwZUmbQs7feZtttom2sRuOU+ZsOm81VV+ttEouXyOcTm7TTflc5hrR5bZVA7mqtVtvvXWw\nJ0+eHI3jNFArO7FrOpeyX6mrOyVrWRkq1QyUvweQltOA+nWJN1esTJSSK6zUlJK17DlOVbq246q9\nQWfuWuQSCTZUItfsNlXmxcLbUuEV9hhT6em5z8qlrtv31GdogTw5QgghhCgkesgRQgghRCGpuCQs\nS1JALBH0798/2Jw5AwC77757sNmtmctCsllTPJYlqrlz50bjFi9eHOxRo0YlP2vatGlYUzp06BBs\n61LPSU3souPvb5tRFrnisSWVBWcbatrroIYdd9wxes3nlaP5rSu+2rOrcnISX492XbHMnMue4Ws9\nV+U6JXUAaTd4bn+cVWL317Vr12Dbhq/V3rAVAHr06FGn91UqhaTkKitdWPm+2sjdm1IV4oHVq3gz\nuey41P75tyxX8ZjnPBfWwmvM/k6y/MWV0+0xrS3VfdcXQgghRGHRQ44QQgghCokecoQQQghRSLIx\nOazX2kqLzzzzTLDbtm0bbNtJesGCBcHmeBqr4/JrW7mR9UrWIK3Gx/FAfEy33HILUqRSWy2ctmxT\nyPk753R+jhGxXdgXLlyYfF9zx54TPl8ch2NTw994441a92djdyqNybExGdVGrhow6+O2DETu2q9L\ntdRKS1Ok4niA+Bri68Rq+3369Enuv9pjtACge/fu0Wu+D+cqWDO5eeLrisfZe0K1dyG355fPFdtT\np06NxnE8rD2nfF/MlW1Ircdcl4Pcuk9ts50Mdtlll1rfAyiFXAghhBDic9FDjhBCCCEKSVauYtf/\ntttuG21LNZu0FUf5fezusm5lfm3dXyxPcFMvmxbHjQVHjx4dbOviYyqtgstS28svv5w8Pq6MbLex\nS85KY/Z8FIklS5ZEr1lGYgnJnpOUXDVp0qTo9Ze+9KVg8zVh5Sp7HNWGLX2QSv+2KeR8TnNu9Uor\nCPNnWRd7ynVu3dd8rbB0xeUJgDjlttolkdqwFdr5/PE5r7Rshp2nVNq4nedqb9CZg6/vmTNnRtu4\n0bWVmfmc8nzZ39dUOnhuLnMp5Lym+ff6hhtuiMaxXNWQ0rE8OUIIIYQoJHrIEUIIIUQh0UOOEEII\nIQpJNiZn3rx5wZ4xY0a07bXXXgs2x6HkdDxOm7Zp4qwZ2hRhfs37t60ReBvHBuViMXIl5hn+jtzh\nHIi7ZdtjZy2b9287qBcZmzrYrVu3YLNubNPobXxFDd776DXH9bAGbGNEipymXwlz5syJXnfp0iXY\nfK5sixE+pzYNn/X9VAdxIB3/Y2M7UjE5tjQFryWO+5syZUo0ju859np49dVXk8dbLXTq1Cl6zbEW\nfL5s/CPf13lu7f2f38fxUXaec61xqoFcuxReVyeddFI07tvf/naw7Tnk9jf8+2XjZiuNg+Nxy5Yt\nC/by5cujcXy8PM6Wnzj11FNr/Vy7j7VFnhwhhBBCFBI95AghhBCikFTchdxW6P3a174W7MmTJwfb\nprix65urj9qusyzz2M64qYqrnDIOAEOHDg02V2TOkau+ysfILjkro3D3c+sS5/1zFeZK022LgO2c\ny3C6YaWdiK2cwtcHyyc23Tklf1ULdm2OHTs22Ly+2cUM5KuC8/X97rvvJj+bXek8L9YtnarSal3x\nvI1la+sS5/uKTZ195ZVXksdbLYwZMyZ6zfclvufZ8ILWrVsHO1UmA4jnetGiRcG2JUm4cm81YmVg\n/q3gazpXasR2JOdSJ7bsSVOA1x+HfACrS2Brgzw5QgghhCgkesgRQgghRCGpWK6aMGFC9JozM1gu\nsBlP7EJjdzk30wRit7WtUMxu5pEjRwbbNg1lV+tXv/rV1b9EmUobz7ELkatJjho1KhrXqlWrYFu3\nOr/m71xNrnKbGdOuXbtgsyu2UrnKzlmqsqeVBHPZc9WArW7LkgNjzy+vdetW5wyMrl27Jj87Jc9a\nFztLH3wcNruH55ylFM76BOJ7k5VS+Hs9/vjjyWMvMt/4xjei12eeeWawO3bsGGwbrsChASwDs8wC\nxNlbF154Ya3/DgAPP/zwGhx18bBrLhUqkcPKsbxm+NrPNeplbChH6nfTVi3ncbksKd7/FltsEW2r\nz8xHeXKEEEIIUUj0kCOEEEKIQqKHHCGEEEIUkopjcmxa6QUXXBBsrmBrO/0654LNlRbtOI5r2Xnn\nnaNtHAcwe/bsYNvYndtvvz39BdaS448/PtjHHXdctI27ZdtUS47l4RRKez6LzD333JPcxnp+XXX5\nBQsWBJtTEa2WbdOLq42HHnooet27d+9gc/q3vTa50zGvdSAuD8DxOTY+IFXxOFchl7X9jz76KBrH\n64y/lx339NNPB9um3z722GOodmwl+6OOOirY7du3D/bee+8djePzOmLEiGDbqtpcnZzvA7ZjdrVz\nxhlnRK+HDx8e7LfffruifdiYw6Yeg3j11VcH25aNuf/+++vtc+TJEUIIIUQh0UOOEEIIIQpJi1zF\nXyGEEEKI5oo8OUIIIYQoJHrIEUIIIUQh0UOOEEIIIQpJxSnk6xrn3H4AfgbgUwCbAZgL4DjvfZ3z\ngJ1z6wP42Htfex3r0phvABjjvf+fun6OKOGcOw/AIAAbA9gFwKTypqu999c32oGJNcY51xmAx6o5\n3ADAfADfS61J59wEAOcA+ATAOd77YbWNE02DxBw/BuBs7/37qfeJpo9zrgOA8wH0BVBTk+Us7/34\nOuzrqwD+5r1P90RqQjTJhxzn3IYA/h+APt77ReV/+x2AbwO4MPde0XTw3p8GhJvnRO/9no16QGJt\neZPn0Dl3PoBfAPhRox2RqG/CHDvnNkbpfnsjgC815kGJuuOcawHgDgDX1fzx7pzrC+AB59xQ7/2c\n7A5W55cAbgagh5y1YBOUvDehYqD3/icA4Jw7BMBpAD5E6fi/7r2fV/6rcTyAPQD0ADDOe3+DK1Uj\n/H8A3gcQqs0559oBuL68j1YA/uC9v67hv5pwzp0FoAuAHQD8L0p/WVyOkny6PoCfeu8nOueuQenh\n6Kry+z5D6a/L4QB+i9KcbgzgFO/9FOfcKADjALQA8DGAY733c51z8wD8HcCO3vsvr6OvWQ08CuC4\n8vkd472f7ZzbExmvjXOuB8xcA1gB4DbvvSuP6QRgMoDtARwG4GSU5vRNAN/x3i9zzq0EcDWA9bz3\npzTYN6xivPcfOudOBTDLOfc9AHsBaA3gIgBPoDSPbVG6f17ovb+xvAajtQngWQBXAXAAPgPwrPf+\nxHX9faqY0QA+897/qeYfvPfTnXM9Aax0zl0CYABKc/OQ9/4M59wXUJrfnQBsBOBJ7/0pzrlfAugG\n4EHn3CHe+8q6hzYiTTImx3v/Nko/VtOcc+Odc6e7VaWTtwRwpPd+FIB/ATiJ3trSe78/Sh6f08r/\nNg7AX7z3IwE8T2O3BfBH7/1eAA5EaeGKdUcXAKO8988AuATAZeW/IE8A8HkPm6cCuKh8DXwDQAfn\n3KYoLcpDy3N9CYAL6D2z9IBTfzjn1gNwKEpyxpqw2lx7718E8IFzrl95zBEAbkJpjZ6O0gPUMAAT\nAPy8PKYlgH/pAadh8d5/DOBpAJsD6A9gf+/9PSjJkPeV758jAJztnGuLWtYmShLJYO/9EO/9Hijd\n11ut/mmigegNYIr9R+/9CpTWWhcAQ1Gax7HOuZEoPcw+770f4b0fXP73Pt77ceW3j24ODzhA0/Xk\nwHv/O+fcVQDGAhgF4Enn3M9QigO4tvyk2R6r9GOgdBNEecxWZbsvgHPLNte1XwjgNOfcaSjF/WwN\nsS6Z7L2vKdI0GMCRQPgLYwvnXJv0W3EjgN845wYBuNN7f1fZ7gDgtvLz8Hoo/WVSwxP1/g2qj7Zl\njylQ+gPpMQAXo/SwUimpub4BwOEo/SFyJID/396ZxktVXGv/Ic7zhBERUJBQqEQRFTVBVFSc8rsa\no3G4GnMTx7waNSYxMXE2E7kOcUg0icZocrmaOFw1KlFQIypBURJFKQVEQBARcJ6H98PuUzy16Cr6\nNN1n2Of5fzmre1fvrt61a+991rPWquMA7IxiTMdUxnQVFLF5QOHZeXh5foyomXVQXCOf8N63rJux\nO4AdnHNHV15/iOJmWW1urgrgVefcXQDuAHBT5R9Z0TZ8jOJ6WI0dAdxXuRZ/7Jx7CMAOAMYD6O2c\nexTA+yjmYe6a3GHpsA85zrnVvfcLUfxHN9o59xcAlwHoBWCI9/5559xJALanj31Edjf626Id8kBf\niOK/+8Odc2tiSTCWaBs+INtWpOxWeS+8X4nTAgB47290zo1B8QB8tnNuIgo5alYm7ueDxPuidhZU\nO74VGbGFle12Q2qsRwO4xzn3BwCreu8nO+c2BTDRe/+lxL40pk2m4iEdjGJ8+Hi/jyLo/HHzkYl2\nbnrvzwSwi3NuCAqv+WOVWJB5EG3BUwCOsW9W4nJS8/EwFA87u3jvP3LO2XHuNHRIuco5tzeAR51z\na9Hb/QDMQ/HAMrPy38EBKP67y/EMiv8IAWBPen8jAFMq9hEAPnHOLWtfojlMALA3ADjntgWwsPKA\n+waA3pU2e6AyhFMrUwAAIABJREFUISu68Are+5sAnIJifJ8D0N05N6jSZrhz7rg2/RVdFx6nEcto\nW3WsvfdzALwK4HsoYuiAwsU+1DnXo9L+EOfcAY3uvKiOc24lFP9Y3oulg0zHo5A64JxbzTn3a+fc\nitXmpnNue+fc0d77J7z35wOYhCJuUrQB3vsHAbzpnPtBy3vOua0A3A7gZQB7Oee6VbKPd0UxRzcq\nPuo/cs5thyIOp+X+2BIb2SnokJ4c7/2YSoDiWOfcOyieLucD+E8AZ6O4+L2IIiXuBudcLtbifADX\nV9o8jCXenisAXO6cOwbAtQDGonC13tGEnyTynAzgKufcCSgmz1GV968FcJNzbjiAvwNocXE/jyIz\nYDEK79w53vt3nXNHArjGOfdepZ0ectqGi1Ac9+ewbAkpNdZAIVldieIfGnjv5zrnTgFwZ+U68A6A\noyGaSYskuQKKuIy/o4h7PMy0OxfA751z41Hc/H5buSEuNTcBTAdwjnPueBQJI9MhqbGt2R/Axc65\npwEsRDEOh6KIt+qJ4qF1BQC3ee8fds7NAnCHc+5BFGP13wAuc87tBOAeAI875/6jjsysNkdrVwkh\nhBCilHRIuUoIIYQQYnnRQ44QQgghSokecoQQQghRSvSQI4QQQohSooccIYQQQpQSPeQIIYQQopRk\n6+R069atXfLLBw4cGL0+88wzgz1lypRgjx07Nmr34YcfBnuNNcLanujfv3/U7thjjw32LbfcEuzL\nLrssavfxxx+3ptsN49NPP+227Fatp73Gs14233zzYG+88cbB/uijj6J2EyZMCHa3bulDx9s++aTt\nFtBtxnjmxpJ/Z65ExAorLCkA3l7nekeh1mOmuVku2nputhcrr7ykEPkHHzS2UPgqq8Q1dN9///1E\ny+aSGkt5coQQQghRSrLFABv9RLrOOksWnj3rrLOibXvvvXewe/ToEW177733gt2rV69gz5sXL33y\nwgsvBHvIkCHBXnXVVaN2c+bMCfa7774b7LXXXjtq98orrwT7Zz/7WbBHjx6NZlK2/xatdyV1zllP\n2oorLnE0nnHGGcG+/PLLo3bz58+v2i7HZz6z5Pne9qfRBTI7w3+LPDf/8Ic/RNv222+/YLO3FAAW\nLlwYbJ5Lb731VtTu7bffDjb/p8djDAC9e/cOdt++fYNtzyH2Qt1zzz3B/slPfhK1Gz9+PFLU48kq\n29zs6nSGudmK741e8/ltvd/M4YcfHuxtt9022NZD88wzzwT76quvbnWfml14WJ4cIYQQQnQp9JAj\nhBBCiFKihxwhhBBClJKmx+Swxn7rrbcG2+p9rNnb6G/W+lnP32CDDaJ2rO9zHM9rr70WteMYnZVW\nWrJivNU0V1tttWB379492LfffnvU7jvf+Q4aSdl0fz7GQBzXcfTRSxaVtllwNm4rxfe///1gz5w5\nM9g33XRT1I41aj7vu2pMzrBhw4LNcWYcnwPE84fnFRAfU87gsBkWnO3I82z11VeP2nEsz9y5c4O9\n5pprRu34nOJYOhvjwxmYBx98MFIou6pr0lHnZj3Yc5/jcPbYY49gX3DBBVE7jrW57bbbkvs76KCD\ngr3LLrsE+9JLL43a/epXv6rav1zmayOuuYrJEUIIIUSXQg85QgghhCglTZerrr322mDvtNNOwbbp\n3ywNsQvcwlKWTWdNpQXb/bGr2+4jhZVcmJ133rmmfdRKGVziPBa5wnucNn7++edH21599dVg59J9\nWUr87ne/G+wf/OAHye9lV2wuvbIRdNRigLyNU8HZBmJ595133om2vfzyy1X3/cYbb0Sve/bsGWyW\nvOx3sVTGv6Nfv35RO/4cXzssXETysMMOi7bdfffdwebfaCU5pgxzUyyhs8tVuevYbrvtFuwbb7wx\n2N/+9rejdrytVo466qhgX3/99dE2vo6fc845Vftarb/Li+QqIYQQQnQp9JAjhBBCiFKSXbuqEXAU\nNmdJ2SrE7Jq2cgS/Zhe7lZBSEol12bObjL+XP2+38T422mijqN0222wT7H/9618Q+Uj6TTbZJNiL\nFi0K9uuvv578TK4iLcta66+/frAPOOCAqN3//d//1dS/zg6fx3zc9t1336gdV/5meYnlPyCet5wl\nBQCf/exng80ZUPb4sjzNWZHrrrtu1I6rnbNsZGVllqH4HLJznbO8Ro0aFW1juSonUQnRUbD3qJzk\nc+WVVwabM01z8pTdP8P31BtuuCHY9nrxy1/+MtiPPPJIsMeMGRO1a6uQAXlyhBBCCFFK9JAjhBBC\niFKihxwhhBBClJKmx+RwNVLWvW3FY9bcbcXjVEq51d9TcRu5FGbet23HcUO8zabCfe5znwu2YnIK\ncqnLG264YbD5nFhrrbWidhxrUWtaNFfnHTBgQLJdmWNyUseHU0qBOB6G5589v1mnt8eNqxzzvLUx\nLnwd4P3Z2DyueDx79uxg9+nTJ2rHqewcM2SroHNMzmabbYZasL+x2asnC1ErNg6Vz29O6wbi2Mfz\nzjuvpn2m4lWBdBmPSy65JGr3jW98o6ptY3KaXbqjBXlyhBBCCFFK9JAjhBBCiFLSdLmKF+BjycdK\nUOz+smlsKRnK7iOVQl5rO+sKZHcdu/Bt/wYOHFi1f12ZXMXjrbbaKtgsV3D6NxDLVam0aAvLGFb+\nYlhaKZs8kZJnt9566+h1ahFbXix3Wftm6ZFT+S08f1gas8eexzlVagCIzxXet53r/F120VCet1On\nTk32qbOfD10Rlibt9YJl0PYkdX8B0pX47TnMHHHEEdHrZ599Nti5uZn6rnrP+1tuuSXYxx57bF37\n4GPD9+VceZkU8uQIIYQQopToIUcIIYQQpaTpclWtEhK766xLnF1X7Fqz+0i513JyFWeH2Ha8WCBH\nglv33uabb171e7syucqZnNXDFXNzMgmPrd03ny8sV2255ZZ19bUWF2hnxMqqfNy4aqmVazh7yR4b\nzobizCabXcUVlVmizFVw5WsCz0UgPldYurJzmH+jzejkBYNZrsqdX2JpcpmPI0eODDZXqZ4+fXrU\njuVSHiceIyCWa/r37x9tY+mUpVkeW2DpLKS2JHU/rHWhaAsfK5Z3gdplOe4T78+e9yzx56SsW2+9\nNdi8SLIdywkTJiT3wfu32datRZ4cIYQQQpQSPeQIIYQQopToIUcIIYQQpaThMTk2FY5TyFn7s9VH\nFyxYEGy7GjVrvjn9l+MFWJu3feKUNN7G8QVAnILMMQA2LsGuSi7yGjNv22GHHYLN50COXNzF/Pnz\ng33wwQfXtL+uEnNhVwvm+Aaec6uttlrULlcNmY/dwoULg73eeusl98Hz6s0334zacWVknmccawXE\ncV2s2dux5BgDG/M1dOjQYF933XXBbqtKrGUhd00ePHhwsHmu2znMY8hjNmjQoKjd3Llzg22rYP/z\nn/8M9iuvvBJsjvdpa2x8W+pac/rpp0ev+dzkivp2/nGsGsfOAXGMEsfn2PtXrqRDqu+5Kugvv/xy\nsGfMmBHs3/3ud1E7TnG39945c+YEe9y4ccF+4IEHkv1LIU+OEEIIIUqJHnKEEEIIUUoaLlfZNM1U\nSqhNMWX3lHV1s1udXWbWNcouNN5mXYT8OpdiyottcpVe6zrPVdYtG+xmzrmpeSysdMVp4yxP5GQC\n3l+uOijLo7aCdYqc/GV/V2eufrvmmmtGrxcvXhxsHhN2cwPArFmzgm1d3SwxcoqwhecWS012bq6x\nxhrB5nG1ad2cNs4Shu0Dp67b82bEiBHJ/oraycm9o0aNasOeLGGfffYJ9hVXXNEufQCWvqfwfW/Y\nsGHB/slPfhK1S0n3LAkD8fnN11IgHheWjew8SF13rQyVqkJsF9lNLahtF8/lEgD22sS/5Ywzzgj2\nPffcE7U74IADqn4XI0+OEEIIIUqJHnKEEEIIUUr0kCOEEEKIUtL0FHLW+3gJBQtrkDYmp9YU31Ss\njYXjKnL9e/7554PN+qmNyeE4grJT65IHuXbbbrttsDmNMAePUy52h+OjOM6r1n2XjU033TTYVufn\necCpqP/4xz+idpzey2MHxLEyfOztuHLJCI4rmDdvXtSOdX/W+nNxcCeeeGKwb7755qgdz02bYmtT\n6kV95NKOOT6j0fMst1r8fvvtF+z2XHYnt2o4l7iw5Q0WLVoUbL6W2iUOuHyJjUFMlV6xcUKp8csd\nX96WisGx32Xvr/ybbVwrl7Tg38xlCGpFnhwhhBBClBI95AghhBCilDRcrrJuJ3ZxsZRl091SrjAg\ndofl2qVSTnOrCvM2607lFHJ2nds0YiuvdRVY4rBjwa5TK5NwuiAfV+tu5dd8jK0kylJGjx49gm3P\nxT322CPY7ALmzwDx6uX3339/tM2+7uj07ds32Pa4pVzY5513XtTuy1/+crD32muvaBunsHJKuk1D\nT825nj17Ru34fOD5mCsH8OSTTwb70UcfjbbtvffeweZKrPa7+BpT1lXom0WurMLySlQ5ySQHn/d8\nHW9r7L2Hzy0u22DhdrnK33wNtqU6WKrla7DtE18XctITz2FulysDwd9lpTue+7ZP3JZ/s63GbtPX\nqyFPjhBCCCFKiR5yhBBCCFFKGi5X2UXToi8jt5h1Q3KktY3+ZtdVTq5ilxe706wrLOWmZ9c7EFeJ\nZKw727rmy8wPf/jDYPOici+99FLUjrNprEzALmzOeBk7dmzUjrNfWOKyC7jy+ZGrxHnuuecGmxe9\ns+POmQ02A6ezyVVbbLFFsO08SGF/42GHHRZse+zZfczjbOXoVDXy3GKgvGgfV0m28AKM9957b7SN\ns2zs72dXP58PU6dOTX6XaFtyldRzGbR2AWim1krojSDXR55LViZKLSJtr0e8wC3LxUB8fnO1YXvf\n5Otx7v6aysKyx5Ovu3x9t9dZlrXsOPM++X7bq1evqJ3NNquGPDlCCCGEKCV6yBFCCCFEKdFDjhBC\nCCFKScNjcjh1D0indb/66qtRu+nTpwd71113jbaxrsdaXU7vzMUfpNLkbEVGu+JrC1abZO3TroJs\nK7p2djjm4dRTTw22TW3kqrlWbx04cGCwOSbK7oOPP+u5tnIta9Y2nouZOXNmsDmGyMZ7sLZ9/fXX\nJ/fXGXDOBdummNqU8hQco8TnOgC88MILweYV4NkG4vgDnmd2DvP85liBV155JWrXu3fvqn0dPXp0\n9PqSSy6p2s72g6vidtaYnFTMRK1p150Bnt/2OnDooYcGm9OT58+fH7XLxXc1mtyx52tQrmwKzyUb\n68ap4blYI55ndt7zPEhVSQbie2quyjXH0PDvqPV6A8THLVV+AqgtvkqeHCGEEEKUEj3kCCGEEKKU\nNFyushVM2eXMacC8+CUAPP3008G2qb/srsq5yVLuNOsSTy3kaaUO7lMOTmHmVFSgfHIVu345Zdim\nbG6yySbBHjNmTLSNKxFvs802wbbVUfm7WCaxlYz5fPnzn/8cbK5wDMSVjTk9+cUXX4zaDRgwINjv\nvfceOjNbbbVVsO3csZJSitxCliyBcTsr9bJEwGmvtk+cEtq/f/9gP/fcc8k+8CKFVppgbLo6z/1+\n/folP9dRyV0LyyRRMVaiYk455ZRg5xbGzJU5aUu4vANfj4C4pApft2yaOGMr7/NcqrV8RO68yclG\nDMtVLIVZuYrH0oaQpPpRz3ktT44QQgghSokecoQQQghRShouV7EkBcQSBLuLrTubsyesbJSSnqzL\nLJU1Zd26KfnLymSpaoo2opvb2eyTssEZAfy77bHncbduSnar8ljYStK8j4kTJwbbuqw5c+fZZ58N\n9k477RS1Y8mE5Q+7qCPLJLNmzUJnZtNNNw22lW05U2P8+PHJfbC7nDOt7DZ2Jdsq4DxmPEdsn/hc\n4YzL3GKGvKBqa+Qqvn5st912yc+1BXwdqtUl39kkqVTVa0uuIjafR/a6whl3LKPa67XNgG0vjjji\niGDPmTMn2sbziq+zLGMBcdVge/3MhWIwKdkzt1JAbiz5u3IV6HlbTl7M9TUnpYe+1rRnIYQQQohO\nhh5yhBBCCFFK9JAjhBBCiFLS8Jgcq5OyTsj66rRp06J2nBpnU1t5H6zjWS2QYzhYM8zF7qRS3C2s\n9VuNl3XC1lR17Iywzstaca5y7Y477pjcH8d4zJ49O9rGeivH8djKvc8880yw99lnn2DbFbO5JAD3\nb/jw4VG7ESNGVO1fZ4TjD2zVYI6bueuuu5L74JgXWy2WK65yuqgdIz6OnMJrU2K5PABr+LlU8733\n3jvYdgX1f//731V/BxCvXp6rnt4W8DUqF/PH/bRxSldddVWwOT7Fnt8c12KPa61xPtxfvp7mrsn1\nHONcyrgdaz43OcaFq1kD7VvR+uCDDw52LsWb5xLHr9oYJT7eubgWvi/l0r9zZQmYWleD599oy3Hw\neWNjhvj6kVtp/Bvf+May+7rMFkIIIYQQnRA95AghhBCilDRcW7GLLKbcn3Pnzo1es1vdpsLVSmpx\nMdsHfs2usFya6rhx44LN7nEgdtfZRT7LBh9XrtLJC2gC8bG0afUsI7Ebdf3114/acbokb7PlB3bZ\nZZdgs3xiv5dlEk41t1WqUwuzdkZYgrVzjqWAyZMnJ/fB1YDtsWH3Ocs/VtbilFhenNfOF55LLHXY\nawLP4S9+8YvJvnPJAzvO7D63kk5bw7+v1uuflR95XrEkYxd1zElAtaayp6rG10uvXr2q2haew/Z6\nzb+Z+87nALB0qERb8tOf/jTYuRIcPJdS5ReAvLyUW2yz1n2kqHXMed+pEi/V9sfHhq85tjL5E088\nscw+yJMjhBBCiFKihxwhhBBClJKmy1UpF5WNjN95552Dbas6ptxp1p2aWqDTtmN3cCojC4gltDvu\nuCPYhx9+eNTuhRdeSO6jbPCxZPc4HwMgXqAzV9XWZpIw7GafOXNmsK0UwplvvECsXfCT98eLiw4Z\nMiRq19kzqlLVTW1mBr+2i6imsPOb5xK70u2xZzhrzmaEsATK42UzM9idn8veY1lyr732irZxH3MV\nYduC008/PdhcxRYAHnnkkWBzJqGVH7mKN1e6Pumkk6J2Z511VrDrrZrMx4ulvn333TdqxwvE8rjb\nOczZNJxda6vk8vnCC70C8fnC/bNZm20JLx4LxJIpS+v2d7IUyWNkr6X8O63Mmbr32ntUrdLT8t7b\ncveBXFZybm7WEh5S7juyEEIIIbosesgRQgghRCnRQ44QQgghSknDY3Ksdp6Kp+FVpQHga1/7WrCt\n7p9a/dTCmh/rmLYPqT7ZWKCRI0cG+957701+L3+X1VbLDMfT2DgWjq2wcRepqtX2+HOl0hkzZgSb\n43OAeLXqwYMHB3vQoEFRO47B4FRUG6ti0+E7G6lzMFc5lOcOH5tcO6D2eLkUVovnvvM+bDVy7kcu\nrourXNvvYq2/vVemnjBhQrC/9a1vRds4BpBjbe65556o3TbbbBPsXNo/H0tO5wfieCAu29C/f/+o\nHZ8jfFw5LggAnnzyyWDzHLapv48//niwjz/++GBfcMEFUTtOp7aV8Rm+T3AV7bbmmGOOiV6n5qCN\nL0rFydhzPXc/TFWitnM2Vb241srI9VTJtt9lnxtyFcCZXLmBsK+aeieEEEII0cnQQ44QQgghSknD\n5Srrukqlf9kUU3avWpdePalrOYkrtXindZnttNNOwf7jH/9Y0/eWfYFOhlMW7bjnqnmylJWT97jd\ngAEDgs3pwwCwwQYbBJvH3Z5j7B7lCrx2zJ5//vlknzoDBx54YLD5GNZa2fSwww5LbrPyBss8LEva\ndFY+9lxuwFat5XNl3rx5weYFJ4Gl52oL9npjz5XUNv4dNtU8J1U3iocffriqbeHj07dv32gby0vb\nbbddsG2VWE755nkAAJMmTQo2l2bgVHwAOPfcc4M9fvz4ZH/rgVPNeRFZID7HclIFj229afKNYOjQ\nodFr7gvbVq5KLXabk3zsdZbnIH+uESsK1PJ+vfsDapfNahlbeXKEEEIIUUr0kCOEEEKIUtJwbcW6\nwthtaCt5MuxqtZk6qQU1LamIdOvu4v2xVGHdxLaKbwqO/s4tQlY2eOE7W9mTpQaufmzh8bQLCfI2\ntnOVe1988cVgz5o1K2rHGSJjx45N9u9f//pXsr+dAXbj87lf6+Kx22+/fXKbzZTj8z2V3QjEc5/P\nlcWLF0fteCx5XHILgzK77bZb9NpW1mU464ZluNw1pi2wbnw+lnx9zS00abNXOwKpivRA/Lsuuuii\nYD/22GNRO65UbuF9cvXt9qxgbiW1VLV3ezxS10V7L8uFZaQyoOw9KldROdWuXomK4e+yIQO8/5Q0\nDdRWqVyeHCGEEEKUEj3kCCGEEKKU6CFHCCGEEKWk6fnOrBPm0jl5Jeinnnoq2sZ6XU7XZXib1SBT\nFR45FRkA9tlnn2CPGjUq2JyWa7+rPatrtjUcs8Qp3kCc9mjTxFlHzY0Tx5CwHmzjODjeg1OBbUwV\njw2vBmyx519ngyvB8jHMpetzfICt/ssVoG1KL8cO8D5s/A+347gp51zUjqtZc5xeLk6FGTZsWPS6\n1nRZrgI8cODAaNuDDz5Y0z4aRXumPDeTWle75hiMv//9783qTpvw0EMPRa/5PsfxbbbcBc9Vvv/l\n4mlyqdY8D+x31UOjz9HcPM3FudZSqVyeHCGEEEKUEj3kCCGEEKKUNFyuyi2GaWUe5ne/+12wbSrx\neuutF2x2yeXkDZZLbAoav2bpwy4M6r2v2le7kCT3oyst0Dl16tRgH3HEEdG2119/Pdg8fkBc3ZPH\nyaYJs7zEKaE2fZgX7eP0ZCsdcgVXlrjs/mxV384Gn998rubc2Xze2irEuc+xy53HyF4HeO7z/J49\ne3ayH+zCtmNpK8S2YCsj8zlqYVmP+2srBAtRLxdffHH0mhdf5XPdLkDLcy4l7wP5iseplO9cSZVa\nJcVGy1U5OTp3/cktJhw+X3+3hBBCCCE6LnrIEUIIIUQp0UOOEEIIIUpJw2NyrKbH2tqMGTOSnzv+\n+OMb3ZWmYWN1eLmARqTndRa4JIAty80xGLYUO58TrEvb5RVSWrEt5c0xKPwZG8fBsTvcdxvfUety\nHh2VCy64oKrNMSjA0jFoLXDqNhAfKxvfxssmcMyTjavj+BqO47Fjyfo7t7PxfPxbOJZr8803j9p9\n/etfD/Y111wTbeNYMV4CJFdGXojWYJeWSS3lwPFsQPp6Z2Nh+H5j7728f459tMuW8LU7tzQLU+sS\nD7n91bPkDC8XBMTHM4U8OUIIIYQoJXrIEUIIIUQpabhcZVPh2NWWSwXrCNRaVdW+z7LIZptt1vB+\ndVQmTJgQbHaHArEL1FYoTrli7QrXVgJLteP98XfZ8eQ+8jjlpLYykZKnLN27d49esxxk5zC7vlm6\nsu7nlGvaus45DZY/Y9P8eWz5HLKVt5kFCxZkXwvRaOx84YreOZmHpXWWZnNylb2O8dziav48T+0+\n+FqaSyfnvueqFdebks6v+XphZT37W6rRsZ86hBBCCCHqRA85QgghhCglDZerrMuMK5ra7I6ORq1V\nHG3F48mTJwebM0y6Ek8++WT0eujQocG2GTQsPfTs2bOh/WD3qJVCFi1aFGweM5sRZ8e3s5Grbsqk\n3Mx33XVX9JpdwrbCKEtbXEXaHns+Bzhby14TWO5OVW4GYlmSF/xsjdSYkkPtcSnrgpmi+djzm2Vy\nlt15EVwgrtxeaxV9e96y9Dt9+vRgjxs3LmrXq1evYPO12UrE/Jr3bWW3lORsM6H42NhtLOvx77IL\ncnJWdmqFAnlyhBBCCFFK9JAjhBBCiFKihxwhhBBClJJuOb25W7duEqPbgU8//TSdW7gctOV4Dhw4\nMNgjR46MtnG8BsfD2FXIOYaGVzW3cRecGs2xG/yZ9qQZ46m52T6UYW6KJbT13Bw0aFBVe+utt47a\ncRV9jnmx8S+5VO4ePXoE+6ijjgo2x7CVidRYypMjhBBCiFKihxwhhBBClJKsXCWEEEII0VmRJ0cI\nIYQQpUQPOUIIIYQoJXrIEUIIIUQpafiyDsuDc24zAB7Ao5W3VgLwIoBvee9fS3zmAQAXAvgIwIXe\n+2HN76loJGbcPwWwJoCxAH7ovVfQWBvinBsFYCiAVQFsiyVz8Rrv/Q3L+OzXAezpvT/SvD8YwDe9\n9ydX+cx+ACZ47xdVXl8CYByAhQBe9t7PWL5fJJaFc25fAD8E8DGANQC8AOD41DW3xn2uCOBD730y\nRTt1vojm0FWvsx3qIafCAu/9bi0vnHO/BPBjAN9ttx6JtiCMe+UC+SyA/wUwOfch0Vi8998HwgVx\nPM/F5djnZABLPeBUOA3AiQBaiiLtDuAsAJcAuBGAHnKaiHNuZQB/AjDIez+v8t4vAHwTwEXt2TfR\nFLrcdbYjPuRY/gHgeOfcTBRP/dOcc7sh47Vxzg0AcBUKOW5FAD8AsBjALd57V2nTG8AEAH0AfAXF\nRbgbgAUAjvHeL3TOvQHgGgAreO+/3bRfKCzro/DizXfOfRnA9wG8h2Isj/Lez3TObQfgtwDeAnAX\ngPMArOm9/yixT9FAnHM/BzACwPsAXgJwdGXT2s65PwHYEoUX9iAAu6IyXyue18kovEQ3AdgFwJ+d\nc/8F4A0ArwHYC8AhAIY6504DMAdmPnvvxzvnrgPwLoB+ADYGcJ33/uIm//SysRoK701YXdV7fwYA\nZObeAwDuA/AFAAMAnOO9/7NzzqF4YHoHwP0t+3PObQTghso+1gHwK+/99c3/aWIZdInrbIeOyXHO\nrYDiIvlQKz96OYDfVJ5YTwRwvfd+CoB3nXMtpSW/CmA0gJ4AfoTiAWoYgAcAnFlpsyaAu/SA0yZs\n6Jx7wDn3DxT/Xfyu8p/lugAO9d7vjmKSnVRp/ysA53nvd0VxY1yl2k5F43HOrQfg/wHY2Xu/C4Bb\nALQsm7wVgOMAbAdgEIAhVXbxlvd+V+/9lQBeBvCf3vtnAOwN4F7v/a0oHoRO996PQ5X5TPvaxHu/\nN4DhAH6IdYVhAAAgAElEQVTsnNugsb+23HjvXwdwDoDJzrn7nHM/qjysAOm5BxQ3uv1QeHy+X3nv\nHADXVubkv6ltTwBXeO9HAPgSAD2Ith9d7jrbER9yWgbhART/DcxF4bpuDTsCuBcAvPdPofjvsjuA\nPwM4uNLmUBT/deyM4r/AMZXvPKzyGig8Ow/X/UtEa1jgvd/Nez8cxfHfwjl3EoD5AP7onHsQwNcB\ndK+0H4zigRQA/trGfe3SeO8XAxgD4EHn3OkAHvHez6psfsx7/05F438JxcXT8khi1yMB/L3K+6n5\njJb2lfiR5wB8ro6f1KXx3v8CwKYovNabAvinc+5EpOcesGTuvYjCIwAAnwcwvmKPo7ZzARzunBuP\nQhrRg2j70eWusx1RropiclpwznFg1MrL2IcNoupWeW80gHucc38AsKr3frJzblMAE733X0rs64PE\n+6JJeO8/cM79BcAJAIYBGOK9f74yGbevNPsMgE8q9sdVdiMaiHPuchQ3sde99wd47w92zg0EsD+K\nh52vVJpaN3a1wNOl5pRz7jMoJKxJVdqn5jMQ/6PWrUpbsQycc6t77xeiuD6Orsy9ywD0QvW5B8Tj\n3I3+tszJFWj7hQCe994f7pxbE8CbzfgdonV0letsR/TkpHgDQO+KPWIZbSegcH3DObctgIXe+4Xe\n+zkAXgXwPRReHAB4DIX236PS/hDn3AGN7rxoNcMBzEQxwWY651YFcACWuEunoogJAApJUzQR7/3J\nlf8AD3DO9XPOnea9n+q9vwiFXLVNnbv+BEVcwPYAnvTef2LeBxLzubJt98r76wHojyJ7RNSIc25v\nAI8659ait/sBmIf03EvxDArPOADsSe9vBGBKxT4CwCfOuU4ne5SU0l9nO6InJ8VFAK5xzj2HZUtI\nJwO4yjl3AooL5VG07c8ArkQxkeG9n+ucOwXAnc65d1AEzR0N0dZsWJELgcJTNwPA8ZXXj6Fwi/8S\nwA3OuUNQZNtd4ZybC+BvKP6D/wSiLZgDYFvn3EQU/5UvRhGQeHD2U9UZA+AOFIGpLFXdC+Bq59yp\nyM/nxc6521DM53OWJ+25K+K9H1NJ1Bhbuf51QyFd/CeAs1F97qU4H8D1lTYPY4m35woAlzvnjgFw\nLYq05f9BMe6ibely11mtXSU6Jc653QEs8t7/yzk3BMDolsw50TWoZFeN997/vr37IkQZKcN1tjN5\ncoRgPgTwe+fceyj+Izl+Ge2FEEK0jk5/nZUnRwghhBClpDMFHgshhBBC1IwecoQQQghRSvSQI4QQ\nQohSooccIYQQQpSSbHZVt27dljsquVu3JQVPP/OZJc9Un3wSp9p3pgDolVZaKXr94YcfJtvy72dy\nv/fTTz+t/qHlpBHjKVpPM8ZTY9k+aG6WizLPzRNPPDHY7777brDnzZsXteN78aJFi4Ldt2/fqN17\n770X7JkzZwb7o4/iIudTp06tr8PLSWos5ckRQgghRCnRQ44QQgghSknTiwGyXPPxx7Wt79WnT5/o\n9VNPPRXsm2++OdgXXnhh1G6LLbYI9uqrrx7s7t27R+023HDDYJ9yyinBPu6446J2/F1MTp6ydCYZ\nTgghROfEhlEce+yxwX7ttSWrnWy99dZRu7XXXjvYr7/+erBXWSVeXuztt98O9mWXXRbsl19+OWo3\nbdq0YFspqz2QJ0cIIYQQpUQPOUIIIYQoJXrIEUIIIUQpya5dVU8qHKeJA0unirdwzDHHRK8POOCA\nYK+zzjrRtoEDBwZ7tdVWC7bt+/vvv1/1e2279ddfP9j3339/sK0GybE7t912W7AvueSSqN2rr76K\nWuD4JKWQdx3KnKba1dDcLBdlmptDhgyJXv/1r38N9p133hnsHj16RO3WWGONYHO8KcfxAMDnPve5\nYD/99NPB/stf/hK1u++++1rT7YahFHIhhBBCdCn0kCOEEEKIUtKQFPJcJWPm3nvvDbZN6+Y0NJt2\n9uabbwZ7/vz5yX0wLA0tWLAg2jZ58uSq++vXr1/Ujn/Xl7/85WAfeOCBUbuvfe1rwZ40aVK0beWV\nVw72Bx98kOyvEEIIUS+777579Hr27NnBXmuttYJtU75ZhuI0dA7rAOL74c477xzsnj17Ru1eeOGF\nYE+fPr2mvjcTeXKEEEIIUUr0kCOEEEKIUtL0isdHH310sPfcc89ge++jdiwV2Synz372s8F+7rnn\ngs3uOCB2r22++ebBfuedd6J2b731VrBXXXXVYK+77rpRO64EyZHmdtHNK6+8Mtg77bRTtI0lqlpl\nPSGEEKI1WNmIF9R85ZVXatoH3zf580B8D+TsKr6fAsBGG20UbMlVQgghhBBNQg85QgghhCglesgR\nQgghRClpSExOLr7kW9/6VrAXL14cbF4lHIirLvKq40Ac1zJx4sRg77rrrlG7W2+9Ndhc1ZHTxAFg\nxx13rNpXTi0HgM022yzYvXv3Tu5vk002CfYee+wRbRs7dmywbSxPe3LaaadFr1lvXWGFFYJtV1yv\n5zesuGJ8mqWqPdt9cykBXsHetktVkuYVdS18ztrzt9Z4Ke4Tx40BcRqlECIPxyvmrjG5uZm6rvC1\nG1g6lrMs2NIrHFPDVfntKuQbbLBBsBctWhRsG6P67rvvBvuNN96oum9g6Zja9kaeHCGEEEKUEj3k\nCCGEEKKUND2FfM011ww2V1rkNDMgdkNylWAgXnhzwIABweb0bwDYbbfdgp2Tq9idxt/Fi38Ccdod\nu+4WLlwYteMU9ZNPPjnaxnIVyxvtAS84euqpp0bb2F3MchVXygRil3Ct0lVuMdJaabTUZxeSZXIu\ncf4cjyef5wDw4IMPLkfvmoOtiMrjMm3atGDbY/P2228Hmyui2n0wdrz4WPH5xbZtl5IrgVjCtpIq\nw/vIjSvPb+v2tymyXZHcODETJkyIXrNse/311wf77rvvjto1s6QGL/4MxCU/yoS9Bs2ZMyfYPB9Z\ndgLi0iss7dnznj/3+OOPB9vKVc65YPMC2O2FPDlCCCGEKCV6yBFCCCFEKalbrkpV7x05cmTUrk+f\nPsFm95mtQszu4sGDB0fbuC1nPFmXKUtgvCgnu8+A2MXObm9bMXLQoEHB5t+bc1/vv//+yW3tDffb\nHn8+DixJWGmuEaQyKXKyVq3t6umDdcXzNivd8HezrGGzBWutMFovtR4PdivbPqXOfc50BGqX73IZ\naywL8/llMzF4LNg9bhe35Yw9/v32msAZJnYb75+38eKDwNLXj85OTvpNnUu5c4wzC22owT777BPs\nbbfdNtgvvfRS1I7nEl+bZs6cGbVjOeWWW26JtnHWFEswd9xxR9SuEdePjog99jw3ee7bsIy//e1v\nwT7ssMOS++ewj1QlfyC/cHZ7IE+OEEIIIUqJHnKEEEIIUUr0kCOEEEKIUlJ3TE5K1zz00EOj16yp\ncprql770pagda+dTpkyJtnGlZE4b50rDQJzuzBqkrWTMui7HHzzzzDNRu6FDhwabdechQ4ZE7ThN\njlPcgTiGiI+F1cXbQifm77RxKBwbwRqrrVZcTyq3/W2p9G0bM5GKjan1WNnfWE/6u90Hx5pwf3nF\nemDpVOtGU+sx4HHlGDMgjtHi35KrAG3Tq2vtB8d28XfZmByOk+FjyFVZgbisRO7c4PPXnsscR8V9\nsvE/p5xyCspEPbFvubisww8/PNj2vOeq4xwLMn78+KidjRNJ9ZWv0RtvvHG0jeNwOD7FXv/Lio1z\n4pIhPG/t+c3z4MUXXwz2DjvsELXj+cP36169ekXtcuU52oOO1RshhBBCiAahhxwhhBBClJKGy1XD\nhw+PXnOV41zaWSqFGYhTw/l7rZts7ty5wWb3nJW12PXNcgS794ClK0hW6wMQu1p5oUsAOPDAA4N9\n6aWXBru95arconV8TGy/6ulnrSmrdt/NrILK5MYi9/tz8lezx5PHyMp8vHAtb7PnJveZZSMr6+Qq\nD1v5qgVbjZyrovJct59nuYqlZFt5m6U2PtatcZXz7+TjafvELvyykTtvc+cYc+SRR1b9jH3NlbNt\nlVyuWs2fsTIwSyu2DAa/ZjnFyjhlxS4KzCEWPH72uHGIxQknnBBsm6LP+2MZ0pbPsPtvb+TJEUII\nIUQp0UOOEEIIIUqJHnKEEEIIUUoavgq5TQllOCXbauesv1tdl2NqnnzyyWC/8cYbUTsuO82lpVlL\nBOLYHY77sLo/xzCwnmxXUOcYH7vCK8cocUxOW8WbLC+5eJp6lmQA4liI3PlST1wLf4Z1eduPWlPI\na/39llSsSqPIxUjwyvdctoHTroF02q7te+5ctXO1Bbsy+FZbbVV1f3aMOfWX98ExALl92P3xGNlY\nI/u62meA9G8sO7lx/+pXvxpsXhJk0aJFUbtUfJ+Nu+TrP8d02KUK+Ly32/i7uE82Zbqs8LJJALDn\nnnsGm891G0PDY/HEE08E28bwrbfeesHma4mdc7NmzWpNt5uOPDlCCCGEKCV6yBFCCCFEKWmIXMVu\nMeuG5DRs615k2DVt3YssFbH0ZN3qXF2Y3XPWTZ9KUZw+fXrUjtNgbSojwzKXdQX269cv+bm2JicT\n8Gsei2bIVSyTsKRkjx33N9cPPl/YnW3TmFlK5HMnV53Yyi6pVGPbv3XXXTe5z0bDKz0DwMMPPxxs\nPja5czGXJs6/zcrMqWrDdm6ytMzjavf35ptvVm1npSUeB+6f7Tu3s+OcSj1fZ511onZcBqNs5OZt\nbtuPf/zjYPPxsZI/w+eE3TfPTb4+WKmQz2d7Xeeq2nwvsKvIz5gxI9nHzoxdsZ2vQSwH2usih28w\nvGoAEIdp8D7snLP30fZGnhwhhBBClBI95AghhBCilDREruLMCesKYxdlTq5irDTEbjJ2Z7MNxC45\nztqxkgO7rXnb1ltvHbXjqqq5Sqr8XbbaY07mamty2Q2p6q+5fTBWruHjao8BZwHwgq72/Egdc/t+\nSvK6+eabo3a8f7ZzWST2u/i4sbvc9v25554L9siRI5P7bwQ/+tGPotdnn312sEeNGhVsKyHxb+Hf\nacef3dG1VnaudWHXnGyakyhT0mtO1sqd11b6YDpaBdcWaq0k3pp9pD53zTXXRK/5Wstyh60an8ps\nst/L84fH0PYnJVPaPnGV6n333Tdqd/fdd1ftU2fHZhvz/Yszj/l+DQDz5s2ruj9blZorKvO+OZO5\nWj/aG3lyhBBCCFFK9JAjhBBCiFKihxwhhBBClJKGxORwJWOrk3JKIWt3d955Z9Ruiy22qPoZIE4h\nX3/99YNtU9c4NoP7YeMlOG4olX5sv2vKlClV+wrE6Y+24qfVK9uTXLwDx2TkUqprrdScW8GYV73l\n42rjKeqJK+C4ExsPwHEXtf6OXEwAx2LZc2z06NHBPumkk2r6rnrp0aNH9Pr+++8P9vbbbx/sp59+\nOmrH45xLE2/0iuq17o/b2fFKVa+2cTf8W+zvyn2O6UhxdUy9cTf8OjcPTjvttGDbuLKnnnoq2Fzx\n3cZ98TWZr+O22jb3gz9jYzxz48mp/3zN2WWXXdAVsOdw6r7E9zUAmDhxYtX92Zic3XffPdhTp04N\n9qBBg6J2qUrq7YU8OUIIIYQoJXrIEUIIIUQpaYhc1atXr2DbKok2zbuFf/7zn9HrnXbaKdi28i27\nOdmVmXMxsyvetmPXaE6uYgmC+8CuWgDYZpttgm2ro7L0NmzYsGCPHz8+2fdmkatIy8eB07/tMeHP\nsbvYur1zqZ682CKPk/0ulq9ybvWUDJeTXXhbaqHGan3i38XuYCsdNNtle9RRRwWbF63NYedBreUA\nUlWN6yVXKTu1LSe58O/KjWVOrsqlY9sFfptNrp+591PHy8713BiecMIJwT755JODbaVOXqwxt8hu\nCtsnHkOukm/vBQ899FCwWSazbVli5AWegbwc35mx9x6WlPjY2GtaKuV79uzZ0Wu+pvE+bIkF+wzQ\n3siTI4QQQohSooccIYQQQpSShshVvPCfzWixLsUWWP4BYhetdWWybJRbjC/lhsy5Rvl7c5H8vNjZ\nmDFjonY77rhj8rv492+55ZbBbg+5KldxOnVcrUyUqkJcqxRisWPIcDZUruJ0SobKVdNl256zqfPD\nfi61WCdQe/ZWveyxxx7B5grHrYGlHe6/7XtuodR6F2lNkZJccpl3tcpOOXkn13eWZtqCZp87HF5w\n6qmnRtv233//YLNExdWEgXhu5uTC1GK89vrPx5/lk0mTJkXtWAphWQuIs6t4/zY7LreIaJnge9bA\ngQODbaX0WbNmVf28za569tlng83nkM0oTlVQbi/kyRFCCCFEKdFDjhBCCCFKiR5yhBBCCFFKGp5C\nPm3atGgbp9kyNq0vFR8AxHotp6RbHZ0rOXI7G6vDmiR/72uvvRa1Y22Y980rTFus/surtfbv3z/5\nubaAYwvsseM4gFrjX+qtuMpwTI7dXyq+ptYU51pXsa41XsTuP/ddNr6r0XDVbU4VBZY/Rdb+ltyq\n0Kl4KEut6eq1xtfk9pH63lx8We5cbvZYLovBgwcHm8tQ2LgWjoXYbLPNgj1ixIioHceu2GM3efLk\nYPPxsqu08za2n3jiiagdnzt8n7ArYaeqkffu3Ttqt9FGGyEFl6bw3gfbprhzuzJh5x+fA3w/tHGy\nqRgljqEC4vsyzxeuqg7EVf5tXE97IE+OEEIIIUqJHnKEEEIIUUoaIlcxVmpauHBh1XbWDZlLF2ZX\nG7serQuVv7s1EkQL1s3PLrmePXsG20py3A+Wp4A4vdIu7NnW5FzyKbkqJ080mnrGLEe9fc1JcrX2\nsdlVVXO/LVUqwPad5wvLhrYcQK6yOH8ud97UuihkrmJ1LZ+x5PbB23L7aOsKuTfeeGP0mq+h1157\nbbC/8IUvRO2GDx8ebK4o/81vfjNqx9dTlsIA4Ljjjgt2TqbjKrdcgmHjjTeO2rEsxbKIrZJrr+Ut\ncPV1IK42b88x7i9/l5W4WDYrE3Ze8THg++38+fOjdnfddVfV/dkyJ2eccUaw99prr2Bb6dHKXO2N\nPDlCCCGEKCV6yBFCCCFEKalbrkotWmddu6kF86yrldtZ1zG739nNyRlPQOw6Zxeqja5nNzVXzbSV\nGlOLLFr3PWe32Mj1l156Kdhbb7111f21Fc3MmmqmjNUa6snOyfW91qrJtl2uunQjSC2qB6TloFxW\nWu641VrJOCdJ1XrcUp9pTSVjJnfOp46TrcJtK+s2A64abyUflsq/853vBPuOO+6I2j366KPBZknK\nLpI8ZMiQYO+www7RNs4wZVnHZo1yBd0+ffoE20oVM2bMCDZnpdpjz5ISX+Pt/YTHwlZhZrlqwYIF\nyb4PHToUXQH+nXzPstcmXryT75s225izcwcNGhRsO1+40rIN32gP5MkRQgghRCnRQ44QQgghSoke\ncoQQQghRSuqOyWENmbn77ruj1wcffHDVdryaLBDr0FYDT1XFzaXEsj5rUyFZd2TN12qVqdRRq+k+\n88wzwf7iF78Ybfv3v/8d7H322afq/toKPnYvv/xytI018lzKcIp6V3+uN9Yi1a7Wfecq9eb2kVqt\n3VafTcWiNYqHHnoouS1VZbzedOp64pysTp+aS7lV6HPfm+pHvSuNc//sqso29qMZcIyDvV5xnEvf\nvn2DfeaZZ0bt+Bzk1GuO4wGAxYsXB9vG63AcIsdg2BhMPpbTp08PNl9bgThukq+N9hrD1yPue646\nsU075/Oe92HvNbmK9WWC7z25cgBcGZkrXttzg8d20003DbaND7TXwvZGnhwhhBBClBI95AghhBCi\nlNTtU08tlGYraLIExG5D62pkuerFF1+MtnEKIKeDW1c3u6rZHWpdqLyN92HlqlQa8K677hq9Zung\nK1/5SrSNf8ucOXOCPWDAgKhdW7hQ+ffYY2cri7Zgx6nWxTBrlY1Sn7HflWuXkjUakcad+118DGs9\nno3iggsuCPaJJ54YbeNSBXz+2VIKqRIJNr2XX9vfmZK5rByROlatSd9P0YhK2bmx5BTpZnH77bcH\n26Z1f/7znw82p+fmFhxlecLOA04Nt+OUWrjWShAsDbGsZffH197rrrsu2KNHj47asUwyYcKEqp8H\n8vImhytw/1hOAzqenNIseBFslijtfKm1RIIdixbsmPB3dQTkyRFCCCFEKdFDjhBCCCFKSd1yVffu\n3YM9c+bMYFvXFbvEJ02aFGzrOk+5woDYhcouSbuPVLtcZDm7eF999dVoW6qq8zrrrBO9Hjt2bLDP\nP//8aBu7cnnBtCOPPDJqd/bZZyf72Ci437YyM2eQ8LFj9zgQH6/cQou1tsuRkkJqXTSz0Qt+Wjjz\nw2bg9O/fv6nfzfz617+OXvNcmj17drBz8h2PuV1klyUIK0fwPu051dGZNWtWsHOZZ1y1vC0466yz\nktt4McSDDjoo2sZViPmax7IFEGf+2bnJ2/habs8Jzga76qqrgn3DDTdE7ViGqhXO9rHXZK6GbM9F\nvi5wf+3Ypq7rZYOzo3ie2uxLW+k/BR9T3p+tjNzRkCdHCCGEEKVEDzlCCCGEKCV6yBFCCCFEKak7\nJodXEeeqljb+JZWmmsOmtHE1UtYTbUxOLr2QSaVB27gK1h05zoFTMIFY17b6L2vIrLVzfFJb8eyz\nzwZ71KhR0TaOO+DfbSvV1htfUwu2SjDHvHBVzVxFZj7+dkXkemJ0aq20a0sC3HLLLcH++c9/3urv\nbQ1HHHFE9HrEiBHB5mPAsQ5AfL7zvLVzltPh7Rjx5y677LJg29i0jg6fN/ac5zIKtaa1N4spU6ZU\ntXPYWKnNN9882BzjA8S/j1fynjZtWtSOY734GtcIDjnkkGBz7CewdIwOwzEjfJ5y6Q4AePzxx5e3\ni50Cvj/yOWyvYQsXLqxpfzxHcjE+HQ15coQQQghRSvSQI4QQQohSUrdcddNNNwWb3ZWcTg3Ei7Jx\neqFdyJMXxbML5K2++urBZleblYZY3ujdu3dN+2N3vk0VZVfp/Pnzq+7b0uy05eWB3Yo/+9nP2rEn\nopHY6rH2tVg2LMPaKt/tDcuz3M9apTMrR/DriRMnLmfvYnKL3dba39tuu62hfeqqsMTPqwbYBUu5\nTEguHZxDNmot0dIRkCdHCCGEEKVEDzlCCCGEKCV6yBFCCCFEKak7JmfMmDFVbUtKX7XaH6ez2jTl\nVIqajaHhVFcuac3LTgCxPsmphjZdkWN8WMfkpRqApZc+EEKIRlFraYyOQDNLTIjWwSUAeEkfWyKi\n1lXIOfaW73m2VEdHQ54cIYQQQpQSPeQIIYQQopTULVelqgbbFOqU+9KufPr5z38+2LaC5vPPPx9s\nruZqZSiWqzhNkuUpAFi8eHGwN9xww2DbCspc1ZFt6+7r2bMnUnDbjl4ZUgghRDlIpYPbivFvvfVW\nq/fN9/+333671Z9vS+TJEUIIIUQp0UOOEEIIIUpJ3XJVPVH0nMlkpRuuQmzlpS233DLYLI1dffXV\nUbs+ffoE+3vf+16yHyypPfTQQ8EeNmxY1O7pp58O9ty5c4Pdv3//qJ3tL6NsAyGEEG0NV/rnkAqu\nhAzE97YcHNrBGcabbLJJvV1sE+TJEUIIIUQp0UOOEEIIIUqJHnKEEEIIUUrqjsmpB1659JRTTom2\ncao1V2e0DBo0KNhTpkxJtrvrrruC7b1PtuOUdE5Bz7HSSitFr3Of4xWNORao1hV5hRBCiNYybty4\nYG+wwQbBtinftVbU5hjYHj16BPuJJ56ot4ttgjw5QgghhCglesgRQgghRCnpJtlECCGEEGVEnhwh\nhBBClBI95AghhBCilOghRwghhBClpE1TyBuBc25fAD8E8DGANQC8AOB4AJMB7Om9n2baXwrgBu/9\nJPP+6gD28d7f0iYdF0mcc5sB8AAeBfApgDUBjAXwQ++9gsY6ERrLjoVzbmMAswH82Hv/8xraPwDg\nQu/9ffTebpX3hqU+Z/ZxHYDx3vvf19Nns68jvfd/Wt79lBnn3CgAQwGsCmBbFHMPAK7x3t/Qbh3r\nIHSqhxzn3MoA/gRgkPd+XuW9XwD4Zuoz3vtTE5u2BXAQAD3kdAwWeO93AwDn3IoAngXwvygeXkXn\nQmPZcTgawDMAvg5gmQ85HQnn3AoAzkZxzRcJvPffB8I/GONb5p4o6FQPOQBWQ+G9WaPlDe/9GQDg\nnDsZwOHOuV0AbAbgW977+1r+MwHwEYCzALwH4C4AJwNYzzk3quUkER2G9QGsBGC+c+7LAL6PYtxW\nBHCU936mc247AL8F8BaK8TwPwJre+48S+xTtg8ayffkGgBMBXOec+4L3/hEAcM7NBPArAPsC6Avg\nBO/9WP6gc+4PKDzl/6D3+gD4NYDVUXjpzmSvDzHUOXcwgF4A/uC9v8g5twaKce6N4py43nv/m8rD\nzKUAtkPh/RvnvT8LwLUANnXO/d17P7IhR6ML4Zw7F8XYbgrgdABvArgKRZjKigB+4L0fbz1vzrlP\nUYzPLigejN9B4SX6tvf+Mefc7gDOAdANwIcAjvXev1A5p24E0M97f0gb/cxl0qlicrz3r6M4uJOd\nc/c5537knHPUZEFlMpwP4JQqu9gexYX1ShSDd68ecDoMGzrnHnDO/QPFf/6/q3jr1gVwqPd+dxQ3\nwJMq7X8F4Dzv/a4AXgOwSnt0WlRFY9kBcM4NR3EzGwfgegD/ZZq8W7leXgjg2+az5wF4y3t/vvnM\nbwBc5L0fAeA/APy+4q2z9ETxADUMwJnOufUr3/Ga9344gBEAznDO9QPwVRQ34y8CGA5gpHNuVxTX\n+gV6wFku+gLYvRKucTmA31Q8PSeiOCdynArg4sp8/TqAjSthHlcBOKgyXy8H8N/0mec70gMO0Mke\ncgDAe/8LFE+m11T+/tM5d2Jl8wOVv3NQXFCrfNwvqvK+aH8WeO93q1wANwawhXPuJADzAfzROfcg\nionWvdJ+MJaM91/buK8ij8ayY/BNANdVYqH+AOCrlZtUCw9U/r6IwuPWwtcB7I/q/yjuDuC8iof8\nf5SYHoEAAAMcSURBVFH8J//ZKu3u895/6r1/DcA0AJ8DsCOAewHAe/8ugMcBDKm839L+YwAPAdih\ntT9WVGUCxcLx8X8KwNrOue7JTwL/A+CnzrmLAGzkvb8dwCAUc/qWyjnwXQAb0mceaXD/l5vOJlfB\nObe6934hgNEARjvn/gLgospmdm93W+rDwAfN7p9Yfrz3H1TG9QQU/wkO8d4/X7lRbl9p9hkAn1Ts\n2hZfEW2OxrJ9cM6tDeArAGY55w6qvL1C5b2WYNTU9XIVACuj8LZYKep9FP/Fv7qMLnxCdjcUMpQN\nPF/W+2L54XveMo9/Je4VAOC9v9E5NwbASABnO+cmopCjZmXifjrcPbZTeXKcc3sDeNQ5txa93Q/F\nfwqt5RMUuqPomAwHMBPFOM10zq0K4AAskTKmAvhCxT5oqU+LjoTGsu05HMCD3vstvfeDvfeDARyH\npSWralwN4D8B/NY5t6HZNh6FvATnXPdK9mo1RlTarIfiGv0cgAkA9q68vwaKGJxJlff3cs51q0hf\nu1be0zW6sfDx3xbAworD4A0UcVIAsAcqDz0VyXIF7/1NKLx6O6MYx+7OuUGVNsOdc8e16a9oJZ3K\nk+O9H+OcGwBgrHPuHRRPovMB/D8sSZurlYkAfuGcu9Z7/40Gd1W0ng0r7k+g+C9yBorSAADwGAqX\n+i8B3OCcOwSFm/QK59xcAH9DMTE/gegIaCzbn2+iiE1k/grg4koWThbv/VPOuYsBXIc45uLbKB5+\nDkfxkHphYhdznXO3AegP4Hzv/WvOucsrn/1H5bPnVwLPZ6F4yB2Pwtt0m/f+4coDz8vOuUkAhnvv\n3058l6iNkwFc5Zw7AcXD41GV968FcFMlhuvvAF6vvP88gHudc4tRjMs53vt3nXNHArjGOfdepV2H\nfsjR2lWiU1KJ8F/kvf+Xc24IgNHee7esz4mOh8ZSCNEsOpUnRwjiQxSZHe+h8BYcv4z2ouOisRRC\nNAV5coQQQghRSjpV4LEQQgghRK3oIUcIIYQQpUQPOUIIIYQoJXrIEUIIIUQp0UOOEEIIIUqJHnKE\nEEIIUUr+P9rPn3cslCmKAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f67cef7f090>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "qMwKN4DHcqXV",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Exploratory Task\n",
        "Run the above cells multiple times to get a good sense of the dataset. Look at the general structure of the images and their corresponding labels. Can you spot some classes that might be difficult for a classifier to distinguish? If so, why do you think they would be difficult to distinguish? Chat to your neighbour about this. \n"
      ]
    },
    {
      "metadata": {
        "id": "S6U63sTpGYvg",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Preparing the data with TensorFlow\n",
        "At the moment, our training data consists of two large tensors. The images are stored in a tensor of shape $[50000, 28, 28]$, consisting of all the $28 \\times 28$ images matrices stacked together. The labels are stored in a 1D vector of shape $[50000]$. We wish to train a model using **mini-batch stochastic gradient descent**. In order to do so, we need to shuffle the data and split it into smaller (mini-)batches. We also convert the data from numpy arrays to TensorFlow Tensors.\n",
        "\n",
        "#### Questions\n",
        "* Can you explain what the difference is between normal stochastic gradient descent and \"batched\" stochastic gradient descent? What is the purpose of chunking data into batches during training? Are there any trade-offs? What are they? **HINT**: Speak to your neighbours and tutors about this. You can also read [this blog post](https://machinelearningmastery.com/gentle-introduction-mini-batch-gradient-descent-configure-batch-size/) for more details.\n",
        "* Why is it important to randomise the data before chunking it into batches? \n",
        "\n",
        "In order to do this batching (and shuffling) we will use the Tensorflow [Dataset API](https://www.tensorflow.org/api_docs/python/tf/data/Dataset), which is a set of simple reusable components that allow you to build efficient data pipelines. Data is said to \"stream\" through the pipeline, meaning that when something at the output of the pipeline wants data, the pipeline will provide that data as soon as it has enough, rather than waiting to process all the data. This allows you to easily build pipelines that work on large datasets without having to load it all into memory in one go!\n",
        "\n",
        "We build this pipeline step-by-step:"
      ]
    },
    {
      "metadata": {
        "id": "HO8Z6SImg_7W",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We being by defining the ```batch_size``` hyperparameter of our model. This hyperparameter controls the sizes of the mini-batches (chunks) of data that we use to train the model. The value you use will affect the memory usage, speed of convergence and potentially also the performance of the model. It also interacts with the *learning rate* used in gradient descent. "
      ]
    },
    {
      "metadata": {
        "id": "x7VwqreOhbfv",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "batch_size = 128"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "3CaT9bWFheRw",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The first component we add will group the image and label tensors together into a tuple and then split them into individual entries - ie. 50000 tuples containing a (28, 28) dimensional image and 1D label associated with that image. The following line adds this splitting component to the pipeline."
      ]
    },
    {
      "metadata": {
        "id": "7wITDBK8h5Z6",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "train_ds = tf.data.Dataset.from_tensor_slices((train_images, train_labels))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "VNVG8Kb3h_CW",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The next thing we do is apply a map operation. This lets us run an arbitrary function on each element. The function we provide returns the image values divided by 255 and converted ('cast') to a float and the label converted to a 32-bit integer.\n",
        "\n",
        "**NOTE**: \"Lambda\" functions are just one-line, anonymous functions. In this case, it defines a function that takes arguments x and y (before the colon), and outputs their manipulated values (after the colon)."
      ]
    },
    {
      "metadata": {
        "id": "4fcE7eTviBOi",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Divide image values and cast to float so that they end up as a floating point number between 0 and 1\n",
        "train_ds = train_ds.map(lambda x, y: (tf.cast(x, tf.float32) / 255.0, tf.cast(y, tf.int32)))  "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "gm5iOdU0iD_0",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Then we add a **shuffle** component. This returns a random element from the pipeline. Can you spot a potential problem here? \n",
        "\n",
        "**QUESTION**:  What might happen if the shuffle component didn't have a *buffer*?  **HINT**: Refer back to the discussion about the data pipeline and \"data streaming through it\" above."
      ]
    },
    {
      "metadata": {
        "id": "bvr5r8k6iJGI",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Shuffle the examples.\n",
        "train_ds = train_ds.shuffle(buffer_size=batch_size * 10) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "OlGI65QIjOgp",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The final component in our pipeline is the batch component. This just requests `batch_size` elements from the previous pipeline component, groups them together into a single tensor and returns that."
      ]
    },
    {
      "metadata": {
        "id": "PB8CZrsmjR7Z",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Now \"chunk\" the examples into batches\n",
        "train_ds = train_ds.batch(batch_size)  \n",
        "\n",
        "# The output of this pipeline will be tuples of tensors containing images and labels. \n",
        "# The images will be of shape (batch_size, 28, 28) and the labels of shape (batch_size, )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "D6Y4EvFijYRR",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Finally, we apply the same pre-processing (converting to Tensors and normalising the values to lie between 0 and 1) to the validation set. Here we don't need to use the Dataset API (but could if we wanted to!) because we don't need batching or shuffling. "
      ]
    },
    {
      "metadata": {
        "id": "EHxnBJjRB_Rl",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Apply the same preprocessing to the validation set\n",
        "# Ignore this for now - it will be used later! \n",
        "validation_images = tf.convert_to_tensor(validation_images, dtype=tf.float32) / 255.0\n",
        "validation_labels = tf.convert_to_tensor(validation_labels, dtype=tf.int32)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "v04ibRFSALQu",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Define the model"
      ]
    },
    {
      "metadata": {
        "id": "KItI--nMHFRQ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "In this section we'll build a classifier. A **classifier** is a function that takes an object's characteristics (or \"features\") as inputs and outputs a prediction of the class (or group) that the object belongs to. It may make a single prediction for each input or it may output some score (for example a probability) for each of the possible classes. Specifically, we will build a classifier that takes in (a batch of) 28 x 28 Fashion MNIST images as we've seen above, and outputs predictions about which class the image belongs to. \n",
        "\n",
        "For each (batch of) input images, we will use a **feed-forward neural network** to compute un-normalised scores (also known as **logits**) for each of the 10 possible classes that the image could belong to. We can then **classify** the image as belonging to the class which receives the highest score, or we can quantify the model's \"confidence\" about the classifications by converting the scores into a probability distribution. \n",
        "\n",
        "A feed-forward neural network consisting of $N$ layers, applied to an input vector $\\mathbf{x}$ can be defined as:\n",
        "\n",
        "\\begin{equation}\n",
        "\\mathbf{f_0} = \\mathbf{x} \\\\\n",
        "\\mathbf{f_i} = \\sigma_i(\\mathbf{W_if_{i-1}} + \\mathbf{b_i}) \\ \\ \\ i \\in [1, N]\n",
        "\\end{equation}\n",
        "\n",
        "Each layer has a particular number, $m_i$, of neurons. The parameters of a layer consist of a matrix $\\mathbf{W_i} \\in \\mathbb{R}^{m_i \\times m_{i-1}}$ and bias vector $\\mathbf{b_i} \\in \\mathbb{R}^{m_i}$. Each layer also has a non-linear activation function $\\sigma_i$. \n",
        "\n",
        "**QUESTION**: Why do you think the activation functions need to be *non-linear*? What would happen if they were *linear*? **HINT**: If you're stuck, consider the very simplest case of an identity activation (which essentially does nothing) and ignore the biases. \n"
      ]
    },
    {
      "metadata": {
        "id": "tHy4bjbTVCzV",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Aside: Activation functions"
      ]
    },
    {
      "metadata": {
        "id": "Ox7VKs3BVCFV",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "Activation functions are a core ingredient in deep neural networks. In fact they are what allows us to make use of multiple layers in a neural network. There are a number of different activation functions, each of which are more or less useful under different circumstances. The four activation functions that you are most likely to encounter are, arguably, [ReLU](https://www.tensorflow.org/api_docs/python/tf/keras/layers/ReLU), [Tanh](https://www.tensorflow.org/api_docs/python/tf/keras/activations/tanh), [Sigmoid](https://www.tensorflow.org/api_docs/python/tf/keras/activations/sigmoid), and [Softmax](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Softmax). \n",
        "\n",
        "ReLU, has in recent years, become the default choice for use in MLPs and Convolutional Neural Networks (CNNs). ReLU has two advantages over Tanh and Sigmoid: it is computationally much more efficient, and, it allows us to use deeper networks because it does not suffer from [vanishing gradients](https://en.wikipedia.org/wiki/Vanishing_gradient_problem). As a result of their success, a number of ReLU variants, such as [LeakyRelu](https://www.tensorflow.org/api_docs/python/tf/keras/layers/LeakyReLU) and [PReLU](https://www.tensorflow.org/api_docs/python/tf/keras/layers/PReLU), have been developed.\n",
        "\n",
        "Sigmoid and Softmax activations are often found after the last layer in binary and multi-class classification networks, respectively, as they transform the outputs of the network in such a way that we can interpret them as class probabilities.\n",
        "\n",
        "Both Tanh and Sigmoid are found in LSTM and GRU recurrent neural networks, which we will find out more about in the coming days. They are also useful in MLPs and CNNs where we want the output to be bounded between -1 and 1 (Tanh) or 0 and 1 (Sigmoid).\n",
        "\n",
        "Read more about activation functions [here](https://towardsdatascience.com/activation-functions-neural-networks-1cbd9f8d91d6). "
      ]
    },
    {
      "metadata": {
        "id": "bDRoliDeAShd",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Configure the feed-forward neural network\n",
        "We configure the feed-forward neural network part of our classifier using the [Keras Layers API](https://www.tensorflow.org/api_docs/python/tf/keras/layers). This API consists of various reusable building-blocks that allow us to define many different neural network architectures (similar to how we defined a data pipeline earlier!). \n",
        "\n",
        "Here we use the [Sequential](https://www.tensorflow.org/api_docs/python/tf/keras/Sequential) component which allows us to wrap together a sequence of layers. An important point to note here is that we are **configuring** our neural network architecture as a pipeline. We can think of the resulting ```model`` variable as a *function* that takes a batch of images as inputs and outputs a batch of logits. "
      ]
    },
    {
      "metadata": {
        "id": "cDPrTMSahRBu",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "model = tf.keras.Sequential([\n",
        "    # Convert the 28x28 image into a flat vector of 28x28 = 784 values\n",
        "    tf.keras.layers.Flatten(input_shape=(28, 28), name='flatten_input'),  \n",
        "    # Create a \"hidden\" layer with 256 neurons and apply the ReLU non-linearity\n",
        "    tf.keras.layers.Dense(256, activation=tf.nn.relu, name='input_to_hidden1'),  \n",
        "    # Create another hidden layer with 128 neurons\n",
        "    tf.keras.layers.Dense(128, activation=tf.nn.relu, name='hidden1_to_hidden2'),\n",
        "    # Create an \"output layer\" with 10 neurons\n",
        "    tf.keras.layers.Dense(10, name='hidden_to_logits'),  \n",
        "])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "W3AU_94YP1st",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The following summary shows how many parameters each layer is made up of (the number of entries in the weight matrics and bias vectors). Note that a value of ```None``` in a particular dimension of a shape means that the shape will dynamically adapt based on the shape of the inputs. In particular, the output shape of the ```flatten_input``` layer will be $[N, 784]$ when the batch of inputs passed to the model has shape $[N, 28, 28]$"
      ]
    },
    {
      "metadata": {
        "id": "fSW4V3HuQd1-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 272
        },
        "outputId": "f605f661-f91e-4ee5-ee7f-1df1d691038b"
      },
      "cell_type": "code",
      "source": [
        "model.summary() "
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "flatten_input (Flatten)      (None, 784)               0         \n",
            "_________________________________________________________________\n",
            "input_to_hidden1 (Dense)     (None, 256)               200960    \n",
            "_________________________________________________________________\n",
            "hidden1_to_hidden2 (Dense)   (None, 128)               32896     \n",
            "_________________________________________________________________\n",
            "hidden_to_logits (Dense)     (None, 10)                1290      \n",
            "=================================================================\n",
            "Total params: 235,146\n",
            "Trainable params: 235,146\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "PwI8FcreWsC_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**QUESTION**: Note that the flattening operation has no parameters. However, the Dense (fully-connected) layer mapping from the 784-D output of the flatten operation to the 256-D hidden layer has 200,960 parameters! Can you explain why the dense layer has 200,960 parameters? **HINT**: Refer back to the equations of a feed-forward network."
      ]
    },
    {
      "metadata": {
        "id": "VwWoSlpDMzNZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Define the loss\n",
        "As we did in the previous practical, we need to specify a loss function for our classifier. This tells us how good our model's predictions are compared to the actual labels (the targets), with a lower loss meaning better predictions. The standard loss function to use with a **multi-class classifier** is the **cross-entropy loss** also known as the \"negative log likelihood\". Suppose we have a classification problem with $C$ classes. A classifier is trained to predict a probability distribution $p(y | X_i)$ for each input $X_i$ from a batch of $N$ examples. The vector $p(y|X_i)$ is $C$ dimensional, sums to one, and we use $p(y|X_i)_c$ to denote the $c$th component of  $p(y|X_i)$. The true class for example $i$ in the batch is $y_i$ and we define the indicator function $\\mathbb{1}[y_i=c]$ to be 1 whenever $y_i = c$ and $0$ otherwise. This classifier has a cross-entropy loss of\n",
        "\n",
        "$- \\frac{1}{N}\\sum_{i=1}^N \\sum_{c=1}^C log( p(y|X_i)_c) \\mathbb{1}[y_i=c]$\n",
        "\n",
        "**NOTE**: The indicator is one for the true class label, and zero everywhere else. So in that sum, the indicator just \"lifts out\" the $log(p(y|X_i))$ values for all true classes. So the above expression is minimised (note the negative at the front) when the model places all its probability mass on the true labels for all the examples. Remember that  log(1)=0 , thus the closer all probabilities of $y_i = c$ are to one, the lower the loss will be and the better the model will be performing.\n",
        "\n",
        "**QUESTION**: \n",
        "* Why do you think this is a *good* loss function?\n",
        "* Can you think of any potential issues with this loss function?\n",
        "\n",
        "Fortunately we don't need to write this function ourselves as Tensorflow provides a version called \n",
        "\n",
        "```tf.nn.sparse_softmax_cross_entropy_with_logits```. \n",
        "\n",
        "**NOTE**: This function actually computes the cross entropy loss directly from the un-normalised logits, rather than from the probability distribution for numerical stability.\n",
        "\n",
        "By the way, for training data in which the labels are themselves distributions rather than exact values, this definition of cross-entropy still works, where the indicator function is replaced with the corresponding probability of each class for that example. This might be important when we are not sure whether the training data has been labelled correctly, or when the data was labelled by a human who gave their answer along with a degree of confidence that the answer was correct"
      ]
    },
    {
      "metadata": {
        "id": "7OHi-HftJy9_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Train the model\n",
        "Now that we have our data, data processing pipeline, our neural network architecture and the corresponding loss that we want to minimise, we need to **train** the model using batched stochastic gradient descent. We do this in multiple **epochs**, which is a single iteration through the entire training dataset. Briefly, in each epoch we loop over all the batches of images and labels, and for each batch we perform the following steps:\n",
        "* Get the **predictions** of the model on the current batch of images\n",
        "* Compute the **average loss** values across the batch, telling us how good these predictions are / how close they are to the true targets.\n",
        "* Compute the **gradient of the average loss** (or the average gradient of the losses in the batch) with respect to each of the model's parameters: This tells us the direction to move in \"parameter space\" to decrease the loss value\n",
        "* **Adjust the parameters** by taking a small step in the direction of each component of the gradient (where the learning rate controls the size of the step)\n",
        "\n",
        "During training we also track some metrics, such as the loss and accuracy to see how well the classifier is doing. Note that the cell below may take a few minutes to run! \n",
        "\n",
        "###ASIDE: Computing gradients in Eager mode\n",
        "\n",
        "In graph mode, TensorFlow builds up a \"computation graph\" which captures all operations of the model and their dependencies. For training, the gradient can then be computed by traversing backwards from every node through its dependents and applying the \"chain rule\" of differentiation. \n",
        "\n",
        "However, in Eager mode, we don't have the concept of the computation graph anymore. Operations are performed imperatively (in the order in which they were executed). TensorFlow therefore uses a mechanism called the \"GradientTape\" for computing gradients in Eager mode. Basically, the gradient tape records the order of all operations as they are executed, and can then be \"run backwards\" (traversed from last to first operation) for computing the gradients."
      ]
    },
    {
      "metadata": {
        "id": "5hpiGc40hYkK",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Create an optimizer\n",
        "# The optimizer is responsible for controlling the learning rate\n",
        "optimizer = tf.train.AdamOptimizer()\n",
        "\n",
        "step_counter = tf.train.get_or_create_global_step()  # Just a variable that keeps track of how many training steps we've run\n",
        "num_epochs = 50  # The number of epochs to run\n",
        "\n",
        "# Lists to store the loss and accuracy of every epoch\n",
        "epoch_losses = []\n",
        "epoch_accuracies = []\n",
        "\n",
        "for epoch in range(num_epochs):\n",
        "  # Tensorflow provides a convenient API for tracking a number of metrics during training/evaluation\n",
        "  loss_avg = tfe.metrics.Mean()\n",
        "  accuracy = tfe.metrics.Accuracy()\n",
        "\n",
        "  # Loop over our data pipeline\n",
        "  for step, (image_batch, label_batch) in enumerate(train_ds):\n",
        "    \n",
        "    # Initialise a GradientTape to track the operations\n",
        "    with tf.GradientTape() as tape:\n",
        "      # Compute the logits (un-normalised scores) of the current batch of examples \n",
        "      # using the neural network architecture we defined earlier\n",
        "      logits = model(image_batch, training=True)\n",
        "      # Compute the cross-entropy loss of the classification outputs on this batch\n",
        "      loss_value = tf.nn.sparse_softmax_cross_entropy_with_logits(\n",
        "          logits=logits, labels=label_batch)  \n",
        "      # Compute the average loss over the batch\n",
        "      loss_value = tf.reduce_mean(loss_value)  \n",
        "      \n",
        "      # Add current batch loss to our loss metric tracker - note the function call semantics\n",
        "      loss_avg(loss_value)  \n",
        "      # Compare most likely predicted label to actual label\n",
        "      accuracy(tf.argmax(logits, axis=1, output_type=tf.int32), label_batch)\n",
        "\n",
        "    # Play the tape backwards and get the gradient of the loss of the current batch\n",
        "    # Note we're now outside the scope of the with-block above\n",
        "    grads = tape.gradient(loss_value, model.variables)\n",
        "    # Use the optimizer to apply the gradients to the model parameters along with\n",
        "    # its internal learning rate\n",
        "    optimizer.apply_gradients(\n",
        "        zip(grads, model.variables), global_step=step_counter)\n",
        "  \n",
        "  # Get the average loss and accuracy for the epoch\n",
        "  epoch_loss = loss_avg.result()\n",
        "  epoch_losses.append(epoch_loss)\n",
        "  epoch_accuracy = accuracy.result()\n",
        "  epoch_accuracies.append(epoch_accuracy)\n",
        "  print(\"Epoch {:03d}: Loss: {:.3f}, Accuracy: {:.3%}\".format(epoch, epoch_loss, epoch_accuracy))\n",
        "  \n",
        "# Plot the loss for all epochs using Matplotlib\n",
        "plt.figure()\n",
        "plt.plot(range(num_epochs), epoch_losses)\n",
        "plt.title('Loss vs epochs')\n",
        "\n",
        "# Plot the accuracy for all epochs using Matplotlib\n",
        "plt.figure()\n",
        "plt.plot(range(num_epochs), epoch_accuracies)\n",
        "plt.title('Accuracy vs epochs')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "i-oib_tntPIY",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The above code block shows a typical training loop. There's actually an easier way to do this using Tenosrflow and Keras, which we'll see in the next prac, but for now we expand it so you can see what's going on.\n",
        "\n",
        "Lets visualise some of the model's prediction on the training set"
      ]
    },
    {
      "metadata": {
        "id": "rotl5kAX7SrW",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "images, labels = next(train_ds.make_one_shot_iterator())  # Get a batch of images and labels\n",
        "\n",
        "_logits = model(images, training=False)  # Pass the images to the model function and get its output logits\n",
        "predicted_labels = tf.argmax(_logits, axis=1, output_type=tf.int32)\n",
        "\n",
        "img_indexs = np.arange(images.numpy().shape[0])\n",
        "np.random.shuffle(img_indexs)\n",
        "\n",
        "plt.figure(figsize=(10,10))\n",
        "for i in range(25):\n",
        "    plt.subplot(5,5,i+1)\n",
        "    plt.xticks([])\n",
        "    plt.yticks([])\n",
        "    plt.grid('off')\n",
        "    \n",
        "    img_index = img_indexs[i]\n",
        "    predicted_label = int(predicted_labels[img_index])\n",
        "    \n",
        "    plt.imshow(images[img_index], cmap=plt.cm.gray)\n",
        "    \n",
        "    actual_label = int(labels[img_index].numpy())\n",
        "    plt.xlabel(\"Actual: {} ({})\\n Predicted: {} ({})\".format(\n",
        "        actual_label, text_labels[actual_label], predicted_label, text_labels[predicted_label]\n",
        "    ))\n",
        "    \n",
        "plt.tight_layout()\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "SIUsSjNwVk_n",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        " ### Aside: Optimisation schemes"
      ]
    },
    {
      "metadata": {
        "id": "ExBhca58VpPH",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "You might have noticed that we are using the Adam optimizer to train our neural networks. Adam is a variant of stochastic gradient descent which often performs well in practice. In fact, there is a whole range of variations on stochastic gradient descent that are often used. Here is an illustration showing how a few of these methods perform on a toy problem: \n",
        "\n",
        "![optimization methods](http://ruder.io/content/images/2016/09/saddle_point_evaluation_optimizers.gif)\n",
        "\n",
        "For a detailed description of various optimization methods read [this](http://ruder.io/optimizing-gradient-descent/) article. For a great visual discussion on how these methods work see [this](https://distill.pub/2017/momentum/) article."
      ]
    },
    {
      "metadata": {
        "id": "zMwZVX1T3zWP",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Testing\n",
        "\n",
        "After 50 epochs of training on our training set, we obtain a loss of around 0.061 and accuracy of 97.7%. That's pretty good, right? \n",
        "\n",
        "It is important to distinguish between data that is **in-sample** and **out-of-sample**. Our training data is all in-sample and we would expect any resonably powerful model (like an MLP) to be very accurate and have a low loss on this (in fact, with a sufficiently large MLP and enough training epochs, we can get the loss arbitrarily close to zero!). This is not a good thing though, because pushing the loss close to zero may mean that the model has fit the **noise** in the training data rather than the true signal. If this is the case, it will not perform well on out-of-sample data that it has not seen before (which is what we really care about!). To assess this, we evaluate our trained model on the held-out test set (but we don't update the parameters of the model when testing it, that would be cheating!)"
      ]
    },
    {
      "metadata": {
        "id": "SmBt_X4B4g3-",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# We need to apply the same pre-processing to the test set as we did to the training set\n",
        "# Since we don't need batching or shuffling, we can do this directly instead of \n",
        "# building a tf.Dataset pipeline\n",
        "\n",
        "tf_test_images = tf.convert_to_tensor(test_images, dtype=tf.float32) / 255.0\n",
        "tf_test_labels = tf.convert_to_tensor(test_labels, dtype=tf.int32)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "j0BaJidP46Bm",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "a2cb8a2e-7dd7-4eee-8f7a-9838cecd414a"
      },
      "cell_type": "code",
      "source": [
        "test_accuracy = tfe.metrics.Accuracy()\n",
        "\n",
        "test_logits = model(tf_test_images, training=False)\n",
        "\n",
        "# Compute the average cross-entropy loss of the classification over the entire test set\n",
        "test_loss = tf.nn.sparse_softmax_cross_entropy_with_logits(\n",
        "    logits=test_logits, labels=tf_test_labels)  \n",
        "test_loss = tf.reduce_mean(test_loss)\n",
        "\n",
        "# Compare predicted labels to actual labels\n",
        "test_accuracy(tf.argmax(test_logits, axis=1, output_type=tf.int32), tf_test_labels)\n",
        "\n",
        "print('Completed testing on', tf_test_images.shape[0], 'examples...')\n",
        "print('Loss: {:.3f}, Accuracy: {:.3%}'.format(test_loss, test_accuracy.result()))"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Completed testing on 10000 examples...\n",
            "Loss: 0.607, Accuracy: 88.920%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "Qn4ptik28aHF",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "And again we visualise some of the model's predictions:"
      ]
    },
    {
      "metadata": {
        "id": "JHVSTjZU8ehm",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "test_predictions = tf.argmax(test_logits, axis=1, output_type=tf.int32)\n",
        "\n",
        "plt.figure(figsize=(10,10))\n",
        "for i in range(25):\n",
        "    plt.subplot(5,5,i+1)\n",
        "    plt.xticks([])\n",
        "    plt.yticks([])\n",
        "    plt.grid('off')\n",
        "    \n",
        "    img_index = np.random.randint(0, 10000)\n",
        "    plt.imshow(test_images[img_index], cmap=plt.cm.gray)\n",
        "    \n",
        "    actual_label = int(test_labels[img_index])\n",
        "    predicted_label = int(test_predictions[img_index])\n",
        "    \n",
        "    plt.xlabel(\"Actual: {} ({})\\n Predicted: {} ({})\".format(\n",
        "        actual_label, text_labels[actual_label], predicted_label, text_labels[predicted_label]\n",
        "    ))\n",
        "    \n",
        "plt.tight_layout()\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "yo6bEKs17a_d",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "What happened? We got 97.7% accuracy on the training set, but only 88.9% on the test set. This is an example of how a model's in-sample performance differs from its out-of-sample performance. It may even be considered **overfitting** (where the model failing to generalise its in-sample performance to unseen out-of-sample data). There are a number of ways we can address this, but first, it's important to realise that we should try and keep the number of evaluations on the test set to a minimum. \n",
        "\n",
        "**QUESTION**: Can you think of WHY we should limit the number of evaluations on the test set? This is an important point, so spend a few minutes thinking about it, discuss it with your neighbour and ask the tutors if you get stuck! \n",
        "\n",
        "The first important method is called **validation**. Using validation, we reserve a portion of our training dataset and call it the validation set. After each epoch of training (or every K epochs where K can be a fraction), we evaluate our model on this validation set (without updating the model parameters). This gives us a good sense of how well we expect our model to do on out-of-sample data. In this way, the validation set acts as a surrogate for the test set which we can safely use during model development and training. We can also use this validation set to tune our model by selecting hyperparameters such as the batch size, sizes of layers or the network architecture. \n"
      ]
    },
    {
      "metadata": {
        "id": "ot-FjimzV6op",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Aside: Underfitting"
      ]
    },
    {
      "metadata": {
        "id": "iQMKAkkFV7bf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "In addition to the problem of _overfitting_ (in which the model is too complex and as a result is able to memorize the training data rather than learning a general pattern) there is the problem of _underfitting_ which is the opposite. Underfitting occurs when the model does not have enough complexity (also known as capacity) to learn a general pattern. While overfitting is characterised by diverging test/validation and training scores, underfitting is characterised by test/validation and training scores that very slowly continue to improve. \n",
        "\n",
        "\n",
        "Unlike overfitting which has a number of solutions, such as collecting more data, data augmentation, dropout, and L1/L2 regularization, underfitting has one simple solution: just make your model incrementally more complex until it no longer underfits.\n",
        "\n",
        "![underandoverfitting](https://i.imgur.com/m2bSP1S.png)"
      ]
    },
    {
      "metadata": {
        "id": "E8KSIHzkWC1N",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##TASKS:\n",
        "\n",
        "1. [**ALL**] Implement **Validation** by adding some code to the loop in the \"Train the model\" section (**TIP**: The code for evaluating on the validation set will be very similar to evaluating on the test set!): \n",
        " * Evaluate your model on the validation set at the end of each training epoch and print the loss and accuracy on the validation set. We have already defined a validation set and done the necessary preprocessing for you.\n",
        "  * Evaluate how closely the loss and accuracy on the validation set matches the eventual performance on the test set. \n",
        "\n",
        "2. [**ALL**] Implement DROPOUT (see [Tensorflow documentation](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dropout) and [research paper](https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf) ) to improve the model's generalisation.\n",
        "\n",
        "3. [**ADVANCED**] Implement BATCH NORMALISATION ([Tensorflow documentation](https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization) and [research paper](http://proceedings.mlr.press/v37/ioffe15.pdf))to improve the model's generalisation.\n",
        "\n",
        "4. [**OPTIONAL**] Remove the shuffling from the dataset pipeline and compare the generalisation performance of the model with and without shuffling. \n",
        "\n",
        "5. [**OPTIONAL**] Adjust the model architecture by adding layers and modifying other parameters to improve the performance on your **validation** set. Once you've settled on an architecture and hyperparameters, evaluate your model on the **test** set. See the Tensorflow documentation for examples of parameters you can change in the model.\n",
        "\n",
        "\n",
        "## Important Tip\n",
        "If you want to re-train your model, you need to rerun all the code in the \"Define the model\" section first. This is to ensure that the Tensorflow variables containing the model parameters get reset. If you don't do this, you're effectively running more epochs on an already-trained model."
      ]
    }
  ]
}