Principal_Component_Analysis.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Principal Component Analysis.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyMKvhGe+3cdIs0fluigs0lA",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/vinayshanbhag/math/blob/master/Principal_Component_Analysis.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x_n0Lj6zNgDK",
        "colab_type": "text"
      },
      "source": [
        "## Principal Component Analysis (PCA)\n",
        "\n",
        "PCA is an unsupervised dimensionality reduction technique that can be used for feature extraction, wherein we can construct a lower dimensional subspace from a dataset with large number of features.\n",
        "\n",
        "The goal of PCA is to find directions of maximum variance in high-dimensional data and project the data into a new subspace with equal or fewer dimensions."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZEQmTIJuBYlN",
        "colab_type": "text"
      },
      "source": [
        "PCA can be summarized in following steps-\n",
        "\n",
        "1. Standardize the $m$-dimensional dataset\n",
        "2. Create covariance matrix\n",
        "3. Perform eigen decomposition of the covariance matrix\n",
        "4. Sort eigenvectors in descending order of their corresponding eigenvalues\n",
        "5. Select $k$ eigenvectors with the largest eigenvalues, where $k$ is the number of principal components we are interested in $(k \\leq m)$\n",
        "6. Create projection matrix $W$ from top $k$ eigenvectors\n",
        "7. Transform the input dataset using $W$ to get new $k$-dimensional features \n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dzSB1awPK0mK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "from sklearn.datasets import load_wine\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "figsize=(10,5)"
      ],
      "execution_count": 35,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fpSv57BHUChk",
        "colab_type": "text"
      },
      "source": [
        "The open source UCI wine dataset built into sklearn has 13 features for each sample. The target is one of 3 cultivars from Italy. The targets are not used by PCA and are used here only for illustrating PCA results in the end."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_A5-L95fLe2v",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "70f06467-3c21-4306-adb0-45e8f0218fcb"
      },
      "source": [
        "X, y = load_wine(return_X_y=True)\n",
        "print(f\"X.shape={X.shape}, y.shape={y.shape}\")"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "X.shape=(178, 13), y.shape=(178,)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tfuGo1hpMrwl",
        "colab_type": "text"
      },
      "source": [
        "### 1. Standardize the  $m$-dimensional dataset\n",
        "\n",
        "Feature-wise standardization of input $X$\n",
        "\n",
        "\\begin{align*}\n",
        "X_{std} = \\frac{(X - \\mu)}{\\sigma}\n",
        "\\end{align*}"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-paqWYWKEdxq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X_std = (X-X.mean(axis=0))/X.std(axis=0)"
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mAPCjPaLOVdq",
        "colab_type": "text"
      },
      "source": [
        "### 2. Create covariance matrix\n",
        "\n",
        "Covariance between features $j$ and $k$ is given by &ndash;\n",
        "\n",
        "\\begin{align*}\n",
        "\\sigma_{jk}= \\frac{1}{n-1}\\displaystyle\\sum_{i=1}^{n}(x_j^{(i)}-\\mu_j)(x_k^{(i)}-\\mu_k)\n",
        "\\end{align*}\n",
        "\n",
        "A covariance matrix with $m$ features will look like this&ndash;\n",
        "\n",
        "\\begin{align*}\n",
        "\\Sigma = \\begin{bmatrix}\\sigma_{1}^2&\\sigma_{12}&\\ldots&\\sigma_{1m}\\\\\\sigma_{21}&\\sigma_{2}^2&\\ldots&\\sigma_{2m}\\\\\\vdots&\\vdots&\\ddots&\\vdots\\\\\\sigma_{m1}&\\sigma_{m2}&\\ldots&\\sigma_{m}^2\\end{bmatrix}\n",
        "\\end{align*}\n",
        "\n",
        "> $\\Sigma \\in \\mathbb{R}^{m\\mathbb{x}m}$\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I8BG8Zi2OS0a",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "59eaecc9-2545-404d-dd02-5686905be4a6"
      },
      "source": [
        "cov_mat = np.cov(X_std.T)\n",
        "print(f\"Covariance matrix shape:{cov_mat.shape}\")"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Covariance matrix shape:(13, 13)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cLHkdq0lOs1v",
        "colab_type": "text"
      },
      "source": [
        "### 3. Perform eigen decomposition of the covariance matrix\n",
        "\n",
        "We want to find eigenpairs of the covariance matrix such that&ndash;\n",
        "\n",
        "\\begin{align*}\n",
        "\\Sigma v = \\lambda v\n",
        "\\end{align*}\n",
        "\n",
        ">$\\lambda$ is a scalar, the eigenvalue\n",
        "\n",
        "Eigenvalues are simply the coefficients attached to eigenvectors, which give the axes magnitude. In this case, they are the measure of the data’s covariance."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8BxooHjjOgUF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Eigen decompose the covariance matrix\n",
        "eigen_values, eigen_vectors = np.linalg.eig(cov_mat)\n",
        "eigen_pairs = [(np.abs(eigen_values[i]), eigen_vectors[:,i]) for i in range(len(eigen_values))]"
      ],
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AG6re8KBMKDZ",
        "colab_type": "text"
      },
      "source": [
        "### 4. Sort eigenvectors \n",
        "\n",
        "Sort eigenvectors in descending order of their corresponding eigenvalues\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NmiYls2JMYAR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Sort the eigen pairs by eigen values in descending order\n",
        "eigen_pairs.sort(key=lambda k: k[0], reverse=True)"
      ],
      "execution_count": 33,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b3Mq1c9MMZ2I",
        "colab_type": "text"
      },
      "source": [
        "### 5. Select $k$ eigenvectors\n",
        "\n",
        "Since our goal is to reduce dimensionality, we want to only select the eigenvectors that contain most of the information (variance)\n",
        "\n",
        "Explained variance ratio = $\\frac{\\lambda_j}{\\sum_{j=1}^{d}\\lambda_j}$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3zt6RKNLQ1IN",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 369
        },
        "outputId": "193f5b3e-2460-48ac-c1d1-cea3cecefe6d"
      },
      "source": [
        "total_var = np.sum([i[0] for i in eigen_pairs])\n",
        "expl_var = [i[0]/total_var for i in eigen_pairs]\n",
        "cumulative_var = np.cumsum(expl_var)\n",
        "\n",
        "plt.figure(figsize=figsize)\n",
        "plt.bar(range(1,X.shape[1]+1), expl_var, alpha=0.5, align='center',label='Individual explained variance')\n",
        "plt.step(range(1,X.shape[1]+1), cumulative_var, where='mid',label='Cumulative explained variance')\n",
        "plt.ylabel('Explained variance ratio')\n",
        "plt.xlabel('Principal component')\n",
        "plt.legend(loc='best')\n",
        "plt.grid();\n",
        "plt.tight_layout();"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsgAAAFgCAYAAACmDI9oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxV1bnw8d8joogBi2K5VLBYK0pQBokIdWjQqlQtaqF1bO3gUKut0tqqrRVt9b3X61BLq/JaB/Rax6pX5GpVWqP3esEyiCgBhyoiOCEVSRBUYL1/5CTvBhI4YM45JPl9P598cvbea6/znIcYHhdrrxUpJSRJkiTV2aLUAUiSJEmbEwtkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkjC1LHcDG6tq1a+rVqxfLli1j2223LXU4bYo5Lw3zXnzmvPjMefGZ89Iw78VXn/Pp06e/l1LaMZ97WlyB3KtXL6ZNm0ZVVRWVlZWlDqdNMeelYd6Lz5wXnzkvPnNeGua9+OpzHhGv53uPUywkSZKkDAtkSZIkKcMCWZIkScpocXOQG/PJJ5+wYMECVqxYUepQWrXtttuOOXPmlDqMNqepvHfo0IEePXrQvn37EkQlSVLr1SoK5AULFtCpUyd69epFRJQ6nFarpqaGTp06lTqMNqexvKeUWLx4MQsWLGCXXXYpUWSSJLVOrWKKxYoVK9hhhx0sjtVmRAQ77LCD/2oiSVIBtIoCGbA4Vpvjz7wkSYXRagpkSZIkqTlYIDeTt99+m+OOO45dd92VQYMGcfjhh/PSSy8V9D0rKyuZNm3aettcc801fPjhhw3Hhx9+OEuWLCloXBsjn89wyimnUF1d3Szv16tXL957771m6SurOWOUJEml1Soe0iu1lBLHHHMMJ598MnfddRcAzz33HO+88w69e/cuaWzXXHMNJ510Eh07dgTg4YcfLmk8m+LGG28sdQjrtWrVqs0+RkmSlL+CjSBHxM0R8W5EvNDE9YiIsRHxSkTMioi9CxVLoT3xxBO0b9+eH/zgBw3n+vfvzwEHHEBVVRVHHnlkw/mzzjqL8ePHA3WjmRdccAEDBgygoqKCGTNmcNhhh7Hrrrsybtw4gPXen3XGGWdQUVFB3759GTNmDABjx47lzTffZNiwYQwbNqzhPd977z3OP/98rr322ob7L774Yq688koArrjiCvbZZx/69evX0NfaHnvsMYYOHcree+/NN77xDWpra3n99dfZbbfdeO+991i9ejUHHHAAjz32GPPmzWOPPfbgxBNPpE+fPowaNWqNUe31fQZYc5S5rKyMX/7yl/Tv358hQ4bwzjvvALBo0SJGjhzJPvvswz777MPTTz8NwOLFizn00EPp27cvp5xyCimldd533Lhx/OxnP2s4Hj9+PGeddRYARx99NIMGDaJv377ccMMNDW3Kysr46U9/Sv/+/Zk8efIaMTb1OXr16sWYMWPYe++92WuvvZg7dy4AtbW1fPe732WvvfaiX79+3HfffWvk+IADDmjIsSRJKrxCjiCPB/4A3NbE9a8Cu+W+9gWuz33/VC55aDbVby79tN2sofxznRnztb5NXn/hhRcYNGjQJvW98847M3PmTEaPHs13vvMdnn76aVasWMGee+65RsG9IZdddhnbb789q1at4uCDD2bWrFn8+Mc/5uqrr+aJJ56ga9eua7Q/9thjOeecczjzzDMBuOeee3j00Ud57LHHePnll/n73/9OSokRI0bw1FNPceCBBzbc+95773HppZcyadIktt12Wy6//HKuvvpqLrroIs477zzOOOMMBg8eTHl5OYceeijz5s3jxRdf5KabbmK//fbje9/7Htdddx3nnnvuBj9Dv3791mizbNkyhgwZwmWXXcbPf/5z/vjHP3LhhRdy9tlnM3r0aPbff3/mz5/PYYcdxpw5c7jkkkvYf//9ueiii/iv//ovbrrppnVyN3LkSIYOHcoVV1wBwN13380vf/lLAG6++Wa23357li9fzj777MPIkSPZYYcdWLZsGfvuuy9XXXVVXn8W9Z+ja9euzJgxg+uuu44rr7ySG2+8kd/85jdst912PP/88wC8//77a+R49erVXHfddQ05liRJhVWwAjml9FRE9FpPk6OA21LdkN6UiPhMRHRPKb1VqJg2RyNGjABgr732ora2lk6dOtGpUye23nrrjZorfM8993DDDTewcuVK3nrrLaqrq9cpLrMGDhzIu+++y5tvvsmiRYvo0qULPXv25He/+x2PPfYYAwcOBOpGN19++eU1CuQpU6ZQXV3NfvvtB8DHH3/M0KFDgbq5uPfeey/jxo1j5syZDff07Nmzof1JJ53E2LFj1ymQ8/kMW221VcOI+qBBg3j88ccBmDRp0hpzgJcuXUptbS1PPfUU999/PwBHHHEEXbp0WScXO+64I1/4wheYMmUKu+22G3Pnzm2IdezYsTzwwAMAvPHGG7z88svssMMOtGvXjpEjR270n8XXv/71htjr45o0aVLD1ByALl26MHHixIYcr169mpUrVzbkWJKktd3xzHwenLmwYP1vaLCwtSnlHOSdgDcyxwty59YpkCPiNOA0gG7dulFVVUVtbS1VVVVA3U5jNTU1APykcueCBFvff2N22WUX7r777kbbfPzxx3z88ccN12pqalixYgU1NTWklPjkk0+oqanh448/JiIa2kUES5YsWe/9q1atYtmyZTz//PP8+7//O1VVVXTp0oUf/OAHLFmypOE9amtr2XrrrQHWOB4xYgS333477777LkcddRQ1NTV89NFHjB49mu9973vrfP5Vq1ZRU1PDhx9+SGVlJbfccss6bT788EPmz5/P6tWrefvttwEapgbUf4YPP/ywoa98PkN9m5qaGtq3b9/Q38cff8zy5csb2jz++ON06NChIZ6UEqtXr6a2trbhvdfOR72jjz6a22+/nd69e3PEEUdQW1vLf//3fzeMqnfs2JHDDz+cf/7zn9TU1NChQ4c1ponk+2dR/+e9YsUKPvroI2pqataJsT5H9TletWoV7dq1a/TncMWKFQ3/Haj5ZH+/qDjMefGZ89IoVN5vfWY582tWs3OnwsyeXbB6KVVViwrSd6FtSs5bxEN6KaUbgBsAKioqUmVlJVVVVVRWVgIwZ86cku7wduSRR3LppZdy5513ctpppwEwa9YsPvjgA/r06cNLL73EVlttxfLly3nqqacYNmwYnTp1IiIoKyujU6dOdOjQga222qrhc9RfW9/97dq1Y9ttt2X16tV06tSJHj16sGjRIiZNmsQhhxxCp06d6Ny5Mymldfrt1KkT3/72tzn11FN57733ePLJJ+nUqRNf+9rX+NWvfsX3v/99ysrKWLhwIe3bt+ezn/1sw45uw4YN49xzz+Wdd97hi1/8IsuWLWPhwoX07t2bX/ziF3zrW9/i85//PKNHj2bixImUlZXxxhtv8MILLzB06FD+8z//k8rKyrw/Q32b+s9Q/32bbbahffv2dOrUicMOO4zx48c3zCWeOXMmAwYMoLKykgkTJnDhhRfyyCOPsGTJkobPn3X88cdTUVHB7Nmzufzyy+nUqROffPIJXbt2pVu3bsydO5epU6fSsWPHdeIA8voc2dxvu+22tGvXriH2W2+9lWuuuQaom2KRzXG3bt3YYostGnKc1aFDh4bRfjWf7O8XFYc5Lz5zvq5Cj8ICLFmynM98ZusNN9xIby7/iH49P8Pdp/uvjWvblJ/1Ui7zthDomTnukTvX4kQEDzzwAJMmTWLXXXelb9++XHDBBfzLv/wLPXv25Jvf/CZ77rkn3/zmNze6mMnn/v79+zNw4ED22GMPTjjhhIbpAQCnnXYaw4cPb3hIL6tv377U1NSw00470b17dwAOPfRQTjjhBIYOHcpee+3FqFGj1hm13HHHHRk/fjzHH388/fr1Y+jQocydO5cnn3ySqVOnct5553HiiSey1VZbNYwy77777lx77bX06dOH999/nzPOOCPvz5CPsWPHMm3aNPr160d5eXnDQ45jxozhqaeeom/fvtx///3svHPj/8LQpUsX+vTpw+uvv87gwYMBGD58OCtXrqRPnz6cf/75DBkyZINxbMrnuPDCC3n//ffZc8896d+/P0888cQaOR46dGhDjiVJhfPgzIVUv9W8zzEVS3n3zhw1YKdSh9FqRGNP9Tdb53VzkCemlPZs5NoRwFnA4dQ9nDc2pTR4Q31WVFSkadOmrTOC3KdPn2aMXI2pH0HeWPPmzePII4/khRcaXdBEG7C+vPuzXxiOrBWfOS8+c76uY//vZICCjsKa9+Krz3lETE8pVeRzT8GmWETEnUAl0DUiFgBjgPYAKaVxwMPUFcevAB8C3y1ULJIkSVK+CrmKxfEbuJ6AMwv1/tp89OrVy9FjSWpDijGXtxCq31pKeffOpQ5DmwG3mpYkSc2qpc7ldR6v6rWIVSwkSVLLUt69sysqqMVyBFmSJEnKsECWJEmSMlrlFIvfPv5Ss/Y3+pDeG2xTVlbWsMNbPqqqqrjyyiuZOHEiEyZMoLq6mvPPP7/J9hdddBEHHnggX/nKV5rsZ1P06tWLadOm0bVr1026f0MqKyu58sorqahoelWVU045hZ/85CeUl5d/6vcr1OdpzhglKR+FftBtyZLlXP/i5IL07cNuaulaZYHc0owYMYIRI0ast82vf/3rIkVTfDfeeGOpQ1ivVatWbfYxSmp96h90a4mFpg+7qaWzQG5mVVVVXHzxxXTt2pUXXniBQYMGcfvttxMR/OUvf+Gcc86hY8eO7L///g33jB8/nmnTpnHZZZfRr18/XnvtNbbYYguWLVvGHnvswauvvsqpp57KkUceyahRo5rs5+KLL6asrIxzzz0XgD333JOJEyfSq1cvjj76aN544w1WrFjB2Wef3bAldlMee+wxxowZw0cffcSuu+7KLbfcwvz58zn66KOZPHky22+/PV/+8pf51a9+Re/evRk+fDiDBg1ixowZ9O3bl9tuu42OHTuu0ecZZ5zB1KlTWb58OaNGjeKSSy4B1hxlLisr4+yzz2bixIlss802PPjgg3Tr1o1Fixbxgx/8gPnz5wNwzTXXsN9++7F48WKOP/54Fi5cyNChQ2ls45tx48bxj3/8gyuuuGKNfP/hD39oMi9lZWWcfvrpTJo0iWuvvZYLL7ywIcamPkevXr04+eSTeeihh/jkk0+499572WOPPaitreVHP/oR06ZNIyIYM2YMI0eObDTHZWVlG/XzJql1K+SDbnWbJ/gQndQY5yAXwLPPPss111xDdXU1r776Kk8//TQrVqzg1FNP5aGHHmL69Om8/fbb69y33XbbMWDAAJ588kkAJk6cyGGHHUb79u0b2uTTT2Nuvvlmpk+fzrRp0xg7diyLFy9usu17773HpZdeyqRJk5gxYwYVFRVcffXV7Lzzzpx33nmcccYZXHXVVZSXl3PooYcC8OKLL/LDH/6QOXPm0LlzZ6677rp1+r3sssuYNm0as2bN4sknn2TWrFnrtFm2bBlDhgzhueee48ADD+SPf/wjAGeffTajR49m6tSp3HfffZxyyikAXHLJJey///7Mnj2bY445pqGAzho5ciQPPPBAw/Hdd9/Ncccdt968LFu2jH333Zfnnntujf8J2dDn6Nq1KzNmzOCMM87gyiuvBOA3v/kN2223Hc8//zyzZs3ioIMOajLHkiSp9CyQC2Dw4MH06NGDLbbYggEDBjBv3jzmzp3LLrvswm677UZEcNJJJzV677HHHsvdd98NwF133cWxxx67xvV8+1nb2LFj6d+/P0OGDOGNN97g5ZdfbrLtlClTqK6uZr/99mPAgAHceuutvP7660DdXNylS5cybty4hgIQoGfPnuy3334AnHTSSfzP//zPOv3ec8897L333gwcOJDZs2dTXV29TputttqKI488EoBBgwYxb948ACZNmsRZZ53FgAEDGDFiBEuXLqW2tpannnqqIQdHHHEEXbp0WafPHXfckS984QtMmTKFxYsXM3fu3IZYm8pLu3btGDlyZKP5Wd/n+PrXv95o7Gee+f/3xOnSpct6cyxJkkrLKRYFsPXWWze8bteuHStXrsz73hEjRvCLX/yCf/7zn0yfPp2DDjoo73u33HJLVq9e3XC8YsUKoO6f0SZNmsTkyZPp2LEjlZWVDdcak1LikEMO4c4771zjfE1NDR9++CELFiwAoLa2lk6dOgEQEWu0Xfv4tdde48orr2Tq1Kl06dKF73znO43G0L59+4Z7s7lbvXo1U6ZMoUOHDnnlYm3HHXcc99xzD3vssQfHHHMMEbHevHTo0IF27dqt08+GPkf9n/2G/tybyrEkSSo9C+Qi2WOPPZg3bx7/+Mc/2HXXXZssjMrKythnn304++yzOfLII9cp0tbXT69evRpWs5gxYwavvfYaAB988AFdunShY8eOzJ07lylTpqw31iFDhnDmmWfyyiuv8MUvfpFly5axcOFCunfvznnnnceJJ57I5z//eU499dSG95s/fz6TJ09m6NCh3HHHHetMS1i6dCnbbrst2223He+88w6PPPIIlZWVeefv0EMP5fe//z0/+9nPAJg5cyYDBgzgwAMP5I477uDCCy/kkUce4f3332/0/mOOOYbLLruMZ599lssvv3yT8rKpn+OQQw7h2muv5ZprrgHg/fffbzLHvXtveMUUSflrqVsegytBSKXUKgvkfJZlK7YOHTpwww03cMQRR9CxY0cOOOAAampqGm177LHH8o1vfIOqqqqN6mfkyJHcdttt9O3bl3333beh2Bo+fDjjxo2jT58+7L777gwZMmS9se64446MHz+e448/no8++giASy+9lH/84x9MnTqVp59+mnbt2nHfffdxyy23MGzYMHbffXeuvfZavve971FeXs4ZZ5yxRp/9+/dn4MCB7LHHHmtMx8jX2LFjOfPMM+nXrx8rV67kwAMPZNy4cYwZM4bjjz+evn378qUvfYmdd9650fu7dOlCnz59qK6uZvDgwZuUl039HBdeeCFnnnkme+65J+3atWPMmDF8/etfbzTHFshS83IlCEmbIhp76n9zVlFRkaZNm5Z7+rYSgDlz5tCnT5/SBtYG1NTUNEypyJo3bx5HHnkkL7zwQgmiav2ayjv4s18o2d8vKo5C5fzY/1u3zq9bHq/Ln/PSMO/FV5/ziJieUmp6Y4YMH9KTJEmSMiyQ9an16tXL0WNJktRqtJoCuaVNFZE+LX/mJUkqjFbxkF6HDh1YvHgxO+ywwzrLi0mtUUqJxYsXb/Kyd9LGKvRqEEuWLOf6Fyc3e78t9QE9SaXVKgrkHj16sGDBAhYtWlTqUFq1FStWWJCVQFN579ChAz169ChBRGqLWupqEK4EIWlTtIoCuX379uyyyy6lDqPVq6qqYuDAgaUOo80x79pclHfvXLDVIOqeMnelCUmbh1YzB1mSJElqDhbIkiRJUoYFsiRJkpRhgSxJkiRlWCBLkiRJGa1iFQtJagkKvZZwIbXEJd4kaVM5gixJRVK/lnBL5HrCktoSR5AlqYgKuZawJKl5OIIsSZIkZVggS5IkSRkWyJIkSVKGBbIkSZKUYYEsSZIkZVggS5IkSRkWyJIkSVKGBbIkSZKUYYEsSZIkZVggS5IkSRluNS2pxbnjmfk8OHNhwfpfsmQ51784udn7rX5rKeXdOzd7v5Kk5uUIsqQW58GZC6l+a2mpw9ho5d07c9SAnUodhiRpAxxBltQilXfvzN2nDy1I31VVVVRWFqZvSdLmzxFkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkDAtkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkDAtkSZIkKaOgBXJEDI+IFyPilYg4v5HrO0fEExHxbETMiojDCxmPJEmStCEFK5Ajoh1wLfBVoBw4PiLK12p2IXBPSmkgcBxwXaHikSRJkvKxZQH7Hgy8klJ6FSAi7gKOAqozbRLQOfd6O+DNAsYjKeOOZ+bz4MyFpQ5jk1S/tZTy7p033FCSpE1QyCkWOwFvZI4X5M5lXQycFBELgIeBHxUwHkkZD85cSPVbS0sdxiYp796Zowas/etEkqTmESmlwnQcMQoYnlI6JXf8LWDflNJZmTY/ycVwVUQMBW4C9kwprV6rr9OA0wC6des26K677qK2tpaysrKCxK7GmfPSKFTe//WZ5QBcsO82zd53S+fPevGZ8+Iz56Vh3ouvPufDhg2bnlKqyOeeQk6xWAj0zBz3yJ3L+j4wHCClNDkiOgBdgXezjVJKNwA3AFRUVKTKykqqqqqorKwsUOhqjDkvjULl/foXJwNQWTm02ftu6fxZLz5zXnzmvDTMe/FtSs4LOcViKrBbROwSEVtR9xDehLXazAcOBoiIPkAHYFEBY5IkSZLWq2AFckppJXAW8Cgwh7rVKmZHxK8jYkSu2U+BUyPiOeBO4DupUHM+JEmSpDwUcooFKaWHqXv4LnvuoszramC/QsYgSZIkbQx30pMkSZIyLJAlSZKkDAtkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkDAtkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkDAtkSZIkKcMCWZIkScrYstQBSC3ZHc/M58GZCwv6HkuWLOf6Fyc3e7/Vby2lvHvnZu9XkqSWzhFk6VN4cOZCqt9aWuowNkl5984cNWCnUochSdJmxxFk6VMq796Zu08fWrD+q6qqqKwsXP+SJGlNjiBLkiRJGRbIkiRJUsYGC+SI6BERD0TEooh4NyLui4gexQhOkiRJKrZ8RpBvASYA3YHPAQ/lzkmSJEmtTj4F8o4ppVtSSitzX+OBHQsclyRJklQS+RTIiyPipIhol/s6CVhc6MAkSZKkUsinQP4e8E3gbeAtYBTw3UIGJUmSJJXKBtdBTim9DowoQiySJElSyTVZIEfEz1NK/x4RvwfS2tdTSj8uaGSSJElSCaxvBHlO7vu0YgQiSZIkbQ6aLJBTSg/lXn6YUro3ey0ivlHQqCRJkqQSyechvQvyPCdJkiS1eOubg/xV4HBgp4gYm7nUGVhZ6MAkSZKkUljfHOQ3qZt/PAKYnjlfA4wuZFCSJElSqaxvDvJzwHMRcUdK6ZMixiRJkiSVzAbXQQZ6RcS/AuVAh/qTKaUvFCwqSZIkqUTyeUjvFuB66uYdDwNuA24vZFCSJElSqeRTIG+TUvorECml11NKFwNHFDYsSZIkqTTymWLxUURsAbwcEWcBC4GywoaltuaOZ+bz4MyFpQ5jo1W/tZTy7p1LHYYkSWpG+Ywgnw10BH4MDAJOAk4uZFBqex6cuZDqt5aWOoyNVt69M0cN2KnUYUiSpGa03hHkiGgHHJtSOheoBb5blKjUJpV378zdpw8tdRiSJKmNW+8IckppFbB/kWKRJEmSSi6fOcjPRsQE4F5gWf3JlNL9BYtKkiRJKpF8CuQOwGLgoMy5BFggS5IkqdXZYIGcUnLesSRJktqMfFaxkCRJktoMC2RJkiQpwwJZkiRJythggRwR3SLipoh4JHdcHhHfL3xokiRJUvHlM4I8HngU+Fzu+CXgnEIFJEmSJJVSPgVy15TSPcBqgJTSSmBVQaOSJEmSSiSfAnlZROxA3drHRMQQ4IOCRiVJkiSVSD4bhfwEmADsGhFPAzsCowoalSRJklQi+WwUMiMivgzsDgTwYkrpk4JHJkmSJJVAPqtYnAmUpZRmp5ReAMoi4oeFD02SJEkqvnzmIJ+aUlpSf5BSeh84NZ/OI2J4RLwYEa9ExPlNtPlmRFRHxOyIuCO/sCVJkqTCyGcOcruIiJRS/UN67YCtNnRTrt21wCHAAmBqRExIKVVn2uwGXADsl1J6PyI+uykfQpIkSWou+Ywg/wW4OyIOjoiDgTtz5zZkMPBKSunVlNLHwF3AUWu1ORW4NjcqTUrp3fxDlyRJkppf5AaGm24QsQVwOnBw7tTjwI0ppfWuhRwRo4DhKaVTcsffAvZNKZ2VafOf1G08sh/QDrg4pbRO8R0RpwGnAXTr1m3QXXfdRW1tLWVlZfl9SjWLQub8X59ZDsAF+25TkP5bMn/Wi8+cF585Lz5zXhrmvfjqcz5s2LDpKaWKfO7JZxWL1cD1ua/mtiWwG1AJ9ACeioi9snOeczHcANwAUFFRkSorK6mqqqKysrIAIakphcz59S9OBqCycmhB+m/J/FkvPnNefOa8+Mx5aZj34tuUnOezisV+EfF4RLwUEa9GxGsR8WoefS8EemaOe+TOZS0AJqSUPkkpvUbdaPJu+QYvSZIkNbd8HtK7CRgNTGfjtpieCuwWEbtQVxgfB5ywVpv/BI4HbomIrkBvIJ/iW5IkSSqIfArkD1JKj2xsxymllRFxFvAodfOLb04pzY6IXwPTUkoTctcOjYhq6orvn6WUFm/se0mSJEnNJZ8C+YmIuAK4H/io/mRKacaGbkwpPQw8vNa5izKvE3VbWf8k34AlSZKkQsqnQN439z371F8CDmr+cCRJkqTSymcVi2HFCESSJEnaHOQzgkxEHAH0BTrUn0sp/bpQQUmSJEmlks8yb+OAY4EfAQF8A/h8geOSJEmSSiKfraa/lFL6NvB+SukSYCh1y7FJkiRJrU4+BfLy3PcPI+JzwCdA98KFJEmSJJVOPnOQJ0bEZ4ArgBnUrWBxY0GjkiRJkkokn1UsfpN7eV9ETAQ6pJQ+KGxY2hR3PDOfB2euvZt381myZDnXvzi5IH1Xv7WU8u6dC9K3JEnSxmiyQI6Ig1JKf4uIrzdyjZTS/YUNTRvrwZkLW2yhWd69M0cN2KnUYUiSJK13BPnLwN+ArzVyLVG3s542M+XdO3P36UML0ndVVRWVlYXpW5IkaXPRZIGcUhoTEVsAj6SU7iliTJIkSVLJrHcVi5TSauDnRYpFkiRJKrl8lnmbFBHnRkTPiNi+/qvgkUmSJEklkM8yb8fmvp+ZOZeALzR/OJIkSVJp5bPM2y7FCESSJEnaHOQzgkxE7AmUAx3qz6WUbitUUJIkSVKpbLBAjogxQCV1BfLDwFeB/wEskCVJktTq5POQ3ijgYODtlNJ3gf7AdgWNSpIkSSqRfArk5bnl3lZGRGfgXdz/snUAABO2SURBVKBnYcOSJEmSSiOfOcjTIuIzwB+B6UAtMLmgUUmSJEklks8qFj/MvRwXEX8BOqeUZhU2LEmSJKk0NjjFIiImRMQJEbFtSmmexbEkSZJas3zmIF8F7A9UR8SfI2JURHTY0E2SJElSS5TPFIsngScjoh1wEHAqcDPQucCxSZIkSUWX70Yh2wBfo27b6b2BWwsZlCRJklQq+WwUcg8wGPgL8Afgydyyb5IkSVKrk88I8k3A8SmlVYUORpIkSSq1fOYgP1qMQCRJkqTNQT6rWEiSJElthgWyJEmSlNHkFIuI2Ht9N6aUZjR/OJIkSVJprW8O8lW57x2ACuA5IIB+wDRgaGFDkyRJkoqvySkWKaVhKaVhwFvA3imlipTSIGAgsLBYAUqSJEnFlM8c5N1TSs/XH6SUXgD6FC4kSZIkqXTyWQd5VkTcCNyeOz4RmFW4kCRJkqTSyadA/i5wBnB27vgp4PqCRSRJkiSVUD4bhayIiHHAwymlF4sQkyRJklQyG5yDHBEjgJnAX3LHAyJiQqEDkyRJkkohn4f0xgCDgSUAKaWZwC6FDEqSJEkqlXwK5E9SSh+sdS4VIhhJkiSp1PJ5SG92RJwAtIuI3YAfA/9b2LAkSZKk0shnBPlHQF/gI+BOYClwTiGDkiRJkkoln1UsPgR+mfuSJEmSWrUNFsgR0Rs4F+iVbZ9SOqhwYUmSJEmlkc8c5HuBccCNwKrChiNJkiSVVj4F8sqUkjvnSZIkqU3I5yG9hyLihxHRPSK2r/8qeGSSJElSCeQzgnxy7vvPMucS8IXmD0eSJEkqrXxWsXDXPEmSJLUZTRbIEXFQSulvEfH1xq6nlO7fUOcRMRz4HdAOuDGl9G9NtBsJ/BnYJ6U0La/IJUmSpAJY3wjyl4G/AV9r5FoC1lsgR0Q74FrgEGABMDUiJqSUqtdq1wk4G3hmI+KWJEmSCqLJAjmlNCb3/bub2Pdg4JWU0qsAEXEXcBRQvVa73wCXs+YcZ0mSJKkk8nlIj4g4grrtpjvUn0sp/XoDt+0EvJE5XgDsu1a/ewM9U0r/FREWyJIkSSq5fHbSGwd0BIZRt1nIKODvn/aNI2IL4GrgO3m0PQ04DaBbt25UVVVRW1tLVVXVpw2jVVmyZDlAwfJizkvDvBefOS8+c1585rw0zHvxbUrO8xlB/lJKqV9EzEopXRIRVwGP5HHfQqBn5rhH7ly9TsCeQFVEAPwLMCEiRqz9oF5K6QbgBoCKiopUWVlJVVUVlZWVeYTRdlz/4mQAKiuHFqR/c14a5r34zHnxmfPiM+elYd6Lb1Nyns9GIctz3z+MiM8BnwDd87hvKrBbROwSEVsBxwET6i+mlD5IKXVNKfVKKfUCpgDrFMeSJElSMeVTIE+MiM8AVwAzgHnAnRu6KaW0EjgLeBSYA9yTUpodEb+OiBGbHrIkSZJUOPlsFPKb3Mv7ImIi0CGl9EE+naeUHgYeXuvcRU20rcynT0mSJKmQ1rdRSKMbhOSu5bVRiCRJktTSrG8EubENQuptcKMQSZIkqSVa30Yhm7pBiCRJktRibfAhvYjYISLGRsSMiJgeEb+LiB2KEZwkSZJUbPmsYnEXsAgYSd0mIYuAuwsZlCRJklQq+WwU0j2zkgXApRFxbKECau0ueWg21W8uLUjf1W8tpbx754L0LUmS1FbkUyA/FhHHAffkjkdRt7Zxm/Lbx19qln6enb+ERTUfNUtfPbpss8ZxeffOHDVgp2bpW5Ikqa3Kp0A+FTgH+I/ccTtgWUScDqSUkkOWG+HLvXdstr5GH9K72fqSJElSnXw2CulUjEAkSZKkzUE+q1h8f63jdhExpnAhSZIkSaWTzyoWB0fEwxHRPSL2BKYAjipLkiSpVcpnisUJuVUrngeWASeklJ4ueGSSJElSCeQzxWI34GzgPuB14FsR0bHQgUmSJEmlkM8Ui4eAX6WUTge+DLwMTC1oVJIkSVKJ5LPM2+CU0lKoW9MNuCoiHipsWJIkSVJpNDmCHBE/B0gpLY2Ib6x1+TuFDEqSJEkqlfVNsTgu8/qCta4NL0AskiRJUsmtr0COJl43dixJkiS1CusrkFMTrxs7liRJklqF9T2k1z8illI3WrxN7jW54w4Fj0ySJEkqgSYL5JRSu2IGIkmSJG0O8lkHWZIkSWozLJAlSZKkDAtkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkDAtkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkDAtkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkDAtkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkDAtkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkDAtkSZIkKcMCWZIkScqwQJYkSZIyLJAlSZKkDAtkSZIkKcMCWZIkScqwQJYkSZIyClogR8TwiHgxIl6JiPMbuf6TiKiOiFkR8deI+Hwh45EkSZI2pGAFckS0A64FvgqUA8dHRPlazZ4FKlJK/YA/A/9eqHgkSZKkfBRyBHkw8EpK6dWU0sfAXcBR2QYppSdSSh/mDqcAPQoYjyRJkrRBkVIqTMcRo4DhKaVTcsffAvZNKZ3VRPs/AG+nlC5t5NppwGkA3bp1G3TXXXdRW1tLWVlZQWJvzLs1HxXtvfL12U5bF/X9ip1z1THvxWfOi8+cF585Lw3zXnz1OR82bNj0lFJFPvdsWeig8hERJwEVwJcbu55SugG4AaCioiJVVlZSVVVFZWVl0WL87eMvFe298vXNyt5Ffb9i51x1zHvxmfPiM+fFZ85Lw7wX36bkvJAF8kKgZ+a4R+7cGiLiK8AvgS+nlDa/YVpJkiS1KYUskKcCu0XELtQVxscBJ2QbRMRA4P9SNxXj3QLG0qY11+j3Tis+ara+Rh9S3NFvSZKkfBXsIb2U0krgLOBRYA5wT0ppdkT8OiJG5JpdAZQB90bEzIiYUKh4JEmSpHwUdA5ySulh4OG1zl2Uef2VQr6/JEmStLHcSU+SJEnKsECWJEmSMiyQJUmSpAwLZEmSJCnDAlmSJEnKsECWJEmSMiyQJUmSpAwLZEmSJCnDAlmSJEnKsECWJEmSMiyQJUmSpAwLZEmSJCnDAlmSJEnKsECWJEmSMiyQJUmSpAwLZEmSJCnDAlmSJEnKsECWJEmSMiyQJUmSpAwLZEmSJCljy1IHIDXlt4+/VOoQGjX6kN6lDkGSJBWQI8iSJElShgWyJEmSlGGBLEmSJGVYIEuSJEkZFsiSJElShgWyJEmSlGGBLEmSJGVYIEuSJEkZFsiSJElShgWyJEmSlGGBLEmSJGVsWeoApNbot4+/1Gx97bTio2bpb/QhvZshGkmSWj9HkCVJkqQMR5AlraE5R7+bi6PfkqRicgRZkiRJyrBAliRJkjIskCVJkqQM5yBLahVcOUSS1FwcQZYkSZIyHEGWpBJz5RBJ2rw4gixJkiRlWCBLkiRJGRbIkiRJUoYFsiRJkpThQ3qSpE2yOS6tBz5gKOnTs0CWJLU5rhwiaX0skCVJakGaq7h31F5qmgWyJEkquM1x1B4s7tU4C2RJkqT12Bzn21vYF1ZBC+SIGA78DmgH3JhS+re1rm8N3AYMAhYDx6aU5hUyJkmSpLZicxy5bwnFfcGWeYuIdsC1wFeBcuD4iChfq9n3gfdTSl8EfgtcXqh4JEmSpHwUch3kwcArKaVXU0ofA3cBR63V5ijg1tzrPwMHR0QUMCZJkiRpvSKlVJiOI0YBw1NKp+SOvwXsm1I6K9PmhVybBbnjf+TavLdWX6cBp+UOdwdeBLoCa7RTwZnz0jDvxWfOi8+cF585Lw3zXnz1Of98SmnHfG5oEQ/ppZRuAG7InouIaSmlihKF1CaZ89Iw78VnzovPnBefOS8N8158m5LzQk6xWAj0zBz3yJ1rtE1EbAlsR93DepIkSVJJFLJAngrsFhG7RMRWwHHAhLXaTABOzr0eBfwtFWrOhyRJkpSHgk2xSCmtjIizgEepW+bt5pTS7Ij4NTAtpTQBuAn4j4h4BfgndUV0vm7YcBM1M3NeGua9+Mx58Znz4jPnpWHei2+jc16wh/QkSZKklqiQUywkSZKkFscCWZIkScpocQVyRAyPiBcj4pWIOL/U8bQFEdEzIp6IiOqImB0RZ5c6prYiItpFxLMRMbHUsbQFEfGZiPhzRMyNiDkRMbTUMbUFETE697vlhYi4MyI6lDqm1iYibo6Id3P7D9Sf2z4iHo+Il3Pfu5QyxtamiZxfkfv9MisiHoiIz5Qyxtaosbxnrv00IlJEdN1QPy2qQM5z+2o1v5XAT1NK5cAQ4EzzXjRnA3NKHUQb8jvgLymlPYD+mPuCi4idgB8DFSmlPal7qHtjHthWfsYDw9c6dz7w15TSbsBfc8dqPuNZN+ePA3umlPoBLwEXFDuoNmA86+adiOgJHArMz6eTFlUgk9/21WpmKaW3Ukozcq9rqCsadiptVK1fRPQAjgBuLHUsbUFEbAccSN3qOqSUPk4pLSltVG3GlsA2ufXwOwJvljieViel9BR1q0VlHQXcmnt9K3B0UYNq5RrLeUrpsZTSytzhFOr2iFAzauJnHeC3wM+BvFanaGkF8k7AG5njBVioFVVE9AIGAs+UNpI24Rrq/mNeXepA2ohdgEXALblpLTdGxLalDqq1SyktBK6kblTnLeCDlNJjpY2qzeiWUnor9/ptoFspg2mDvgc8Uuog2oKIOApYmFJ6Lt97WlqBrBKKiDLgPuCclNLSUsfTmkXEkcC7KaXppY6lDdkS2Bu4PqU0EFiG/+RccLl5r0dR9z8onwO2jYiTShtV25PbpMt1X4skIn5J3fTFP5U6ltYuIjoCvwAu2pj7WlqBnM/21SqAiGhPXXH8p5TS/aWOpw3YDxgREfOom0p0UETcXtqQWr0FwIKUUv2/jvyZuoJZhfUV4LWU0qKU0ifA/cCXShxTW/FORHQHyH1/t8TxtAkR8R3gSOBEdw8uil2p+x/w53J/p/YAZkTEv6zvppZWIOezfbWaWUQEdfMy56SUri51PG1BSumClFKPlFIv6n7O/5ZSclStgFJKbwNvRMTuuVMHA9UlDKmtmA8MiYiOud81B+PDkcUyATg59/pk4MESxtImRMRw6qbOjUgpfVjqeNqClNLzKaXPppR65f5OXQDsnfud36QWVSDnJrbXb189B7gnpTS7tFG1CfsB36JuFHNm7uvwUgclFcCPgD9FxCxgAPB/ShxPq5cbsf8zMAN4nrq/l9yKt5lFxJ3AZGD3iFgQEd8H/g04JCJepm4k/99KGWNr00TO/wB0Ah7P/V06rqRBtkJN5H3j+3F0X5IkSfr/WtQIsiRJklRoFsiSJElShgWyJEmSlGGBLEmSJGVYIEuSJEkZFsiS2ryIWJVbcumFiLg3t/NSY+3+dxP7r4iIsZ8ivtpNvbcliYhzmsq9JBWTy7xJavMiojalVJZ7/SdgenZTnIjYMrcOe8nja81yu1xVpJTeK3Uskto2R5AlaU3/DXwxIioj4r8jYgK5HfXqR3Jz16oi4s8RMTci/pTbBY6I2Cci/jcinouIv0dEp1z7ibnrF0fEf0TE5Ih4OSJOzZ0vi4i/RsSMiHg+Io7aUKAR8e2ImJV7r//InesVEX/Lnf9rROycOz8+Iq6PiCkR8WouppsjYk5EjM/0WRsRv42I2bn7d8ydH5C7d1ZEPBARXXLnqyLi8txnfSkiDsidbxcRV0TE1Nw9p68vdxHxY+BzwBMR8UQz/DlK0iazQJaknIjYEvgqdTu6AewNnJ1S6t1I84HAOUA58AVgv4jYCrg7d09/6nYnW97Ivf2Ag4ChwEUR8TlgBXBMSmlvYBhwVX3R3USsfYELgYNy73V27tLvgVtTSv2APwHZqR1dcu85mrpthn8L9AX2iogBuTbbAtNSSn2BJ4ExufO3Aefl+n0+cx5gy5TS4Fw+6s9/H/ggpbQPsA9wakTs0lTuUkpjgTeBYSmlYU19bkkqBgtkSYJtImImMA2YD9yUO//3lNJrTdzz95TSgpTSamAm0AvYHXgrpTQVIKW0tImpGQ+mlJbnphI8AQwGAvg/uW2uJwE7Ad3WE/NBwL310xFSSv/MnR8K3JF7/R/A/pl7Hkp18+qeB95JKT2fi392Ln6A1dQV+QC3A/tHxHbAZ1JKT+bO3wocmOn3/tz36Zl+DgW+ncvrM8AOwG65a43lTpI2G1uWOgBJ2gwsTykNyJ7IDd4uW889H2Ver2Ljfp+u/fBHAk4EdgQGpZQ+yc3H7bARfeajPubVrBn/apqOP58HVer7yuYhgB+llB7NNoyISj5d7iSp4BxBlqTm8yLQPSL2AcjNP26s+DsqIjpExA5AJTAV2A54N1ccDwM+v4H3+hvwjVwfRMT2ufP/CxyXe30idXOqN8YWwKjc6xOA/0kpfQC8Xz+/GPgWddMv1udR4IyIaJ+Lr3dEbLuBe2qAThsZryQ1O/+vXZKaSUrp44g4Fvh9RGxD3fzjrzTSdBZ1Uyu6Ar9JKb2ZWz3joYh4nrqpHnM38F6zI+Iy4MmIWAU8C3wH+BFwS0T8DFgEfHcjP8YyYHBEXAi8CxybO38yMC63DNurefR7I3VTJ2bk5lIvAo7ewD03AH+JiDedhyyplFzmTZKKKCIuBmpTSleWOpbGtJUl5SRpfZxiIUmSJGU4gixJkiRlOIIsSZIkZVggS5IkSRkWyJIkSVKGBbIkSZKUYYEsSZIkZfw/OiHIg0YrhKAAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 720x360 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qE93F-_wR1MX",
        "colab_type": "text"
      },
      "source": [
        "We can see that first two principal components account for roughly 60% of the explained variance"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MHOO5j41Oyd1",
        "colab_type": "text"
      },
      "source": [
        "### 6. Create projection matrix $W$ from top $k$ eigenvectors\n",
        "\n",
        "Create the projection matrix $W$ using top $k$ eigenvectors. \n",
        "\n",
        "> $W \\in \\mathbb{R}^{m \\mathbb{x} k}$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CcrymzqmSFi1",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "29786f99-baf3-439b-d53f-42b260bfa0dd"
      },
      "source": [
        "w = np.hstack((eigen_pairs[0][1][:, np.newaxis],eigen_pairs[1][1][:, np.newaxis]))\n",
        "print(f\"Transformation matrix shape:{w.shape}\")"
      ],
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Transformation matrix shape:(13, 2)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kCcDWaBhQPRP",
        "colab_type": "text"
      },
      "source": [
        "### 7. Transform the input dataset using $W$\n",
        "\n",
        "\\begin{align*}\n",
        "X^\\prime = XW\n",
        "\\end{align*}\n",
        "\n",
        "> $X \\in \\mathbb{R}^{n\\mathbb{x}m}$ $n$ samples and $m$ features \\\\\n",
        "> $W \\in \\mathbb{R}^{m\\mathbb{x}k}$ $m$ features and $k$ components \\\\\n",
        "> $X^\\prime \\in \\mathbb{R}^{n\\mathbb{x}k}$ $n$ samples and $k$ principal components "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kXhR7rmvSO8G",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 369
        },
        "outputId": "60816b4f-bcd8-4c78-a5f9-5a7054edeb72"
      },
      "source": [
        "X_pca = X_std.dot(w)\n",
        "plt.figure(figsize=figsize)\n",
        "colors =['tab:blue','tab:orange','tab:green']\n",
        "markers=list('svo')\n",
        "for l,c,m in zip(np.unique(y), colors, markers):\n",
        "    plt.scatter(X_pca[y==l, 0],\n",
        "                X_pca[y==l, 1],\n",
        "                color='white',\n",
        "                edgecolor=c,\n",
        "                label=l,\n",
        "                marker=m\n",
        "               )\n",
        "plt.xlabel('Principal Component 1')\n",
        "plt.ylabel('Principal Component 2')\n",
        "plt.grid();\n",
        "plt.legend()\n",
        "plt.tight_layout();"
      ],
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsgAAAFgCAYAAACmDI9oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZzcdZX/+/fJggmkCXdYOpkEDQElRLKRJjLEwQ4KwyQsE5eriAuDJteLY4I6LhF/ATK/MSqDE1B/eonMCI6S0Rm5YQkRVBoxDDYJIYRVoQc0IQTEHyENgWzn90dVJdWdqupvVX33ej0fj3506tvVVaeqv+k+dep8zsfcXQAAAAAKBiUdAAAAAJAmJMgAAABAGRJkAAAAoAwJMgAAAFCGBBkAAAAoMyTpAOpxxBFH+Lhx45IOIzSvvPKKDjnkkKTDQEZwvqAenC+oF+cM6pGX82XdunV/dPcj+x/PVII8btw4rV27NukwQtPV1aXOzs6kw0BGcL6gHpwvqBfnDOqRl/PFzJ6pdJwWCwAAAKAMCTIAAABQhgQZAAAAKJOpHmQAAACkx65du7Rp0ya99tprSYdS07BhwzR27FgNHTo00PVJkAEAANCQTZs2qa2tTePGjZOZJR1ORe6uF198UZs2bdIxxxwT6HtosQAAAEBDXnvtNR1++OGpTY4lycx0+OGH11XlJkEGAABAw9KcHJfUGyMJMgAAAFCGBBkAAACZtnr1ah1//PE67rjj9NWvfrXp2yNBBgC0nFU9qzR35VxNuWGK5q6cq1U9q5IOCUCD9uzZo09+8pO6/fbb9eijj+rGG2/Uo48+2tRtMsUCANBSVvWs0jXrr9GSU5doWvs0rd+6XovvXSxJmj1+dsLRAfk15Yo7tG3HrgOOjxw+VBsuO7Ph2+3u7tZxxx2n8ePHS5I+8IEPaOXKlZo4cWLDt0mCDABoKcs3LteSU5doxugZkqQZo2doyalLtLR7KQkyEKFtO3bp6a/OOeD4uC/e1tTtbt68WUcfffS+y2PHjtVvfvObpm6TFgsAQEvp2dajae3T+hyb1j5NPdt6EooIQNqQIAMAWsr4keO1fuv6PsfWb12v8SPHJxQRgGaMGTNGf/jDH/Zd3rRpk8aMGdPUbZIgAwBayrxJ87T43sXq3tKtXXt3qXtLtxbfu1jzJs1LOjQADTj55JP1u9/9Tv/93/+tnTt3asWKFTr33HObuk16kAEALaXUZ7y0e6l6tvVo/MjxWjBtAf3HQEYNGTJE3/rWt/RXf/VX2rNnjy666CK99a1vbe42Q4oNAIDMmD1+NgkxELORw4dWXJA3cvjQpm979uzZmj07vP/TJMgAAACIXDOj3OJGDzIAAABQhgQZAAAAKEOCDAAAAJQhQQYAAADKkCADAAAAZUiQAQAAkFkXXXSRjjrqKJ144omh3SYJMgAAADLrwgsv1OrVq0O9TeYgAwAAIHrLJksvPXPg8cPeJF3yUMM3e9ppp+npp59uPK4KSJABAAAQvQlzCp/PWrr/2OpFkiyRcGqhxQIAAADRm7lQevBH0vbnCpe3PydtuLFwPGVIkAEAABC9tlHS1A9Ka64uXF5ztTTlg1Jbe7JxVZBYgmxmw8ys28w2mNkjZnZFUrEAAAAgBqUq8pYNqa0eS8lWkF+XdLq7T5E0VdJZZnZKgvEAAAAgSqUq8vXnhlY9Pv/88/UXf/EXeuKJJzR27Fhdd911Td9mYov03N0l9RYvDi1+eFLxAAAAIAYzF0q/uzO06vGNN94Yyu2Us0KemgwzGyxpnaTjJH3b3b9Q4TrzJc2XpPb29ukrVqyIN8gI9fb2asSIEUmHgYzgfEE9OF9QL84Z1KN0vowcOVLHHXdc0uEE8uSTT2rbtm19js2aNWudu3f0v26iY97cfY+kqWZ2mKSbzOxEd3+433WulXStJHV0dHhnZ2f8gUakq6tLeXo8iBbnC+rB+YJ6cc6gHqXz5bHHHlNbW1vS4QQybNgwTZs2LdB1UzHFwt1fknSXpLOSjgUAAACtLckpFkcWK8cys+GSzpD0eFLxAAAAAFKyLRajJV1f7EMeJOnH7n5rgvEAAAAAiU6xeEhSsEYQAAAAICaJLtJDY6ZccYe27dh1wPGRw4dqw2VnJhARAADxW9WzSss3LlfPth6NHzle8ybN0+zxs4mpxfzhD3/QRz7yEW3dulVmpvnz52vhwuZGyJEgZ9C2Hbv09FfnHHB83BdvSyAaAADit6pnla5Zf42WnLpE09qnaf3W9Vp872JJSiwhTWNMrWDIkCG66qqrdNJJJ2n79u2aPn26zjjjDE2cOLHh20zFFAsAAIB6LN+4XEtOXaIZo2do6KChmjF6hpacukTLNy4nphRb1bNKc1fO1ZQbpmjuyrla1bOq6dscPXq0TjrpJElSW1ubTjjhBG3evLmp26SCDAAAMqdnW4+mtfddyjStfZp6tvUkFFE6Y0qTOCrsTz/9tNavX6+3ve1tTd0OFWQAAJA540eO1/qt6/scW791vcaPHJ9QROmMKU2irrD39vbqPe95j5YtW6ZDDz20qdsiQQYAAJkzb9I8Lb53sbq3dGvX3l3q3tKtxfcu1rxJ84gppaKssO/atUvvec97dMEFF+jd735307dHi0UGjRw+tOKCvJHDhyYQDQAAA9s33eGlHo04aIS279yuYw87tuEpD6XvWdq9dN/EiAXTFiS6GC6NMaVJqcI+Y/SMfcfCqLC7uz72sY/phBNO0Gc+85lmw5REgpxJjHIDAGRJtd7T048+Xdesv0ZSYz2os8fPTl3ymcaY0qJUYe9/HiyYtqCp212zZo1+8IMfaNKkSZo6daok6Stf+Ypmz27850CCDAAAIlXeeyppX+/p0u6l+z6TVOZfVBX2t7/97XL3MELchwS5xbDJCAAgbrV6T5ny0FqyUmEnQW4xbDICAIhbrd5TpjwgjZhiAQAAIlVtusPpR5/OlIccCLu9IQr1xkgFGQAARKpP72nZFItf/uGXTHnIuGHDhunFF1/U4YcfLjNLOpyK3F0vvviihg0bFvh7SJABAEiRfePQiouYGh2DljZZ6T1FfcaOHatNmzbphRdeSDqUmoYNG6axY8cGvj4JMgAAKRHHVrxAmIYOHapjjjkm6TBCR4LcYthkBADSq9Y4tKwlyHmthKM1kCC3GEa5JYsxewBqiXIr3jhRCUfWkSADMWLMHoBaotqKN27VKuGX/vpSqsrIBMa8AQCQEtXGoWVtDFqlSvjWV7dq195dWjRjkdZ+aK0WzVika9Zfo1U9qxKKEqiOCjIAACkR1Va8catUCf/Ohu/o66d9PRf91cg/EmQAAFIkD+PQSpXw8h7kTds35aK/Gq2BBBkAAISqUiX8zw/584r91aMPHq25K+e2bF8y0z7SiQQZiBFj9gC0iv6V8FU9qw6oKl+65lLt3L1TS2a25rQLpn2kFwkyECNGuQFoVZWqynLpyndcGXpfclaqsnmae503JMhIDDOBAaC19K8qT7lhSuh9yVmqyuZl7nUekSBXQfIWPWYCA0Bri2Luc5aqsnmZe51HJMhVkLwBAHCgMNsXKk27WHzvYi2YtqDh+LJUlY3i8SMcJMgAACCQsNsXas19bjQRr6cqm3Svcl7mXudRYgmymR0t6QZJ7ZJc0rXufnVS8QAAgNqiaF+oNPe5mUQ8aFU2Lb3KeZh7nUdJVpB3S/qsuz9gZm2S1pnZne7+aIIxAQCAKuJqX2gmEQ9alc1SrzLil1iC7O5bJG0p/nu7mT0maYwkEuQWwUxgAMiWuBaVNZuIB6nKZqlXGfFLRQ+ymY2TNE3Sb5KNZD+St+gxDQQAsiWuRWVxJOJMkEAt5u7JBmA2QtLdkv7R3X9a4evzJc2XpPb29ukrVqyIOcLo9Pb2asSIEUmHgYzgfEE9OF9Qr6DnzMs7X9YLO17Q67tf1xuGvEFHDj9Shx50aKixvLzzZW19davGHDJGw4cO145dO7T5lc1qP7g9tPt67pXn9NLrL2mv79UbBr9BbQe1advObaHeR57l5XfMrFmz1rl7R//jiSbIZjZU0q2Sfubu3xjo+h0dHb527droA4tJV1eXOjs7kw4DEQpznjbnC+rB+YJ6pe2ciXLCRKUFel+45ws6441n6EunfCmU+8i7tJ0vjTKziglyklMsTNJ1kh4LkhwDWcQ8bQBoTJTTHSot0PvaX35NS7uXRnJ/yJ5BCd73TEkflnS6mT1Y/GDZKAAAiBQL9DCQJKdY/FqSJXX/QH9sLw4ArSFPC/SS3uwkr1IxxQJIA9oh0oEXKgCilpctntOy2UkekSAjt0i0sokXKgAqCbNSmpctntnsJDokyIhUkklqGhIt5mkDQPOiqJTmYYtneqmjQ4KMSKUhSU0SlWoAaB6V0sry1EudNiTIQEho6QCAaFAprSwvvdRpRIIMFDXbDtHq1XIAiAqV0sry0kudRiTILYyKZ19peMy1fiZXv+OgPseO+9Iq7d574E6YQwaZnvxKdn850rcNoD8qpdXloZc6jUiQW1jeK55ZTLRq/0z6Jsi793ouf35peKECIF2olCJuJMiIVJJJKokWAORHo5VSNtJAI0iQESmSVABAUuoZD0cijXIkyEBIstjSASA/SPAOFHQ8HDvSoT8SZCAkVMsBJIUEr7Kg4+GYs4z+SJBbWKtUPLM0raOen8mQQVbxukMGWSSxAUgvErzKgo6HY84y+iNBbmGNJIdpTzbriS+N0x5qPYddXV19Lmd5lBuAcJHgVRZ0PBxzltEfCTLqkvbRcGmPDwCiQIJXWdDxcMxZRn8kyAAAZBwJXnVBxsMxZxn9kSADAJBxJHjNY0c6lCNBBqpIe781AJQjwQPCQ4Lc4lopCezfhzzQtA76mQEAcWCGdfqQILe4epPAtI+GqxVf3hJ+AEA4kkxQm5lhTWIdHRJk1CXtSWYY8VWrqiehPJbPTtqtC4vJPwk/AIQj6U1WGp1hnXTceUeCDPRTqqpH0UpRb0tLeYW/q6tLT1/QKYk2DwAIS9KbrDQ6wzrpuPOOBBm5k+a+avqaASBdkt5kpdEZ1knHnXdVE2QzmyRpuaQxkm6X9AV3/9/Fr3W7+4xq3wskKawktFo/8+DiFs9pSLgBICml/tenXnpKbQe1qXdnr8Yflr0+2KQ3WWl0hnXSceddrQrydyRdLuk+SR+X9GszO9fdn5KUjhVZaFoSi+7SXOEtV2lr6vLEm6ovgFZV6n+dc8wc7di9I9N9sElvstLoDOuk4867Wglym7uvLv77n8xsnaTVZvZhSR59aIhDKQnsn7Ru27ErsiopbQYAkG2l/tel3Usz3webhk1WGplhnYa486xmD7KZjXT3bZLk7neZ2Xsk/aekP4sjOMSHpHW/NI2yK4+l/xQLAEhKqf81L32wWd1kJatxZ0GtBPlrkk5QocVCkuTuD5nZOyX9j6gDA5IyUGtFM6ol30OKfc2Vrl9pigUAJKnU/0ofLPKqaoLs7j+qcvz3kuZFFhHQpGYqwFHPQK7WrlItCW/FCj6A9Cv1v845Zg59sMilRMe8mdm/SDpb0vPufmKSsSA/mumZrtRqMuWKO0JvuaiUiDMZA0BWlN7WX75xuZ7tfVaf7vr0vikW9MEiD5Keg/x9Sd+SdEPCcSBGaerxDWLDZWeG2mYh0fMNIPvof0WeDZggm9lMd18z0LFGuPuvzGxcs7eD5sWZtFIhBQAAaRakgvxNSScFOIYMI2lF1mRlnjYAIHvMvfJIYzP7C0mnSrpE0j+XfelQSXPdfUooARQqyLdW60E2s/mS5ktSe3v79BUrVoRxt6nQ29urESNGJB0Gyjy8+WV5hTHfJtOJYw4N7X42bt6mSWNGVjxebvAg08TRhfvlfOmr1nNY6Xir4XxBvThnUI+8nC+zZs1a5+4d/Y/XqiAfJGlE8TptZcdflvTecMOrzt2vlXStJHV0dHhnZ2dcdx25rq4u5enx5MGFNaZJhDli7cIqt1ft/iXOl3KF6vGQ6s8h4/A4X1A3zpn0K23vXdoYJMltvfN+vtQa83a3pLvN7Pvu/kyMMQG5l7WFimkT5Sg+AEij0vbeQbb1TlMinVVBepDfYGbXShpXfn13P73ZOzezGyV1SjrCzDZJuszdr2v2doG0o0cWAFCP0vbeA23rXU8ijeqCJMg/kfRdSd+TtCfMO3f388O8PbQ2Fm0BAPIq6LbeQRNp1BYkQd7t7t+JPBLkVlyJa72zhavFNeWKO0ioM4A2FQCtJOi23kETadQWJEG+xcwulnSTpNdLB939T5FFhVwJc1OMWsl2mHH1j42kK30qvYgZ98XbeHEDIFZx9fuWtvceaFvvoIk0aguSIH+0+PlzZcdcEs80YhfXDnRh7pqH8FE9BpAGcfb7lm5vaffSfcl4pW29gybSqG3ABNndj4kjECDvkuiRjus+435sVImB1pLWqQxx9/sG2d47aCKN2oJsNX2wpM9IeqO7zzezN0s63t1vjTw6IEfqbekIIwmMq+K+bccujRw+9IAkeduOXfR0A2hKmqcypLXfN0giHURaX5jEIUiLxb9KWqfCrnqStFmFyRYkyEidcV+8TYMHmfbs3b8bXhbedu+fxIadwMYhrmQcQGtJ81SGPPf7xvHCJM0JeJAE+Vh3f7+ZnS9J7v6qmVnEcSFH4uoXLSVn42rsRpdEXGlFZRdAFqS1Sivlu9836hcmaX5nQAqWIO80s+EqLMyTmR2rsmkWwEDCTMLCTGpbPTlkNzoAWZDmKm2e+32jfmGS5ncGpGAJ8mWSVks62sx+KGmmpAujDAqopjypDVopBgBkV6Uq7ed/9XkNHTRUq3pW1ZVMRfGWflj9vmkT9QuTNL8zIAWbYnGnmT0g6RRJJmmhu/8x8siAnEmipaPWfYZZQQ779gCgpJR8Ll6zWM++8qzGto3VZzs+q/aD2+t6Sz7tb+mnTdTtI2l+Z0AKVkGWpGGS/nfx+hPNTO7+q+jCAvKnUktHaTxaVBuTbLjszKqV9jAXz2247ExNueKOlu7pBhCd2eNna/nG5fremd/rk1DV85Z82t/ST5uo20fS3r8dZMzb1yS9X9IjkvYWD7skEmQkKg+L7PLUB52nxwIgfM22NzT7lnw935/m6QpxirJ9JO3920EqyH+jwtxjFuYhVUjIgsnDCwkA2RZGe0Ozb8kH/X5aMeKT5v7tIAlyj6ShYnIFkEm8kACQtDDaG5p9Sz7o99OKASlYgvyqpAfN7BcqS5LdPR1NIgAAINXCmFgQ5C35Wq0RQd/ST/t0BcQjSIJ8c/EDAAC0qKB9uZWuV629YfTBo+uKodZb8kFaI4K8pZ/26QqIR5Axb9eb2UGS3lI89IS7M88JAIAWEbQvt9r1Thtzmi5dc6n+ceY/7jv+5TVf1uu7X697lnE1YbVGVGvFOG3MaZq7cq6eeukptR3Upt6dvRp/WOsu4Mu7IFMsOiVdL+lpFeYgH21mH2XMGwD0VRrb19/I4UPpBUemBU0+a11PLl3+X5drc+9mjR85XpecdImOGH5EaL29YbVGVGrFOG3MafrV5l9pzjFztGP3DhbwtYAgLRZXSTrT3Z+QJDN7i6QbJU2PMjAAyJptO3ZFPnMaSELQ5LPW9dz9gDaFMHt7w2yN6N+KMXfl3H2JPgv4WsOgANcZWkqOJcndf6vCVAsAANACSslnuUrJZ7XrjT54tA4ffri+9LYvae2H1mrRjEW6Zv01+u6D3w2tt7fUGtG9pVu79u5S95ZufeGeL+ipl57S3JVztapnVdXvXdWzSnNXztWUG6ZUvG4p8WcBX+sIUkFea2bfk/RvxcsXSFobXUhAzJZNll565sDjh71JuuSh+OPJixCeV1oWgHQIOiKt2vX2+B597S+/dkDl9ZKuS/Tlt305lBj7tEa81KNDDjpE5x9/vj4x9RM1WyGC9FeXEn8W8LWOIAny/yvpk5JK/wvukfS/IosIiNuE4lviZy3df2z1IhVa7tGwEJ5XWhaAdAg6Iq3a9Rb9elHFyusrO18JtTWh1Boxd+VcLZqxKFArRJD+6lLiP+eYOaneHhnhCTLF4nUz+5akX6iw1fQT7r4z8siAuMxcKH37bYXPbaOk7c9JG26ULv5NwzdJ5VORPK8AkhN017NK11u+cXnlyuth0VRe62mFCHLd0uNZvnG5nu19Vp/u+vS+KRZp2h4Z4QkyxWKOpO9KekqF0s8xZvb/uPvtUQeHbMlsUtg2SjKTrjq+7/Gr3tJwm0Uclc/UP99to6SpH5TWXF2oIq+5WpryQamtPenIBtToc8u23q0p6HzgVtbsLnj1qqcVIuh107wtMsIXdIrFLHd/UpLM7FhJt0kiQUYfmX47/ITzpEGDpbO/sf9YytssMvF8l6rIUz6Qqepxo89tKl6YIFZB5wO3uqAtGmGpJyGPO3nPilZ/4RckQd5eSo6LeiRtjygeIBmzFknfnC694/O0A4SpVEW+/lxp6gWBq8flFdzypDQ11XGgKKzNKVpBnBXYehLyuJP3LOCFX/ApFqsk/ViSS3qfpPvN7N2S5O4/jTA+IB5to6QT3yv96kppzlWZagdIvZkLpd/dWfgcUK0K7rgv3kbLAlKDsV/pVU9CTvtEX7zwC5YgD5O0VdI7ipdfkDRc0jkqJMwkyMiHWYsK7QAnfaTlqseR9jO3jZI+Fd5kyEqJM5AUxn4hj3jhF2yKxd/GEQiQuAbbASrJ2mKtTPQzAylE/yryiBd+waZYHCPpU5LGlV/f3c9t9s7N7CxJV0saLOl77v7VZm8TyclaUlhRA+0AlcTRJ5uL5zuleG4RFP2ryCNe+AVrsfj/JV0n6RYV5iCHwswGS/q2pDMkbVKhr/lmd380rPtAvHKxeCrkdoAo5eL5TimeW9SD/lXkDS/8giXIr7n7NRHc9wxJT7p7jySZ2QpJ50kiQQZaHBVcAEhWq7/wM3evfQWzD0p6s6Q7JL1eOu7uDzR1x2bvlXSWu3+8ePnDkt7m7n/X73rzJc2XpPb29ukrVqxo5m5Tpbe3VyNGjEg6DCTs0S0va8/eA/8fDh5kmjj60H2XozxfNm7epkljRgY+jvTj9wvqxTmDeuTlfJk1a9Y6d+/ofzxIBXmSpA9LOl37Wyy8eDly7n6tpGslqaOjwzs7O+O421h0dXUpT48n65Lame7CL95WdYHc0+d37rsc5fmy8Io7tG3jKwccHzl8uDZcEP59pn4XwBzg9wvqlZpzZtlk6aVnDjze4M6miEZqzpeIBEmQ3ydpvLvvDPm+N0s6uuzy2OIxIBGtPMkh7qS0lZ9rAAOYUPzdcNbS/cdSvrMp8mdQgOs8LOmwCO77fklvNrNjzOwgSR+QdHME9wMAALJi5kLpwR8VdjSV9u9s2uR0IaAeQSrIh0l63MzuV98e5KbGvLn7bjP7O0k/U2HM27+4+yPN3CYAAMi40kz6NVcXqsjsbIoEBEmQL4vqzt19laRVUd0+AADIoJkLCzubTvlAy+1sinQIspPe3WbWLunk4qFud38+2rCA1sFIM6AOLOBqDSHubAo0IshOev+3pCsldanQIf9NM/ucu/9HxLEBsUoqUW3FqQ28KEDDWMDVOkLa2TQJq3pWafnG5fs22Zg3aV5LzxTOoiAtFpdKOrlUNTazIyX9XBIJMnKlFRPVpPBco2Glt95nLixUGUsLuHgLPn8ytLNpuVU9q3TN+msO2KZZEklyhgRJkAf1a6l4UcGmXwBIAWYOI1dYwIWUW75xuZacukQzRs+QJM0YPUNLTl2ipd1LSZAzJEiCvNrMfibpxuLl90u6PbqQgPxJMkll5jByhwVcSLGebT2a1j6tz7Fp7dPUs60noYjQiCCL9D5nZu+W9PbioWvd/aZowwLyhSQVLS/MxXUs4EKKjR85Xuu3rt9XQZak9VvXa/zI8QlGhXpVTZDN7DhJ7e6+xt1/KumnxeNvN7Nj3f2puIIEAGRc2IvrMryAC/k2b9I8Lb538QE9yAumLUg6NNShVgV5maRFFY5vK37tnEgiAgDkT9iL6zK6gAv5V+ozXtq9dN8UiwXTFtB/nDG1EuR2d9/Y/6C7bzSzcZFFBADIHxbXoYXMHj+bhDjjaiXIh9X42vCwAwEQDWYOIzVYXNfSmA2MLKmVIK81s3nuvrz8oJl9XNK6aMMC8iXJJJVRbkgNFte1LGYDI2tqJciXSLrJzC7Q/oS4Q9JBkuZGHRiQJ2lOUpmTjFixuK4lMRsYWVM1QXb3rZJONbNZkk4sHr7N3X8ZS2QAYsEIOsSKxXUtidnAyJogc5DvknRXDLEAAIAcYjYwsoYtowEAQKRKs4G7t3Rr195d6t7SrcX3Lta8SfOSDg2oKMhW0wAABBPmjnnIDWYDI2tIkAEA4Ql7x7ykkOiHjtnAyJJaW01vl+SVviTJ3f3QyKICEBvmJCNUYe+Yl5S8JPoAGlJrikVbnIEASAaj3BCqvOyYl5dEH0BDAi/SM7OjzOyNpY8ogwIAZNjMhdKDP5K2bCgklVmceVye6EvZTfQBNGTAHmQzO1fSVZL+XNLzkt4k6TFJb402NABpkpkNRegdTV5edswbaGtszjUgt4Is0vsHSadI+rm7TytuHPKhaMMCkDahbCgSR0JB72g65GHHvIESfc41ILeCJMi73P1FMxtkZoPc/S4zWxZ5ZADyJ46Egt7RdMjLjnm1En3ONSC3giTIL5nZCEn3SPqhmT0v6ZVowwKQS3EkFHlZJIZ0qJXop/Fco+0DCEWQBPk8Sa9JukTSBZJGSloSZVAAciquhGKg3lEgLGk712j7AEIxYILs7q+Y2ShJMyT9SdLP3P3FyCMDkE9xJBR5WSSG9AvrXAur8kvbBxCKIFMsPi5psaRfqvAS9JtmtsTd/yXq4ACkR2gbisSVvOZhkRiyIYxzLazKbxrbPoAMCtJi8TlJ00pVYzM7XNK9kkiQgRYS6ii3OJLXvCwSQ/qFca6FWflNW9sHkEFBEuQXJW0vu7y9eAxA1iW1oIfkFegrzMovLUZA04IkyE9K+o2ZrZTkKizae8jMPiNJ7v6Neu/UzN4n6XJJJ0ia4e78pQSSwIIeID3CrPzSYgQ0JUiC/FTxo2Rl8XNbE/f7sKR3S/r/mrgNAGG5xwAAABjNSURBVM1iQU88GL2FIMKs/PIuDdCUIFMsrgj7Tt39MUkyo0oFJIoFPfGgUo+gqPwCqWDuXvkLZsvc/RIzu0WF1oo+3P3cpu/crEvS39dqsTCz+ZLmS1J7e/v0FStWNHu3qdHb26sRI0YkHQYyIrLzZe9u6fnHpMOPk158UjrqBGlQkDeXEFjpOS49t/0vRyDzv1+ef1Tas/PA44MPko6aGH88LSDz5wxilZfzZdasWevcvaP/8Vq/mX9Q/PxPjdyhmf1c0qgKX7rU3VdWOF6Ru18r6VpJ6ujo8M7OzkbCSaWuri7l6fEgWpGeL6t/Ia37p8Lbuqd/Kpr7aHWrfyHtfLZQRV69SDrMIn2uM//7ZfXPCp8rVd07L04kpLzL/DmDWOX9fKmaILv7uuI/10ra4e57JcnMBkt6w0A37O7vCiVCANHjbd3oMXqrPvTHA0hQkPf2fiHpXZJ6i5eHS7pD0qlRBQUgZizoiV4aR2+lefEg/fHBpfnnCGRUkAR5mLuXkmO5e6+ZHdzMnZrZXEnflHSkpNvM7EF3/6tmbhMAUi9tlfq0Lx6k6h5M2n+OQAYFSZBfMbOT3P0BSTKz6ZJ2NHOn7n6TpJuauQ0AyJy0VerT3saQxqp7lBqtBKf95whkUJAE+RJJPzGzZ1V4OTpK0vsjjQoAEL0stDGkreoepUYrwVn4OQIZE2QO8v1mNkHS8cVDT7j7rmjDAgAE9j9HSbv7vbF3/BXS//yA9OXnan9v2tsY0lZ1j1IzleC0/xz7o28aKRd0AOfJksYVr3+Smcndb4gsKgCN4Y9OazrqBGlshzT7yv3Hbv63YPOCW62NIc2aqQRn7edI3zRSbsAE2cx+IOlYSQ9K2lM87JJIkIG04Y9Oazrnaum6M6S//Oz+yuOrf5LOvSbY97dSG0PaNVMJztLPkb5ppNygANfpkDTT3S92908VPxZEHRiABsxcKD34o8IfG2n/H50s/MFE40ZPLlSL77mqcPmeq6Qhw6RRJwb7/lIbQ9qrjq2gvBJcbx9xln6O5dVyib5ppE6QBPlhVd4RD0Da8EendZ1ztfTADdKWDdL6H0iHvTHpiNCoR1dKO3ul+74tXT5y/8eyyUlHFq7SC/otG3ghj9QJ0oN8hKRHzaxb0uulg+5+bmRRAehr2WSp/SLp8vP6Hq/UW5y1xToIR6mK/P2zpSMnFirISI9q6wOGDD9wgeXgg6TpH5XmfGP/sTy2SmWtbxotJUiCfHnUQQAYwIQ50s4jpcu37T9W7Q9m3H90sr4wMOvxlzvnaum6Mwu9x4//MeloUK7a+oDf3ye98ZS+x29eIG34d+m0z+e/PzdLfdNoKUHGvN0dRyBA7oSZeM1cKN32k8IfyiB/MOP8oxPnwsD+z+ngodKeClMn63mO87SwcfTk/WPdHu9KNBT0U21R2kdvKVT9y48/drN04ntbY65xK43xQ6ZUTZDN7Nfu/nYz267C1Ip9X5Lk7n5o5NEBWRZm4tU2Sjr4z4L/wWz2j049yX2cq9H7P6erF0l7d/cdb1bpOa71eD52B6vpEb1qI9xGTap8fOYCWqWABFVdpOfuby9+bnP3Q8s+2kiOgQDCnigxoj2+BS0T5kinXFxo6Sh9nHKxNOHsA68b58LA/s/plPOlB34w8HNc6/GwsBFxqbYordLxZqZZAGhazSkWZjbYzB6PKxggV8JOvAYNie8PZr3JfVyr0fs/pxtuLGySMdBzPNDjYTU94lAt6a12fOZC6ZAjOR+BBNRMkN19j6QnzIx5QUAjwk684vqDWW9yH2e1q/9zes41Az/HAz0eqnWIS7X/w5WOhz3XeNnkvmPj8jo+DghBkCkW/5ekR4pj3l4pHWTMGxBA2BMl4lzQUu+4uLgWBvZ/TkdPCvYcD/R4WE2POFT7PxzH/+08LUgFIhYkQf4fkUcB5FlWE696k/u4k/fy5zTIczzQ42E1PfKO7Z2BwKq2WJjZMDO7RNL7JE2QtMbd7y59xBYhkHVZ2v61v7T2QPZ/ToM+x2l9PEAcWJAKBFarB/l6SR2SNkr6a0lXxRIRgPTIcnJfSd4eD1AvFqQCgdRqsZjo7pMkycyuk9QdT0gAACASbO8MBFIrQd63PZW77zajiR8AgFRpZMfOrK6LAGJUK0GeYmYvF/9tkoYXL7OTHgAAadDIZAoWpAIDqrWT3uB+u+cNYSc9AABSJOwdOwFIGmCjEAAAkGJMpgAiEWQOMgAA+dZIL29a1LupD4ABkSADAJDkLnPNJudMpgBCR4IMAECSu8yFkZwzmQIIFQkyEJUsv2ULtJryXt6zlsbbyxtGcs5kCiBUJMhAVBqtClVLrCculdQZUnBoSfW+aIvyRV4aX0Am1cubZHIOoCISZCAqjVaFqiXWO0dGFytaQ70v2sJ+kTd4qNT254UEOMme32qS7OVloR2QKox5A6LS6PilanNNR1BNQpPqnZnb6IzdCXOkGfOly7ft/zjlYql9kjTh7OZuO2ozF0qHHBl/HOXJOdVjIHGJJMhmdqWZPW5mD5nZTWZ2WBJxAJErJQFbNgT/418tsR7EGz5oUr0v2pp5kffQj/smvw/+SPrTU/v/D6R1fm+plzeJOJJKzgEcIKkK8p2STnT3yZJ+K2lRQnEA0Wq0KtRIYo1glk2WLh954MeyyUlHFo96z61mXuTdc1Xh8pqrpT8bL039UN//A5znfSWZnAPoI5GSlLvfUXbxPknvTSIOIBaNjF+q2Av5WGQhtpQ09r7Gqd4+20b7cks9tdM+VEiC5dL5K8K5bQCImLl7sgGY3SLp393936p8fb6k+ZLU3t4+fcWKFZWulkm9vb0aMWJE0mEgrfbulv74O+mIN0uDhnC+hGXvbun5x6SjTii0rfS/nBM1z5d+59aA6r1+ycubpVf/JMmlgw+XDh0T3m0jdPyOQT3ycr7MmjVrnbt39D8eWYJsZj+XNKrCly5195XF61wqqUPSuz1AIB0dHb52bX7mPHZ1damzszPpMJARnC8hWl3s6jpr6f7q8Vlfiea+EhpndsD5kkQc25+T/vWvJXfpop/lu0KcxrF1deJ3DOqRl/PFzComyJG9XHf3dw0Q0IWSzpb0ziDJMQCEJs6RWmlp6UgijrZR0oL10d1+mqTl5wwgFIm8n2VmZ0n6vKR3uPurScQAIERZq57F2fs6c6F0zdS+87DX/ou0+zXp8Vvje37C3Eo5az/vOCS5VTWA0CXV8PUtSW+QdKeZSdJ97v6JhGIB0KwsVs8aWTzZiLZR0pEnFCY6zL6yMNGh42OSXLE+P2Hu1pbFn3fU2A0PyJWkplgcl8T9AohIktWzRquZpZFacTjnaum6MwoTHTbcKH30Fun6c+KvLobVWtLIz7sVqs7shgfkBkuGATQvyepZFqqZoydLR00sJMVTi2PPkqguhtVa0sjPO6yfU5oTbcbWAblBggwgHElVz7LS+3nO1dL33im9+V3Sf1yUXHxhtZZU+nnXSl4/dkc4P6e0vyCKq3UHQKSS2kkPQN40umtgWPebti2L+xs9WTr549JP/jbZ+MLara3Sz3vCHOmUi6XLt+3/OOViacLZ0nVnSrtele7+euH77/66tPOVQutJPUq775VvY52mXfjYDQ/IBSrIAMKTVPWs0ep13G/Xh/n8pKHVoP/jqVnN90JC/PB/StM/Kj3y00JifdAh9d0ni+EAxIAEGUB44lz41v9+G+n9jPvt+jCfnzS0GvR/PLWS11LyPPFvCj+nt/6N9OjKxlpNWAwHIGIkyADyIUh1tlrV9bFbpE8/nN7+5UrS2ntdnryW5j3f9+3C14YMk3a/Lh1yZOHzzlekq95Sf9WbxXAAIkaCDCAfglRnK1VdV31O2lT8viy9XZ/WVoN9yes5hckd8+/a/7XvvF16/Gbpb2+XbjhPuuRhac0yNVT1ZjEcgAiRIANoHZWqrg/9WPK90pYN6ajAlhuozzitrQYzF0q/XS29+GThOS4919t+L2nQ/sqvvPG4k2rnAdASSJABtI5KVddSopbGt+sH6jNOa6tB2yhpwfpCrH2e6w9JO3ulx28pJNFrljVf9U7DYkUAuUOCDKC1VKy6ejrfrg/SZ5zmVoNqz/WjK6XekHqm07BYEUDuMAcZQGupNL83rbNrg8x4TmvsUvXnOsx52Wmfiwwgk0iQAbSemQsLkxSykESVEsBSj3QWYi5X6bkO8/nPykYxADKFFgsA2RFWv2mWFniltc84qErPddjPf1oXKwLILBJkANnRqv2mae4zToOsv4gAkDq0WADIjlbtN01zn3FaZKltBkDqkSADyA76TVENLyIAhIgWCwDZQr9pNjGvGECGkCADyBb6TbNpoP5xEmgAKUKCDCB7WLSWPQNtetKqCzABpBI9yACyh37T7Bmof7xVF2ACSCUSZABAPGptesICTAApQoIMAIjHQNtMZ33XQAC5QQ8yAKCvKBfM1eofZwEmgJQgQQYA9BXlgrmBtplmASaAFKDFAgDQV5IL5liACSAFSJABAH2xYA5AiyNBBgAciAVzAFoYCTIA4EADTZwAgBwjQQYAVDZzoXTIkVSPAbScRKZYmNk/SDpP0l5Jz0u60N2fTSIWAEAVA02cAICcSqqCfKW7T3b3qZJulbQ4oTgAAACAPhJJkN395bKLh0jyJOIAAAAA+jP3ZHJTM/tHSR+RtE3SLHd/ocr15kuaL0nt7e3TV6xYEV+QEevt7dWIESOSDgMZwfmCerTs+fL8o9KenQceH3yQdNTE+OPJkJY9Z9CQvJwvs2bNWufuHf2PR5Ygm9nPJY2q8KVL3X1l2fUWSRrm7pcNdJsdHR2+dm1++uG6urrU2dmZdBjICM4X1KNlz5fViwqfK+0CeNZXEgkpK1r2nEFD8nK+mFnFBDmyRXru/q6AV/2hpFWSBkyQAQCoaeZC6dtvK3xuG7V/F8CLf5N0ZAAyJJEeZDN7c9nF8yQ9nkQcAICcYRdAACFIZMybpK+a2fEqjHl7RtInEooDAJA3pSrylA9QPQbQkEQSZHd/TxL3CwBoAeW7AE69gOoxgLqxkx4AIH/YBRBAE5JqsQAAIDrsAgigCVSQAQAAgDIkyAAAAEAZEmQAAACgDD3IAFrTssnSS88cePywN0mXPBR/PACA1CBBBtCaJswpfK60JTEAoKXRYgGgNc1cKD34o8JWxNL+LYkZCwYALY8EGUBrYktiAEAVJMgAWlepirxlA9VjAMA+JMgAWlf5lsRUjwEARSTIAFobWxIDAPphigWA1saWxACAfqggAwAAAGVIkAEAAIAyJMgAAABAGRJkAAAAoAwJMgAAAFCGBBkAAAAoQ4IMAAAAlCFBBgAAAMqQIAMAAABlSJABAACAMiTIAAAAQJkhSQcAALmwbLL00jMHHj/sTdIlD8UfDwCgYSTIABCGCXMKn89auv/Y6kWSLJFwAACNo8UCAMIwc6H04I+k7c8VLm9/TtpwY+E4ACBTSJABIAxto6SpH5TWXF24vOZqacoHpbb2ZOMCANQt0QTZzD5rZm5mRyQZBwCEolRF3rKB6jEAZFhiCbKZHS3pTEm/TyoGAAhVqYp8/blUjwEgw5KsIP+zpM9L8gRjAIBwzVwoHXIk1WMAyLBEpliY2XmSNrv7BjNWeAPIkbZR0qfWJh0FAKAJ5h5NAdfMfi5pVIUvXSrpS5LOdPdtZva0pA53/2OV25kvab4ktbe3T1+xYkUk8Saht7dXI0aMSDoMZATnC+rB+YJ6cc6gHnk5X2bNmrXO3Tv6H48sQa7GzCZJ+oWkV4uHxkp6VtIMd3+u1vd2dHT42rX5qcx0dXWps7Mz6TCQEZwvqAfnC+rFOYN65OV8MbOKCXLsLRbuvlHSUaXLA1WQAQAAgDgxBxkAAAAok/hW0+4+LukYAAAAgBIqyAAAAEAZEmQAAACgDAkyAAAAUIYEGQAAACgT+xzkZpjZC5KeSTqOEB0hifF2CIrzBfXgfEG9OGdQj7ycL29y9yP7H8xUgpw3Zra20nBqoBLOF9SD8wX14pxBPfJ+vtBiAQAAAJQhQQYAAADKkCAn69qkA0CmcL6gHpwvqBfnDOqR6/OFHmQAAACgDBVkAAAAoAwJMgAAAFCGBDklzOyzZuZmdkTSsSC9zOxKM3vczB4ys5vM7LCkY0L6mNlZZvaEmT1pZl9MOh6kl5kdbWZ3mdmjZvaImS1MOiakn5kNNrP1ZnZr0rFEhQQ5BczsaElnSvp90rEg9e6UdKK7T5b0W0mLEo4HKWNmgyV9W9JfS5oo6Xwzm5hsVEix3ZI+6+4TJZ0i6ZOcLwhgoaTHkg4iSiTI6fDPkj4viRWTqMnd73D33cWL90kam2Q8SKUZkp509x533ylphaTzEo4JKeXuW9z9geK/t6uQ9IxJNiqkmZmNlTRH0veSjiVKJMgJM7PzJG129w1Jx4LMuUjS7UkHgdQZI+kPZZc3iYQHAZjZOEnTJP0m2UiQcstUKOrtTTqQKA1JOoBWYGY/lzSqwpculfQlFdorAEm1zxd3X1m8zqUqvDX6wzhjA5BPZjZC0n9KusTdX046HqSTmZ0t6Xl3X2dmnUnHEyUS5Bi4+7sqHTezSZKOkbTBzKTC2+UPmNkMd38uxhCRItXOlxIzu1DS2ZLe6Qwyx4E2Szq67PLY4jGgIjMbqkJy/EN3/2nS8SDVZko618xmSxom6VAz+zd3/1DCcYWOjUJSxMyeltTh7n9MOhakk5mdJekbkt7h7i8kHQ/Sx8yGqLCA850qJMb3S/qguz+SaGBIJStUZ66X9Cd3vyTpeJAdxQry37v72UnHEgV6kIFs+ZakNkl3mtmDZvbdpANCuhQXcf6dpJ+psODqxyTHqGGmpA9LOr34O+XBYnUQaGlUkAEAAIAyVJABAACAMiTIAAAAQBkSZAAAAKAMCTIAAABQhgQZAAAAKEOCDKAlmdme4kirh83sJ2Z2cJXr3dvg7XeY2TVNxNdb5fgoM1thZk+Z2TozW2Vmb2n0ftLAzDrN7NQqX5tgZv9lZq+b2d/HHRuA1kSCDKBV7XD3qe5+oqSdkj5R/sXihhty94qJ20Dcfa27L2g+zD4xmaSbJHW5+7HuPl3SIkntYd5PAjolVXue/yRpgaR/ii0aAC2PBBkApHskHVesZN5jZjdLelTaX8ktfq3LzP7DzB43sx8WE1aZ2clmdq+ZbTCzbjNrK17/1uLXLzezHxQrob8zs3nF4yPM7Bdm9oCZbTSz8waIc5akXe6+b4MYd9/g7vdYwZXFivhGM3t/Wdx3m9lKM+sxs6+a2QXFODea2bHF633fzL5rZmvN7Ldmdnbx+DAz+9fiddeb2azi8QvN7Kdmtrr4mL5eisnMziw+1geK1fkRxeNPm9kVZY93gpmNU+HFyaeLFf2/LH/A7v68u98vaVcDP1cAaMiQpAMAgCQVK8V/LWl18dBJkk509/+ucPVpkt4q6VlJayTNNLNuSf8u6f3ufr+ZHSppR4XvnSzpFEmHSFpvZrdJel7SXHd/2cyOkHSfmd3s1XdwOlHSuipfe7ekqZKmSDpC0v1m9qvi16ZIOkGFamyPpO+5+wwzWyjpU5JKWwyPkzRD0rGS7jKz4yR9UpK7+yQzmyDpjrKWjqnF5+R1SU+Y2TeLj/3Lkt7l7q+Y2RckfUbSkuL3/NHdTzKzi1XYpvbjxR0he92dKjGAVCBBBtCqhpvZg8V/3yPpOhXe5u+ukhyr+LVNklT83nGStknaUqxyyt1fLn69//eudPcdknaY2V0qJKK3SfqKmZ0maa+kMSq0SzzXwON5u6Qb3X2PpK1mdrekkyW9LOl+d99SjOspSXcUv2ejClXpkh+7+15JvzOzHkkTirf7zeJje9zMnpFUSpB/4e7birf7qKQ3STpM0kRJa4rPwUGS/qvsPn5a/LxOhaQeAFKHBBlAq9rh7lPLDxQTuldqfM/rZf/eo/p+h/avCrukCyQdKWm6u+8ys6clDatxG49Iem8d91lSHvfesst71fcxVIox6O2Wng+TdKe7nz/A99T7/AFAbOhBBoDmPCFptJmdLEnF/uNKid95xX7ew1VYlHa/pJGSni8mx7NUqMDW8ktJbzCz+aUDZja52Ld7j6T3m9lgMztS0mmSuut8LO8zs0HFvuTxxcd2jwqJvIqtFW8sHq/mPhVaT44rfs8hNvCUje2S2uqMFQAiQ4IMAE1w952S3i/pm2a2QdKdqlwFfkjSXSokkP/g7s9K+qGkDjPbKOkjkh4f4L5c0lxJ77LCmLdHJC1VoSXjpuJ9bFAhkf68u9fbqvF7FZLq2yV9wt1fk/S/JA0qxvjvki5099er3YC7vyDpQkk3mtlDKrRXTBjgfm+RNLfSIj0rjLXbpEIf85fNbFOxzxsAImPV14IAAMJgZpcr5YvQzOz7km519/9IOhYASBoVZAAAAKAMFWQAAACgDBVkAAAAoAwJMgAAAFCGBBkAAAAoQ4IMAAAAlCFBBgAAAMr8H2AWXpO/Fx4nAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 720x360 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    }
  ]
}