Hockey_games.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Predicting the Outcome of NHL Games\n",
    "\n",
    "Sports over time have been largely dominated by the use of numbers and statistics. Moneyball, the 2011 movie demonstrating a statistical approach in developing a competitive team using baseball statistics, shows one approach of how statistics, and machine learning to an extent, can predict and optimize the outcome of sports (in this case it was choosing players for a team).\n",
    "\n",
    "Today I will be attempting to see if I can predict the outcome of NHL matches using machine learning tools, particularly the Random Forest Classifier.\n",
    "\n",
    "<img src=\"files/SportsReference.png\">\n",
    "\n",
    "The datasets I will be using are from www.hockey-reference.com, part of the www.sports-reference.com sites. Sports-Reference has other excellent data including:\n",
    "- College Football (and NFL)\n",
    "- College Basketball (and NBA)\n",
    "- Olympic Sports\n",
    "- NHL\n",
    "\n",
    "The huge amount of data available on these sites are an excellent source to practice machine learning as much of the data is cleaned and well organized."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Here are the direct links if you wish to view the two datasets I use to build my model:\n",
    "\n",
    "- Games 2014 data: http://www.hockey-reference.com/leagues/NHL_2014_games.html\n",
    "- Standings 2013 data: http://www.hockey-reference.com/leagues/NHL_2013_standings.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Getting Started: Preliminary Setup\n",
    "\n",
    "Let's begin by exporting the data as well as fixing any issues such as renaming columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "os.chdir('C:/Users/Kevin/My Project/Hockey')\n",
    "games_filename = ('leagues_NHL_2014_games_games.csv')\n",
    "playoff_filename = ('leagues_NHL_2014_games_games_playoffs.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Visitor</th>\n",
       "      <th>G</th>\n",
       "      <th>Home</th>\n",
       "      <th>G.1</th>\n",
       "      <th>Unnamed: 5</th>\n",
       "      <th>Unnamed: 6</th>\n",
       "      <th>Att.</th>\n",
       "      <th>LOG</th>\n",
       "      <th>Notes</th>\n",
       "      <th>Notes.1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Washington Capitals</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Chicago Blackhawks</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Winnipeg Jets</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Edmonton Oilers</td>\n",
       "      <td>4.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Toronto Maple Leafs</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Montreal Canadiens</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2013-10-02</td>\n",
       "      <td>Anaheim Ducks</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Colorado Avalanche</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2013-10-02</td>\n",
       "      <td>Buffalo Sabres</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Detroit Red Wings</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date              Visitor    G                Home  G.1 Unnamed: 5  \\\n",
       "0  2013-10-01  Washington Capitals  4.0  Chicago Blackhawks  6.0        NaN   \n",
       "1  2013-10-01        Winnipeg Jets  5.0     Edmonton Oilers  4.0        NaN   \n",
       "2  2013-10-01  Toronto Maple Leafs  4.0  Montreal Canadiens  3.0        NaN   \n",
       "3  2013-10-02        Anaheim Ducks  1.0  Colorado Avalanche  6.0        NaN   \n",
       "4  2013-10-02       Buffalo Sabres  1.0   Detroit Red Wings  2.0        NaN   \n",
       "\n",
       "   Unnamed: 6  Att.  LOG Notes Notes.1  \n",
       "0         NaN   NaN  NaN   NaN     NaN  \n",
       "1         NaN   NaN  NaN   NaN     NaN  \n",
       "2         NaN   NaN  NaN   NaN     NaN  \n",
       "3         NaN   NaN  NaN   NaN     NaN  \n",
       "4         NaN   NaN  NaN   NaN     NaN  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's take a look at the datafile containing all the matches:\n",
    "results = pd.read_csv(games_filename)\n",
    "results_playoff = pd.read_csv(playoff_filename)\n",
    "\n",
    "results.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Date    object\n",
       "dtype: object"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Did Pandas interpret the \"Date\" column as actual dates?\n",
    "results.dtypes.head(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Pandas did not interpret the 'Date' column as actual dates, we will need to add an extra parameter in our \"read_csv\" function to fix this issue.\n",
    "\n",
    "The dataset for the 2014 games are split up into two different datasets:\n",
    "1. Regular Season\n",
    "2. Playoff Season\n",
    "\n",
    "I will combine both datasets into a single dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Import both datasets\n",
    "results1 = pd.read_csv(games_filename, parse_dates=[\"Date\"])\n",
    "results2 = pd.read_csv(playoff_filename, parse_dates=[\"Date\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# We will be using \"results\" as our main dataframe when building our model\n",
    "results = results1.append(results2, ignore_index=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Let us clean and fix some of the issues in the dataframe above\n",
    "# Fix the names of the columns\n",
    "results.columns = [\"Date\", \"Visitor Team\", \"Visitor Goals\", \"Home Team\", \"Home Goals\", \\\n",
    "                   \"Overtime\", \"Blank\", \"Att\", \"LOG\", \"Notes1\", \"Notes2\"]\n",
    "\n",
    "# Delete the columns we do not need in our analysis\n",
    "results = results.drop(results.columns[[5,6,7,8,9,10]], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Visitor Team</th>\n",
       "      <th>Visitor Goals</th>\n",
       "      <th>Home Team</th>\n",
       "      <th>Home Goals</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Washington Capitals</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Chicago Blackhawks</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Winnipeg Jets</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Edmonton Oilers</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Toronto Maple Leafs</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Montreal Canadiens</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2013-10-02</td>\n",
       "      <td>Anaheim Ducks</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Colorado Avalanche</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2013-10-02</td>\n",
       "      <td>Buffalo Sabres</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Detroit Red Wings</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2013-10-02</td>\n",
       "      <td>Toronto Maple Leafs</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Philadelphia Flyers</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Date         Visitor Team  Visitor Goals            Home Team  \\\n",
       "0 2013-10-01  Washington Capitals            4.0   Chicago Blackhawks   \n",
       "1 2013-10-01        Winnipeg Jets            5.0      Edmonton Oilers   \n",
       "2 2013-10-01  Toronto Maple Leafs            4.0   Montreal Canadiens   \n",
       "3 2013-10-02        Anaheim Ducks            1.0   Colorado Avalanche   \n",
       "4 2013-10-02       Buffalo Sabres            1.0    Detroit Red Wings   \n",
       "5 2013-10-02  Toronto Maple Leafs            3.0  Philadelphia Flyers   \n",
       "\n",
       "   Home Goals  \n",
       "0         6.0  \n",
       "1         4.0  \n",
       "2         3.0  \n",
       "3         6.0  \n",
       "4         2.0  \n",
       "5         1.0  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Lets view our cleaned dataframe\n",
    "results.ix[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Team Name Change\n",
    "\n",
    "In preparing our model, we will be working with team names from the 2013 and 2014 seasons.\n",
    "\n",
    "- There is a change in a team name moving forward in the 2015 season and we will need to address this issue early on\n",
    "- The Phoenix Coyotes changed their name to the Arizona Coyotes\n",
    "\n",
    "<img src=\"coyotes.png\">\n",
    "Let's change their team name in our dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# The team names appear in both the 'Visitor Team' and 'Home Team' columns so we need to make changes in both columns\n",
    "results[\"Visitor Team\"] = results[\"Visitor Team\"].replace(\"Phoenix Coyotes\", \"Arizona Coyotes\")\n",
    "results[\"Home Team\"] = results[\"Home Team\"].replace(\"Phoenix Coyotes\", \"Arizona Coyotes\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Create the class to predict\n",
    "\n",
    "We are trying to predict whether a certain team won their match. In our model, we will predict whether the Home Team won their match. Lets create the appropriate attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Visitor Team</th>\n",
       "      <th>Visitor Goals</th>\n",
       "      <th>Home Team</th>\n",
       "      <th>Home Goals</th>\n",
       "      <th>HomeWin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Washington Capitals</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Chicago Blackhawks</td>\n",
       "      <td>6.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Winnipeg Jets</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Edmonton Oilers</td>\n",
       "      <td>4.0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Toronto Maple Leafs</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Montreal Canadiens</td>\n",
       "      <td>3.0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2013-10-02</td>\n",
       "      <td>Anaheim Ducks</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Colorado Avalanche</td>\n",
       "      <td>6.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2013-10-02</td>\n",
       "      <td>Buffalo Sabres</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Detroit Red Wings</td>\n",
       "      <td>2.0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2013-10-02</td>\n",
       "      <td>Toronto Maple Leafs</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Philadelphia Flyers</td>\n",
       "      <td>1.0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Date         Visitor Team  Visitor Goals            Home Team  \\\n",
       "0 2013-10-01  Washington Capitals            4.0   Chicago Blackhawks   \n",
       "1 2013-10-01        Winnipeg Jets            5.0      Edmonton Oilers   \n",
       "2 2013-10-01  Toronto Maple Leafs            4.0   Montreal Canadiens   \n",
       "3 2013-10-02        Anaheim Ducks            1.0   Colorado Avalanche   \n",
       "4 2013-10-02       Buffalo Sabres            1.0    Detroit Red Wings   \n",
       "5 2013-10-02  Toronto Maple Leafs            3.0  Philadelphia Flyers   \n",
       "\n",
       "   Home Goals HomeWin  \n",
       "0         6.0    True  \n",
       "1         4.0   False  \n",
       "2         3.0   False  \n",
       "3         6.0    True  \n",
       "4         2.0    True  \n",
       "5         1.0   False  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a new attribute called \"HomeWin\" to show which team won that game\n",
    "results[\"HomeWin\"] = results[\"Visitor Goals\"] < results[\"Home Goals\"]\n",
    "\n",
    "# This will be the class we are trying to predict for our model, whether the home team won or not\n",
    "y_true = results[\"HomeWin\"].values\n",
    "\n",
    "# Lets view the results\n",
    "results.ix[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Create the Performance Indicator\n",
    "\n",
    "We will be using the F1 score as our main indicator of overall model performance. The F1 score is an overall measure of a test's accuracy which considers precision and recall in its formula as shown below:\n",
    "\n",
    "<img src=\"f1formula.png\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# We will use the classification report and the F1 score metric to show the performance of our model\n",
    "from sklearn.metrics import f1_score, make_scorer, classification_report\n",
    "\n",
    "# Lets designate a scorer object with the F1 score\n",
    "scorer = make_scorer(f1_score, pos_label=None, average='weighted')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Creating a baseline to beat\n",
    "In general, home teams win games more frequently - this is shown throughout many other sports as well.\n",
    "\n",
    "We need our model to beat the default dataset we were given without any major changes. The performance for the dataset is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overall Home Win Percentage in Dataset: 54.0%\n"
     ]
    }
   ],
   "source": [
    "# Calculate the number of times home teams won in the dataset\n",
    "n_games = results[\"HomeWin\"].count()\n",
    "n_homewins = results[\"HomeWin\"].sum()\n",
    "win_percentage = n_homewins / n_games\n",
    "\n",
    "print(\"Overall Home Win Percentage in Dataset: {0:.1f}%\".format(100 * win_percentage))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**Note**: 54.0% is the chance we'll guess correctly if we were to randomly pick a game and guess that the home team won"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "scrolled": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "F1 Score: 0.3782\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Kevin\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1074: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import f1_score\n",
    "y_pred = [1] * len(y_true)\n",
    "print(\"F1 Score: {:.4f}\".format(f1_score(y_true, y_pred, pos_label=None, average=\"weighted\")))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**We need to build a model that produces a better F1 Score than 0.3782.**\n",
    "\n",
    "Let us create some more attributes to produce a better model!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## New Feature: Whether the home or visitor team won their last game"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let us create features that indicates whether each team won their last game. \n",
    "\n",
    "- Losing a game can easily cause stress and may demotivate players to do better in successive games\n",
    "- On the flipside, winning a game can add to players' egos and make them more lax in successive games\n",
    "\n",
    "Whether these statements are true or not, lets see if this new feature can improve our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "# Create the new features with a default value of \"False\"\n",
    "results[\"HomeLastWin\"] = False\n",
    "results[\"VisitorLastWin\"] = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "won_last = defaultdict(int)\n",
    "\n",
    "for index, row in results.iterrows(): # for each row\n",
    "    home_team = row[\"Home Team\"]\n",
    "    visitor_team = row[\"Visitor Team\"]\n",
    "    row[\"HomeLastWin\"] = won_last[home_team]\n",
    "    row[\"VisitorLastWin\"] = won_last[visitor_team]\n",
    "    results.ix[index] = row\n",
    "    # Set which team won\n",
    "    won_last[home_team] = row[\"HomeWin\"] # set HomeLastWin as True if \"HomeWin\" is True\n",
    "    won_last[visitor_team] = not row[\"HomeWin\"] # otherwise set \"VisitorLastWin\" as True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "scrolled": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Visitor Team</th>\n",
       "      <th>Visitor Goals</th>\n",
       "      <th>Home Team</th>\n",
       "      <th>Home Goals</th>\n",
       "      <th>HomeWin</th>\n",
       "      <th>HomeLastWin</th>\n",
       "      <th>VisitorLastWin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Detroit Red Wings</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Boston Bruins</td>\n",
       "      <td>4.0</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Tampa Bay Lightning</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Chicago Blackhawks</td>\n",
       "      <td>2.0</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Washington Capitals</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Dallas Stars</td>\n",
       "      <td>2.0</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Anaheim Ducks</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Minnesota Wild</td>\n",
       "      <td>3.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Philadelphia Flyers</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Montreal Canadiens</td>\n",
       "      <td>4.0</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Columbus Blue Jackets</td>\n",
       "      <td>3.0</td>\n",
       "      <td>New York Islanders</td>\n",
       "      <td>2.0</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date           Visitor Team  Visitor Goals           Home Team  \\\n",
       "20 2013-10-05      Detroit Red Wings            1.0       Boston Bruins   \n",
       "21 2013-10-05    Tampa Bay Lightning            3.0  Chicago Blackhawks   \n",
       "22 2013-10-05    Washington Capitals            1.0        Dallas Stars   \n",
       "23 2013-10-05          Anaheim Ducks            4.0      Minnesota Wild   \n",
       "24 2013-10-05    Philadelphia Flyers            1.0  Montreal Canadiens   \n",
       "25 2013-10-05  Columbus Blue Jackets            3.0  New York Islanders   \n",
       "\n",
       "    Home Goals HomeWin HomeLastWin VisitorLastWin  \n",
       "20         4.0    True        True           True  \n",
       "21         2.0   False        True          False  \n",
       "22         2.0    True       False           True  \n",
       "23         3.0   False       False          False  \n",
       "24         4.0    True       False          False  \n",
       "25         2.0   False        True          False  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results.ix[20:25]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Classification with Decision Trees\n",
    "\n",
    "Lets use the Decision Tree Classifier as our model to predict which team won the game."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using only our new attributes, whether each team won their last match, we get an F1 Score of:\n",
      "F1 Score: 0.4407\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Kevin\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1074: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "# Note: Cross validation score uses multiple folds\n",
    "from sklearn.cross_validation import cross_val_score\n",
    "\n",
    "# Create a new dataframe with just the necessary features\n",
    "X_previousWins = results[[\"HomeLastWin\", \"VisitorLastWin\"]].values\n",
    "\n",
    "# The object, tree_clf, has the Decision Tree Classifier ready\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "# random_state is for reproducibility if you would like to follow along\n",
    "tree_clf = DecisionTreeClassifier(random_state = 10)\n",
    "\n",
    "# Compute our F1 Score\n",
    "scores = cross_val_score(tree_clf, X_previousWins, y_true, scoring=scorer)\n",
    "print(\"Using only our new attributes, whether each team won their last match, we get an F1 Score of:\")\n",
    "print(\"F1 Score: {0:.4f}\".format(np.mean(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Our F1 Score has improved significantly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "scrolled": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>F1 Score</th>\n",
       "      <th>Overall Performance Boost</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Baseline</th>\n",
       "      <td>0.3728</td>\n",
       "      <td>0.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Last Win</th>\n",
       "      <td>0.4407</td>\n",
       "      <td>0.0679</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          F1 Score  Overall Performance Boost\n",
       "Baseline    0.3728                     0.0000\n",
       "Last Win    0.4407                     0.0679"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(index=['Baseline', 'Last Win'],data=[(0.3728, 0),(0.4407, 0.4407-0.3728)], \\\n",
    "                  columns=['F1 Score', 'Overall Performance Boost'])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## New Feature: Win Streaks\n",
    "\n",
    "Similar to before, having a win streak can easily boost morale where a lose streak can cause ripples/ problems within the team leading to poor performance for each successive game.\n",
    "\n",
    "Can we improve our model by incorporating win streaks?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Visitor Team</th>\n",
       "      <th>Visitor Goals</th>\n",
       "      <th>Home Team</th>\n",
       "      <th>Home Goals</th>\n",
       "      <th>HomeWin</th>\n",
       "      <th>HomeLastWin</th>\n",
       "      <th>VisitorLastWin</th>\n",
       "      <th>HomeWinStreak</th>\n",
       "      <th>VisitorWinStreak</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Columbus Blue Jackets</td>\n",
       "      <td>3.0</td>\n",
       "      <td>New York Islanders</td>\n",
       "      <td>2.0</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Buffalo Sabres</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Pittsburgh Penguins</td>\n",
       "      <td>4.0</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Arizona Coyotes</td>\n",
       "      <td>1.0</td>\n",
       "      <td>San Jose Sharks</td>\n",
       "      <td>4.0</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Florida Panthers</td>\n",
       "      <td>0.0</td>\n",
       "      <td>St. Louis Blues</td>\n",
       "      <td>7.0</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Ottawa Senators</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Toronto Maple Leafs</td>\n",
       "      <td>5.0</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>2013-10-05</td>\n",
       "      <td>Edmonton Oilers</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Vancouver Canucks</td>\n",
       "      <td>6.0</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date           Visitor Team  Visitor Goals            Home Team  \\\n",
       "25 2013-10-05  Columbus Blue Jackets            3.0   New York Islanders   \n",
       "26 2013-10-05         Buffalo Sabres            1.0  Pittsburgh Penguins   \n",
       "27 2013-10-05        Arizona Coyotes            1.0      San Jose Sharks   \n",
       "28 2013-10-05       Florida Panthers            0.0      St. Louis Blues   \n",
       "29 2013-10-05        Ottawa Senators            4.0  Toronto Maple Leafs   \n",
       "30 2013-10-05        Edmonton Oilers            2.0    Vancouver Canucks   \n",
       "\n",
       "    Home Goals HomeWin HomeLastWin VisitorLastWin  HomeWinStreak  \\\n",
       "25         2.0   False        True          False              1   \n",
       "26         4.0    True        True          False              1   \n",
       "27         4.0    True        True           True              1   \n",
       "28         7.0    True        True           True              1   \n",
       "29         5.0    True        True           True              2   \n",
       "30         6.0    True       False          False              0   \n",
       "\n",
       "    VisitorWinStreak  \n",
       "25                 0  \n",
       "26                 0  \n",
       "27                 1  \n",
       "28                 1  \n",
       "29                 1  \n",
       "30                 0  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Set our new features with a default value of 0\n",
    "results[\"HomeWinStreak\"] = 0\n",
    "results[\"VisitorWinStreak\"] = 0\n",
    "\n",
    "from collections import defaultdict\n",
    "win_streak = defaultdict(int)\n",
    "\n",
    "for index, row in results.iterrows(): # Similar to before\n",
    "    home_team = row[\"Home Team\"]\n",
    "    visitor_team = row[\"Visitor Team\"]\n",
    "    row[\"HomeWinStreak\"] = win_streak[home_team]\n",
    "    row[\"VisitorWinStreak\"] = win_streak[visitor_team]\n",
    "    results.ix[index] = row\n",
    "    \n",
    "    if row[\"HomeWin\"]:\n",
    "        win_streak[home_team] += 1 # if \"HomeWin\" is True, increase home_team by 1\n",
    "        win_streak[visitor_team] = 0\n",
    "    else:\n",
    "        win_streak[home_team] = 0\n",
    "        win_streak[visitor_team] += 1 # if \"HomeWin\" is False, increase visitor_team by 1\n",
    "\n",
    "results.ix[25:30]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "By adding win streaks, our new F1 score is:\n",
      "F1 Score: 0.4940\n"
     ]
    }
   ],
   "source": [
    "# Lets run the decision tree classifier again to see if the F1 Score has improved\n",
    "tree_clf = DecisionTreeClassifier(random_state = 10)\n",
    "X_winstreak = results[[\"HomeLastWin\", \"VisitorLastWin\", \"HomeWinStreak\", \"VisitorWinStreak\"]].values\n",
    "scores = cross_val_score(tree_clf, X_winstreak, y_true, scoring=scorer)\n",
    "print(\"By adding win streaks, our new F1 score is:\")\n",
    "print(\"F1 Score: {0:.4f}\".format(np.mean(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Our F1 Score has improved again albeit at a lower rate.\n",
    "\n",
    "**Note:** The Overall Performance Boost for \"Last Win\" is the difference between \"Last Win\" and \"Baseline\" whereas the Overall Performance Boost for successive tests is a comparison with \"Last Win\" and not Baseline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>F1 Score</th>\n",
       "      <th>Overall Performance Boost</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Baseline</th>\n",
       "      <td>0.3728</td>\n",
       "      <td>0.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Last Win</th>\n",
       "      <td>0.4407</td>\n",
       "      <td>0.0679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Win Streaks &amp; Last Win</th>\n",
       "      <td>0.4940</td>\n",
       "      <td>0.0533</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        F1 Score  Overall Performance Boost\n",
       "Baseline                  0.3728                     0.0000\n",
       "Last Win                  0.4407                     0.0679\n",
       "Win Streaks & Last Win    0.4940                     0.0533"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = pd.DataFrame(index=['Win Streaks & Last Win'], data=[(0.4940, 0.4940-0.4407)], \\\n",
    "                   columns=['F1 Score', 'Overall Performance Boost'])\n",
    "df = df.append(df2)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "So far, all our features have been based on the same dataset. The more we build on the same dataset, the more we risk overfitting our data and not being able to predict instances when using unseen or new data\n",
    "\n",
    "One dataset we can consider is the standings of all the teams from the previous year."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Avoiding Overfitting: Team Rankings from the previous Season"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "If a team placed high last season, their momentum/ spirit is most likely to be very high. Imagine a team that was essentially in last place that lost every game and was put up against the team that had won last season in a landslide, would the losing team be nervous about their upcoming game? This information could potentially be a strong indicator in determining the outcome of a game.\n",
    "\n",
    "Let us create a new feature that checks whether the home team was ranked higher than the visitor team from the last season."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "scrolled": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Rk</th>\n",
       "      <th>Overall</th>\n",
       "      <th>Home</th>\n",
       "      <th>Road</th>\n",
       "      <th>EAS</th>\n",
       "      <th>WES</th>\n",
       "      <th>ATL</th>\n",
       "      <th>NEA</th>\n",
       "      <th>SEA</th>\n",
       "      <th>CEN</th>\n",
       "      <th>NWE</th>\n",
       "      <th>PAC</th>\n",
       "      <th>≤1</th>\n",
       "      <th>≥3</th>\n",
       "      <th>Jan</th>\n",
       "      <th>Feb</th>\n",
       "      <th>Mar</th>\n",
       "      <th>Apr</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Team</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Chicago Blackhawks</th>\n",
       "      <td>1</td>\n",
       "      <td>36-7-5</td>\n",
       "      <td>18-3-3</td>\n",
       "      <td>18-4-2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>36-7-5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16-1-1</td>\n",
       "      <td>10-3-2</td>\n",
       "      <td>10-3-2</td>\n",
       "      <td>19-3-5</td>\n",
       "      <td>10-1-0</td>\n",
       "      <td>6-0-1</td>\n",
       "      <td>11-0-2</td>\n",
       "      <td>9-5-0</td>\n",
       "      <td>10-2-2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pittsburgh Penguins</th>\n",
       "      <td>2</td>\n",
       "      <td>36-12-0</td>\n",
       "      <td>18-6-0</td>\n",
       "      <td>18-6-0</td>\n",
       "      <td>36-12-0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12-6-0</td>\n",
       "      <td>12-3-0</td>\n",
       "      <td>12-3-0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13-2-0</td>\n",
       "      <td>10-5-0</td>\n",
       "      <td>4-3-0</td>\n",
       "      <td>9-5-0</td>\n",
       "      <td>15-0-0</td>\n",
       "      <td>8-4-0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Anaheim Ducks</th>\n",
       "      <td>3</td>\n",
       "      <td>30-12-6</td>\n",
       "      <td>16-7-1</td>\n",
       "      <td>14-5-5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30-12-6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10-2-3</td>\n",
       "      <td>12-3-0</td>\n",
       "      <td>8-7-3</td>\n",
       "      <td>14-2-6</td>\n",
       "      <td>9-5-0</td>\n",
       "      <td>3-1-1</td>\n",
       "      <td>11-2-0</td>\n",
       "      <td>9-4-4</td>\n",
       "      <td>7-5-1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Montreal Canadiens</th>\n",
       "      <td>4</td>\n",
       "      <td>29-14-5</td>\n",
       "      <td>14-7-3</td>\n",
       "      <td>15-7-2</td>\n",
       "      <td>29-14-5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7-6-2</td>\n",
       "      <td>9-6-3</td>\n",
       "      <td>13-2-0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12-6-5</td>\n",
       "      <td>14-6-0</td>\n",
       "      <td>4-2-0</td>\n",
       "      <td>9-2-3</td>\n",
       "      <td>9-3-2</td>\n",
       "      <td>7-7-0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Boston Bruins</th>\n",
       "      <td>5</td>\n",
       "      <td>28-14-6</td>\n",
       "      <td>16-5-3</td>\n",
       "      <td>12-9-3</td>\n",
       "      <td>28-14-6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7-6-2</td>\n",
       "      <td>10-6-2</td>\n",
       "      <td>11-2-2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13-7-6</td>\n",
       "      <td>7-2-0</td>\n",
       "      <td>5-1-1</td>\n",
       "      <td>8-1-1</td>\n",
       "      <td>9-6-2</td>\n",
       "      <td>6-6-2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     Rk  Overall    Home    Road      EAS      WES     ATL  \\\n",
       "Team                                                                         \n",
       "Chicago Blackhawks    1   36-7-5  18-3-3  18-4-2      NaN   36-7-5     NaN   \n",
       "Pittsburgh Penguins   2  36-12-0  18-6-0  18-6-0  36-12-0      NaN  12-6-0   \n",
       "Anaheim Ducks         3  30-12-6  16-7-1  14-5-5      NaN  30-12-6     NaN   \n",
       "Montreal Canadiens    4  29-14-5  14-7-3  15-7-2  29-14-5      NaN   7-6-2   \n",
       "Boston Bruins         5  28-14-6  16-5-3  12-9-3  28-14-6      NaN   7-6-2   \n",
       "\n",
       "                        NEA     SEA     CEN     NWE     PAC      ≤1      ≥3  \\\n",
       "Team                                                                          \n",
       "Chicago Blackhawks      NaN     NaN  16-1-1  10-3-2  10-3-2  19-3-5  10-1-0   \n",
       "Pittsburgh Penguins  12-3-0  12-3-0     NaN     NaN     NaN  13-2-0  10-5-0   \n",
       "Anaheim Ducks           NaN     NaN  10-2-3  12-3-0   8-7-3  14-2-6   9-5-0   \n",
       "Montreal Canadiens    9-6-3  13-2-0     NaN     NaN     NaN  12-6-5  14-6-0   \n",
       "Boston Bruins        10-6-2  11-2-2     NaN     NaN     NaN  13-7-6   7-2-0   \n",
       "\n",
       "                       Jan     Feb     Mar     Apr  \n",
       "Team                                                \n",
       "Chicago Blackhawks   6-0-1  11-0-2   9-5-0  10-2-2  \n",
       "Pittsburgh Penguins  4-3-0   9-5-0  15-0-0   8-4-0  \n",
       "Anaheim Ducks        3-1-1  11-2-0   9-4-4   7-5-1  \n",
       "Montreal Canadiens   4-2-0   9-2-3   9-3-2   7-7-0  \n",
       "Boston Bruins        5-1-1   8-1-1   9-6-2   6-6-2  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import data\n",
    "ladders_filename = ('leagues_NHL_2013_standings_standings.csv')\n",
    "ladder = pd.read_csv(ladders_filename, skiprows = [0,1], index_col=\"Team\")\n",
    "ladder[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Visitor Team</th>\n",
       "      <th>Visitor Goals</th>\n",
       "      <th>Home Team</th>\n",
       "      <th>Home Goals</th>\n",
       "      <th>HomeWin</th>\n",
       "      <th>HomeTeamRanksHigher</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Washington Capitals</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Chicago Blackhawks</td>\n",
       "      <td>6.0</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Winnipeg Jets</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Edmonton Oilers</td>\n",
       "      <td>4.0</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2013-10-01</td>\n",
       "      <td>Toronto Maple Leafs</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Montreal Canadiens</td>\n",
       "      <td>3.0</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2013-10-02</td>\n",
       "      <td>Anaheim Ducks</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Colorado Avalanche</td>\n",
       "      <td>6.0</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2013-10-02</td>\n",
       "      <td>Buffalo Sabres</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Detroit Red Wings</td>\n",
       "      <td>2.0</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Date         Visitor Team  Visitor Goals           Home Team  \\\n",
       "0 2013-10-01  Washington Capitals            4.0  Chicago Blackhawks   \n",
       "1 2013-10-01        Winnipeg Jets            5.0     Edmonton Oilers   \n",
       "2 2013-10-01  Toronto Maple Leafs            4.0  Montreal Canadiens   \n",
       "3 2013-10-02        Anaheim Ducks            1.0  Colorado Avalanche   \n",
       "4 2013-10-02       Buffalo Sabres            1.0   Detroit Red Wings   \n",
       "\n",
       "   Home Goals HomeWin HomeTeamRanksHigher  \n",
       "0         6.0    True                True  \n",
       "1         4.0   False               False  \n",
       "2         3.0   False                True  \n",
       "3         6.0    True               False  \n",
       "4         2.0    True                True  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create our new feature - HomeTeamRanksHigher\n",
    "\n",
    "def home_team_ranks_higher(row):\n",
    "    home_team = row[\"Home Team\"]\n",
    "    visitor_team = row[\"Visitor Team\"]\n",
    "    \n",
    "    # Need to address the team name changes\n",
    "    if home_team == \"Arizona Coyotes\":\n",
    "        home_team = \"Phoenix Coyotes\"\n",
    "    if visitor_team == \"Arizona Coyotes\":\n",
    "        visitor_team = \"Phoenix Coyotes\"\n",
    "    \n",
    "    home_rank = ladder.loc[home_team][\"Rk\"]\n",
    "    visitor_rank = ladder.loc[visitor_team][\"Rk\"]\n",
    "    return home_rank < visitor_rank # Remember, a higher ranking is a lower number (in other words, 1st place > 2nd place)\n",
    "\n",
    "# This will return a boolean result, either True or False and the line below will take whatever this boolean result is\n",
    "# and place it into \"HomeTeamRanksHigher\"\n",
    "results[\"HomeTeamRanksHigher\"] = results.apply(home_team_ranks_higher, axis=1)\n",
    "results[:5][['Date', 'Visitor Team', 'Visitor Goals', 'Home Team', 'Home Goals', 'HomeWin', 'HomeTeamRanksHigher']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "scrolled": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "By considering last seasons' rankings, the new F1 score is:\n",
      "F1: 0.5488\n"
     ]
    }
   ],
   "source": [
    "# Lets build our Decision Tree Classifier to see if there are any improvements\n",
    "X_homehigher = results[[\"HomeLastWin\", \"VisitorLastWin\", \"HomeTeamRanksHigher\"]].values\n",
    "\n",
    "tree_clf = DecisionTreeClassifier(random_state=10)\n",
    "scores = cross_val_score(tree_clf, X_homehigher, y_true, scoring=scorer)\n",
    "print(\"By considering last seasons' rankings, the new F1 score is:\")\n",
    "print(\"F1: {0:.4f}\".format(np.mean(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "When compared to win streaks, a teams' ranking from last season provides a much larger boost in performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>F1 Score</th>\n",
       "      <th>Overall Performance Boost</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Baseline</th>\n",
       "      <td>0.3728</td>\n",
       "      <td>0.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Last Win</th>\n",
       "      <td>0.4407</td>\n",
       "      <td>0.0679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Win Streaks &amp; Last Win</th>\n",
       "      <td>0.4940</td>\n",
       "      <td>0.0533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Team Rankings &amp; Last Win</th>\n",
       "      <td>0.5488</td>\n",
       "      <td>0.1081</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          F1 Score  Overall Performance Boost\n",
       "Baseline                    0.3728                     0.0000\n",
       "Last Win                    0.4407                     0.0679\n",
       "Win Streaks & Last Win      0.4940                     0.0533\n",
       "Team Rankings & Last Win    0.5488                     0.1081"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = pd.DataFrame(index=['Team Rankings & Last Win'], data=[(0.5488,0.5488-0.4407)], \\\n",
    "                   columns=['F1 Score', 'Overall Performance Boost'])\n",
    "df = df.append(df2)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Moving away from default parameters\n",
    "\n",
    "So far we've been using the default parameters when using DecisionTreeClassifier. Could we potentially find a better model if we tweak the parameters?\n",
    "\n",
    "- **GridSearchCV** is a great function in Scikit-Learn that allows you to quickly try many different combinations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "F1 Score: 0.5665\n"
     ]
    }
   ],
   "source": [
    "from sklearn.grid_search import GridSearchCV\n",
    "i = list(range(1,21))\n",
    "\n",
    "parameter_space = {\"max_depth\": i}\n",
    "\n",
    "tree_clf = DecisionTreeClassifier(random_state=10)\n",
    "grid = GridSearchCV(tree_clf, parameter_space, scoring=scorer)\n",
    "grid.fit(X_homehigher, y_true)\n",
    "print(\"F1 Score: {0:.4f}\".format(grid.best_score_))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "There seems to be a parameter setting that is better than the default! We will use GridSearchCV again later on down to find our best model once we have defined all our features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## New Feature: Did the home team win against the visiting team last game?\n",
    "\n",
    "- Our previous feature, LastWin, only looked at the teams' literal last game they played (so the previous opponent would have been different from the current visiting team)\n",
    "- This feature looks at if the home team won the LAST game against the current visiting team, this last match could easily have happened several games ago"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "last_match_winner = defaultdict(int)\n",
    "\n",
    "def home_team_won_last(row):\n",
    "    home_team = row[\"Home Team\"]\n",
    "    visitor_team = row[\"Visitor Team\"]\n",
    "    \n",
    "    # Sort team names for a consistent ordering (sorts by alphabetical order)\n",
    "    teams = tuple(sorted([home_team, visitor_team]))\n",
    "    \n",
    "    # If the home team won the previous game, the \"result\" will be 1\n",
    "    result = 1 if last_match_winner[teams] == row[\"Home Team\"] else 0\n",
    "    \n",
    "    # Update record for next encounter\n",
    "    winner = row[\"Home Team\"] if row[\"HomeWin\"] else row[\"Visitor Team\"]\n",
    "    \n",
    "    last_match_winner[teams] = winner\n",
    "    return result\n",
    "\n",
    "results[\"HomeTeamWonLast\"] = results.apply(home_team_won_last, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Visitor Team</th>\n",
       "      <th>Visitor Goals</th>\n",
       "      <th>Home Team</th>\n",
       "      <th>Home Goals</th>\n",
       "      <th>HomeWin</th>\n",
       "      <th>HomeTeamWonLast</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>2013-10-24</td>\n",
       "      <td>Winnipeg Jets</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Nashville Predators</td>\n",
       "      <td>3.0</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>2013-10-24</td>\n",
       "      <td>New York Rangers</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Philadelphia Flyers</td>\n",
       "      <td>2.0</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>2013-10-24</td>\n",
       "      <td>Chicago Blackhawks</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Tampa Bay Lightning</td>\n",
       "      <td>6.0</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>2013-10-25</td>\n",
       "      <td>Toronto Maple Leafs</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Columbus Blue Jackets</td>\n",
       "      <td>5.0</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>2013-10-25</td>\n",
       "      <td>Carolina Hurricanes</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Colorado Avalanche</td>\n",
       "      <td>4.0</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Date         Visitor Team  Visitor Goals              Home Team  \\\n",
       "145 2013-10-24        Winnipeg Jets            2.0    Nashville Predators   \n",
       "146 2013-10-24     New York Rangers            1.0    Philadelphia Flyers   \n",
       "147 2013-10-24   Chicago Blackhawks            5.0    Tampa Bay Lightning   \n",
       "148 2013-10-25  Toronto Maple Leafs            2.0  Columbus Blue Jackets   \n",
       "149 2013-10-25  Carolina Hurricanes            2.0     Colorado Avalanche   \n",
       "\n",
       "     Home Goals HomeWin  HomeTeamWonLast  \n",
       "145         3.0    True                1  \n",
       "146         2.0    True                0  \n",
       "147         6.0    True                1  \n",
       "148         5.0    True                0  \n",
       "149         4.0    True                0  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results[145:150][['Date', 'Visitor Team', 'Visitor Goals', 'Home Team', 'Home Goals', 'HomeWin', 'HomeTeamWonLast']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "By considering if the home team won against the visiting team last game, we get a new F1 score of:\n",
      "F1 Score: 0.5351\n"
     ]
    }
   ],
   "source": [
    "# Let us calculate the F1 Score now\n",
    "X_last_win = results[[\"HomeLastWin\", \"VisitorLastWin\", \"HomeTeamRanksHigher\", \"HomeTeamWonLast\"]].values\n",
    "\n",
    "tree_clf = DecisionTreeClassifier(random_state=10)\n",
    "scores = cross_val_score(tree_clf, X_last_win, y_true, scoring=scorer)\n",
    "print(\"By considering if the home team won against the visiting team last game, we get a new F1 score of:\")\n",
    "print(\"F1 Score: {0:.4f}\".format(np.mean(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The performance fell as compared to our previous model however the fall in performance is very small. This could be potentially due to noise or other randomness factors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>F1 Score</th>\n",
       "      <th>Overall Performance Boost</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Baseline</th>\n",
       "      <td>0.3728</td>\n",
       "      <td>0.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Last Win</th>\n",
       "      <td>0.4407</td>\n",
       "      <td>0.0679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Win Streaks &amp; Last Win</th>\n",
       "      <td>0.4940</td>\n",
       "      <td>0.0533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Team Rankings &amp; Last Win</th>\n",
       "      <td>0.5488</td>\n",
       "      <td>0.1081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Won Last, Team Rankings, &amp; Last Win</th>\n",
       "      <td>0.5351</td>\n",
       "      <td>0.0944</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     F1 Score  Overall Performance Boost\n",
       "Baseline                               0.3728                     0.0000\n",
       "Last Win                               0.4407                     0.0679\n",
       "Win Streaks & Last Win                 0.4940                     0.0533\n",
       "Team Rankings & Last Win               0.5488                     0.1081\n",
       "Won Last, Team Rankings, & Last Win    0.5351                     0.0944"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = pd.DataFrame(index=[\"Won Last, Team Rankings, & Last Win\"], data=[(0.5351,0.5351-0.4407)], \\\n",
    "                  columns=[\"F1 Score\", \"Overall Performance Boost\"])\n",
    "df = df.append(df2)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Summary of Features and their Performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x280f6ad1d30>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm0AAAG4CAYAAAAXENC/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtU1NXi///XcAtIw0TPScsMcHnJNMELGCRpdjxFkqSE\nqZ0yb2Fm2dFTaSlmnazM7Ha0sIsVZeXtk/X5VX76FGWJSiZ1hD6mkPdbYHgbHC77+4er+TkHbDAH\nacPzsZZrMW8379m8NzM8ec8M4zDGGAEAAOAPza++JwAAAADviDYAAAALEG0AAAAWINoAAAAsQLQB\nAABYgGgDAACwQK2iLT8/X6mpqYqOjlZKSory8vJqHJebm6sbb7xR0dHRSk5OVk5Ojk8nCwAA0Fh5\njTaXy6X09HQNGTJEubm5GjFihNLT0+V0Oj3G7d+/X+PHj9f48eP17bffaty4cZo4caJcLledTR4A\nAKCx8BptOTk58vf3V1pamvz9/TV48GCFh4crOzvbY9yKFSsUHx+v/v37S5KSkpK0aNEiORyOupk5\nAABAI+I12goLCxUVFeWxLSIiQoWFhR7b8vPz9ac//UkTJkxQbGyshg4dqvLycgUGBvp2xgAAAI2Q\n12hzOp0KCQnx2BYSEqKysjKPbaWlpXrvvfc0fPhwff3110pOTta4ceN0+PBh384YAACgEfIabTUF\nmtPpVGhoqMe2oKAgJSYmqnfv3vL399ewYcMUGhqqDRs2+HbGAAAAjZDXaIuMjFRRUZHHtqKiIrVr\n185jW0RERLUXHVRVVam270fP+9YDAACcWoC3AXFxcXK5XMrKylJaWppWrFihkpISJSQkeIy74YYb\nNHToUGVnZ6tPnz5688035XK5FBsbW6uJFBcflR9/NQ4AADRyzZs3qXG7w9TiFNfmzZs1ffp0/fjj\nj2rbtq0yMjLUtWtXzZgxQw6HQxkZGZKkr7/+Wk8++aS2b9+uSy65RBkZGerSpUutJnjgAM99AwAA\naNmyaY3baxVtZwPRBgAAcOpo4wFJAAAACxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsADRBgAAYAGi\nDQAAwAJEGwAAgAWINgAAAAsQbQAAABYg2gAAACxAtAEAAFiAaAMAALAA0QYAAGABog0AAMACRBsA\nAIAFiDYAAAALEG0AAAAWINoAAAAsQLQBAABYgGgDAACwANEGAABgAaINAADAAkQbAACABYg2AAAA\nCxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsEBAfU8AAPDHVV5eroKCTfU9jUanU6fOCgwMrO9p4A+G\naAMAnFJBwSa9+Ox7atniwvqeSqNx4OddGjdR6tq1W31PBX8wRBsA4De1bHGhLrwgsr6nATR6PKcN\nAADAAkQbAACABYg2AAAACxBtAAAAFiDaAAAALEC0AQAAWIA/+QEAQCPBH0uuH776Y8lEGwAAjURB\nwSZ9/v89rbYXtazvqTQa23YekDTJJ38smWgDAKARaXtRS7Vv16q+p4Hfgee0AQAAWIBoAwAAsADR\nBgAAYAGiDQAAwAJEGwAAgAWINgAAAAsQbQAAABYg2gAAACxAtAEAAFiAaAMAALAA0QYAAGABog0A\nAMACRBsAAIAFiDYAAAALEG0AAAAWINoAAAAsUKtoy8/PV2pqqqKjo5WSkqK8vLwax40bN06XX365\nYmJiFB0drZiYGJ9OFgAAoLHyGm0ul0vp6ekaMmSIcnNzNWLECKWnp8vpdFYbW1BQoLffflsbNmzQ\nt99+qw0bNtTJpAEAABobr9GWk5Mjf39/paWlyd/fX4MHD1Z4eLiys7M9xpWUlKikpETt2rWrs8kC\nAAA0Vl6jrbCwUFFRUR7bIiIiVFhY6LEtPz9f5557rsaNG6fevXtr2LBh2rhxo29nCwAA0EgFeBvg\ndDoVEhLisS0kJERlZWUe244fP67o6GhNmTJFF198sZYsWaIxY8boo48+Unh4uNeJOBwO+fGyCAD4\nQ/Hzc9T3FBolPz+H/P19f+xZz/rhq/X0Gm01BZrT6VRoaKjHtquvvlpXX321+/LNN9+st956S2vX\nrtV1113ndSLh4efK4eCbCQD+SMLCQr0Pgs+FhYWqefMmdbJfnH2+Wk+v0RYZGamsrCyPbUVFRUpO\nTvbY9vHHH6uqqkrXXnute5vL5VJQUFCtJlJcfJQzbQDwB1Naeqy+p9AolZYeU0nJkTrZL86+013P\nUwWe12iLi4uTy+VSVlaW0tLStGLFCpWUlCghIcFj3LFjx/TUU0+pffv2atu2rV577TUdP3682rhT\nMcaosrJWQwEAZ0lVlanvKTRKVVVGlZW+P/asZ/3w1Xp6jbagoCBlZmZq+vTpmjt3rtq2bav58+cr\nODhYM2bMkMPhUEZGhlJSUnTgwAGNHj1av/zyizp37qzMzEwFBwef8SQB2KO8vFwFBZvqexqNTqdO\nnRUYGFjf0wBQh7xGmyS1b99eixcvrrZ95syZHpfHjh2rsWPH+mZmAKxUULBJH82YpjZhzep7Ko3G\njtJfpJmPqmvXbvU9FQB1qFbRBgCno01YM7WrxavGAQC1x1P/AQAALEC0AQAAWIBoAwAAsADRBgAA\nYAGiDQAAwAJEGwAAgAWINgAAAAsQbQAAABYg2gAAACxAtAEAAFiAaAMAALAA0QYAAGABog0AAMAC\nRBsAAIAFiDYAAAALEG0AAAAWINoAAAAsQLQBAABYgGgDAACwANEGAABgAaINAADAAkQbAACABYg2\nAAAACxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsADRBgAAYAGiDQAAwAJEGwAAgAWINgAAAAsQbQAA\nABYg2gAAACwQUN8TAMrLy1VQsKm+p9HodOrUWYGBgfU9DQBALRFtqHcFBZs0851HFNa6eX1PpdEo\n3V2iGWkPqmvXbvU9FQBALRFt+EMIa91c4W3/VN/TAADgD4vntAEAAFiAaAMAALAA0QYAAGABog0A\nAMACRBsAAIAFiDYAAAALEG0AAAAWINoAAAAsQLQBAABYgGgDAACwANEGAABgAaINAADAAkQbAACA\nBYg2AAAACxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsADRBgAAYIFaRVt+fr5SU1MVHR2tlJQU5eXl\n/eb4NWvWqFOnTnI6nT6ZJAAAQGPnNdpcLpfS09M1ZMgQ5ebmasSIEUpPTz9lkB06dEjTpk3z+UQB\nAAAaM6/RlpOTI39/f6Wlpcnf31+DBw9WeHi4srOzaxyfkZGhpKQkn08UAACgMfMabYWFhYqKivLY\nFhERocLCwmpj33//fR0+fFhDhw6VMcZ3swQAAGjkArwNcDqdCgkJ8dgWEhKisrIyj227d+/Wc889\np7ffflvHjx+Xw+Hw7UwBAAAaMa/RVlOgOZ1OhYaGui8bY3T//fdr0qRJatGihXbu3OneXlsOh0N+\nvJa1UfLzI/Drg5+fQ/7+vj/2rGf9YD0bFtazYfHVenqNtsjISGVlZXlsKyoqUnJysvvy3r179d13\n3+mHH35QRkaGqqqqZIzRVVddpQULFigmJsbrRMLDz+XsXCMVFhbqfRB8LiwsVM2bN6mT/eLsYz0b\nFtazYfHVenqNtri4OLlcLmVlZSktLU0rVqxQSUmJEhIS3GNatWqljRs3ui/v2rVLV199tb744gsF\nBwfXaiLFxUc509ZIlZYeq+8pNEqlpcdUUnKkTvaLs4/1bFhYz4bldNfzVIHnNdqCgoKUmZmp6dOn\na+7cuWrbtq3mz5+v4OBgzZgxQw6HQxkZGdU+z+FwnNbDo8YYVVbWejgakKoqXrRSH6qqjCorfX/s\nWc/6wXo2LKxnw+Kr9fQabZLUvn17LV68uNr2mTNn1jj+wgsvVEFBwZnNDAAAAG48IAkAAGABog0A\nAMACRBsAAIAFiDYAAAALEG0AAAAWINoAAAAsQLQBAABYgGgDAACwANEGAABgAaINAADAAkQbAACA\nBYg2AAAACxBtAAAAFgio7wn8HuXl5Soo2FTf02h0OnXqrMDAwPqeBgAAjZKV0VZQsEkPPf22zgtv\nXd9TaTQOFe/WrEk3q2vXbvU9FQAAGiUro02SzgtvreZ/jqjvaQAAAJwVPKcNAADAAkQbAACABYg2\nAAAACxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsADRBgAAYAGiDQAAwAJEGwAAgAWINgAAAAsQbQAA\nABYg2gAAACxAtAEAAFiAaAMAALAA0QYAAGABog0AAMACRBsAAIAFiDYAAAALEG0AAAAWINoAAAAs\nQLQBAABYgGgDAACwANEGAABgAaINAADAAkQbAACABYg2AAAACxBtAAAAFiDaAAAALEC0AQAAWIBo\nAwAAsADRBgAAYAGiDQAAwAJEGwAAgAWINgAAAAsQbQAAABYg2gAAACxAtAEAAFiAaAMAALAA0QYA\nAGABog0AAMACtYq2/Px8paamKjo6WikpKcrLy6tx3DPPPKMrr7xS3bt316233qotW7b4dLIAAACN\nlddoc7lcSk9P15AhQ5Sbm6sRI0YoPT1dTqfTY9x7772nVatWadmyZfrmm2/UvXt3/eMf/6iziQMA\nADQmXqMtJydH/v7+SktLk7+/vwYPHqzw8HBlZ2d7jEtNTdWSJUvUsmVLHTlyRIcOHVLz5s3rbOIA\nAACNSYC3AYWFhYqKivLYFhERocLCwmpjg4ODtXz5ck2dOlVNmzbVK6+84ruZAgAANGJeo83pdCok\nJMRjW0hIiMrKymocf/3112vgwIF6/fXXNWrUKK1atUrnnXee14k4HA751fJlEX5+jtoNhE/5+Tnk\n7+/7Y8961g/Ws2FhPRsW1rNh8dV6eo22mgLN6XQqNDS0xvGBgYGSpNtvv11vvvmm1q1bp/79+3ud\nSHj4uXI4avcFhYXVfN2oW2FhoWrevEmd7BdnH+vZsLCeDQvr2bD4aj29RltkZKSysrI8thUVFSk5\nOdlj23PPPaeKigpNmjTJva28vFxNmzat1USKi4/W+kxbaemx2g2ET5WWHlNJyZE62S/OPtazYWE9\nGxbWs2E53fU8VeB5jba4uDi5XC5lZWUpLS1NK1asUElJiRISEjzGXX755ZoyZYqSkpIUERGhBQsW\nqGnTpoqOjq7VBI0xqqys1VBVVZnaDYRPVVUZVVb6/tiznvWD9WxYWM+GhfVsWHy1nl7PbQUFBSkz\nM1MrV65UbGys3nrrLc2fP1/BwcGaMWOGMjIyJEl9+vTR3//+d40fP14JCQnatGmTFi5cqKCgoDOe\nJAAAQGPn9UybJLVv316LFy+utn3mzJkel2+66SbddNNNvpkZAAAA3HgbKwAAAAsQbQAAABYg2gAA\nACxAtAEAAFiAaAMAALAA0QYAAGABog0AAMACRBsAAIAFiDYAAAALEG0AAAAWINoAAAAsQLQBAABY\ngGgDAACwANEGAABgAaINAADAAkQbAACABYg2AAAACxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsADR\nBgAAYAGiDQAAwAJEGwAAgAWINgAAAAsQbQAAABYg2gAAACxAtAEAAFiAaAMAALAA0QYAAGABog0A\nAMACRBsAAIAFiDYAAAALEG0AAAAWINoAAAAsQLQBAABYgGgDAACwANEGAABgAaINAADAAkQbAACA\nBYg2AAAACxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsADRBgAAYAGiDQAAwAJEGwAAgAWINgAAAAsQ\nbQAAABYg2gAAACxAtAEAAFiAaAMAALAA0QYAAGABog0AAMACRBsAAIAFahVt+fn5Sk1NVXR0tFJS\nUpSXl1fjuHfffVcDBgxQjx49lJqaqtzcXJ9OFgAAoLHyGm0ul0vp6ekaMmSIcnNzNWLECKWnp8vp\ndHqMW7t2rZ5++mk9++yzys3N1fDhw5Wenq7S0tI6mzwAAEBj4TXacnJy5O/vr7S0NPn7+2vw4MEK\nDw9Xdna2x7i9e/dq9OjR6tChgyRp0KBB8vPz048//lg3MwcAAGhEArwNKCwsVFRUlMe2iIgIFRYW\nemy74YYbPC5/8803OnbsmNq1a+eDaQIAADRuXs+0OZ1OhYSEeGwLCQlRWVnZKT9ny5Ytuvvuu3X3\n3XerWbNmZz5LAACARs7rmbaaAs3pdCo0NLTG8atXr9a9996rUaNGafTo0bWeiMPhkF8tX8vq5+eo\n9X7hO35+Dvn7+/7Ys571g/VsWFjPhoX1bFh8tZ5eoy0yMlJZWVke24qKipScnFxt7NKlS/XYY4/p\n4Ycf1nXXXXdaEwkPP1cOR+2+oLCwmoMRdSssLFTNmzepk/3i7GM9GxbWs2FhPRsWX62n12iLi4uT\ny+VSVlaW0tLStGLFCpWUlCghIcFj3Jo1a/Twww/rlVdeUffu3U97IsXFR2t9pq209Nhp7x9nrrT0\nmEpKjtTJfnH2sZ4NC+vZsLCeDcvpruepAs9rtAUFBSkzM1PTp0/X3Llz1bZtW82fP1/BwcGaMWOG\nHA6HMjIytHDhQlVUVGjMmDGSJGOMHA6Hnn322WqBVxNjjCora/fFVFWZ2g2ET1VVGVVW+v7Ys571\ng/VsWFjPhoX1bFh8tZ5eo02S2rdvr8WLF1fbPnPmTPfHL7/88hlPBgAAADXjbawAAAAsQLQBAABY\ngGgDAACwANEGAABgAaINAADAAkQbAACABYg2AAAACxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsADR\nBgAAYAGiDQAAwAJEGwAAgAWINgAAAAsQbQAAABYg2gAAACxAtAEAAFiAaAMAALAA0QYAAGABog0A\nAMACRBsAAIAFiDYAAAALEG0AAAAWINoAAAAsQLQBAABYgGgDAACwANEGAABgAaINAADAAkQbAACA\nBYg2AAAACxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsADRBgAAYAGiDQAAwAJEGwAAgAWINgAAAAsQ\nbQAAABYg2gAAACxAtAEAAFiAaAMAALAA0QYAAGABog0AAMACRBsAAIAFiDYAAAALEG0AAAAWINoA\nAAAsQLQBAABYgGgDAACwANEGAABgAaINAADAAkQbAACABYg2AAAACxBtAAAAFiDaAAAALFCraMvP\nz1dqaqqio6OVkpKivLy83xz/2muvaeLEiT6ZIAAAAGoRbS6XS+np6RoyZIhyc3M1YsQIpaeny+l0\nVhvrdDr1xBNP6PHHH5fD4aiTCQMAADRGXqMtJydH/v7+SktLk7+/vwYPHqzw8HBlZ2dXGzthwgTt\n2LFDQ4cOrZPJAgAANFZeo62wsFBRUVEe2yIiIlRYWFht7OzZs/Xcc88pPDzcdzMEAACAArwNcDqd\nCgkJ8dgWEhKisrKyamNbtmz5uyficDjkV8uXRfj58dBrffDzc8jf3/fHnvWsH6xnw8J6NiysZ8Pi\nq/X0Gm01BZrT6VRoaOgZX/nJwsPPrfXz4MLCfHvdqJ2wsFA1b96kTvaLs4/1bFhYz4aF9WxYfLWe\nXqMtMjJSWVlZHtuKioqUnJx8xld+suLio7U+01Zaesyn143aKS09ppKSI3WyX5x9rGfDwno2LKxn\nw3K663mqwPMabXFxcXK5XMrKylJaWppWrFihkpISJSQk1H62tWCMUWVl7cZWVRmfXjdqp6rKqLLS\n98ee9awfrGfDwno2LKxnw+Kr9fR6bisoKEiZmZlauXKlYmNj9dZbb2n+/PkKDg7WjBkzlJGRccaT\nAAAAwG/zeqZNktq3b6/FixdX2z5z5swax0+YMOHMZgUAAAAPvI0VAACABYg2AAAACxBtAAAAFiDa\nAAAALEC0AQAAWIBoAwAAsADRBgAAYAGiDQAAwAJEGwAAgAWINgAAAAsQbQAAABYg2gAAACxAtAEA\nAFiAaAMAALAA0QYAAGABog0AAMACRBsAAIAFiDYAAAALEG0AAAAWINoAAAAsQLQBAABYgGgDAACw\nANEGAABgAaINAADAAkQbAACABYg2AAAACxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsADRBgAAYAGi\nDQAAwAJEGwAAgAWINgAAAAsQbQAAABYg2gAAACxAtAEAAFiAaAMAALAA0QYAAGABog0AAMACRBsA\nAIAFiDYAAAALEG0AAAAWINoAAAAsQLQBAABYgGgDAACwANEGAABgAaINAADAAkQbAACABYg2AAAA\nCxBtAAAAFiDaAAAALEC0AQAAWIBoAwAAsADRBgAAYAGiDQAAwAK1irb8/HylpqYqOjpaKSkpysvL\nq3HcBx98oP79+ys6Olp33HGHiouLfTpZAACAxsprtLlcLqWnp2vIkCHKzc3ViBEjlJ6eLqfT6THu\nhx9+UEZGhp5++mmtXbtWLVq00AMPPFBnEwcAAGhMvEZbTk6O/P39lZaWJn9/fw0ePFjh4eHKzs72\nGPfrWbYuXbooKChIkydP1pdffqmSkpI6mzwAAEBj4TXaCgsLFRUV5bEtIiJChYWFvzmuWbNmCgsL\nqzYOAAAAp89rtDmdToWEhHhsCwkJUVlZ2e8aBwAAgNMX4G3AqQItNDTUY1twcHCtxp2Kw+GQXy1f\ny+rn59Ch4t21GwyfOFS8W35+Dvn7O3y+bz8/h0p38zD62VS6u6RO13NH6S8+3y9ObUfpL+pSh+t5\n4OddPt8vTu3Az7vq9Pa5becBn+8Xp7Zt5wFFXe6b9XQYY8xvDfjiiy80a9YsrVq1yr1t4MCBuvvu\nu9W/f3/3tjlz5ujgwYN69NFHJUklJSVKSEjQmjVrFBYWdsYTBQAAaMy8ntuKi4uTy+VSVlaWKioq\ntGTJEneQnez666/XJ598og0bNuj48eOaO3eu+vTpQ7ABAAD4gNczbZK0efNmTZ8+XT/++KPatm2r\njIwMde3aVTNmzJDD4VBGRoYk6aOPPtLTTz+t4uJi9ejRQ//85z/VvHnzuv4aAAAAGrxaRRsAAADq\nF29jBQAAYAGiDQAAwAJEGwAAgAWINgAAAAsQbcDvVFZWpuLi4vqeBuoAaws0Ljt37qzvKdQK0fYf\nOnbsqOjoaMXExCg6Olr9+vXTiy++WKfXuXz5cg0ePFiStHLlSt1yyy11en2NUceOHbVlyxaf7nP4\n8OH697//rcrKSvXs2VNffPGF+//Ky8sVHR3tXtdfvfDCCxo1apT27NmjmJgY3ubtN4wbN05z5szx\n2DZq1Ch17txZR44ccW/Lzc1VdHS0kpKStHr1aq/7rays1Ny5c9WvXz9FR0crMTFRM2bM0KFDh9xj\nfl1bX3v++ec1ceJEn+/3bBozZoz7PrJz58667LLL3Jd//fNPde3mm29W165dFRMT476v7tOnj2bP\nnn3G+96+fbs6duwol8tV7f8efPBBPf/882d8HQ3R6d5ey8vLf9f1nPzz0lcKCgo0dOhQn+6zrnh9\nG6vGxuFwaMmSJYqKipIkbdu2TTfffLOioqI83gGiLq5XOvFuEwMHDqyz62msfj2+vnTw4EFJkr+/\nv2JjY5Wbm6s+ffpIOnHH1LZtW23dulXFxcUKDw+XJK1bt059+/ZVq1attGHDBp/PqSGJj4/Xf//3\nf7svO51Offvtt+rQoYO+/PJLXXvttZKktWvXKi4uTvPnz6/Vfl944QWtW7dOb731li644AIdOHBA\n06ZN03333efex69rWxfq4nvxbMrMzHR/PHHiRLVv314TJkw4q3NwOBx68MEHddNNN7m35efna+TI\nkWrXrp2GDBlyxvuvySOPPHJG+23ITvf2GhgY+Luvy9e3oUOHDqmystKn+6wrnGn7D8YYnfyn69q2\nbasePXqooKDA/f/z5s3Ttddeq5iYGPXt21fvvPOOe/yTTz6phIQExcfHa/To0dqxY4ckqaqqSs8/\n/7z69eun+Ph4TZs2TUePHq12/cuWLXP/FvH8889rypQpuuOOOxQdHa3rr79eX331lXvs+vXrNWTI\nEPXs2VNpaWn67rvv6uSYNAS/9ecIf72zT0hIUHR0tEaNGqWSkhPvhfrDDz8oLS1NvXr10rXXXqtX\nX31VkjRhwgTt2bNHd999t958803Fx8dr/fr17n1mZ2fr6quvVvfu3ZWdnS1Jcrlc2rhxo/r06aNd\nu3apY8eOcjqdWrdunZKTkzV79mzFxsbqqquu0sKFC+vwaNghPj5emzZt0vHjxyVJa9asUefOnfXX\nv/5Vn3/+uXvc2rVr1adPH/Xr1899rDt27Kg33nhD/fr1U2xsrKZMmaKKigpJ0r///W/Fx8frggsu\nkCS1bNlSU6dOVatWrSRVX9vly5dr+PDhuummm9S7d2/t2LFDe/bsUXp6umJjYzVgwAAtW7bMPZ+d\nO3cqPT1dV111lbp166abb75ZRUVF1b6+zZs364orrtB//dd/1cnxqy85OTkaPHiwevbsqaFDh3qc\nsVy9erWGDh2quLg49ejRQ5MmTXKf0Ro2bJheeuklDRw4UNHR0Zo4caJyc3M1cOBAde/eXZMnT/a4\nnv+8TV966aXq2bOnfvzxR0knwvvvf/+7+vXrp8svv1wpKSnu+8j33ntPY8eO1eTJk9W9e3cNGDBA\nH3zwQbWvpby8XKNHj9b48eNVUVGhKVOmaO7cue75Pvvss0pOTlb37t11yy23aO/eve7PmzFjhnr1\n6qUBAwYoMzNTnTt3lnTifuD+++9XXFycEhMTdc8996i0tNQXh75ene7tdfv27brjjjvUq1cvXXPN\nNR73ebfccovmzZunQYMGKSYmRrfccot2767d+42//vrrGjhwoHr06KGEhASPM6Ovvfaa+vbtq7i4\nON1yyy3atGmTSkpKNHbsWB08eFAxMTF//LUw8NChQwfz448/ui/n5+ebxMREk5eXZ4wxZsWKFSYp\nKckUFxcbY4x5//33TdeuXc2xY8fM119/bfr27WtKS0tNZWWlefDBB819991njDEmMzPTDBo0yOzd\nu9ccPXrU3Hvvve7/W7ZsmRk8eHC1j5977jlz2WWXmZycHFNeXm5mz55t/vrXvxpjjNm1a5eJiYkx\nn376qamsrDQfffSRiY2NNaWlpWfnQFnmP9f1ZNdcc4158803jTHG/PLLLyY1NdU888wzxhhjhg0b\nZl599VVSu6N4AAAPjUlEQVRjjDFbtmwxPXr0MD/99JMxxpi+ffuazz//3BhjzPbt202XLl3M8ePH\njTHGDBgwwOTl5ZlXX33V3HXXXcYYY3Jyckzfvn2NMcbs3LnTdOzY0Rw7dsysXbvWdOjQwSxYsMBU\nVlaaVatWmU6dOpm9e/fWzcGwSN++fc2aNWuMMcY89NBD5qWXXjL5+fkmLi7OGGNMWVmZ6dq1q9mx\nY4fHenTo0MGMHz/eHDt2zBQVFZm4uDjz4YcfGmNO3Ma6dOliHnjgAfPhhx/WeJxP3teyZctMp06d\nTE5Ojjl8+LCprKw0ycnJZu7cuaaiosL88MMP5sorrzRr1641xhhz6623mjlz5piqqirjdDpNenq6\n+cc//mGMOXGbnjhxovnpp5/MlVdeaZYvX163B7CO3XXXXea5555zX96xY4eJjo42n3/+uamsrDQf\nfvihiYuLM4cPHzZHjhwx3bt3N9nZ2cYYY/bs2WMSExPdx+Dmm2821113nTlw4ID5+eefTY8ePUxy\ncrL5+eefza5du0yvXr3M6tWr3WMXL17sMZcvv/zS9OjRw71u9913n7n//vuNy+Uy5eXl5qGHHjJ/\n+9vfjDHGvPvuu6Zjx47mww8/NJWVlebVV181PXv2NOXl5Wbbtm3u2+aECRNMenq6qaioMMYYM3ny\nZPPUU0+55/CXv/zF7N692xw+fNgMHTrUPPzww8YYY5566imTmppqiouLzc8//2yGDBliOnfubIwx\nZvHixWbYsGGmrKzMlJWVmdtvv9288MILdbI+Z1ttb68//fST6devn5kzZ44pLy83W7duNf3793ev\n6YgRI8w111xjdu3aZQ4fPmyGDx9upk+fbozx/Bn5n9avX2/i4+PN9u3b3Zc7duxotm/fbrZt22a6\ndetm9uzZY4w5cVscMWKEMcaYtWvXuuf4R8eZthoMHTpUvXr1Urdu3XTjjTeqffv2at++vSSpf//+\nWrRokZo3b659+/YpKChILpdLpaWlCgoK0sGDB/XOO+9o27Ztevjhh93PsVi6dKnuvPNO/fnPf1Zo\naKjuvfdevf/++zU+b+Jk3bp1U2xsrAICApScnKxt27ZJkj744APFxcWpX79+8vPz04ABA9S+fXt9\n/PHHdXtwGqCXX35Zw4YNk9Pp1J49e3T++edr3759kqRzzjlHn332mT7//HNdeOGFWr9+vdq2bVtt\nH23atFGrVq20ceNG7dixQ6Wlperatav69Omjr7/+WpWVlVq3bp2uvPLKGucQEBCg0aNHy8/PT/37\n91doaKj7LG1jdsUVVyg3N1eS9OWXXyoxMVGdOnVSYGCgvvvuO23cuFGtW7fWRRddVO1zb7vtNoWE\nhOiSSy5RdHS0fvrpJ0lSSkqKMjMzdfz4cT366KNKTEzUDTfcoJycnFPOo2XLloqNjVWTJk30/fff\na+/evZo0aZL8/f3VoUMH3XTTTXr33XclSY8//rgmTJig8vJy7dy5U82aNXN/P0nSvn37NHLkSKWm\npmrQoEE+PFr1b+XKlUpISFBiYqL8/Px03XXXKTIyUp988olCQ0O1YsUK9enTR0eOHNH+/fvVvHlz\n7d+/3/35ycnJatGihcLDwxUVFaXrr79e4eHhat26tSIiIjzOtsyePdt9P33ppZdqwYIFmjlzphIT\nEyVJU6ZM0bRp0+RwOLRr1y6dd955HuvQpk0bXXfddfLz81NycrIOHz7sfljcGKOpU6eqsLBQzz77\nrPz9/Wv8egcNGqRWrVqpSZMm6t+/v/t7bOXKlbrzzjvVvHlzhYeHezx8fM4556ioqEjLli3TwYMH\ntXDhQo0fP95na1Cfant73b17t44cOaJJkyYpICBAkZGRGj16tJYvX+7eV3Jyslq3bu0+tr/+7Pst\nXbp00dKlS9WmTRsVFxervLxcwcHB2rdvnwICAlRRUaG3335bP/zwg+6880698cYbdXYs6grPaavB\nO++8435OW3FxsR544AFNmjRJ8+fPV3l5uWbNmqU1a9aodevW6tixo6QTD392795djz32mLKysvTM\nM8/ooosu0gMPPKDExETt2bNH9913n/vGb4xRUFCQ9uzZ85tzOfm9WwMCAtwPCezZs0dffPGFevXq\n5d5fRUWFevTo4fPj0dDl5eVpzJgxOnbsmNq3b69Dhw65j/tTTz2lefPmaebMmSouLlZSUpKmT5+u\nkJCQavv59Q7rvPPOU0JCgiQpMjJS559/vvLy8rR27VrddtttNc6hadOmHj8YTl7rxiw+Pl7vvPOO\nNm/eLGOM+5enK6+8Ul9//bVcLtcpQ/j88893fxwQEKCqqir35djYWMXGxkqSioqK9Pbbb2vcuHH6\n9NNP1aJFi2r7Onnb7t27dfjwYY/bXlVVlfvhry1btmjOnDnav3+/2rVr5x7zq++++069e/fWxx9/\nrPT0dAUENJy74T179ujzzz+vdr90xRVXyOFwaNWqVXr99dfl5+fnfnrAyesSFhbm/tjPz0/nnXee\nx+WTx95///1KS0vTkSNHlJGRoe3bt7ufU/rrXB599FEVFRUpKipKTZs29ViHk+9bAwMD3ev4q59/\n/lm7d+/W999/r+jo6Bq/3lPdP+/fv9/98LsktW7d2v3xoEGDdPToUS1ZskSzZs1Sp06dNHPmTF12\n2WW/dWitUNvba0lJif70pz/Jz+//P2/UunVr98PLkuftNzAw0GNtTsXhcOiFF17QJ598ohYtWriP\nqTFGrVu3VmZmpl5++WUtWrRIzZo108SJE3XjjTf66ss/KzjTVoOTb9jh4eEaNmyY+7fwp556SsYY\nrV69WsuXL/d4JdjevXt1ySWX6I033tD69es1ePBg3XPPPaqqqlLLli31r3/9S+vWrdO6deuUk5Oj\n5cuX6+KLL/5dc2zZsqWSkpLc+1u/fr1WrlypUaNGndkX38js27dP999/v5588kl98cUXWrhwofsH\nrXTieUdTp07VZ599piVLlui7777Tm2++WeO+4uPjtWHDBq1evdr9274kJSYm6quvvlJBQYGuuOKK\nOv+aGpIrrrhC33//vbKzsz1+IPfp00fr16/3ePFHbVRVValnz55as2aNe1tERISmTp2q0NBQFRYW\n1vh5Jz/xuWXLlrrgggs8bnuffPKJnn76aZWXl+uuu+7S+PHj9dVXX2nRokXugPnVVVddpZdeekkO\nh0P/+te/aj13G7Rs2VIDBw6sdr80cuRI5ebmasGCBXrjjTf06aef6oUXXnC/QOdXv+cJ5k2aNNFj\njz2myspK3Xvvve7t99xzj5KSkpSTk6OsrCxdc801v7mfk6/b4XBowYIFGjNmjKZNm3bar3S84IIL\nPH4hP/njoqIixcfHa+nSpVqzZo26deumBx544LT2/0dV29trq1attH//fo8Q27FjR7Xvh9P16quv\nasuWLfr000/1wQcf6JFHHnE/l7WkpEShoaHKzMzUunXrdO+992ratGk6cODAGV3n2Ua0eXHo0CEt\nXbpUMTExkqSjR4/qnHPOkZ+fnw4ePOh++LOiokJ5eXm64447tGPHDoWEhKhp06YKCwuTn5+fBg0a\npOeff14HDhxQeXm55s6dq7Fjx5722ZRfxyclJemzzz5z//D55ptvlJycrO+//96HX33DcuDAAe3b\nt8/9r6SkxP1ikODgYEknXkDw0UcfuW/ojzzyiF566SVVVlaqRYsW8vPzc/8GGBQU5PFS9ri4OBUU\nFOjbb791n2mTTvyWuXTpUnXu3FmhoaHu7ZxJ8y4sLEyRkZFavHixxw+B+Ph4/d///Z82b95cLYp+\ni5+fn/7yl7/oiSeecN9WDh8+rEWLFikgIEBdunSRVH1tT9atWzcFBwfr5ZdfVkVFhfbu3avbbrtN\nWVlZKi8vl8vlcn8/bdy4Ue+88477+0k6cdbA399fGRkZyszMdD9xviFISkrS//zP/2jdunWS5H4h\nQX5+vo4cOaKAgAAFBQWpoqJCy5Yt04YNGzyOze8VGBioJ554QmvWrNF7770n6cR99a/rsGXLFr38\n8su/GV8n3x6NMQoMDNSYMWMk6bTjOiUlRQsWLFBJSYlKSkq0YMEC9/+tWrVKkydPVklJiZo0aaKQ\nkBA1a9bstPb/R1Xb22vXrl3VokULzZs3Ty6XS1u3btUrr7yi5OTkWl1PeXm5x335vn37VFZWpiNH\njigwMFABAQE6evSoZs+erYqKClVUVGj37t0aOXKk8vPzFRQUpGbNmik4OFghISEKCgrS8ePHf/ef\nITmbGs55eR9xOBxKTU2Vw+GQw+FQYGCgevfurccff1zSiZe433ffferZs6fCwsKUnJysiy++WFu3\nbtWAAQO0efNmDRs2TEePHlVkZKSeffZZSSf+hk1FRYXS0tJ0+PBhXXrppXrxxRc9Tg/Xdn7SiVe1\nzps3T3PmzNFPP/2k8PBwTZ06VXFxcb49IA2Ew+HQ7bff7rEtJiZGWVlZSk9P19/+9jdVVVUpKipK\nQ4cOdZ9ZnTt3rjIyMrRo0SIFBQUpOTnZ/erelJQUPfjgg9qxY4fuuOMONWnSRJdccokqKys97oTj\n4uL0yy+/aMSIEdXm9FvzxQkJCQlauHChx1nKJk2aKCIiwn2HK1U/U3Kyky/PnDlTCxYs0JQpU7Rv\n3z4FBgaqV69eeuONN9wPe5+8tn/+85899hUQEKAXX3zRHfSBgYFKSkrSnXfe6Y6xadOmyel0qk2b\nNkpLS9Nbb71V7eGd7t27u69n8eLFVq75f845MjJSc+fO1ezZs7Vt2za1aNFCDz30kHr06CFjjK6+\n+molJSUpMDBQl112mVJSUrR169Ya9/Vbl2s6VlFRURo3bpyefPJJJSYm6pFHHtFjjz2m2bNn64IL\nLtCQIUM0b968U8Z4TfsPDAxURkaGRo8e7f6TFb81h1+NHTtWe/bs0TXXXKMWLVqob9++7r9AMHLk\nSO3atUvXX3+9jh8/ri5duuif//znKfdlm9reXhcsWKBZs2YpISFBISEhGj58uG699VZJ3u//Nm/e\nrKuuuspj26xZszRy5EhNnjxZvXv31rnnnqt+/fopJiZGW7du1YgRIzR58mTdddddKikp0YUXXqh5\n8+apSZMm6tChg9q1a6e4uDitWLFCbdq08e1B8SGH4dd9AAB8Ji8vT1FRUWrSpIkk6bPPPtOsWbP0\nv//7v/U8M9iOM20AAPjQe++9p6qqKs2cOVPHjh3T66+/fsoXzACngzNtAAD40MGDBzV9+nT3c/v6\n9++vqVOn6txzz63nmcF2RBsAAIAFePUoAACABYg2AAAACxBtAAAAFiDaAAAALEC0AQAAWIBoAwAA\nsMD/A7GlA3kmmXqrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x280f51605f8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Let's visually summarize our performance\n",
    "%matplotlib inline\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.axes_grid1.axes_divider import make_axes_area_auto_adjustable\n",
    "\n",
    "labels = [\"Baseline\", \"LastWin\", \"WinStreak\", \"TeamRankings\", \"WonLast\"]\n",
    "scores = [0.3728, 0.4407, 0.4940, 0.5488, 0.5351]\n",
    "\n",
    "sns.set(style=\"darkgrid\", context=\"talk\")\n",
    "sns.barplot(labels, scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "By adding new features, we see improvements in our model however by continuously adding features that were derived from the  same dataset, our performance improvements were marginally less from each successive addition\n",
    "\n",
    "- By adding a feature outside of our original dataset, that is \"Rankings\", we saw a noticable performance boost\n",
    "- Adding our final feature \"WonLast\" provided negligible improvements"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Encoding\n",
    "\n",
    "- One of the most important pieces of information in our dataset we have been ignoring up till now is the actual team themselves\n",
    "- Currently, the team names are stored as strings (e.g. \"Calgary Flames\" or \"Boston Bruins\") and classification methods don't work too well with these\n",
    "- By utilizing the 'one hot encoder' feature provided by Scikit-Learn, we can label the team names in a numeric fashion so that our classifier can understand them and potentially provide much more information to our model in determining winners"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What will happen is we will be creating a feature that will ask a question:\n",
    "- Was this team the Calgary Flames?\n",
    "\n",
    "And the answer will result in values like:\n",
    "- 1 (for yes) and 0 (for no)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LabelEncoder()"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import LabelEncoder, OneHotEncoder\n",
    "encoding = LabelEncoder() # encoding object has LabelEncoder ready to use\n",
    "encoding.fit(results[\"Home Team\"].values) # give each team a number to identify them by"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[ 6, 28],\n",
       "        [11, 29],\n",
       "        [15, 26],\n",
       "        [ 7,  0],\n",
       "        [10,  3]], dtype=int64), (1327, 2))"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "home_teams = encoding.transform(results[\"Home Team\"].values)\n",
    "visitor_teams = encoding.transform(results[\"Visitor Team\"].values)\n",
    "\n",
    "# vstack allows you to stack a sequence of arrays vertically - in this case we do it to make easy comparisons\n",
    "X_teams = np.vstack([home_teams, visitor_teams]).T # transpose\n",
    "\n",
    "X_teams[:5], X_teams.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, we can see the Chicago Blackhawks (6) is paired with the Washington Capitals (28) in the first pair [6, 28].\n",
    "We know it is these two teams by running the two commands below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Chicago Blackhawks', 'Edmonton Oilers', 'Montreal Canadiens', ...,\n",
       "       'New York Rangers', 'New York Rangers', 'Los Angeles Kings'], dtype=object)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results[\"Home Team\"].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Washington Capitals', 'Winnipeg Jets', 'Toronto Maple Leafs', ...,\n",
       "       'Los Angeles Kings', 'Los Angeles Kings', 'New York Rangers'], dtype=object)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results[\"Visitor Team\"].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Prepare the One Hot Encoder\n",
    "onehot = OneHotEncoder()\n",
    "X_teams = onehot.fit_transform(X_teams).todense() # todense returns a matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1327, 60)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_teams.shape # see matrix dimensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Home: [[ 0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.\n",
      "   0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]]\n",
      "Visitor: [[ 0.  0.  0.  0.  0.  0.  1.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.\n",
      "   0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.]]\n"
     ]
    }
   ],
   "source": [
    "print(\"Home:\", X_teams[0, :30])\n",
    "print(\"Visitor:\", X_teams[0, :30])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above is large matrix (with dimensions 1327 x 60) that has the list of team names and will mark as \"1\" if the team name is found within it.\n",
    "\n",
    "By doing this, our model can now understand team names when doing its calculations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The F1 score using encoded teams is the following: 0.5371\n"
     ]
    }
   ],
   "source": [
    "# Lets build our model using the Decision Tree Classifier\n",
    "tree_clf = DecisionTreeClassifier(random_state=10)\n",
    "scores = cross_val_score(tree_clf, X_teams, y_true, scoring=scorer)\n",
    "\n",
    "# Note: The only feature is the encoder we just made\n",
    "print(\"The F1 score using encoded teams is the following: {0:.4f}\".format(np.mean(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is not a bad score, recall our F1 scores from this dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>F1 Score</th>\n",
       "      <th>Overall Performance Boost</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Baseline</th>\n",
       "      <td>0.3728</td>\n",
       "      <td>0.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Last Win</th>\n",
       "      <td>0.4407</td>\n",
       "      <td>0.0679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Win Streaks &amp; Last Win</th>\n",
       "      <td>0.4940</td>\n",
       "      <td>0.0533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Team Rankings &amp; Last Win</th>\n",
       "      <td>0.5488</td>\n",
       "      <td>0.1081</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Won Last, Team Rankings, &amp; Last Win</th>\n",
       "      <td>0.5351</td>\n",
       "      <td>0.0944</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     F1 Score  Overall Performance Boost\n",
       "Baseline                               0.3728                     0.0000\n",
       "Last Win                               0.4407                     0.0679\n",
       "Win Streaks & Last Win                 0.4940                     0.0533\n",
       "Team Rankings & Last Win               0.5488                     0.1081\n",
       "Won Last, Team Rankings, & Last Win    0.5351                     0.0944"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Random Forest Classifier\n",
    "\n",
    "Up to now we've been using only the Decision Tree Classifier as our tool to building our model. Could we potentially produce a better model if we were to use a Random Forest Classifier?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The F1 score using Random Forest Classifier is the following: 0.5460\n"
     ]
    }
   ],
   "source": [
    "# Random Forest Classifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "forest_clf = RandomForestClassifier(random_state=10)\n",
    "scores = cross_val_score(forest_clf, X_teams, y_true, scoring=scorer)\n",
    "print(\"The F1 score using Random Forest Classifier is the following: {0:.4f}\".format(np.mean(scores)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There was a marginal drop in performance (as compared to our \"Team Rankings & Last Win\" model) which could potentially be explained by overfitting or other randomness factors\n",
    "\n",
    "Could we do better moving away from the default values using GridSearchCV **with** Random Forest Classifier?\n",
    "\n",
    "## GridSearchCV with Random Forest Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "F1: 0.5840\n",
      "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='entropy',\n",
      "            max_depth=None, max_features=50, max_leaf_nodes=None,\n",
      "            min_samples_leaf=4, min_samples_split=2,\n",
      "            min_weight_fraction_leaf=0.0, n_estimators=100, n_jobs=1,\n",
      "            oob_score=False, random_state=10, verbose=0, warm_start=False)\n"
     ]
    }
   ],
   "source": [
    "# Will take some time to run\n",
    "parameter_space = {\n",
    "    \"max_features\": [2, 10, 50, 'auto'],\n",
    "    \"n_estimators\": [50, 100, 200],\n",
    "    \"criterion\": [\"gini\", \"entropy\"],\n",
    "    \"min_samples_leaf\": [1,2,4,6],\n",
    "}\n",
    "\n",
    "forest_clf = RandomForestClassifier(random_state=10)\n",
    "grid = GridSearchCV(forest_clf, parameter_space, scoring=scorer)\n",
    "grid.fit(X_teams, y_true)\n",
    "\n",
    "# View results\n",
    "print(\"F1: {0:.4f}\".format(grid.best_score_))\n",
    "print(grid.best_estimator_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our best result was an F1 score of 0.5840 with the following parameters:\n",
    "\n",
    "- **Max_features** = 50\n",
    "- **n_estimators** = 100 (the number of tress in the forest)\n",
    "- **Criterion** = entropy (information gain/ certainty)\n",
    "- **min_samples_leaf** = 4 (minimum number of samples in a leaf; a split is discarded if after a split, at least one of the leaves contains less than \"min_samples_leaf\" samples)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combine Encoded attribute with Previous Attributes\n",
    "\n",
    "Let us combine our previous attributes and our new encoded attribute to see the overall performance change:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1327, 64)\n"
     ]
    }
   ],
   "source": [
    "X_all = np.hstack([X_last_win, X_teams])\n",
    "print(X_all.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "F1 score: 0.5398\n"
     ]
    }
   ],
   "source": [
    "# Decision Tree Classifier\n",
    "tree_clf = DecisionTreeClassifier(random_state=10)\n",
    "scores = cross_val_score(tree_clf, X_all, y_true, scoring=scorer)\n",
    "print(\"F1 score: {0:.4f}\".format(np.mean(scores)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "F1 score: 0.5904\n",
      "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
      "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
      "            min_samples_leaf=6, min_samples_split=2,\n",
      "            min_weight_fraction_leaf=0.0, n_estimators=100, n_jobs=1,\n",
      "            oob_score=False, random_state=10, verbose=0, warm_start=False)\n"
     ]
    }
   ],
   "source": [
    "# Random Forest Classifier\n",
    "parameter_space = {\n",
    "    \"max_features\": [2, 10, 50, 'auto'],\n",
    "    \"n_estimators\": [50, 100, 200],\n",
    "    \"criterion\": [\"gini\", \"entropy\"],\n",
    "    \"min_samples_leaf\": [1,2,4,6],\n",
    "}\n",
    "\n",
    "forest_clf = RandomForestClassifier(random_state=10)\n",
    "grid = GridSearchCV(forest_clf, parameter_space, scoring=scorer)\n",
    "grid.fit(X_all, y_true)\n",
    "print(\"F1 score: {0:.4f}\".format(grid.best_score_))\n",
    "print(grid.best_estimator_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Final Results\n",
    "\n",
    "Let us take a look at our previous and final results\n",
    "\n",
    "Note:\n",
    "- DT = Decision Tree\n",
    "- RF = Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm0AAAG4CAYAAAAXENC/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xtc1GXe//H3zCgBHfCG1LSDod1kJ2NQERUtD5t3qZgi\nYUl3jy0PYQczs7I2Re3ktmmmrba2bd2KWZ5YbUsrtyxLTELpvpVuLbA8a+DiaXAQvr8//DG3E+AM\nMDRe8Xo+Hj4ecnFxzWeuz8x833zngM2yLEsAAAA4p9mDXQAAAAB8I7QBAAAYgNAGAABgAEIbAACA\nAQhtAAAABiC0AQAAGMCv0LZt2zalpKTI6XRq8ODBysvLq3ZeTk6OhgwZIqfTqaSkJGVnZwe0WAAA\ngMbKZ2hzu91KT0/X0KFDlZOTo7S0NKWnp8vlcnnNO3jwoMaMGaMxY8Zo8+bNGj16tB5++GG53e4G\nKx4AAKCx8BnasrOz5XA4lJqaKofDoeTkZEVFRWndunVe87KystS9e3f17dtXktS/f3+9/fbbstls\nDVM5AABAI+IztBUUFKhdu3ZeY9HR0SooKPAa27Ztm1q0aKEHH3xQXbp00bBhw1RWVqamTZsGtmIA\nAIBGyGdoc7lcCgsL8xoLCwtTaWmp11hJSYmWLFmi4cOH66uvvlJSUpJGjx6to0ePBrZiAACARshn\naKsuoLlcLoWHh3uNhYSE6KabblLXrl3lcDh01113KTw8XLm5uX4Vwp9ABQAAqFkTXxPatm2rzMxM\nr7HCwkIlJSV5jUVHR2vXrl1eYxUVFX6HsaKi47LzASQAAKCRi4y8oNpxn6EtISFBbrdbmZmZSk1N\nVVZWloqLi5WYmOg1b9CgQRo2bJjWrVunnj17auHChXK73erSpYtfBVqWpfJyv6YCAAA0OjbLj1Nh\n27dv16RJk7Rjxw61adNGGRkZ6tChgyZPniybzaaMjAxJ0ldffaWXXnpJP/30k6688kplZGTohhtu\n8KuQQ4d47RsAAEDz5hdWO+5XaPs1ENoAAABqDm28igwAAMAAPl/TBgAAcKaysjLl528NdhnGueaa\n6+r1+bWENgAAUCv5+Vv1p1VrFHlZm2CXYozi3T/qMUkdOsTWeQ1CGwAAqLXIy9qoZbuYYJfRqPCa\nNgAAAAMQ2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAADENoAAAAMQGgDAAAwAKENAADAAIQ2AAAAAxDa\nAAAADEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAADENoAAAAMQGgD\nAAAwAKENAADAAIQ2AAAAAxDaAAAADEBoAwAAMAChDQAAwABNgl0AzFJWVqb8/K3BLsM411xznZo2\nbRrsMgAABiO0oVby87dqyrvPKqJ1ZLBLMUbJ3mJNTv2DOnSIDXYpAACDEdpQaxGtIxXVpkWwywCC\ngrPNdccZZ6B+CG0AUAv5+Vv1+qtL1PziS4NdilEO/bxHox8WZ5yBeiC0AUAtNb/4Ul16SdtglwGg\nkeHdowAAAAbgTBsAwCi8rrBueE2h+QhtAACj5Odv1WcfzlSby5oHuxRj/Lj7kKRxvKbQcIQ2wCCc\nYag7zjL8trS5rLlirmoV7DKAXxWhDTBIfv5WrZ78tC6PaBbsUoyyq+Rf0pTnOMsAwGiENsAwl0c0\n01VRUcEuAwDwK+PdowAAAAYgtAEAABiA0AYAAGAAQhsAAIABCG0AAAAGILQBAAAYgNAGAABgAEIb\nAACAAQhtAAAABiC0AQAAGIDQBgAAYABCGwAAgAEIbQAAAAYgtAEAABiA0AYAAGAAQhsAAIABCG0A\nAAAGILQBAAAYwK/Qtm3bNqWkpMjpdGrw4MHKy8urdt7o0aN14403Ki4uTk6nU3FxcQEtFgAAoLHy\nGdrcbrfS09M1dOhQ5eTkKC0tTenp6XK5XFXm5ufn65133lFubq42b96s3NzcBikaAACgsfEZ2rKz\ns+VwOJSamiqHw6Hk5GRFRUVp3bp1XvOKi4tVXFysq666qsGKBQAAaKx8hraCggK1a9fOayw6OloF\nBQVeY9u2bdP555+v0aNHq2vXrrrrrru0ZcuWwFYLAADQSDXxNcHlciksLMxrLCwsTKWlpV5jJ0+e\nlNPp1IQJE3TFFVdo6dKlGjlypFavXq2oqCifhdhsNtl5W8Q5z263BbsEI9ntNjkc9d879r/u6EHw\n0YPgCtT+V66F2qtvD3yGtuoCmsvlUnh4uNdYnz591KdPH8/Xd955pxYtWqSNGzfqtttu81lIVNT5\nstm4EZzrIiLCfU9CFRER4YqMvCAg66Bu6EHw0YPgCtT+V66F2qtvD3yGtrZt2yozM9NrrLCwUElJ\nSV5ja9asUUVFhW699VbPmNvtVkhIiF+FFBUd50ybAUpKTgS7BCOVlJxQcfGxgKyDuqEHwUcPgitQ\n+1+5FmrP3x7UFOx8hraEhAS53W5lZmYqNTVVWVlZKi4uVmJiote8EydO6OWXX1ZMTIzatGmjt956\nSydPnqwyryaWZam83K+pCKKKCivYJRiposJSeXn99479rzt6EHz0ILgCtf+Va6H26tsDn6EtJCRE\n8+fP16RJkzRjxgy1adNGc+fOVWhoqCZPniybzaaMjAwNHjxYhw4d0ogRI/Svf/1L1113nebPn6/Q\n0NA6FwcAAIDTfIY2SYqJidHixYurjE+ZMsXr61GjRmnUqFGBqQwAAAAevIoMAADAAIQ2AAAAAxDa\nAAAADEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAADENoAAAAMQGgD\nAAAwAKENAADAAIQ2AAAAAxDaAAAADEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADAAoQ0A\nAMAAhDYAAAADENoAAAAMQGgDAAAwAKENAADAAIQ2AAAAAxDaAAAADEBoAwAAMAChDQAAwACENgAA\nAAMQ2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAADENoAAAAMQGgDAAAwAKENAADAAIQ2AAAAAxDaAAAA\nDEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAADENoAAAAMQGgDAAAw\nAKENAADAAIQ2AAAAAxDaAAAADEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADCAX6Ft27Zt\nSklJkdPp1ODBg5WXl3fW+Rs2bNA111wjl8sVkCIBAAAaO5+hze12Kz09XUOHDlVOTo7S0tKUnp5e\nYyA7cuSInn766YAXCgAA0Jj5DG3Z2dlyOBxKTU2Vw+FQcnKyoqKitG7dumrnZ2RkqH///gEvFAAA\noDHzGdoKCgrUrl07r7Ho6GgVFBRUmbty5UodPXpUw4YNk2VZgasSAACgkWvia4LL5VJYWJjXWFhY\nmEpLS73G9u7dq9mzZ+udd97RyZMnZbPZalWIzWaTnbdFnPPs9tr1FafZ7TY5HPXfO/a/7uhB8NGD\n4ArU/leuhdqrbw98hrbqAprL5VJ4eLjna8uy9OSTT2rcuHG6+OKLtXv3bs+4v6Kizq910MOvLyIi\n3PckVBEREa7IyAsCsg7qhh4EHz0IrkDtf+VaqL369sBnaGvbtq0yMzO9xgoLC5WUlOT5ev/+/fr2\n22/13XffKSMjQxUVFbIsSzfffLPmzZunuLg4n4UUFR3nTJsBSkpOBLsEI5WUnFBx8bGArIO6oQfB\nRw+CK1D7X7kWas/fHtQU7HyGtoSEBLndbmVmZio1NVVZWVkqLi5WYmKiZ06rVq20ZcsWz9d79uxR\nnz599Pnnnys0NNSf6yHLslRe7tdUBFFFBa9VrIuKCkvl5fXfO/a/7uhB8NGD4ArU/leuhdqrbw98\nntsKCQnR/PnztWrVKnXp0kWLFi3S3LlzFRoaqsmTJysjI6Pan7PZbLwZAQAAIEB8nmmTpJiYGC1e\nvLjK+JQpU6qdf+mllyo/P79+lQEAAMCDV5EBAAAYgNAGAABgAEIbAACAAQhtAAAABiC0AQAAGIDQ\nBgAAYABCGwAAgAEIbQAAAAYgtAEAABiA0AYAAGAAQhsAAIABCG0AAAAG8OsPxp8rysrKlJ+/Ndhl\nGOeaa65T06ZNg10GAACoB6NCW37+Vj0z8x1dFNU62KUY40jRXk0bd6c6dIgNdikAAKAejAptknRR\nVGtFtowOdhkAAAC/Kl7TBgAAYABCGwAAgAEIbQAAAAYgtAEAABiA0AYAAGAAQhsAAIABCG0AAAAG\nILQBAAAYgNAGAABgAEIbAACAAQhtAAAABiC0AQAAGIDQBgAAYABCGwAAgAEIbQAAAAYgtAEAABiA\n0AYAAGAAQhsAAIABCG0AAAAGILQBAAAYgNAGAABgAEIbAACAAQhtAAAABiC0AQAAGIDQBgAAYABC\nGwAAgAEIbQAAAAYgtAEAABiA0AYAAGAAQhsAAIABCG0AAAAGILQBAAAYgNAGAABgAEIbAACAAQht\nAAAABiC0AQAAGIDQBgAAYABCGwAAgAEIbQAAAAYgtAEAABiA0AYAAGAAQhsAAIAB/Apt27ZtU0pK\nipxOpwYPHqy8vLxq582aNUs9evRQx44ddc899+j7778PaLEAAACNlc/Q5na7lZ6erqFDhyonJ0dp\naWlKT0+Xy+XymrdkyRJ9/PHHWr58ub755ht17NhRjz/+eIMVDgAA0Jj4DG3Z2dlyOBxKTU2Vw+FQ\ncnKyoqKitG7dOq95KSkpWrp0qZo3b65jx47pyJEjioyMbLDCAQAAGpMmviYUFBSoXbt2XmPR0dEq\nKCioMjc0NFQrVqzQU089pQsvvFBvvvlm4CoFAABoxHyGNpfLpbCwMK+xsLAwlZaWVjt/wIABGjhw\noP7rv/5L9913nz7++GNddNFFPgux2Wyy+zjvZ7fbfK6Dqux2mxyOwOwdPaibQPWA/a87ehB89CC4\nOBYEX3174DO0VRfQXC6XwsPDq53ftGlTSdK9996rhQsX6uuvv1bfvn19FhIVdb5strNfkYiI6i8T\nZxcREa7IyAsCthZqL1A9YP/rjh4EHz0ILo4FwVffHvgMbW3btlVmZqbXWGFhoZKSkrzGZs+erVOn\nTmncuHGesbKyMl144YV+FVJUdNznmbaSkhN+rQVvJSUnVFx8LGBrofYC1QP2v+7oQfDRg+DiWBB8\n/vagpmDnM7QlJCTI7XYrMzNTqampysrKUnFxsRITE73m3XjjjZowYYL69++v6OhozZs3TxdeeKGc\nTqdfV8SyLJWXn31ORYXl11rwVlFhqbw8MHtHD+omUD1g/+uOHgQfPQgujgXBV98e+Hz3aEhIiObP\nn69Vq1apS5cuWrRokebOnavQ0FBNnjxZGRkZkqSePXtq/PjxGjNmjBITE7V161a98cYbCgkJqXNx\nAAAAOM3nmTZJiomJ0eLFi6uMT5kyxevrO+64Q3fccUdgKgMAAIAHf8YKAADAAIQ2AAAAAxDaAAAA\nDEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAADENoAAAAMQGgDAAAw\nAKENAADAAIQ2AAAAAxDaAAAADEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADAAoQ0AAMAA\nhDYAAAADENoAAAAMQGgDAAAwAKENAADAAIQ2AAAAAxDaAAAADEBoAwAAMAChDQAAwACENgAAAAMQ\n2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAADENoAAAAMQGgDAAAwAKENAADAAIQ2AAAAAxDaAAAADEBo\nAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAADENoAAAAMQGgDAAAwAKEN\nAADAAIQ2AAAAAxDaAAAADEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADCAX6Ft27ZtSklJ\nkdPp1ODBg5WXl1ftvPfee0/9+vVTp06dlJKSopycnIAWCwAA0Fj5DG1ut1vp6ekaOnSocnJylJaW\npvT0dLlcLq95Gzdu1MyZM/Xqq68qJydHw4cPV3p6ukpKShqseAAAgMbCZ2jLzs6Ww+FQamqqHA6H\nkpOTFRUVpXXr1nnN279/v0aMGKGrr75aknT77bfLbrdrx44dDVM5AABAI9LE14SCggK1a9fOayw6\nOloFBQVeY4MGDfL6+ptvvtGJEyd01VVXBaBMAACAxs1naHO5XAoLC/MaCwsLU2lpaY0/8/3332vs\n2LEaO3asmjVr5lchNptNdh/n/ex2m19rwZvdbpPDEZi9owd1E6gesP91Rw+Cjx4EF8eC4KtvD3yG\ntuoCmsvlUnh4eLXz169fr0cffVT33XefRowY4XchUVHny2Y7+xWJiKj+MnF2ERHhioy8IGBrofYC\n1QP2v+7oQfDRg+DiWBB89e2Bz9DWtm1bZWZmeo0VFhYqKSmpytxly5bphRde0NSpU3XbbbfVqpCi\nouM+z7SVlJyo1Zo4raTkhIqLjwVsLdReoHrA/tcdPQg+ehBcHAuCz98e1BTsfIa2hIQEud1uZWZm\nKjU1VVlZWSouLlZiYqLXvA0bNmjq1Kl688031bFjRz/L/z+WZam8/OxzKiqsWq+L0/tWXh6YvaMH\ndROoHrD/dUcPgo8eBBfHguCrbw98vns0JCRE8+fP16pVq9SlSxctWrRIc+fOVWhoqCZPnqyMjAxJ\n0htvvKFTp05p5MiRiouLk9PpVFxcnNavX1/n4gAAAHCazzNtkhQTE6PFixdXGZ8yZYrn/3/9618D\nVxUAAAC88GesAAAADEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAAD\nENoAAAAMQGgDAAAwAKENAADAAIQ2AAAAAxDaAAAADEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxA\naAMAADAAoQ0AAMAAhDYAAAADENoAAAAMQGgDAAAwAKENAADAAIQ2AAAAAxDaAAAADEBoAwAAMACh\nDQAAwACENgAAAAMQ2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAADENoAAAAMQGgDAAAwAKENAADAAIQ2\nAAAAAxDaAAAADEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMAADAAoQ0AAMAAhDYAAAADENoA\nAAAMQGgDAAAwAKENAADAAIQ2AAAAAxDaAAAADEBoAwAAMAChDQAAwACENgAAAAMQ2gAAAAxAaAMA\nADAAoQ0AAMAAhDYAAAADENoAAAAM4Fdo27Ztm1JSUuR0OjV48GDl5eWddf5bb72lhx9+OCAFAgAA\nwI/Q5na7lZ6erqFDhyonJ0dpaWlKT0+Xy+WqMtflcumPf/yjpk+fLpvN1iAFAwAANEY+Q1t2drYc\nDodSU1PlcDiUnJysqKgorVu3rsrcBx98ULt27dKwYcMapFgAAIDGymdoKygoULt27bzGoqOjVVBQ\nUGXuiy++qNmzZysqKipwFQIAAEBNfE1wuVwKCwvzGgsLC1NpaWmVuc2bN69zITabTXYfEdJu5ynX\nurDbbXI4ArN39KBuAtUD9r/u6EHw0YPg4lgQfPXtgc/QVl1Ac7lcCg8Pr/OFVicq6nyfr4OLiAjs\nZTYWERHhioy8IGBrofYC1QP2v+7oQfDRg+DiWBB89e2Bz9DWtm1bZWZmeo0VFhYqKSmpzhdanaKi\n4z7PtJWUnAjoZTYWJSUnVFx8LGBrofYC1QP2v+7oQfDRg+DiWBB8/vagpmDnM7QlJCTI7XYrMzNT\nqampysrKUnFxsRITE2tf7VlYlqXy8rPPqaiwAnqZjUVFhaXy8sDsHT2om0D1gP2vO3oQfPQguDgW\nBF99e+DzjQghISGaP3++Vq1apS5dumjRokWaO3euQkNDNXnyZGVkZNT5wgEAAOAfn2faJCkmJkaL\nFy+uMj5lypRq5z/44IP1qwoAAABe+DNWAAAABiC0AQAAGIDQBgAAYABCGwAAgAEIbQAAAAYgtAEA\nABiA0AYAAGAAQhsAAIABCG0AAAAGILQBAAAYgNAGAABgAEIbAACAAQhtAAAABiC0AQAAGIDQBgAA\nYABCGwAAgAEIbQAAAAYgtAEAABiA0AYAAGAAQhsAAIABCG0AAAAGILQBAAAYgNAGAABgAEIbAACA\nAQhtAAAABiC0AQAAGIDQBgAAYABCGwAAgAEIbQAAAAYgtAEAABiA0AYAAGAAQhsAAIABCG0AAAAG\nILQBAAAYgNAGAABgAEIbAACAAQhtAAAABiC0AQAAGIDQBgAAYABCGwAAgAEIbQAAAAYgtAEAABiA\n0AYAAGAAQhsAAIABCG0AAAAGILQBAAAYgNAGAABgAEIbAACAAQhtAAAABiC0AQAAGIDQBgAAYABC\nGwAAgAEIbQAAAAYgtAEAABiA0AYAAGAAQhsAAIAB/Apt27ZtU0pKipxOpwYPHqy8vLxq573//vvq\n27evnE6n7r//fhUVFQW0WAAAgMbKZ2hzu91KT0/X0KFDlZOTo7S0NKWnp8vlcnnN++6775SRkaGZ\nM2dq48aNuvjiizVx4sQGKxwAAKAx8RnasrOz5XA4lJqaKofDoeTkZEVFRWndunVe8yrPst1www0K\nCQnRY489pi+++ELFxcUNVjwAAEBj4TO0FRQUqF27dl5j0dHRKigoOOu8Zs2aKSIioso8AAAA1F4T\nXxNcLpfCwsK8xsLCwlRaWlqneTWx2Wyy+4iQdrtNR4r2+rUeTjtStFd2u00Ohy0g69ntNpXs5exp\nbZTsLQ5YD+x2m3aV/CsAVTUuu0r+pRsC2INDP+8JQFWNy6Gf9wT0fvDj7kMBqKrx+HH3IbW7MbDH\nguLdPwZkrcaiePePsne8pl498Bnaagpo4eHhXmOhoaF+zavJxRdf4HNOr16J+qpXol/roWH06pWo\nXr3WBLuMRqtXr0T1Wv95sMto1E7fB3gcCiZ6EHz0IDh8Pj3atm1bFRYWeo0VFhbqqquu8hpr166d\n17zi4mIdOXKkylOrAAAAqD2foS0hIUFut1uZmZk6deqUli5dquLiYiUmeifsAQMG6KOPPlJubq5O\nnjypGTNmqGfPnoqIiGiw4gEAABoLm2VZlq9J27dv16RJk7Rjxw61adNGGRkZ6tChgyZPniybzaaM\njAxJ0urVqzVz5kwVFRWpU6dOev755xUZGdnQ1wEAAOA3z6/QBgAAgODiz1gBAAAYgNAGAABgAEIb\nAACAAQhtAAAABiC0AQAAGKBRhbb27dvL6XQqLi5OTqdTvXv31uuvv96gl7lixQolJydLklatWqW7\n7767QS+vsSstLVVRUVGwy2gQZ95+K2/DcXFxeuKJJ361Gj777DP17t1bkpSTk6M+ffo0yOX89a9/\n1TvvvCNJ6t27t2688UbPde7WrZvGjx+v/fv3e+qo3IvY2Fi1b9/ea3/279+vSZMmaePGjfWuix7E\neT1+zps3zzN/zpw5uu666zxzKv899NBDkhSwHjSkCRMm6Prrr9ehQ//3J7LmzJmjsWPHev7/8MMP\n1/jzu3fv1u9//3tJ0uzZs732o2PHjkpNTdXatWs98wcMGOD5/rXXXqsOHTp4blN/+ctflJubq4kT\nJzbQtT33NOT+O51OxcfHa8yYMZ5jxJ49ezyPF7+8TxcXF5+b+281Iu3bt7e+//57z9c7d+60unbt\nan388ccNdpnLly+3kpOTG2x9f1199dVWbGys5XQ6rdjYWKtXr17WvHnz6r3upEmTrJkzZ9bqZ5Yv\nX25dc801ltPp9PyLjY21hgwZUu96hgwZYn322Wf1Xqe+pkyZYq1bt86yrKp7n5iYaD3zzDNWSUmJ\nZVmWtXLlSs/3O3ToYLVv394z1+l0WpZlWaNHj65y+w2GTz/91Ordu3eDXsZPP/1kDRo0yPN1r169\nvHp6+PBh66mnnrL69u1ruVwur5/dvn271b59+yprHjp0yBowYIBVVlZWr9rowf/56quvrBtuuMH6\n4osvLMuyrNmzZ1sPP/xwjWsGqgcNpaSkxIqPj7cee+wxa9asWZ7xM6+Xr+t47733Wjk5OdXOPXXq\nlPXhhx9asbGx1T5GDRkyxFqxYkWV8Yceesj66quv6ny9TNHQ+29Zp2+Dd955p/Xoo49almVZu3fv\nttq3b1/lceRM59r+N6ozbZZlyTrjY+natGmjTp06KT8/3/P9V155Rbfeeqvi4uLUq1cvvfvuu575\nL730khITE9W9e3eNGDFCu3btkiRVVFRozpw56t27t7p3766nn35ax48fr3L5y5cv95x1mzNnjiZM\nmKD7779fTqdTAwYM0JdffumZu2nTJg0dOlSdO3dWamqqvv3223pdd5vNpqVLlyo3N1ebN2/W3/72\nN7399tv65JNP6rXulClT9Mgjj9T656699lrl5uZ6/m3evFnLli2rVy2SdPjw4XqvUV+bN2/Wzp07\n1bNnT0lV937p0qU6ePCgRo0aJUkaOHCgNm/erNzcXM2aNUutW7f2zM3NzZUkPfLII6qoqPC6/f6S\nr9vUmjVrPL/Z33HHHdq6daskqaioSOPHj1dCQoJ69eqll156SW63W5Lkdrv1hz/8QZ06dVLfvn29\nzpRs3LhRCQkJkk6fUR4xYoQef/xxdezYUbfccov+/ve/e+Z+9NFH6tevnxISEvT000/rzjvvVFZW\nVrXX44033tDAgQNrvJ7NmjXTtGnTZLPZ/L7NXHzxxYqJifGqqS5++RjyS42lB5LUtWtXxcTEaMeO\nHT527bRA9aChZGVlqXPnzho+fLjee+89nTp1qlY/v2XLFh04cEAdO3as9vsOh0P/8R//ofvuu0+z\nZs3ye92UlBTNmTOnVrWYqKH3Xzp9G+zfv7+2b9/uNX62+/S5tv+NKrT9Un5+vr799lvddNNNkqSV\nK1fqk08+UWZmpnJzc/Xoo4/q+eefl8vl0oYNG/Thhx/qgw8+0BdffKFWrVrptddekyS9+eabWrt2\nrd555x19/PHHKi0t1bRp06pcns1mk81m83y9evVq/f73v9emTZvUo0cPPfvss5KkvXv36v7779eY\nMWO0ceNb0oQjAAAPG0lEQVRG3XvvvRo1apSOHDlS5+sa6MC6e/duSdLEiRP1xz/+UZJ09OhRTZgw\nQZ07d1b37t310ksv1anW6kLwsWPHaqzzvffekyQ9+OCD2rdvn8aOHauFCxd6PTUtSSdOnFD79u21\nd+9e7dmzR506ddLEiRMVHx+vVatW6eTJk3r22WfVs2dP9ezZU9OnT/c8cOzbt0/33HOP4uPjdcst\nt5z1ur322mu64447atz7li1basaMGdqxY4c+++wzv/akffv2kqS8vLyzzqvpNrV9+3Y9/vjjmjhx\nonJzczVo0CA99NBDsixLDzzwgOx2uz799FO9++67+vrrrz0PUjNnztQPP/ygtWvXKjMz0yuA/PL2\nvH79evXo0UObNm1SWlqapk2bJrfbrcLCQj3++ON65pln9OWXX+qKK67Qli1bqq2/rKxMK1euVL9+\n/c56Pe12u7p166ZvvvnG9+b9f7fccouWLFni9/y6agw9qKio0AcffKAdO3aoS5cufu/Nr9WDuliy\nZImGDh2q2NhYRUZGavXq1bX6+WXLlvm83UpSz5499d1336m0tNSvdbt166YdO3boxx9/rFU9pvk1\n9v/HH3/UkiVL1K1bN6/xs4W2c23/G11oGzZsmOLj4xUbG6shQ4YoJiZGMTExkqS+ffvq7bffVmRk\npA4cOKCQkBC53W6VlJQoJCREhw8f1rvvvqsff/xRU6dO1Ysvvijp9I3lgQceUMuWLRUeHq5HH31U\nK1eu9PymXJPY2Fh16dJFTZo0UVJSkudG8f777yshIUG9e/eW3W5Xv379FBMTozVr1gRsH+obWKv7\nzWPSpEk6duyYPv30U61atUqff/55nR6gqwvBlQe+6up87rnn5HK5NGfOHLVq1UqzZs1SWlqaJHkd\n0H759bFjx3TZZZfpyy+/1O9+9zu9+OKLKiws1Pvvv6+///3v2rp1q+c1OzNnztTVV1+tr7/+WgsW\nLNAHH3ygDRs2VKn9wIEDysnJUa9evc56HcPDwxUXF1er0CGd3uP4+HjFx8erc+fOio+P16effur5\nfk23qTVr1qhnz57q3r27JGn48OGaOXOmdu7cqby8PP3hD39QWFiYWrRoobFjx2r58uWSTgeQ0aNH\nKyIiQi1bttTIkSNrrK1169YaOHCg7Ha7br/9dh0/flzFxcX64IMPlJiYqMTERDkcDo0ePVrNmzev\ndo2tW7cqNDRUl112mc+9iIiIUElJid97d/311+t//ud/fN4vfal8DGlsPRg3bpznsfOGG27Q8uXL\n9dprr+naa6/1zFm7dm2VvTkznASqB4GWm5uro0ePeh4Phw0bpoULF9ZqjU2bNqlDhw4+50VERMiy\nLL9/CXc4HJ7Hnt+qhtr/ytuj0+nU9ddfr1GjRumWW27RY4895pljWZZuvvlmr9vsmcetc23/mwS7\ngF/bu+++q3bt2kk6/ZTExIkTNW7cOM2dO1dlZWWaNm2aNmzYoNatW3vOblRUVKhjx4564YUXlJmZ\nqVmzZumyyy7TxIkTddNNN2nfvn164okn5HA4JJ2+EYSEhGjfvn1nreXMv8vapEkTT9rft2+fPv/8\nc8XHx3vWO3XqlDp16lSv6z5s2DDZ7Xa53W6dPHlSPXr08AqsiYmJPgNr3759NXXq1CphyO1265NP\nPtGyZct0wQUXSJL+/Oc/KyQkpNpa8vPzva6fzWbT4sWL1bZtWy1btkzjx49Xy5YtJUmPPvqofve7\n32nq1KlnrTMsLMznHvzyN6qBAweqadOmatq0qVasWKHFixfroosuknT6zN348eP14IMP6rzzztOm\nTZu0evVqde/e3esgfaacnBxdddVVOu+883zWUtvQYbPZ1KxZM68zLb9U023q559/9uxnpRtvvFFb\ntmxRWFiYIiIiPOOtW7dWUVGRTp06pUOHDqlFixae71166aV+X7Z0+r5z8OBBXXLJJV5zW7VqVe0a\n+/fvrzFM/NLhw4fVrFkzv+ZKUvPmzT3X6WzXw5czH0Oq81vtwcyZM3XTTTepuLhYEyZMkN1u9zw1\nW6lPnz5nfeovUD0ItPfee0+HDx9Wjx49JEmnTp1SSUmJtm3b5vcaBw4c8Ou2e/jwYdntdq9++9Ki\nRQsdOHDA7/mmaaj9r7w9WpalzMxMzZs3TzfffLOaNm3qmWOz2fT5558rNDS0xrXPpf1vdKHtzIN2\nVFSU7rrrLo0bN06S9PLLL8uyLK1fv15NmzbVvn37PK/52L9/v6688kotWLBALpdLCxcu1COPPKJv\nvvlGzZs317PPPut5mqC8vFw//fSTrrjiCs9rkmqjefPm6t+/v+dMnnT6XTH/9m//Vp+r3iCBtdKR\nI0dUVlbmdVC6/PLLa6zlmmuu0dKlS6v93tlCcEREhKZOnars7OwqddaWzWbTxRdfLEkqLi5WaWmp\n7r77bk8grQzLbrdbTz/9tGbPnq2ZM2dq/PjxuummmzRt2jRFRUV5rVnb0FHbA9fhw4c9Ibc2WrZs\nqe+++85r7KWXXtLw4cPlcrlUUlLiOYjs2rVLERERatKkiVq0aKG9e/d6zqZUvmOzNlq1alXlNZk1\nPQDa7Xa/emlZlr766iuNHj3a7zoq17Xb6/cEw9meSjmb30oPIiMjNWvWLA0aNEhTpkyp9qUgNQlU\nDwLp2LFjWr16td5++22vx6znnntOCxYs8Ps+arPZVF5e7nPe559/rhtuuMGvX+wqlZeX1/o+b4pf\nY/9tNpvS0tK0e/du3X///crKyvL6BcfXffpc2v9z554TBEeOHNGyZcsUFxcnSTp+/LjOO+882e12\nHT582BOaTp06pby8PN1///3atWuXwsLCdOGFFyoiIsLzNMScOXN06NAhlZWVacaMGRo1alStH9wr\n5/fv31+ffvqp5+m3b775RklJSfrv//7vel3f6gJrdna2JO/AumLFCq+3VZ8ZWDdt2qTk5GQ98sgj\nXutFRkaqadOmXgeC9evX6/333691nS1atNCf//xnff311/r666+VnZ2trKwsXX755ZoxY4Zn7co6\na9pnu92usrIyz9eHDx+u8enSZs2aKSQkRCtWrPBc7hdffKGVK1cqJCRE//u//6uRI0dqzZo1Wr16\ntY4eParZs2dXe5n+hI5jx45p8+bNtXo9UGW9tXnwqNybW2+9VevXr1d2drbnt87Vq1erVatWSkhI\n0PPPP68TJ07owIEDmj17tpKSkiRJgwYN0ty5c3Xo0CEdOnRIb7zxRq0ve8CAAdqwYYO+/PJLlZeX\n66233qoxMFxyySVeb/evzs8//6ynnnpK5513nqfO6i73lw4ePKgmTZr4HaoD5bfYgwsuuEDPP/+8\nli5dqi+++MLveoLVg7PJysrSlVdeqdjYWEVFRXn+JScn6x//+Iffb25q1arVWfetrKxMq1at0oIF\nCzwfYeGvgwcP1nhm1HS/1v5Lp5+1Of/8871+0fDnOH0u7X+jOtNms9mUkpLiOfA1bdpUXbt21fTp\n0yVJDz/8sJ544gl17txZERERSkpK0hVXXKEffvhB/fr10/bt23XXXXfp+PHjatu2rV599VVJ0ujR\no3Xq1Cmlpqbq6NGjuvbaa/X666/X+rfJyoNxmzZt9Morr+hPf/qTdu7cqaioKD311FNVnoqoj9oE\n1vz8fL3wwgtasGCBLr/8ck9gPTM82O123XbbbXr11Vf14osvyuVyafr06brnnntqXdugQYM0Z84c\ntW3bVs2aNdMrr7yijz76SGvWrNGxY8eq1Gmz2TxvGAgJCfG8aSE6Olo7d+5UQUGBLr30Uv3lL3/x\nqvnMO6vdbtfAgQP1pz/9SdOmTZPD4dCkSZO0f/9+LVy4UHPnzlVUVJQmT56syMhINWnSpNqn5vw5\n4O3atUvPPfecOnToUOUFsb6Ul5d7elZ5HS655BJ9+OGH1c6vvL7R0dGaOXOmnn/+ee3du1dXX321\nZz9efvllPfvss+rTp49sNpsGDRrkOfv8wAMP6Pjx4+rfv7/Cw8M1cOBAffDBB37VWnnZl112mV54\n4QU988wzOn78uPr166dWrVp5PUVR6brrrpMk7dy5U1deeaVnfOzYsbLb7bLZbLrooovUvXt3LViw\noNqzFTWF2ry8PHXq1MnztGFdnPkYUqkx9KC6Pe3SpYuGDh2qyZMn+/3LWSB6EGhLliyp9p2y3bp1\nU2RkpJYsWeL5XLyz6dq1q/Ly8rzmrl271nN/Pe+88/Tv//7vevXVV9W1a9cqP1/T7bbyMbi6n/kt\naMj9/6WQkBA9++yzuvvuu7V27Vq1b9/e5y/B59z+B/gjRHCOat++vedzv+Li4qwuXbpYjzzyiHXo\n0CHLsiyrsLDQuuOOOyyn02ndfPPN1owZM6xbbrnF+uc//2lZlmW9+uqrVmJiouV0Oq3k5GQrLy/P\nsizLevLJJ63p06dblmVZR48etZ588kmrW7duVvfu3a3Zs2dXW4uvz64rKyuzXnnlFatXr15Wp06d\nrP/8z/+0fvjhB7/qnDdvnhUbG2vNnTvXsizLmj59utW9e3crMTHR+tvf/mYlJCRYe/bs8Xw+z4kT\nJzyXe/z4cSsjI8Pq0aOHFR8fb6Wnp1sHDx60LMuy9u/fb40YMcLq3LmzFR8fbz355JPVfrZPUVGR\n1aFDB6/vnbn3HTt2tPr06WO9+OKL1f782T6Da+7cudZTTz1V476dq/bu3Vvls826detmffnll9XO\nz8jIsF5//fWA1/HQQw9Zy5YtC/i6JqAHDS8vL8+67bbbAr7uP//5TystLS3g6/7WNJb9J7QBATZy\n5EjrH//4R8DXHTJkiJWdnR3wdRvali1brO7du1t79uyxysvLrUWLFlmdO3e2jh07Vu38PXv2WP37\n97fKy8sDVsP+/futW2+91Tp16lTA1jQJPfh1jBgxIuAfxDpy5Ehrw4YNAV3zt6ox7H+jfk0b0BDG\njBnj+fM/gfLtt98qLCys1q+BOxfceOONGjlypIYPH67OnTtrxYoVmjdvns4///xq57du3Vq33357\nQPdw9uzZmjRpkufNLY0NPfh1TJo0yetPe9VXTk6OIiMjA/rSmN+yxrD/Nsuq41uhANRoypQpuvnm\nm73eYVsfo0eP1uOPP37Wj5oAAPy2EdoAAAAMwNOjAAAABiC0AQAAGIDQBgAAYABCGwAAgAEIbQAA\nAAb4fxQYorU412LmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x280f6f2e6d8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "labels = [\"Baseline\", \"Basic Features (DT)\", \"Encoding (DT)\", \"Encoding (RF)\", \"All (DT)\", \"All (RF)\"]\n",
    "scores = [0.3728, 0.5488, 0.5371, 0.5460, 0.5398, 0.5904]\n",
    "sns.set(style=\"darkgrid\", context=\"talk\")\n",
    "sns.barplot(labels, scores);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our initial F1 score was 0.3782 and by including new features, the overall F1 score improved to 0.5904\n",
    "\n",
    "Now that we have built a model that successfully improved the performance of predicting winning teams, let us test to see if our model is able to do the same on the 2015 season."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing our Model on the 2015 Season\n",
    "\n",
    "Let us predict the results of the 2015 season by using our previous model. One thing to note is we will need to create all the features again on the 2015 dataset. In the future if the model is sufficient, a data pipeline could be built so that we could feed the data directly to the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Parse files\n",
    "games_filename = ('leagues_NHL_2015_games_games.csv')\n",
    "playoff_filename = ('leagues_NHL_2015_games_games_playoffs.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Combine files\n",
    "results1 = pd.read_csv(games_filename, parse_dates=[\"Date\"])\n",
    "results2 = pd.read_csv(playoff_filename, parse_dates=[\"Date\"])\n",
    "games_2015 = results1.append(results2, ignore_index= True)\n",
    "\n",
    "# Fix the names of the columns\n",
    "games_2015.columns = [\"Date\", \"Visitor Team\", \"Visitor Goals\", \"Home Team\", \"Home Goals\", \"Overtime\", \"Blank\", \"Att\", \"LOG\", \"Notes1\", \"Notes2\"]\n",
    "\n",
    "# Delete the columns we do not need\n",
    "games_2015 = games_2015.drop(games_2015.columns[[5,6,7,8,9,10]], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create feature HomeWin\n",
    "games_2015[\"HomeWin\"] = games_2015[\"Visitor Goals\"] < games_2015[\"Home Goals\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Create HomeLastWin and VisitorLastWin\n",
    "games_2015[\"HomeLastWin\"] = False\n",
    "games_2015[\"VisitorLastWin\"] = False\n",
    "\n",
    "won_last = defaultdict(int)\n",
    "\n",
    "for index, row in games_2015.iterrows(): # for each row\n",
    "    home_team = row[\"Home Team\"]\n",
    "    visitor_team = row[\"Visitor Team\"]\n",
    "    row[\"HomeLastWin\"] = won_last[home_team]\n",
    "    row[\"VisitorLastWin\"] = won_last[visitor_team]\n",
    "    games_2015.ix[index] = row\n",
    "    # Set which team won\n",
    "    won_last[home_team] = row[\"HomeWin\"] # set HomeLastWin as True if \"HomeWin\" is True\n",
    "    won_last[visitor_team] = not row[\"HomeWin\"] # otherwise set \"VisitorLastWin\" as True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create HomeWinStreak and VisitorWinStreak\n",
    "games_2015[\"HomeWinStreak\"] = 0\n",
    "games_2015[\"VisitorWinStreak\"] = 0\n",
    "\n",
    "from collections import defaultdict\n",
    "win_streak = defaultdict(int)\n",
    "\n",
    "for index, row in games_2015.iterrows():\n",
    "    home_team = row[\"Home Team\"]\n",
    "    visitor_team = row[\"Visitor Team\"]\n",
    "    row[\"HomeWinStreak\"] = win_streak[home_team]\n",
    "    row[\"VisitorWinStreak\"] = win_streak[visitor_team]\n",
    "    games_2015.ix[index] = row\n",
    "    \n",
    "    if row[\"HomeWin\"]:\n",
    "        win_streak[home_team] += 1 # if \"HomeWin\" is True, increase home_team by 1\n",
    "        win_streak[visitor_team] = 0\n",
    "    else:\n",
    "        win_streak[home_team] = 0\n",
    "        win_streak[visitor_team] += 1 # if \"HomeWin\" is False, increase visitor_team by 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Parse ranking file\n",
    "ladders_filename = ('leagues_NHL_2014_standings_standings.csv')\n",
    "ladder_2014 = pd.read_csv(ladders_filename, skiprows = [0,1], index_col=\"Team\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Create HomeTeamRanksHigher\n",
    "def home_team_ranks_higher(row):\n",
    "    home_team = row[\"Home Team\"]\n",
    "    visitor_team = row[\"Visitor Team\"]\n",
    "    \n",
    "    if home_team == \"Arizona Coyotes\":\n",
    "        home_team = \"Phoenix Coyotes\"\n",
    "    if visitor_team == \"Arizona Coyotes\":\n",
    "        visitor_team = \"Phoenix Coyotes\"\n",
    "    \n",
    "    home_rank = ladder_2014.loc[home_team][\"Rk\"]\n",
    "    visitor_rank = ladder_2014.loc[visitor_team][\"Rk\"]\n",
    "    return home_rank < visitor_rank # Remember, a higher ranking is a lower number (in other words, 1st place > 2nd place)\n",
    "\n",
    "# This will return a boolean result, either True or False and the command below will take whatever this boolean result is\n",
    "# and place it into \"HomeTeamRanksHigher\"\n",
    "games_2015[\"HomeTeamRanksHigher\"] = games_2015.apply(home_team_ranks_higher, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create HomeTeamWonLast\n",
    "last_match_winner = defaultdict(int)\n",
    "\n",
    "def home_team_won_last(row):\n",
    "    home_team = row[\"Home Team\"] # get the strings of team names into the respective variables\n",
    "    visitor_team = row[\"Visitor Team\"]\n",
    "    \n",
    "    # Sort team names for a consistent ordering (sorts by alphabetical order)\n",
    "    teams = tuple(sorted([home_team, visitor_team]))\n",
    "    \n",
    "    # So if the home team won the previous game, the \"result\" of this function will be 1, otherwise 0 if they didn't win\n",
    "    result = 1 if last_match_winner[teams] == row[\"Home Team\"] else 0\n",
    "    \n",
    "    # Update record for next encounter and place the string of the winning team into the dictionary \"last_match_winner\"\n",
    "    winner = row[\"Home Team\"] if row[\"HomeWin\"] else row[\"Visitor Team\"]\n",
    "    \n",
    "    last_match_winner[teams] = winner\n",
    "    return result\n",
    "\n",
    "games_2015[\"HomeTeamWonLast\"] = games_2015.apply(home_team_won_last, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Extract all the features we need\n",
    "X_basic_features_2015 = games_2015[[\"HomeLastWin\", \"VisitorLastWin\", \"HomeTeamRanksHigher\", \"HomeTeamWonLast\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Important note:\n",
    "\n",
    "We **need to use the same encoder as before** because the team names have already been encoded and attempting to encode them again will result in different numbers for different teams."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "home_teams_2015 = encoding.transform(games_2015[\"Home Team\"].values)\n",
    "visitor_teams_2015 = encoding.transform(games_2015[\"Visitor Team\"].values)\n",
    "X_teams_2015 = np.vstack([home_teams_2015, visitor_teams_2015]).T\n",
    "X_teams_2015 = onehot.transform(X_teams_2015).todense()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1319, 64)"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Combine everything\n",
    "X_all_2015 = np.hstack([X_basic_features_2015, X_teams_2015])\n",
    "X_all_2015.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2015 Baseline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# This will be the class we are trying to predict for our model, whether the home team won or not\n",
    "y_test = games_2015[\"HomeWin\"].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Home team wins 54.36% of matches\n",
      "F1 Score: 0.3829\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "      False       0.00      0.00      0.00       602\n",
      "       True       0.54      1.00      0.70       717\n",
      "\n",
      "avg / total       0.30      0.54      0.38      1319\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Kevin\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1074: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n",
      "C:\\Users\\Kevin\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1074: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "print(\"Home team wins {:.2f}% of matches\".format(100*np.mean(y_test)))\n",
    "y_pred = [1] * len(y_test)\n",
    "print(\"F1 Score: {:.4f}\".format(f1_score(y_test, y_pred, pos_label=None, average='weighted')))\n",
    "print(classification_report(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**We are attempting to outperform an F1 score of 0.38.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "      False       0.51      0.44      0.47       602\n",
      "       True       0.58      0.65      0.61       717\n",
      "\n",
      "avg / total       0.55      0.55      0.55      1319\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Predict 2015 results using our 2014 model\n",
    "y_pred = grid.predict(X_all_2015)\n",
    "\n",
    "from sklearn.metrics import classification_report\n",
    "print(classification_report(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This results in getting 55.4% of predictions correct!\n"
     ]
    }
   ],
   "source": [
    "print(\"This results in getting {:.1f}% of predictions correct!\".format(100*np.mean(y_pred == y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusions\n",
    "\n",
    "Our final model we produced based off of the 2014 NHL data uses the Random Forest Classifier with the following parameters:\n",
    "- **n_features** = 'auto'\n",
    "- **n_estimators** = 100\n",
    "- **criterion** = 'gini'\n",
    "- **min_samples_leaf** = 4\n",
    "\n",
    "By using this model, we attempt to predict for the 2015 season. Here are the results for the 2015 season:\n",
    "- **Baseline F1 score**: 0.38 \n",
    "- **Model F1 score**: 0.55\n",
    "\n",
    "\n",
    "- **Baseline accuracy**: 54.36%\n",
    "- **Model accuracy**: 55.4%\n",
    "\n",
    "While we significantly improved our F1 score, the overall accuracy has barely improved; guessing at random is almost just as good as using our model to predict."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Special Thanks\n",
    "\n",
    "Special thanks to Robert Layton for the inspiration of creating this project. Some of the earlier steps were similarly replicated from his book, Learning Data Mining with Python.\n",
    "\n",
    "<img src=\"book_cover.jpg\">\n",
    "\n",
    "Here is a link to an excerpt to his book: https://www.packtpub.com/books/content/predicting-sports-winners-decision-trees-and-pandas"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}