logistic_regression.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "logistic_regression.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XaoTTFUduK8L",
        "colab_type": "text"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1kqG3UhIcnscMC3sm4hdEHFmIMpmZWEEG?usp=sharing)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "laB2JbdN0cvV",
        "colab_type": "text"
      },
      "source": [
        "# Introducing Logistic Regression\n",
        "Logistic Regression is a classification algorithm. It is used to predict a binary outcome (1 / 0, Yes / No, True / False) given a set of independent variable/s. You can also think of logistic regression as a special case of linear regression when the outcome variable is categorical. Logistic Regression is a supervised machine learning algorithm/model.\n",
        "\n",
        "## Agenda\n",
        "*  About Dataset\n",
        "*  Loading Libraries\n",
        "*  Loading Data\n",
        "*  Understanding Data\n",
        "*  Separating Input/Independent and Output/Dependent Variables\n",
        "*  Splitting the data\n",
        "*  Building Model\n",
        "*  Prediction\n",
        "*  Model Performance\n",
        "\n",
        "## About Dataset\n",
        "The dataset has two columns - age (age of the person/customer) and bought_insurance (whether the customer bought insurance or not). If bought_insurance = 1, the customer bought insurance and if bought_insurance = 0, the customer did not buy the insurance.\n",
        "\n",
        "Dataset Link: [insurance_data](https://raw.githubusercontent.com/codebasics/py/master/ML/7_logistic_reg/insurance_data.csv)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MVGxkd1K3xrW",
        "colab_type": "text"
      },
      "source": [
        "## Loading Libraries\n",
        "All Python capabilities are not loaded to our working environment by default (even they are already installed in your system). So, we import each and every library that we want to use.\n",
        "\n",
        "In data science, numpy and pandas are most commonly used libraries. Numpy is required for calculations like means, medians, square roots, etc. Pandas is used for data processin and data frames. We chose alias names for our libraries for the sake of our convenience (numpy --> np and pandas --> pd).\n",
        "\n",
        "**We can import all the libraries that we think might be needed or can import as we go along.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8Zo1haly0Rap",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zp46CaAG4-79",
        "colab_type": "text"
      },
      "source": [
        "## Loading Data\n",
        "Pandas module is used for reading files. We have our data in '.csv' format. We will use 'read_csv()' function for loading the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cDs7FZ-Y4bMN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# In read_csv() function, we have passed the raw data link at github\n",
        "data_location = \"https://raw.githubusercontent.com/codebasics/py/master/ML/7_logistic_reg/insurance_data.csv\"\n",
        "data = pd.read_csv(data_location)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sQgP0WXi5pl0",
        "colab_type": "text"
      },
      "source": [
        "## Understanding Data\n",
        "Let's check how our data looks. This can be done using head() method."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-gwOLzhX5njJ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "fc58f689-88a2-4a50-91e3-67817ade0aa9"
      },
      "source": [
        "data.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>bought_insurance</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>22</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>25</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>47</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>52</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>46</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   age  bought_insurance\n",
              "0   22                 0\n",
              "1   25                 0\n",
              "2   47                 1\n",
              "3   52                 0\n",
              "4   46                 1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rA4wZUbx51KM",
        "colab_type": "text"
      },
      "source": [
        "There are two columns:\n",
        "\n",
        "*  age: The age of the customer\n",
        "*  bought_insurance: If the customer bought insurance (1) or not (0). This is our target variable which we are interested to know.\n",
        "\n",
        "Since our target variable has only two different classes/values, we can say it as a binary classification problem. And Logistic Regression is used for binary classification problems."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qEYG8who6aXb",
        "colab_type": "text"
      },
      "source": [
        "Looking the relationship between age and bought_insurance using scatter plot."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UOyg4-0K5xDv",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "outputId": "2c5c7a82-ee96-45c6-85dc-f8d154c84580"
      },
      "source": [
        "plt.scatter(data.age,data.bought_insurance,marker='+',color='red')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.collections.PathCollection at 0x7fb2e5c9fb38>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAOoUlEQVR4nO3df6zdd13H8eeLlokCUqFXsqyFzljEBmFbbgoEopUfppukTRSXVWeQII0JNRjxx1AzccZEJAEhTnQgokSYdYo2s1rJ3OaPuLlbx6+2Fi9l2FvBXWCbiQRm9e0f51s4uz295/T23J67z3k+kpt7Pj/u+b7vp+e87iffc06/qSokSY9/T5h0AZKk8TDQJakRBrokNcJAl6RGGOiS1Ij1kzrwxo0ba8uWLZM6vCQ9Lh0+fPgLVTUzaGxigb5lyxbm5uYmdXhJelxK8tlzjXnKRZIaYaBLUiMMdElqhIEuSY0w0CWpEUMDPcn7kjyY5JPnGE+SdyWZT/LxJFeNv8zG7NjR+9Las5b+bZarZaVjq3G8cVtLv/dKf25CaznKDv39wM5lxq8GtnZfe4F3X3hZkqTzNfR96FX1d0m2LDNlN/CH1ft/eO9JsiHJpVX1uTHV2I4zf5Xvvvux7bvumkAxeoy19G+zXC0rHVuN443bWvq911Kd52Ec59AvA072tRe6vrMk2ZtkLsnc4uLiGA4tSTojo1zgotuh315Vzxswdjvw61X1D137DuDnq2rZj4HOzs7W1H5S1J352rWW/m2Wq2WlY6txvHFbS7/3Sn9uFdcyyeGqmh00No4d+ilgc197U9cnSbqIxrFD/35gH3AN8ELgXVW1fdh9TvUOXZJWaLkd+tAXRZN8CNgBbEyyAPwy8ESAqvod4CC9MJ8Hvgy8djxlS5LOxyjvctkzZLyAN4ytIknSivhJUUlqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGjFSoCfZmeR4kvkkNwwYf1aSO5Pcn+TjSa4Zf6mSpOUMDfQk64CbgauBbcCeJNuWTPslYH9VXQlcB/z2uAuVJC1vlB36dmC+qk5U1aPArcDuJXMK+Obu9tOA/xhfiZKkUYwS6JcBJ/vaC11fv7cA1ydZAA4CPznojpLsTTKXZG5xcXEF5UqSzmVcL4ruAd5fVZuAa4APJDnrvqvqlqqararZmZmZMR1akgSjBfopYHNfe1PX1+91wH6Aqvon4EnAxnEUKEkazSiBfh+wNcnlSS6h96LngSVz/h14OUCS76QX6J5TkaSLaGigV9VpYB9wCDhG790sR5LclGRXN+1NwOuTfAz4EPBjVVWrVbQk6WzrR5lUVQfpvdjZ33dj3+2jwEvGW5ok6Xz4SVFJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUiJECPcnOJMeTzCe54Rxzrk1yNMmRJB8cb5mSpGHWD5uQZB1wM/BKYAG4L8mBqjraN2cr8GbgJVX1UJJvXa2CJUmDjbJD3w7MV9WJqnoUuBXYvWTO64Gbq+ohgKp6cLxlSpKGGSXQLwNO9rUXur5+zwGek+Qfk9yTZOegO0qyN8lckrnFxcWVVSxJGmhcL4quB7YCO4A9wHuSbFg6qapuqarZqpqdmZkZ06ElSTBaoJ8CNve1N3V9/RaAA1X1P1X1GeBT9AJeknSRjBLo9wFbk1ye5BLgOuDAkjl/Tm93TpKN9E7BnBhjnZKkIYYGelWdBvYBh4BjwP6qOpLkpiS7ummHgC8mOQrcCfxsVX1xtYqWJJ0tVTWRA8/Oztbc3NxEji1Jj1dJDlfV7KAxPykqSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjRgr0JDuTHE8yn+SGZeb9YJJKMju+EiVJoxga6EnWATcDVwPbgD1Jtg2Y91TgjcC94y5SkjTcKDv07cB8VZ2oqkeBW4HdA+b9KvBW4CtjrE+SNKJRAv0y4GRfe6Hr+5okVwGbq+ovl7ujJHuTzCWZW1xcPO9iJUnndsEviiZ5AvB24E3D5lbVLVU1W1WzMzMzF3poSVKfUQL9FLC5r72p6zvjqcDzgLuSPAC8CDjgC6OSdHGNEuj3AVuTXJ7kEuA64MCZwap6pKo2VtWWqtoC3APsqqq5ValYkjTQ0ECvqtPAPuAQcAzYX1VHktyUZNdqFyhJGs36USZV1UHg4JK+G88xd8eFlyVJOl9+UlSSGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1YqRAT7IzyfEk80luGDD+00mOJvl4kjuSPHv8pUqSljM00JOsA24Grga2AXuSbFsy7X5gtqqeD9wG/Ma4C5UkLW+UHfp2YL6qTlTVo8CtwO7+CVV1Z1V9uWveA2wab5mSpGFGCfTLgJN97YWu71xeB/zVoIEke5PMJZlbXFwcvUpJ0lBjfVE0yfXALPC2QeNVdUtVzVbV7MzMzDgPLUlTb/0Ic04Bm/vam7q+x0jyCuAXge+pqq+OpzxJ0qhG2aHfB2xNcnmSS4DrgAP9E5JcCfwusKuqHhx/mZKkYYYGelWdBvYBh4BjwP6qOpLkpiS7umlvA54C/EmSjyY5cI67kyStklFOuVBVB4GDS/pu7Lv9ijHXJUk6T35SVJIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRowU6El2JjmeZD7JDQPGvyHJH3fj9ybZMu5Cv2bDht7XIDt29L7O13I/t9Kxi3m81ahRulCr8djz8bysoYGeZB1wM3A1sA3Yk2TbkmmvAx6qqm8H3gG8ddyFSpKWt36EOduB+ao6AZDkVmA3cLRvzm7gLd3t24DfSpKqqrFVemZX/sgjj20//PDX/2LffXfv+5n2XXctf5/L/dxKxy7m8c4YZ43ShVqNx56P55GMcsrlMuBkX3uh6xs4p6pOA48Az1h6R0n2JplLMre4uLiyiiVJA2XYJjrJq4GdVfXjXftHgRdW1b6+OZ/s5ix07U93c75wrvudnZ2tubm586+4f2e+1Er/ai/3cysdu5jHW40apQu1Go89H88kOVxVs4PGRtmhnwI297U3dX0D5yRZDzwN+OL5lypJWqlRdujrgU8BL6cX3PcBP1xVR/rmvAH4rqr6iSTXAT9QVdcud78r3qFL0hRbboc+9EXRqjqdZB9wCFgHvK+qjiS5CZirqgPA7wEfSDIPfAm4bnzlS5JGMcq7XKiqg8DBJX039t3+CvBD4y1NknQ+/KSoJDXCQJekRhjoktQIA12SGjH0bYurduBkEfjsRT7sRuCcH3aaUq7J2VyTwVyXs01iTZ5dVTODBiYW6JOQZO5c79+cVq7J2VyTwVyXs621NfGUiyQ1wkCXpEZMW6DfMukC1iDX5GyuyWCuy9nW1JpM1Tl0SWrZtO3QJalZBrokNaLZQE+yOcmdSY4mOZLkjV3/05N8JMm/dd+/ZdK1XixJnpTkn5N8rFuTX+n6L+8u7j3fXez7kknXerElWZfk/iS3d+2pXpMkDyT5RJKPJpnr+qb2uQOQZEOS25L8a5JjSV681tak2UAHTgNvqqptwIuAN3QXt74BuKOqtgJ3dO1p8VXgZVX1AuAKYGeSF9G7qPc7uot8P0Tvot/T5o3Asb62awLfW1VX9L3PepqfOwDvBP66qp4LvIDe42VtrUlVTcUX8BfAK4HjwKVd36XA8UnXNqH1+CbgX4AX0vuk2/qu/8XAoUnXd5HXYhO9J+PLgNuBuCY8AGxc0je1zx16V2H7DN0bSdbqmrS8Q/+aJFuAK4F7gWdW1ee6oc8Dz5xQWRPRnVr4KPAg8BHg08DD1bu4Nwy+CHjrfhP4OeD/uvYzcE0K+Jskh5Ps7fqm+blzObAI/H53au69SZ7MGluT5gM9yVOAPwV+qqr+q3+sen9Wp+p9m1X1v1V1Bb1d6XbguRMuaaKSvAp4sKoOT7qWNealVXUVcDW905Xf3T84hc+d9cBVwLur6krgv1lyemUtrEnTgZ7kifTC/I+q6s+67v9Mcmk3fim9nerUqaqHgTvpnU7Y0F07FgZfBLxlLwF2JXkAuJXeaZd3Mt1rQlWd6r4/CHyY3h//aX7uLAALVXVv176NXsCvqTVpNtCThN61To9V1dv7hg4Ar+luv4beufWpkGQmyYbu9jfSe03hGL1gf3U3barWpKreXFWbqmoLvWvh/m1V/QhTvCZJnpzkqWduA98HfJIpfu5U1eeBk0m+o+t6OXCUNbYmzX5SNMlLgb8HPsHXz43+Ar3z6PuBZ9H773uvraovTaTIiyzJ84E/oHex7ycA+6vqpiTfRm93+nTgfuD6qvrq5CqdjCQ7gJ+pqldN85p0v/uHu+Z64INV9WtJnsGUPncAklwBvBe4BDgBvJbuecQaWZNmA12Spk2zp1wkadoY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakR/w+oGVyw4/IRsgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m003O7uL6xc8",
        "colab_type": "text"
      },
      "source": [
        "We can easily observe from the scatter plot that generally the customer who is of age less than 30 years has not bought the insurance."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uWzLw-d07EQs",
        "colab_type": "text"
      },
      "source": [
        "## Separating Input and Output Variables\n",
        "Before building any machine learning model, we always separate the input variables and output variables. Input variables are those quantities whose values are changed naturally in an experiment, whereas output variable is the one whose values are dependent on the input variables. So, input variables are also known as independent variables as its values are not dependent on any other quantity, and output variable/s are also known as dependent variables as its values are dependent on other variable i.e. input variables. Like here in this data, we can see that whether a person will buy insurance or not is dependent on the age of that person\n",
        "\n",
        "By convention input variables are represented with 'X' and output variables are represented with 'y'."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BslsxtTY6sd3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "X = data[['age']]     # input variable\n",
        "\n",
        "y = data['bought_insurance']    # output variable"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UsgEP0tH-ArH",
        "colab_type": "text"
      },
      "source": [
        "If you notice the above code cell, I have used two square brackets while taking input variables and only one square bracket while taking output variable. Why?\n",
        "\n",
        "All machine learning algorithm accepts input variables as a 2D array and output variable as 1D array. Using two square brackets while selecting the input variables gives you the shape of input variable/s as 2D, but if you use only one square bracket, the shape will be 1D as you can see in the case of y.\n",
        "\n",
        "Let's check the shapes of X and y."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UnfWV_JW9_Br",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "2f58a923-c447-4195-ed57-8ea3c40ec8d6"
      },
      "source": [
        "print(\"Shape: \", X.shape, \"Dimension: \", X.ndim)\n",
        "print(\"Shape: \", y.shape, \"Dimension: \", y.ndim)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Shape:  (27, 1) Dimension:  2\n",
            "Shape:  (27,) Dimension:  1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9J3XXD4A_K61",
        "colab_type": "text"
      },
      "source": [
        "## Splitting the data into Train and Test Set\n",
        "We want to check the performance of the model that we built. For this purpose, we always split (both input and output data) the given data into training set  which will be used to train the model, and test set which will be used to check how accurately the model is predicting outcomes.\n",
        "\n",
        "For this purpose we have a class called 'train_test_split' in the 'sklearn.model_selection' module."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XQ1snUZP-4_e",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# import train_test_split\n",
        "from sklearn.model_selection import train_test_split"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ev1mslygAtL5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# split the data\n",
        "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.3, random_state = 42)\n",
        "\n",
        "# X_train: independent/input feature data for training the model\n",
        "# y_train: dependent/output feature data for training the model\n",
        "# X_test: independent/input feature data for testing the model; will be used to predict the output values\n",
        "# y_test: original dependent/output values of X_test; We will compare this values with our predicted values to check the performance of our built model.\n",
        " \n",
        "# test_size = 0.30: 30% of the data will go for test set and 70% of the data will go for train set\n",
        "# random_state = 42: this will fix the split i.e. there will be same split for each time you run the code"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J9tTegE3Bn0D",
        "colab_type": "text"
      },
      "source": [
        "## Building Model\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OiJazgZFjGQu",
        "colab_type": "text"
      },
      "source": [
        "Now we are finally ready, and we can train the model.\n",
        "\n",
        "First, we need to import our model - Logistic Regression (again, using the sklearn library).\n",
        "\n",
        "Then we would feed the model both with the data (X_train) and the answers for that data (y_train)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nZHXi5CfBlns",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# import Logistic Regression from sklearn.linear_model\n",
        "from sklearn.linear_model import LogisticRegression"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7FCfCfawjDyK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "log_model = LogisticRegression()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "caHe1oKSjuq-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        },
        "outputId": "2cbc400c-39f4-4997-95da-db909797678a"
      },
      "source": [
        "# Fit the model\n",
        "log_model.fit(X_train, y_train)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
              "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
              "                   multi_class='auto', n_jobs=None, penalty='l2',\n",
              "                   random_state=None, solver='lbfgs', tol=0.0001, verbose=0,\n",
              "                   warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pkxVjh8wkB91",
        "colab_type": "text"
      },
      "source": [
        "The training happens in the third line (the \"fit\" function)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UON4736ekMZw",
        "colab_type": "text"
      },
      "source": [
        "## Prediction\n",
        "Now logistic regression model (i.e. log_model) is trained using X_train and y_trian data. Let's predict the target value (i.e. bought_insurance) for the X_test data. We use \"predict()\" method for prediction."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4Fy1LxR_kA-f",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "predictions = log_model.predict(X_test)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eiA_FMiwkxN4",
        "colab_type": "text"
      },
      "source": [
        "We already have actual target values (i.e. y_test) for X_test. Let's compare y_test and the predicted value for X_test by our log_model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3kSld42hkvcc",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "bc3348a3-0ad5-46ee-ba64-77f8fd29ffcf"
      },
      "source": [
        "y_test.values"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1, 0, 1, 0, 0, 0, 1, 1, 0])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Csji5Qu9lJW7",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "6798ae5b-3845-46d9-e724-572cd118cdd3"
      },
      "source": [
        "predictions"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1, 0, 1, 0, 0, 0, 0, 1, 0])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aBpanTWzlZmU",
        "colab_type": "text"
      },
      "source": [
        "There is one person who had actually bought insurance but our model predicted that the person had not bought insurance. So, there is one misclassified data by our model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dz0ebyPjlzkp",
        "colab_type": "text"
      },
      "source": [
        "## Model Performance\n",
        "We can also check how accurate our model is performing using the 'accuracy_score' class from 'sklearn.metrics'."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K2TpjY1NlNlS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.metrics import accuracy_score"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v0KrwUNsmLV6",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "8391adea-f34d-40df-e9dc-ef78ae6ac09f"
      },
      "source": [
        "accuracy_score(y_test, predictions)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.8888888888888888"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lBMN-wOamc7i",
        "colab_type": "text"
      },
      "source": [
        "Our model is predicting 88.9% correct results."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dPR3NvaJmSIF",
        "colab_type": "text"
      },
      "source": [
        "### Thanks for reading the Notebook!!!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G2SmEVskR65o",
        "colab_type": "text"
      },
      "source": [
        "## Exercise\n",
        "**Instruction:**\n",
        "\n",
        "Use the raw data github link: https://raw.githubusercontent.com/dphi-official/Datasets/master/HR_comma_sep.csv \n",
        "\n",
        "Or you can download it here from [here](https://www.kaggle.com/giripujar/hr-analytics)\n",
        "\n",
        "**Exercise:**\n",
        "\n",
        "*  Load libraries and data.\n",
        "*  Do some exploratory data analysis to figure out which variables have direct and clear impact on employee retention (i.e. whether they leave the company or continue to work)\n",
        "*  Plot bar charts showing impact of employee salaries on retention\n",
        "*  See the correlation between department and employee retention\n",
        "*  Separate dependent and independent variables.\n",
        "*  Split the data into train set and test set\n",
        "*  Now build Logistic Regression model and do prediction for test data\n",
        "*  Measure the accuracy of the model\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y2MZWSKNnX6P",
        "colab_type": "text"
      },
      "source": [
        "**References:**\n",
        "\n",
        "https://github.com/codebasics/py/blob/master/ML/7_logistic_reg/7_logistic_regression.ipynb"
      ]
    }
  ]
}