Balance Data Set.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f5a5c477",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b02b3a69",
   "metadata": {},
   "source": [
    "# Loading Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "338f687a",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('balance-scale.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "6178ce98",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Class</th>\n",
       "      <th>L-Weight</th>\n",
       "      <th>L-Distance</th>\n",
       "      <th>R-Weight</th>\n",
       "      <th>R-Distance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>B</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>R</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>R</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>R</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>R</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>620</th>\n",
       "      <td>L</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>621</th>\n",
       "      <td>L</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>622</th>\n",
       "      <td>L</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>623</th>\n",
       "      <td>L</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>624</th>\n",
       "      <td>B</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>625 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    Class  L-Weight  L-Distance  R-Weight  R-Distance\n",
       "0       B         1           1         1           1\n",
       "1       R         1           1         1           2\n",
       "2       R         1           1         1           3\n",
       "3       R         1           1         1           4\n",
       "4       R         1           1         1           5\n",
       "..    ...       ...         ...       ...         ...\n",
       "620     L         5           5         5           1\n",
       "621     L         5           5         5           2\n",
       "622     L         5           5         5           3\n",
       "623     L         5           5         5           4\n",
       "624     B         5           5         5           5\n",
       "\n",
       "[625 rows x 5 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42b3cf84",
   "metadata": {},
   "source": [
    "# Data Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "95071f9d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Class         0\n",
       "L-Weight      0\n",
       "L-Distance    0\n",
       "R-Weight      0\n",
       "R-Distance    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f37eacad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(625, 5)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "47991908",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['B', 'R', 'L'], dtype=object)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Class'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e609e90e",
   "metadata": {},
   "outputs": [],
   "source": [
    "class_ids = {\"B\":0, \"R\":1, \"L\":2}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5320741b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>L-Weight</th>\n",
       "      <th>L-Distance</th>\n",
       "      <th>R-Weight</th>\n",
       "      <th>R-Distance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>620</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>621</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>622</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>623</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>624</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>625 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     L-Weight  L-Distance  R-Weight  R-Distance\n",
       "0           1           1         1           1\n",
       "1           1           1         1           2\n",
       "2           1           1         1           3\n",
       "3           1           1         1           4\n",
       "4           1           1         1           5\n",
       "..        ...         ...       ...         ...\n",
       "620         5           5         5           1\n",
       "621         5           5         5           2\n",
       "622         5           5         5           3\n",
       "623         5           5         5           4\n",
       "624         5           5         5           5\n",
       "\n",
       "[625 rows x 4 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_trimmed = df.drop(['Class'],axis=1)\n",
    "df_trimmed"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "528b1b2d",
   "metadata": {},
   "source": [
    "# Normalization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "905a6ea7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "scaler = MinMaxScaler(feature_range=(0,1))\n",
    "normalized_data = scaler.fit_transform(df_trimmed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "409f17d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>620</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>621</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>622</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>623</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>624</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>625 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       0    1    2     3\n",
       "0    0.0  0.0  0.0  0.00\n",
       "1    0.0  0.0  0.0  0.25\n",
       "2    0.0  0.0  0.0  0.50\n",
       "3    0.0  0.0  0.0  0.75\n",
       "4    0.0  0.0  0.0  1.00\n",
       "..   ...  ...  ...   ...\n",
       "620  1.0  1.0  1.0  0.00\n",
       "621  1.0  1.0  1.0  0.25\n",
       "622  1.0  1.0  1.0  0.50\n",
       "623  1.0  1.0  1.0  0.75\n",
       "624  1.0  1.0  1.0  1.00\n",
       "\n",
       "[625 rows x 4 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_nd = pd.DataFrame(normalized_data)\n",
    "df_nd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "82686afd",
   "metadata": {},
   "outputs": [],
   "source": [
    "df['balance_ids'] = df.Class.map(class_ids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "9f0f3143",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Class</th>\n",
       "      <th>L-Weight</th>\n",
       "      <th>L-Distance</th>\n",
       "      <th>R-Weight</th>\n",
       "      <th>R-Distance</th>\n",
       "      <th>balance_ids</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>B</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>R</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>R</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>R</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>R</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>620</th>\n",
       "      <td>L</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>621</th>\n",
       "      <td>L</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>622</th>\n",
       "      <td>L</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>623</th>\n",
       "      <td>L</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>624</th>\n",
       "      <td>B</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>625 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    Class  L-Weight  L-Distance  R-Weight  R-Distance  balance_ids\n",
       "0       B         1           1         1           1            0\n",
       "1       R         1           1         1           2            1\n",
       "2       R         1           1         1           3            1\n",
       "3       R         1           1         1           4            1\n",
       "4       R         1           1         1           5            1\n",
       "..    ...       ...         ...       ...         ...          ...\n",
       "620     L         5           5         5           1            2\n",
       "621     L         5           5         5           2            2\n",
       "622     L         5           5         5           3            2\n",
       "623     L         5           5         5           4            2\n",
       "624     B         5           5         5           5            0\n",
       "\n",
       "[625 rows x 6 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e418c9b8",
   "metadata": {},
   "source": [
    "# Testing and Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6ec260c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "df245d27",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train , X_test , Y_train , Y_test = train_test_split(df[['L-Weight', 'L-Distance', 'R-Weight' , 'R-Distance']], df['balance_ids'], test_size = 0.30 , random_state = 0, shuffle = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "3c277965",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>L-Weight</th>\n",
       "      <th>L-Distance</th>\n",
       "      <th>R-Weight</th>\n",
       "      <th>R-Distance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>432</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>433</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>434</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>435</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>436</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>437 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     L-Weight  L-Distance  R-Weight  R-Distance\n",
       "0           1           1         1           1\n",
       "1           1           1         1           2\n",
       "2           1           1         1           3\n",
       "3           1           1         1           4\n",
       "4           1           1         1           5\n",
       "..        ...         ...       ...         ...\n",
       "432         4           3         2           3\n",
       "433         4           3         2           4\n",
       "434         4           3         2           5\n",
       "435         4           3         3           1\n",
       "436         4           3         3           2\n",
       "\n",
       "[437 rows x 4 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "8ab670a9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>L-Weight</th>\n",
       "      <th>L-Distance</th>\n",
       "      <th>R-Weight</th>\n",
       "      <th>R-Distance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>437</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>438</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>439</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>440</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>441</th>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>620</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>621</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>622</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>623</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>624</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>188 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     L-Weight  L-Distance  R-Weight  R-Distance\n",
       "437         4           3         3           3\n",
       "438         4           3         3           4\n",
       "439         4           3         3           5\n",
       "440         4           3         4           1\n",
       "441         4           3         4           2\n",
       "..        ...         ...       ...         ...\n",
       "620         5           5         5           1\n",
       "621         5           5         5           2\n",
       "622         5           5         5           3\n",
       "623         5           5         5           4\n",
       "624         5           5         5           5\n",
       "\n",
       "[188 rows x 4 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "126b01fb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      0\n",
       "1      1\n",
       "2      1\n",
       "3      1\n",
       "4      1\n",
       "      ..\n",
       "432    2\n",
       "433    2\n",
       "434    2\n",
       "435    2\n",
       "436    2\n",
       "Name: balance_ids, Length: 437, dtype: int64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1c987e7a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "437    2\n",
       "438    0\n",
       "439    1\n",
       "440    2\n",
       "441    2\n",
       "      ..\n",
       "620    2\n",
       "621    2\n",
       "622    2\n",
       "623    2\n",
       "624    0\n",
       "Name: balance_ids, Length: 188, dtype: int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y_test"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa4e56d6",
   "metadata": {},
   "source": [
    "# Decision Tree Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "39bead43",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "11bc60a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = DecisionTreeClassifier(random_state = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "cb9beeb0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(random_state=0)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "6c3213b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "Y_predict = clf.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "145efa4a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 1, 2, 2, 1, 1, 1, 2, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 0, 1, 2, 2, 0, 1, 1, 2, 2, 1, 1, 1, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 0, 2, 2, 2, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2,\n",
       "       0, 1, 2, 0, 1, 1, 1, 2, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 0, 1, 2, 2, 1, 1, 1, 2, 0, 1, 1, 1, 2, 1,\n",
       "       1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 0, 1, 1, 2, 2, 1, 1,\n",
       "       1, 2, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 0, 1, 2,\n",
       "       2, 0, 1, 1, 2, 2, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 0, 2, 2, 2, 1, 1, 2, 2, 0, 1, 1], dtype=int64)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y_predict"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3a64297",
   "metadata": {},
   "source": [
    "# Evaluation Methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "b172b56c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import confusion_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "770306a1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.46808510638297873"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_squared_error(Y_test , Y_predict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "5ec5b5b4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0,  14,   0],\n",
       "       [  0,  38,   0],\n",
       "       [ 15,  14, 107]], dtype=int64)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cm = confusion_matrix(Y_test, Y_predict)\n",
    "cm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "7d4328a8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(69.0, 0.5, 'Truth')"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjQAAAGpCAYAAACam6wDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgH0lEQVR4nO3deZgU5bn38d89LKKCCzsCKlHUuMWFV40ao6CCK2iIclyCCTJuUUlyzhFNrmjOccFo9OgLvoDCEYxCFMkrKooGNYoLioACchIWFYcAQ0AQFxRm7vPHFGaCMDM03V1dz/P9eNU13VXdVU8lc/X8uO+nqs3dBQAAkGVlaQ8AAABgexFoAABA5hFoAABA5hFoAABA5hFoAABA5jVOewBbs77qcy6/Ql6t+rIy7SEgMK12aJv2EBCYZo12smIez07plLe/tf58RVHHvjkqNAAAIPNKtkIDAAAKzFItquQVgQYAgFgF1KcJ6FQAAECsqNAAABArWk4AACDzwskztJwAAED2UaEBACBWtJwAAEDmBdSnCehUAABArKjQAAAQK1pOAAAg88LJM7ScAABA9lGhAQAgVmXhlGgINAAAxCqcPEPLCQAAZB8VGgAAYsVVTgAAIPPCyTO0nAAAQPZRoQEAIFZc5QQAADIvnDxDywkAABSemY02s0ozm1trXUsze97MFiQ/d0/Wm5nda2YLzexdMzuivv0TaAAAiJVZ/pb6PSip12brBkua6u5dJU1NnkvSaZK6Jku5pP9X384JNAAAxKrM8rfUw91flrR6s9W9JY1JHo+R1KfW+rFe4w1Ju5lZhzpPZVvOGwAAYEvMrNzMZtRayhvwtnbuvix5vFxSu+RxR0kf1XpdRbJuq5gUDABArPI4KdjdR0oauR3vdzPzXN9PoAEAIFbp3yl4hZl1cPdlSUupMlm/VFLnWq/rlKzbKlpOAAAgLZMk9U8e95f0RK31P0qudjpG0tparaktokIDAECsiligMbNxkk6U1NrMKiTdKGmIpEfNbICkDyWdl7x8sqTTJS2U9LmkH9e3fwINAACxKuKdgt39X7ayqccWXuuSrtqW/RNoAACIVepTaPKHOTQAACDzqNAAABCr9K9yyhsCDQAAsQqoTxPQqQAAgFhRoQEAIFa0nAAAQOaFk2doOQEAgOyjQgMAQKxoOQEAgMwLqE8T0KkAAIBYUaEBACBWtJwAAEDmhZNnaDkBAIDso0IDAECsysIp0RBoAACIVUBzaGg5AQCAzKNCAwBArMIp0BBoAACIldFyAgAAKB1UaAAAiFRIFRoCDQAAkQooz9ByAgAA2UeFBgCASJUFVKIh0AAAEKmQ5tDQcgIAAJlHhQYAgEiFVKEh0AAAEKmQAg0tJwAAkHkEmgx79ZVXdfbpfXRmz7M16v7RaQ8HGfTbm36nc7ufp5/0Lf/GtkfHTlD3w3tq7cdrUxgZQsHnVGkzy9+SNgJNRlVVVenWm4fovhFD9ccnH9ezk5/VooWL0h4WMqbnWadqyLBbvrG+cnmlZrwxU23bt01hVAgFn1Olz8zytqSNQJNRc+fMVec9O6tT505q0rSJep3WUy+98FLaw0LGfOfIQ7TLri2+sf6+O0fosmsHlMSHFLKLzykUU8EmBZvZAZJ6S+qYrFoqaZK7zy/UMWNSuaJS7du3+/p52/btNOfduSmOCKF49cXX1Lpta+2z/z5pDwUZx+dU6QvpHy0FqdCY2XWSxksySW8mi0kaZ2aD63hfuZnNMLMZ9FqB4lv/xXo9PHq8LrniR2kPBUARWB7/S1uhKjQDJB3k7htqrzSzuyTNkzRkS29y95GSRkrS+qrPvUBjC0Lbdm21fPmKr59XLl+hdm3bpDgihOBvFcu0fOlyDTz/CknSysqVuuyCq3TfQ/eqZeuWKY8OWcPnFIqpUHNoqiXtsYX1HZJt2E4HHXyQlny4RBUVS7Xhqw169pkp+v5JJ6Y9LGTct7p20cQXHtW4yWM1bvJYtWnbRiMeGUaYQU74nCp9IU0KLlSFZpCkqWa2QNJHybo9Je0r6acFOmZUGjdurOt/eZ2uGHilqqur1eec3tq3K3MesG3+c/Bteuftd7V2zVqd1/NCXXL5xTr9nF5pDwuB4HOq9JVADskbcy9MZ8fMyiQdpX+eFPyWu1c15P20nJBvq76sTHsICEyrHbisHfnVrNFORY0Yu95wdN7+1q69dXqq8ahgVzm5e7WkNwq1fwAAsH3KAirR8F1OAABEqhTmvuQLN9YDAACZR4UGAIBIhVShIdAAABCpgPIMLScAAJB9VGgAAIgULScAAJB5IQUaWk4AACDzqNAAABCpkCo0BBoAACIVUqCh5QQAADKPCg0AAJEKqEBDoAEAIFa0nAAAAEoIFRoAACIVUoWGQAMAQKTKCDQAACDrAsozzKEBAADZR4UGAIBIMYcGAABknimcQEPLCQAAZB4VGgAAIkXLCQAAZF5IgYaWEwAAKDgz+5mZzTOzuWY2zsyamVkXM5tuZgvN7A9m1jTX/RNoAACIlFn+lrqPYx0lXSOpm7sfLKmRpH6Sbpd0t7vvK+ljSQNyPRcCDQAAkTKzvC0N0FjSjmbWWNJOkpZJ6i5pQrJ9jKQ+uZ4LgQYAAGw3Mys3sxm1lvJN29x9qaQ7JS1RTZBZK+ltSWvcfWPysgpJHXM9PpOCAQCIVD4nBbv7SEkjt3Kc3SX1ltRF0hpJj0nqlbeDi0ADAEC0iniV08mS3nf3lclxJ0o6TtJuZtY4qdJ0krQ01wPQcgIAAIW2RNIxZraT1aSoHpLek/SipL7Ja/pLeiLXAxBoAACIVLGucnL36aqZ/DtT0hzV5I+Rkq6T9HMzWyiplaRRuZ4LLScAACJVzBvrufuNkm7cbPViSUflY/9UaAAAQOZRoQEAIFIhffUBgQYAgEiFFGhoOQEAgMyjQgMAQKQCKtAQaAAAiBUtJwAAgBJChQYAgEiFVKEh0AAAEKmQAg0tJwAAkHlUaAAAiFRABRoCDQAAsaLlBAAAUEKo0AAAEKuAKjQEGgAAIkXLCQAAoIRQoQEAIFIBFWgINAAAxIqWEwAAQAmhQgMAQKRCqtAQaAAAiFRIgYaWEwAAyDwqNAAARCqgAg2BBgCAWNFyAgAAKCFUaBCNacteSXsICEzvvX+Q9hCA7RJShYZAAwBApEIKNLScAABA5lGhAQAgUiFVaAg0AABEKqA8Q8sJAABkHxUaAAAiRcsJAABkXkiBhpYTAADIPCo0AABEKqQKDYEGAIBIBZRnaDkBAIDso0IDAECkaDkBAIDsI9AAAICsC6lCwxwaAACQeVRoAACIVFk4BRoCDQAAsaLlBAAAUEKo0AAAEKmygCo0BBoAACJFywkAAKCEUKEBACBSIVU1CDQAAEQqpDk0IYUzAAAQKSo0AABEKqRJwQQaAAAiRcsJAACghFChAQAgUrScAABA5oXUpgnpXAAAQKSo0AAAEKmQJgUTaAAAiFRIc2hoOQEAgMyjQgMAQKRoOQEAgMwLJ87QcgIAAAEg0AAAEKkys7wt9TGz3cxsgpn9j5nNN7PvmllLM3vezBYkP3fP+VxyfSMAAMi2YgYaSfdIetbdD5D0HUnzJQ2WNNXdu0qamjzP7VxyfSMAAEBDmNmukk6QNEqS3P0rd18jqbekMcnLxkjqk+sxCDQAAETKzPK5lJvZjFpLea1DdZG0UtJ/m9ksM3vAzHaW1M7dlyWvWS6pXa7nwlVOAABEKp+Xbbv7SEkjt7K5saQjJF3t7tPN7B5t1l5ydzczz/X4VGgAAEChVUiqcPfpyfMJqgk4K8ysgyQlPytzPQCBBgCASFkel7q4+3JJH5nZ/smqHpLekzRJUv9kXX9JT+R6LrScAACIVJHvFHy1pIfNrKmkxZJ+rJrCyqNmNkDSh5LOy3XnBBoAAFBw7j5bUrctbOqRj/0TaAAAiBTf5QQAADLPAgo0TAoGAACZR4UGAIBI0XICAACZF06coeUEAAACQIUGAIBIRddyMrNjJe1d+/XuPrZAYwIAAEUQVaAxs4ck7SNptqSqZLVLItAAAICS0JAKTTdJB7p7zt+ACQAASk9I96FpSKCZK6m9pGUFHgsAACiikK4M2mqgMbMnVdNaaiHpPTN7U9KXm7a7+9mFHx4AAED96qrQ3Fm0UQAAgKKLouXk7n+WJDO73d2vq73NzG6X9OcCjw31ePWVV3X7bXeouqpa5/TtowEDf5L2kJAxG77aqNH/PlYbN2xUdVW1Djr+2+p+0fe1aPb7em7UVLm7mjZronN+frZa7dEy7eEig/icKm0hXeXUkPbZKVtYd1q+B4JtU1VVpVtvHqL7RgzVH598XM9OflaLFi5Ke1jImMZNGumS2y7SVcPKdeXQgVowY5E++p8KPTX0GfX9tz66cuhAHXriwfrz+GlpDxUZxOcUimmrgcbMrjCzOZIOMLN3ay3vS5pTvCFiS+bOmavOe3ZWp86d1KRpE/U6radeeuGltIeFjDEz7bBjU0lS1cZqVVdVSzLJpPWf10yZW//Zl2rRsnmKo0RW8TlV+srM8rakra45NI9IekbSbZIG11q/zt1XF3RUqFflikq1b9/u6+dt27fTnHfnpjgiZFV1VbWGXztKq/+2Wked2U2dD+io3teeqd/fOF5NmjbWDjvtoIF3/zjtYSKD+JwqfbHMoVkraa2ZXbfZpuZm1tzdlxR2aACKoaxRma4cOlBffLpe425+TCs+qNTr/3+6LvpNP3U+oKOmTXhdz458Xn0GnZn2UAFgqxoyh+ZpSU8lP6dKWqyayk1OzGyr/9Qzs3Izm2FmM0bdPzrXQ0Shbbu2Wr58xdfPK5evULu2bVIcEbJux+bN1OXQvbRgxiItX7xCnQ/oKEk6+IQD9dH8ipRHhyzic6r0lcnytqSt3kDj7oe4+6HJz66SjpL0+nYc8zd1HGuku3dz927MhK/bQQcfpCUfLlFFxVJt+GqDnn1mir5/0olpDwsZ89naz/TFp+slSRu+3KBFs95Xm86t9eXnX+rvFaskSYtmLVabzq3THCYyis+p0mdmeVvSts3ftu3uM83s6LpeY2bvbm2TpHZb2YZt0LhxY13/y+t0xcArVV1drT7n9Na+XfdJe1jImHWrP9XE302SV7vcXQd979va/+iuOvuaMzT+lgmyMtOOzZupz6Cz0h4qMojPKRST1fcVTWb281pPyyQdIamVu/es4z0rJPWU9PHmmyS95u571Dew9VWf891RyKsnPng87SEgML33/kHaQ0BgmjXaqailjutfvyFvf2tv++6tqZZpGlKhaVHr8UbVzKWp7y/DU5Kau/vszTeY2UsNHRwAACgcK4G5L/lSZ6Axs0aSWrj7v27LTt19QB3bLtiWfQEAgMIohbkv+VLXjfUau3uVpOOKOB4AAIBtVleF5k3VzJeZbWaTJD0m6bNNG919YoHHBgAACqgU7vCbLw2ZQ9NM0ipJ3SW5aib2uiQCDQAAGWYNuh1dNtQVaNomVzjN1T+CzCZcgQQAAEpGXYGmkaTm0hanQBNoAADIuFhaTsvc/T+KNhIAAFBUUVzlpC1XZgAAAEpOXRWaHkUbBQAAKLoobqzn7quLORAAAFBcIc2hCed6LQAAEK1t/rZtAAAQhpAmBRNoAACIVFlAjZpwzgQAAESLCg0AAJGi5QQAADIvpEBDywkAAGQeFRoAACJVFsON9QAAQNhoOQEAAJQQKjQAAEQqpK8+INAAABCpkL6ckpYTAADIPCo0AABEqszCqWsQaAAAiBRXOQEAAJQQKjQAAEQqpEnBBBoAACIV0mXbtJwAAEDmUaEBACBStJwAAEDm0XICAAAoIVRoAACIlHFjPQAAkHUhzaEJJ5oBAIBoUaEBACBSIU0KJtAAABApvssJAACghFChAQAgUmVMCgYAAFlnZnlbGni8RmY2y8yeSp53MbPpZrbQzP5gZk1zPRcCDQAAKJZrJc2v9fx2SXe7+76SPpY0INcdE2gAAIiUWVnelvqPZZ0knSHpgeS5SeouaULykjGS+uR6LsyhAQAgUkWeQ/Nfkv5dUovkeStJa9x9Y/K8QlLHXHdOhQYAAGw3Mys3sxm1lvJa286UVOnubxfq+FRoAACIVD7vQ+PuIyWN3Mrm4ySdbWanS2omaRdJ90jazcwaJ1WaTpKW5np8KjQAAETK8vhfXdz9enfv5O57S+on6QV3v1DSi5L6Ji/rL+mJXM+FQAMAANJynaSfm9lC1cypGZXrjmg5AQAQqTS++sDdX5L0UvJ4saSj8rFfAg0AAJHiTsEAAAAlhAoNAACRasgN8bKCQAMAQKTquzopS8KJZgAAIFpUaAAAiFQaVzkVCoEGAIBIhdRyItAAABCpkCo0zKEBAACZR4UGAIBIhXRjvZINNCvXL097CAjM8R2+l/YQEJgde+2X9hAQGH++oqjHo+UEAABQQkq2QgMAAArLAqprEGgAAIgULScAAIASQoUGAIBIcWM9AACQeWW0nAAAAEoHFRoAACJFywkAAGQeVzkBAACUECo0AABEihvrAQCAzKPlBAAAUEKo0AAAEKkyrnICAABZR8sJAACghFChAQAgUtxYDwAAZB4tJwAAgBJChQYAgEhxYz0AAJB5ZbScAAAASgcVGgAAIsVVTgAAIPO4ygkAAKCEUKEBACBStJwAAEDm0XICAAAoIVRoAACIVFlAdQ0CDQAAkaLlBAAAUEKo0AAAECmucgIAAJlHywkAAKCEUKEBACBStJwAAEDmhRRoaDkBAIDMo0IDAECsApoUTKABACBStJwAAABKCBUaAAAiFdJ9aAg0AABEipYTAABACaFCAwBApEKq0BBoAACIVEhzaGg5AQCAzKNCAwBApGg5AQCAzAsp0NByAgAAmUeFBgCASIU0KZhAAwBApEJqORFoAACIVEgVGubQAACAgjKzzmb2opm9Z2bzzOzaZH1LM3vezBYkP3fP9RgEGgAAImV5/K8eGyX9wt0PlHSMpKvM7EBJgyVNdfeukqYmz3NCoAEAIFLFCjTuvszdZyaP10maL6mjpN6SxiQvGyOpT67nQqABAADbzczKzWxGraV8K6/bW9LhkqZLaufuy5JNyyW1y/X4TAoGACBS+ZwU7O4jJY2s53jNJT0uaZC7f1L7+O7uZua5Hp9AAwBApIp52baZNVFNmHnY3Scmq1eYWQd3X2ZmHSRV5rp/Wk4ZcsdNd6lvj3669IeXf71uzPDf6/yeF+myflfpsn5Xafq0N1McIbLmtzf9Tud2P08/6fvNyvCjYyeo++E9tfbjtSmMDGka9Ys7teLR2Zoz8k9b3L5/53302j1PaP3Ti/SLvpfl5ZhNmzTV+F/epwUPTtMb9z6pvdp1kiSdfMT3NGPYZL078k+aMWyyTjrs2LwcD8VlNaWYUZLmu/tdtTZNktQ/edxf0hO5HoNAkyE9zzpFtw29+Rvrf3BhH40YP0wjxg/T0ccflcLIkFU9zzpVQ4bd8o31lcsrNeONmWrbvm0Ko0LaHnzuMfW64aKtbl+9bo2uGfZr3TlhxDbve692nfTinY99Y/2AXv308adr1fWS43X3xPt1+6U3SJL+vna1zvr1j3Vo+cnqf8fP9NB1927zMbF1RbzK6ThJF0vqbmazk+V0SUMknWJmCySdnDzPCYEmQw498hC12LVF2sNAQL5z5CHaZQu/U/fdOUKXXTsgqJtuoeFemTNdq9et2er2lWtWacZf39GGjRu/se3CHudq+v99SrOGT9Hwa4eorKxhf2Z6H3uqxjxXE3QmvPy0ehx+vCRp9qJ5WrZqhSRp3gd/0Y5Nm6lpk6bbeEbYGjPL21IXd5/m7ubuh7r7Ycky2d1XuXsPd+/q7ie7++pcz6VggcbMDjCzHskEoNrrexXqmLF64g9PauB5V+iOm+7Suk/WpT0cZNyrL76m1m1ba5/990l7KMiYA/bcV+d//ywdN6iPDr+8p6qqq3Rh93Ma9N6Ordrro5U1F7tUVVdp7WefqNUu/3yPtR987wzNXDhHX234Ku9jR/YVJNCY2TWq6YNdLWmumfWutfnWOt739SVfD48eV4ihBefsH56hsZNGa8T4YWrVuqWG33V/2kNChq3/Yr0eHj1el1zxo7SHggzqcfjxOnK/Q/TWsKc1a/gU9Tj8eH2rw16SpIk3PqBZw6do8i1j1W2/QzVr+BTNGj5Fl/Q8r0H7PnCv/XT7pdfrsv/K+b5r2CLL45KuQl3lNFDSke7+aXK9+QQz29vd71EdZ137kq+PPluc86VbMdm91T/+BXP6uafpV9femOJokHV/q1im5UuXa+D5V0iSVlau1GUXXKX7HrpXLVu3THl0KHUm05jnJuiG0d+cBnHuby6VVDOH5sF/u1sn/esP/2n70lXL1blNBy39+zI1KmukXXfeRas++ViS1LF1B/3xpgf0o98O0uJlHxb+RCISUlu5UC2nMnf/VJLc/QNJJ0o6zczuUinEuICsWvmPduO0F17T3vvsleJokHXf6tpFE194VOMmj9W4yWPVpm0bjXhkGGEGDTJ11jT1PeEMtdmtlSRp9xa7ac+2HRv03kmvP6/+p9aEnL4nnKEXZr8qSdp151309M1jNHjUbXpt3ozCDBxBKFSFZoWZHebusyUpqdScKWm0pEMKdMzg3XL9EL3z9rtau+YT9et1kfpffrHemfGuFv51sUxS+z3aadAvr0l7mMiQ/xx8W/I7tVbn9bxQl1x+sU4/h2lusXvkhqE68dDvqvWuLfXRI2/pxrG/U5PGNX8uRjz1e7XbvY1mDJusXXZqrmqv1qBzL9WBl56k+UsW6Ff//Vs9N+QRlVmZNmzcoKuG/kpLKpfWe8xRz4zXQ4Pv0YIHp2n1ujXqd8uVkqSf9r5E++6xt3590SD9+qJBkqRTB1+glWtWFez8Y1LM+9AUmrnnv7NjZp0kbXT35VvYdpy7v1rfPmg5Id/KjIv6kF+deh+f9hAQGH++oqgJ4/11f83b39ouLfZLNR0VpELj7hV1bKs3zAAAAGwLvvoAAIBIhTQpmEADAECkQppDw6QCAACQeVRoAACIVEgVGgINAACRCmkODS0nAACQeVRoAACIFC0nAACQebScAAAASggVGgAAIkXLCQAABCCcQEPLCQAAZB4VGgAAIhVOfYZAAwBAtLjKCQAAoIRQoQEAIFrhVGgINAAARCqcOEPLCQAABIAKDQAA0QqnRkOgAQAgUlzlBAAAUEIINAAAIPNoOQEAEKmQvpySCg0AAMg8KjQAAESKCg0AAEAJIdAAAIDMo+UEAECkuA8NAABACSHQAACAzKPlBABApEK6yolAAwBAtMIJNLScAABA5lGhAQAgUuHUZwg0AABEi8u2AQAASggVGgAAohVOhYZAAwBApMKJMwQaAAAiFk6kYQ4NAADIPCo0AABEiqucAAAASgiBBgAAZB4tJwAAIsWXUwIAgACEE2hoOQEAgMyjQgMAQKTCqc8QaAAAiBaXbQMAAJQQKjQAAEQrnAoNgQYAgEiFE2doOQEAgABQoQEAIFrh1GgINAAARIqrnAAAAEoIgQYAABScmfUys7+Y2UIzG5zv/dNyAgAgUsX6ckozayRpmKRTJFVIesvMJrn7e/k6BhUaAABQaEdJWujui939K0njJfXO5wFKtkLTeedvhTNTqcDMrNzdR6Y9DoSB36eG8+cr0h5CJvA7VbqaNdopb39rzaxcUnmtVSNr/f/eUdJHtbZVSDo6X8eWqNCEorz+lwANxu8T8o3fqQi4+0h371ZrKWqIJdAAAIBCWyqpc63nnZJ1eUOgAQAAhfaWpK5m1sXMmkrqJ2lSPg9QsnNosE3oTSOf+H1CvvE7FTl332hmP5U0RVIjSaPdfV4+j2Huns/9AQAAFB0tJwAAkHkEGgAAkHkEmgwr9G2kERczG21mlWY2N+2xIAxm1tnMXjSz98xsnpldm/aYEC7m0GRUchvpv6rWbaQl/Us+byONuJjZCZI+lTTW3Q9OezzIPjPrIKmDu880sxaS3pbUh88pFAIVmuwq+G2kERd3f1nS6rTHgXC4+zJ3n5k8XidpvmruGAvkHYEmu7Z0G2k+KACUJDPbW9LhkqanPBQEikADACgoM2su6XFJg9z9k7THgzARaLKr4LeRBoDtZWZNVBNmHnb3iWmPB+Ei0GRXwW8jDQDbw8xM0ihJ8939rrTHg7ARaDLK3TdK2nQb6fmSHs33baQRFzMbJ+l1SfubWYWZDUh7TMi84yRdLKm7mc1OltPTHhTCxGXbAAAg86jQAACAzCPQAACAzCPQAACAzCPQAACAzCPQAACAzCPQABllZlXJZbBzzewxM9tpO/b1oJn1TR4/YGYH1vHaE83s2ByO8YGZtc51jABQFwINkF1fuPthyTdjfyXp8tobzaxxLjt190vr+TbkEyVtc6ABgEIi0ABheEXSvkn15BUzmyTpPTNrZGZ3mNlbZvaumV0m1dzB1cyGmtlfzOxPktpu2pGZvWRm3ZLHvcxsppm9Y2ZTky8YvFzSz5Lq0PfMrI2ZPZ4c4y0zOy55bysze87M5pnZA5KsyP+bAIhITv+CA1A6kkrMaZKeTVYdIelgd3/fzMolrXX3/2NmO0h61cyeU823Hu8v6UBJ7SS9J2n0ZvttI+l+SSck+2rp7qvNbLikT939zuR1j0i6292nmdmeqrl79bcl3Shpmrv/h5mdIYk7DwMoGAINkF07mtns5PErqvnOnGMlvenu7yfrT5V06Kb5MZJ2ldRV0gmSxrl7laS/mdkLW9j/MZJe3rQvd1+9lXGcLOnAmq/tkSTtkny78gmSzk3e+7SZfZzbaQJA/Qg0QHZ94e6H1V6RhIrPaq+SdLW7T9nsdfn8Pp0ySce4+/otjAUAioI5NEDYpki6wsyaSJKZ7WdmO0t6WdL5yRybDpJO2sJ735B0gpl1Sd7bMlm/TlKLWq97TtLVm56Y2WHJw5clXZCsO03S7vk6KQDYHIEGCNsDqpkfM9PM5koaoZrK7B8lLUi2jVXNt2z/E3dfKalc0kQze0fSH5JNT0o6Z9OkYEnXSOqWTDp+T/+42uo3qglE81TTelpSoHMEAL5tGwAAZB8VGgAAkHkEGgAAkHkEGgAAkHkEGgAAkHkEGgAAkHkEGgAAkHkEGgAAkHn/CzafKbzEcj04AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x504 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sn\n",
    "plt.figure(figsize=(10,7))\n",
    "sn.heatmap(cm, annot=True, cmap=\"Greens\")\n",
    "plt.xlabel('Predicted')\n",
    "plt.ylabel('Truth')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "6545ec43",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7712765957446809"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy_score(Y_test, Y_predict)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3c969c9",
   "metadata": {},
   "source": [
    "# HyperParameter Tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "64113cd4",
   "metadata": {},
   "outputs": [],
   "source": [
    "param_dist = {\n",
    "    \"criterion\":[\"gini\", \"entropy\"],\n",
    "    \"max_depth\":[1,2,3,4,5,6,7,None]\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "ee890785",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "fd444a57",
   "metadata": {},
   "outputs": [],
   "source": [
    "grid = GridSearchCV(clf, param_grid = param_dist, cv = 18, n_jobs = -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "576f2b05",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=18, estimator=DecisionTreeClassifier(random_state=0), n_jobs=-1,\n",
       "             param_grid={'criterion': ['gini', 'entropy'],\n",
       "                         'max_depth': [1, 2, 3, 4, 5, 6, 7, None]})"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "01aa00cf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'criterion': 'gini', 'max_depth': 4}"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "9a6ba3a3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7666666666666666"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# For Calculating Best Accuracy\n",
    "grid.best_score_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e9cd7a8",
   "metadata": {},
   "source": [
    "# RandomizedSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "c08b445a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import RandomizedSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "31898be9",
   "metadata": {},
   "outputs": [],
   "source": [
    "param_rnd = {\n",
    "    \"criterion\":[\"gini\", \"entropy\"],\n",
    "    \"max_depth\":[1,2,3,4,5,6,7,None],\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "717118f9",
   "metadata": {},
   "outputs": [],
   "source": [
    "rndm_grid = RandomizedSearchCV(clf, param_rnd, cv = 18, n_jobs = -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "919ccd8b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomizedSearchCV(cv=18, estimator=DecisionTreeClassifier(random_state=0),\n",
       "                   n_jobs=-1,\n",
       "                   param_distributions={'criterion': ['gini', 'entropy'],\n",
       "                                        'max_depth': [1, 2, 3, 4, 5, 6, 7,\n",
       "                                                      None]})"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rndm_grid.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "1b3cddc2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7666666666666666"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rndm_grid.best_score_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8559cd23",
   "metadata": {},
   "source": [
    "# Random Forest Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "f423b6e7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(n_estimators=40)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "rfc = RandomForestClassifier(n_estimators=40)\n",
    "rfc.fit(X_train, Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "fd32d0f8",
   "metadata": {},
   "outputs": [],
   "source": [
    "Y_predict_rfc = rfc.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "de79882c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 1, 2, 2, 1, 1, 1, 2, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 0, 1, 2, 2, 0, 1, 1, 2, 2, 1, 1, 1, 2, 2, 2, 2, 2, 2,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 0, 2, 2, 2, 1, 1, 2, 2, 0, 1, 1, 2, 2, 2,\n",
       "       0, 1, 2, 0, 1, 1, 1, 2, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       2, 2, 2, 2, 2, 2, 2, 2, 0, 1, 2, 2, 1, 1, 1, 2, 0, 1, 1, 1, 2, 1,\n",
       "       1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 0, 1, 1, 2, 2, 1, 1,\n",
       "       1, 2, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 0, 1, 2,\n",
       "       2, 0, 1, 1, 2, 2, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,\n",
       "       2, 0, 2, 2, 2, 1, 1, 2, 2, 0, 1, 1], dtype=int64)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y_predict_rfc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "d95d8cd6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7712765957446809"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy_score(Y_test, Y_predict_rfc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "32107340",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0,  14,   0],\n",
       "       [  0,  38,   0],\n",
       "       [ 15,  14, 107]], dtype=int64)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "confusion_matrix(Y_test, Y_predict_rfc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a45178cb",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41ae0775",
   "metadata": {},
   "source": [
    "Some obseravations which were seen :-\n",
    "\n",
    "1)HyperParameter Tuning didn't increased the accuracy instead of that it decreased the accuracy from 77.12% to 76.66% .\n",
    "Reasons for that may be that since hyperparameter is a subjective thing it varies from data set to data set and it finds the optimal parameteres for the data set so that we can get the best accuracy .There may be a possibility that the data set was already in it's best accuracy as the default parameters would be the best parameters for this data set because of that using HyperParameter was not that efficient .\n",
    "\n",
    "2)Using Random Forest Classifier the accuracy was exactly same (i.e 77.12% ) as that of Decision Tree Classifier the reasons for that may be because  the Data set was having very less parameters or a single output parameter so splitting it into multiple trees didn't affected the accuracy .\n",
    "\n",
    "The difficult part for me was adding more parameters to the Randomized Search CV because changing the max_depth of the parameter decreased the accuracy of Data set and I was not able to get extra parameters to make it more discrete ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ee9d04c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}