Hand Gesture Recognition 🤙

Given an image containing a single hand 🖐, our system classifies the hand gesture into one of six digits from 0️⃣ to 5️⃣. It handles different lighting effects⚡ and different hand poses as much as possible.

A complete machine learning pipeline, composed of the following modules:

• Preprocessing
• Feature extraction
• Classification

Wait for the classification stage to see our magical touch ✨

Preprocessing

This module aims to handle the variation in the images color, texture, size, orientation and lighting conditions. It bridges the gap between different hands of same gesture by applying the following steps:

• Resize the image for faster processing
• Convert the image to HSV
• Extract S and V channels
• Get the thresholds of hand and apply it in both channels
• Bitwise AND the two thresholded images
• Filling the empty regions
• Resize region filled mask to be compatable with original image size
• Remove padding
• Horizontally flip the image, if required
• Extract OCR of the image from the previous step

Note: We tried about 🔟 other methods. This was the most robust one🤖

Feature Extraction

This module aims to describe the image with a constant size feature vector. We tried various options to achieve this. Here they are:

• Using OCR extracted from the previous stage
• Histogram of Oriented Gradients
• SIFT (failed)
• ORB (failed)

Classification

This module includes our trained model, or should we say our trained models

We mainly depend on two classical ML algorithms [Support Vector Machines - Random Forest]

• Loads 3 different models
  • HOG descriptor, SVM
  • HOG descriptor, RF
  • OCR descriptor, SVM
• Runs each of them on a single thread
• Collects votes from them
• Classifies the image according to the majority of votes

The model ouputs the HOG_SVM result in case a tie happens

How to run our model on your test set

# Clone this project
$ git clone https://github.com/{{YOUR_GITHUB_USERNAME}}/hand-gesture-recognition

# Access
$ cd Hand-Gesture-Recognition

# Install dependencies
$ pip install -r requirements.txt

# Go to the run directory
$ cd .\src\final

Put the expected.txt in the same directory

Put your images in the 'data' folder

It's better to label your images from 1 to n

# Go back to the root directory
$ cd ..\..\
# Run the following batch script
$ .\src\final\run.bat

# The results.txt and time.txt will be generated in the same directory
# The script will output the accuracy as well

Contributors ✨

Basma Elhoseny

Yasmine Ghanem

Mohab Zaghloul

Yasmin Elgendi

Feedback

If you have any feedback, please reach out to us at [email protected]