Gesture Recognition using OpenCV + Python

This python script can be used to analyse hand gestures by contour detection and convex hull of palm region using OpenCV, a library used fo...

This python script can be used to analyse hand gestures by contour detection and convex hull of palm region using OpenCV, a library used for computer vision processes.

code: https://github.com/vipul-sharma20/gesture-opencv

The video below shows the working of the code:


 

How?

1. Capture frames and convert to grayscale

  • Our ROI is the the hand region, so we capture the images of the hand and convert them to grayscale. 
  • Q. Why grayscale ?
    A. We convert an image from RGB to grayscale and then to binary in order to find the ROI i.e. the portion of the image we are further interested for image processing. By doing this our decision becomes binary: "yes the pixel is of interest" or "no the pixel is not of interest".
 

 2. Blur image


  • I've used Gaussian Blurring on the original image. We blur the image for smoothing and to reduce noise and details from the image. We are not interested in the details of the image but in the shape of the object to track.
  • By blurring, we create smooth transition from one color to another and reduce the edge content. We use thresholding for image segmentation, to create binary images from grayscale images.


3. Thresholding

  • In very basic terms, thresholding is like a Low Pass Filter by allowing only particular color ranges to be highlighted as white while the other colors are suppressed by showing them as black.
  • I've used Otsu's Binarization method. In this method, OpenCV automatically calculates/approximates the threshold value of a bimodal image from its image histogram. But for optimal results, we may need a clear background in front of the webcam which sometimes may not be possible.

4. Draw contours

5. Find convex hull and convexity defects

  • We now find the convex points and the defect points. The convex points are generally, the tip of the fingers. But there are other convex point too. So, we find convexity defects, which is the deepest point of deviation on the contour. By this we can find the number of fingers extended and then we can perform different functions according to the number of fingers extended.






42 comments

  1. github repo link does not work

    ReplyDelete
    Replies
    1. Sorry, I didn't add the link properly. I've corrected it now. Here is the link: https://github.com/vipul-sharma20/gesture-opencv

      Delete
  2. while(cap.isOpened()):
    SystemError: null argument to internal routine

    ReplyDelete
    Replies
    1. Hi. Apologies for the late reply but I don't know why this error is coming. Can you please post the full Traceback ?

      Delete
  3. thank you so much for sharing the source code

    ReplyDelete
  4. Thanks a lot for the code ...how to change the rectangle size so that whole image is covered under the box ??

    ReplyDelete
    Replies
    1. You mean the dimensions of the square where I place my hand ? I've specified the dimensions myself, you can change as per your need. The problem is that I am using Otsu's Binarization method for thresholding the palm region so I need minimum interference because of the surrounding colors. Therefore, I've used small square large enough to contain a hand gesture.

      Delete
  5. Error >
    line 16, in
    cv2.CHAIN_APPROX_NONE)
    ValueError: too many values to unpack

    ReplyDelete
    Replies
    1. Hi. The findContours() method has changed in OpenCV 3.0 (I am assuming that you are using 3.0 because that can be the only reason) so you can try this :
      _, contours, hierarchy = cv2.findContours(...)

      see discussion here: http://answers.opencv.org/question/32248/open-cv-30-findcontours-return-too-many-value-to-unpack/

      Delete
    2. Yeah, worked for me. Thanks

      Delete
    3. Awesome ! Glad that it worked :)

      Delete
  6. Hi Vipul
    I'd like to make contact with you about gesture recognition. I am looking to pay a developer (you?) to code a hand gesture recognition python script for opencv and raspberry pi. I currently have opencv 3 and python 2.7 installed on a pi 2. will your code work? Thanks. Andre (andrebrown1@gmail.com)

    ReplyDelete
  7. thank you for your cod and explaining . but I have always this error
    Traceback (most recent call last):
    File "C:\Python27\test.py", line 16, in
    cv2.CHAIN_APPROX_NONE)
    ValueError: too many values to unpack
    can you help me

    ReplyDelete
    Replies
    1. change to lines 15 and 16 to _, contours, hierarchy = cv2.findContours(thresh1.copy(),cv2.RETR_TREE, \
      cv2.CHAIN_APPROX_NONE)

      Delete
  8. Hello vipul,
    Im working on my project finger recognition with raspberry pi and python.
    I already try your given code. And its work without error.
    But, nothing appear. Camera not functioning. Video not opening. No detection.
    Or i need press a key or what?
    Please help me sir.

    ReplyDelete
    Replies
    1. Hello Sir, I am facing the same problem? Did you find a solution?

      Delete
    2. Hello Sir, I am facing the same problem? Did you find a solution?

      Delete
  9. Error: 'NoneType' object has no attribute '__getitem__' in the line crop_img = img[100:300, 100:300]

    ReplyDelete
    Replies
    1. Hi,
      It is strange that it is giving this error. Can you post the code you are using and the complete error ? Maybe via pastebin or something similar.

      Delete
    2. got the error corrected by using out = cv2.VideoWriter('output.avi', -1, 20.0, (640,480)) after capturing the video ...thankyou anyway :)

      Delete
  10. This comment has been removed by the author.

    ReplyDelete
  11. Hi
    sometime the code works fine but most of the time it is giving erroranious outnut also its not forming contours properly.I think its due to the background .Please suggest a solution

    ReplyDelete
    Replies
    1. Hi,
      The thresholding technique which I've used here (Otsu's Binarization method) may give some erroneous results for non uniform background. You should try it in front of some clean uniform background.
      Thanks.

      Delete
  12. Hello Vipul,
    I have been trying the same on Raspberry Pi. I changed the code a little bit but I am getting an error that says out of resources(other than memory) or incorrect buffer length for resolution.. Kindly help me out.

    Thank you

    ReplyDelete
  13. can you send me code for this
    ashishkanaujiya36@gmail.com

    ReplyDelete
  14. Hello sir,
    This code works properly.
    How can we exit the output window.
    In this code how can we display the number of the counts of the fingers.

    ReplyDelete
  15. iam on project to recognise people that smoking and using a phone during fueling their vehicle. so is it possible for me to recognise the ciggarates and handphone using this opencv and phython
    thank you very much.

    ReplyDelete
  16. Replies
    1. https://github.com/vipul-sharma20/gesture-opencv

      Delete
  17. can you suggest any book to learn open cv using python please

    ReplyDelete
  18. hi vipul,
    I need to know some facts regarding the project CURSOR CONTROL USING HAND GESTURE RECOGNITION
    -which window library should i use to control mouse movements.
    -can u suggest some ways for background substraction so only the hand can be captured.

    ReplyDelete
  19. Hello. My project is hand gesture recognition for sign language. Is it possible to change the gesture classification process? I need to use earth mover's distance algorithm for template matching. Thanks.

    ReplyDelete
  20. hey m getting the following error... please help

    line 16, in
    (version, _, _) = cv2.__version__.split('.')
    ValueError: too many values to unpack
    >>>

    ReplyDelete
  21. how to increase size of the rectangle widow i didnt get that please help me you can just send me a mail at gurudattbushan@gmail.com please its very urgent.

    ReplyDelete
  22. hiii sir,
    can u plz send me the source code at 23komalmaheshwari@gmail.com

    ReplyDelete
  23. hey can you provide me with your source paper if there is any?

    ReplyDelete
  24. Hello Sir, My project is hand gesture recognition for drawing a shapes and fill with selected colors using python and opencv. can you share the source code at suchita.mca@gmail.com ???

    ReplyDelete
  25. Hello Vipul, I'm trying to build a system where i can recognize where my finger is pointing, Can you guide me as to how can i do that.

    ReplyDelete
  26. Hey! Thanks so much for this. Works like a gem.
    I was wondering if I could add a timer to this.

    For example, say I want to trigger an action after the number "4" has been held for 5 seconds, do you know how I could go about doing that?

    Because this runs in a loop, a normal timer would keep resetting if i included it in the else-if statements, right? How do i ensure continued gesture recognition while implementing a timer?

    Your help and code is much appreciated!

    ReplyDelete
  27. Great site for these post and i am seeing the most of contents have useful for my Carrier.Thanks to such a useful information.Any information are commands like to share him.
    Python Training in Chennai

    ReplyDelete
  28. i really likes your blog and You have shared the whole concept really well. and Very beautifully written,
    soulful read! thanks for sharing.
    gclub casino
    goldenslot casino
    goldenslot

    ReplyDelete