Gesture Recognition using OpenCV + Python

This python script can be used to analyse hand gestures by contour detection and convex hull of palm region using OpenCV, a library used fo...

This python script can be used to analyse hand gestures by contour detection and convex hull of palm region using OpenCV, a library used for computer vision processes.

code: https://github.com/vipul-sharma20/gesture-opencv

The video below shows the working of the code:


 

How?

1. Capture frames and convert to grayscale

  • Our ROI is the the hand region, so we capture the images of the hand and convert them to grayscale. 
  • Q. Why grayscale ?
    A. We convert an image from RGB to grayscale and then to binary in order to find the ROI i.e. the portion of the image we are further interested for image processing. By doing this our decision becomes binary: "yes the pixel is of interest" or "no the pixel is not of interest".
 

 2. Blur image


  • I've used Gaussian Blurring on the original image. We blur the image for smoothing and to reduce noise and details from the image. We are not interested in the details of the image but in the shape of the object to track.
  • By blurring, we create smooth transition from one color to another and reduce the edge content. We use thresholding for image segmentation, to create binary images from grayscale images.


3. Thresholding

  • In very basic terms, thresholding is like a Low Pass Filter by allowing only particular color ranges to be highlighted as white while the other colors are suppressed by showing them as black.
  • I've used Otsu's Binarization method. In this method, OpenCV automatically calculates/approximates the threshold value of a bimodal image from its image histogram. But for optimal results, we may need a clear background in front of the webcam which sometimes may not be possible.

4. Draw contours

5. Find convex hull and convexity defects

  • We now find the convex points and the defect points. The convex points are generally, the tip of the fingers. But there are other convex point too. So, we find convexity defects, which is the deepest point of deviation on the contour. By this we can find the number of fingers extended and then we can perform different functions according to the number of fingers extended.






54 comments

  1. github repo link does not work

    ReplyDelete
    Replies
    1. Sorry, I didn't add the link properly. I've corrected it now. Here is the link: https://github.com/vipul-sharma20/gesture-opencv

      Delete
  2. while(cap.isOpened()):
    SystemError: null argument to internal routine

    ReplyDelete
    Replies
    1. Hi. Apologies for the late reply but I don't know why this error is coming. Can you please post the full Traceback ?

      Delete
  3. thank you so much for sharing the source code

    ReplyDelete
  4. Thanks a lot for the code ...how to change the rectangle size so that whole image is covered under the box ??

    ReplyDelete
    Replies
    1. You mean the dimensions of the square where I place my hand ? I've specified the dimensions myself, you can change as per your need. The problem is that I am using Otsu's Binarization method for thresholding the palm region so I need minimum interference because of the surrounding colors. Therefore, I've used small square large enough to contain a hand gesture.

      Delete
  5. Error >
    line 16, in
    cv2.CHAIN_APPROX_NONE)
    ValueError: too many values to unpack

    ReplyDelete
    Replies
    1. Hi. The findContours() method has changed in OpenCV 3.0 (I am assuming that you are using 3.0 because that can be the only reason) so you can try this :
      _, contours, hierarchy = cv2.findContours(...)

      see discussion here: http://answers.opencv.org/question/32248/open-cv-30-findcontours-return-too-many-value-to-unpack/

      Delete
    2. Yeah, worked for me. Thanks

      Delete
    3. Awesome ! Glad that it worked :)

      Delete
  6. Hi Vipul
    I'd like to make contact with you about gesture recognition. I am looking to pay a developer (you?) to code a hand gesture recognition python script for opencv and raspberry pi. I currently have opencv 3 and python 2.7 installed on a pi 2. will your code work? Thanks. Andre (andrebrown1@gmail.com)

    ReplyDelete
  7. thank you for your cod and explaining . but I have always this error
    Traceback (most recent call last):
    File "C:\Python27\test.py", line 16, in
    cv2.CHAIN_APPROX_NONE)
    ValueError: too many values to unpack
    can you help me

    ReplyDelete
    Replies
    1. change to lines 15 and 16 to _, contours, hierarchy = cv2.findContours(thresh1.copy(),cv2.RETR_TREE, \
      cv2.CHAIN_APPROX_NONE)

      Delete
  8. Hello vipul,
    Im working on my project finger recognition with raspberry pi and python.
    I already try your given code. And its work without error.
    But, nothing appear. Camera not functioning. Video not opening. No detection.
    Or i need press a key or what?
    Please help me sir.

    ReplyDelete
    Replies
    1. Hello Sir, I am facing the same problem? Did you find a solution?

      Delete
    2. Hello Sir, I am facing the same problem? Did you find a solution?

      Delete
  9. Error: 'NoneType' object has no attribute '__getitem__' in the line crop_img = img[100:300, 100:300]

    ReplyDelete
    Replies
    1. Hi,
      It is strange that it is giving this error. Can you post the code you are using and the complete error ? Maybe via pastebin or something similar.

      Delete
    2. got the error corrected by using out = cv2.VideoWriter('output.avi', -1, 20.0, (640,480)) after capturing the video ...thankyou anyway :)

      Delete
  10. This comment has been removed by the author.

    ReplyDelete
  11. Hi
    sometime the code works fine but most of the time it is giving erroranious outnut also its not forming contours properly.I think its due to the background .Please suggest a solution

    ReplyDelete
    Replies
    1. Hi,
      The thresholding technique which I've used here (Otsu's Binarization method) may give some erroneous results for non uniform background. You should try it in front of some clean uniform background.
      Thanks.

      Delete
  12. Hello Vipul,
    I have been trying the same on Raspberry Pi. I changed the code a little bit but I am getting an error that says out of resources(other than memory) or incorrect buffer length for resolution.. Kindly help me out.

    Thank you

    ReplyDelete
  13. can you send me code for this
    ashishkanaujiya36@gmail.com

    ReplyDelete
  14. Hello sir,
    This code works properly.
    How can we exit the output window.
    In this code how can we display the number of the counts of the fingers.

    ReplyDelete
  15. iam on project to recognise people that smoking and using a phone during fueling their vehicle. so is it possible for me to recognise the ciggarates and handphone using this opencv and phython
    thank you very much.

    ReplyDelete
  16. Replies
    1. https://github.com/vipul-sharma20/gesture-opencv

      Delete
  17. can you suggest any book to learn open cv using python please

    ReplyDelete
  18. hi vipul,
    I need to know some facts regarding the project CURSOR CONTROL USING HAND GESTURE RECOGNITION
    -which window library should i use to control mouse movements.
    -can u suggest some ways for background substraction so only the hand can be captured.

    ReplyDelete
  19. Hello. My project is hand gesture recognition for sign language. Is it possible to change the gesture classification process? I need to use earth mover's distance algorithm for template matching. Thanks.

    ReplyDelete
  20. hey m getting the following error... please help

    line 16, in
    (version, _, _) = cv2.__version__.split('.')
    ValueError: too many values to unpack
    >>>

    ReplyDelete
  21. how to increase size of the rectangle widow i didnt get that please help me you can just send me a mail at gurudattbushan@gmail.com please its very urgent.

    ReplyDelete
  22. hiii sir,
    can u plz send me the source code at 23komalmaheshwari@gmail.com

    ReplyDelete
  23. hey can you provide me with your source paper if there is any?

    ReplyDelete
  24. Hello Sir, My project is hand gesture recognition for drawing a shapes and fill with selected colors using python and opencv. can you share the source code at suchita.mca@gmail.com ???

    ReplyDelete
  25. Hello Vipul, I'm trying to build a system where i can recognize where my finger is pointing, Can you guide me as to how can i do that.

    ReplyDelete
  26. Hey! Thanks so much for this. Works like a gem.
    I was wondering if I could add a timer to this.

    For example, say I want to trigger an action after the number "4" has been held for 5 seconds, do you know how I could go about doing that?

    Because this runs in a loop, a normal timer would keep resetting if i included it in the else-if statements, right? How do i ensure continued gesture recognition while implementing a timer?

    Your help and code is much appreciated!

    ReplyDelete
  27. Great site for these post and i am seeing the most of contents have useful for my Carrier.Thanks to such a useful information.Any information are commands like to share him.
    Python Training in Chennai

    ReplyDelete
  28. i really likes your blog and You have shared the whole concept really well. and Very beautifully written,
    soulful read! thanks for sharing.
    gclub casino
    goldenslot casino
    goldenslot

    ReplyDelete
  29. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  30. it is really amazing...thanks for sharing....provide more useful information...
    Mobile app development company

    ReplyDelete
  31. how did you trained your machine to recognize hand have you done any training?

    ReplyDelete
  32. Hi there I am so thrilled I found your website, its a fantastic post , Besant technology offerPython training in chennai

    ReplyDelete
  33. I and my friends were going through the nice, helpful tips from the
    blog then the sudden came up with an awful suspicion I never
    expressed respect to the website owner for those secrets.


    Selenium Training in Chennai

    ReplyDelete
  34. It’s great to come across a blog every once in a while that isn’t the same out of date rehashed material. Fantastic read.

    Java Training in Bangalore|

    ReplyDelete
  35. Thank you a lot for providing individuals with a very spectacular possibility to read critical reviews from this site.

    Selenium training in bangalore

    ReplyDelete
  36. My developer is trying to convince me to move to .net from PHP. I have always disliked the idea because of the expenses. But he’s trying none the less. I’ve been using Movable-type on several websites for about a year and am anxious about switching to another platform. I have heard great things about blogengine.net. Is there a way I can transfer all my Word Press posts into it? Any help would be appreciated. DevOps Training in Bangalore

    ReplyDelete
  37. I simply wanted to thank you so much again. I am not sure the things that I might have gone through without the type of hints revealed by you regarding that situation. Besant Technologies offers the best Hadoop Training in Bangalore

    ReplyDelete
  38. This is an awesome post.Really very informative and creative contents. These concept is a good way to enhance the knowledge. AWS Training in Bangalore I like it and help me to development very well.Thank you for this brief explanation and very nice information.Well, got a good knowledge.So join as Start You Career .

    ReplyDelete
  39. Hi, id like to ask for your help please. On what code to add tat if only 1 finger shows it will click the mouse. Thanks

    ReplyDelete