Gesture Recognition using OpenCV + Python

This python script can be used to analyse hand gestures by contour detection and convex hull of palm region using OpenCV, a library used fo...

This python script can be used to analyse hand gestures by contour detection and convex hull of palm region using OpenCV, a library used for computer vision processes.

code: https://github.com/vipul-sharma20/gesture-opencv

The video below shows the working of the code:


 

How?

1. Capture frames and convert to grayscale

  • Our ROI is the the hand region, so we capture the images of the hand and convert them to grayscale. 
  • Q. Why grayscale ?
    A. We convert an image from RGB to grayscale and then to binary in order to find the ROI i.e. the portion of the image we are further interested for image processing. By doing this our decision becomes binary: "yes the pixel is of interest" or "no the pixel is not of interest".
 

 2. Blur image


  • I've used Gaussian Blurring on the original image. We blur the image for smoothing and to reduce noise and details from the image. We are not interested in the details of the image but in the shape of the object to track.
  • By blurring, we create smooth transition from one color to another and reduce the edge content. We use thresholding for image segmentation, to create binary images from grayscale images.


3. Thresholding

  • In very basic terms, thresholding is like a Low Pass Filter by allowing only particular color ranges to be highlighted as white while the other colors are suppressed by showing them as black.
  • I've used Otsu's Binarization method. In this method, OpenCV automatically calculates/approximates the threshold value of a bimodal image from its image histogram. But for optimal results, we may need a clear background in front of the webcam which sometimes may not be possible.

4. Draw contours

5. Find convex hull and convexity defects

  • We now find the convex points and the defect points. The convex points are generally, the tip of the fingers. But there are other convex point too. So, we find convexity defects, which is the deepest point of deviation on the contour. By this we can find the number of fingers extended and then we can perform different functions according to the number of fingers extended.






84 comments

  1. github repo link does not work

    ReplyDelete
    Replies
    1. Sorry, I didn't add the link properly. I've corrected it now. Here is the link: https://github.com/vipul-sharma20/gesture-opencv

      Delete
  2. while(cap.isOpened()):
    SystemError: null argument to internal routine

    ReplyDelete
    Replies
    1. Hi. Apologies for the late reply but I don't know why this error is coming. Can you please post the full Traceback ?

      Delete
  3. thank you so much for sharing the source code

    ReplyDelete
  4. Thanks a lot for the code ...how to change the rectangle size so that whole image is covered under the box ??

    ReplyDelete
    Replies
    1. You mean the dimensions of the square where I place my hand ? I've specified the dimensions myself, you can change as per your need. The problem is that I am using Otsu's Binarization method for thresholding the palm region so I need minimum interference because of the surrounding colors. Therefore, I've used small square large enough to contain a hand gesture.

      Delete
  5. Error >
    line 16, in
    cv2.CHAIN_APPROX_NONE)
    ValueError: too many values to unpack

    ReplyDelete
    Replies
    1. Hi. The findContours() method has changed in OpenCV 3.0 (I am assuming that you are using 3.0 because that can be the only reason) so you can try this :
      _, contours, hierarchy = cv2.findContours(...)

      see discussion here: http://answers.opencv.org/question/32248/open-cv-30-findcontours-return-too-many-value-to-unpack/

      Delete
    2. Yeah, worked for me. Thanks

      Delete
    3. Awesome ! Glad that it worked :)

      Delete
  6. Hi Vipul
    I'd like to make contact with you about gesture recognition. I am looking to pay a developer (you?) to code a hand gesture recognition python script for opencv and raspberry pi. I currently have opencv 3 and python 2.7 installed on a pi 2. will your code work? Thanks. Andre (andrebrown1@gmail.com)

    ReplyDelete
  7. thank you for your cod and explaining . but I have always this error
    Traceback (most recent call last):
    File "C:\Python27\test.py", line 16, in
    cv2.CHAIN_APPROX_NONE)
    ValueError: too many values to unpack
    can you help me

    ReplyDelete
    Replies
    1. change to lines 15 and 16 to _, contours, hierarchy = cv2.findContours(thresh1.copy(),cv2.RETR_TREE, \
      cv2.CHAIN_APPROX_NONE)

      Delete
  8. Hello vipul,
    Im working on my project finger recognition with raspberry pi and python.
    I already try your given code. And its work without error.
    But, nothing appear. Camera not functioning. Video not opening. No detection.
    Or i need press a key or what?
    Please help me sir.

    ReplyDelete
    Replies
    1. Hello Sir, I am facing the same problem? Did you find a solution?

      Delete
    2. Hello Sir, I am facing the same problem? Did you find a solution?

      Delete
  9. Error: 'NoneType' object has no attribute '__getitem__' in the line crop_img = img[100:300, 100:300]

    ReplyDelete
    Replies
    1. Hi,
      It is strange that it is giving this error. Can you post the code you are using and the complete error ? Maybe via pastebin or something similar.

      Delete
    2. got the error corrected by using out = cv2.VideoWriter('output.avi', -1, 20.0, (640,480)) after capturing the video ...thankyou anyway :)

      Delete
  10. This comment has been removed by the author.

    ReplyDelete
  11. Hi
    sometime the code works fine but most of the time it is giving erroranious outnut also its not forming contours properly.I think its due to the background .Please suggest a solution

    ReplyDelete
    Replies
    1. Hi,
      The thresholding technique which I've used here (Otsu's Binarization method) may give some erroneous results for non uniform background. You should try it in front of some clean uniform background.
      Thanks.

      Delete
  12. Hello Vipul,
    I have been trying the same on Raspberry Pi. I changed the code a little bit but I am getting an error that says out of resources(other than memory) or incorrect buffer length for resolution.. Kindly help me out.

    Thank you

    ReplyDelete
  13. can you send me code for this
    ashishkanaujiya36@gmail.com

    ReplyDelete
  14. Hello sir,
    This code works properly.
    How can we exit the output window.
    In this code how can we display the number of the counts of the fingers.

    ReplyDelete
  15. iam on project to recognise people that smoking and using a phone during fueling their vehicle. so is it possible for me to recognise the ciggarates and handphone using this opencv and phython
    thank you very much.

    ReplyDelete
  16. Replies
    1. https://github.com/vipul-sharma20/gesture-opencv

      Delete
  17. can you suggest any book to learn open cv using python please

    ReplyDelete
  18. hi vipul,
    I need to know some facts regarding the project CURSOR CONTROL USING HAND GESTURE RECOGNITION
    -which window library should i use to control mouse movements.
    -can u suggest some ways for background substraction so only the hand can be captured.

    ReplyDelete
  19. Hello. My project is hand gesture recognition for sign language. Is it possible to change the gesture classification process? I need to use earth mover's distance algorithm for template matching. Thanks.

    ReplyDelete
  20. hey m getting the following error... please help

    line 16, in
    (version, _, _) = cv2.__version__.split('.')
    ValueError: too many values to unpack
    >>>

    ReplyDelete
  21. how to increase size of the rectangle widow i didnt get that please help me you can just send me a mail at gurudattbushan@gmail.com please its very urgent.

    ReplyDelete
  22. hiii sir,
    can u plz send me the source code at 23komalmaheshwari@gmail.com

    ReplyDelete
  23. hey can you provide me with your source paper if there is any?

    ReplyDelete
  24. Hello Sir, My project is hand gesture recognition for drawing a shapes and fill with selected colors using python and opencv. can you share the source code at suchita.mca@gmail.com ???

    ReplyDelete
  25. Hello Vipul, I'm trying to build a system where i can recognize where my finger is pointing, Can you guide me as to how can i do that.

    ReplyDelete
  26. Hey! Thanks so much for this. Works like a gem.
    I was wondering if I could add a timer to this.

    For example, say I want to trigger an action after the number "4" has been held for 5 seconds, do you know how I could go about doing that?

    Because this runs in a loop, a normal timer would keep resetting if i included it in the else-if statements, right? How do i ensure continued gesture recognition while implementing a timer?

    Your help and code is much appreciated!

    ReplyDelete
  27. Great site for these post and i am seeing the most of contents have useful for my Carrier.Thanks to such a useful information.Any information are commands like to share him.
    Python Training in Chennai

    ReplyDelete
  28. i really likes your blog and You have shared the whole concept really well. and Very beautifully written,
    soulful read! thanks for sharing.
    gclub casino
    goldenslot casino
    goldenslot

    ReplyDelete
  29. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  30. it is really amazing...thanks for sharing....provide more useful information...
    Mobile app development company

    ReplyDelete
  31. how did you trained your machine to recognize hand have you done any training?

    ReplyDelete
  32. Hi there I am so thrilled I found your website, its a fantastic post , Besant technology offerPython training in chennai

    ReplyDelete
  33. I and my friends were going through the nice, helpful tips from the
    blog then the sudden came up with an awful suspicion I never
    expressed respect to the website owner for those secrets.


    Selenium Training in Chennai

    ReplyDelete
  34. It’s great to come across a blog every once in a while that isn’t the same out of date rehashed material. Fantastic read.

    Java Training in Bangalore|

    ReplyDelete
  35. Thank you a lot for providing individuals with a very spectacular possibility to read critical reviews from this site.

    Selenium training in bangalore

    ReplyDelete
  36. My developer is trying to convince me to move to .net from PHP. I have always disliked the idea because of the expenses. But he’s trying none the less. I’ve been using Movable-type on several websites for about a year and am anxious about switching to another platform. I have heard great things about blogengine.net. Is there a way I can transfer all my Word Press posts into it? Any help would be appreciated. DevOps Training in Bangalore

    ReplyDelete
  37. I simply wanted to thank you so much again. I am not sure the things that I might have gone through without the type of hints revealed by you regarding that situation. Besant Technologies offers the best Hadoop Training in Bangalore

    ReplyDelete
  38. This is an awesome post.Really very informative and creative contents. These concept is a good way to enhance the knowledge. AWS Training in Bangalore I like it and help me to development very well.Thank you for this brief explanation and very nice information.Well, got a good knowledge.So join as Start You Career .

    ReplyDelete
  39. Hi, id like to ask for your help please. On what code to add tat if only 1 finger shows it will click the mouse. Thanks

    ReplyDelete
  40. Hi Vipul,

    Are you available for such projects.

    Thanks

    ReplyDelete
  41. Amazing post. Keep it up. Much thanks to you such an incredible sum for sharing your beneficial blog. Duplicate Payment Review | Daily Transaction Monitoring | Duplicate Invoice Audit


    ReplyDelete
  42. Those guidelines additionally worked to become a good way to recognize that other people online have the identical fervor like mine to grasp great deal more around this condition.

    Selenium Training in Chennai

    Selenium Training in Porur



    ReplyDelete
  43. It’s great to come across a blog every once in a while that isn’t the same out of date rehashed material. Fantastic read.
    Best Python training Institute in chennai

    ReplyDelete
  44. Its nice blog with lot of information thanks for sharing keep doing it.

    Final Year Dotnet Projects Chennai | Final Year NS2 Projects Chennai.

    ReplyDelete
  45. AWS Training in Bangalore - Live Online & Classroom
    myTectra Amazon Web Services (AWS) certification training helps you to gain real time hands on experience on AWS. myTectra offers AWS training in Bangalore using classroom and AWS Online Training globally. AWS Training at myTectra delivered by the experienced professional who has atleast 4 years of relavent AWS experince and overall 8-15 years of IT experience. myTectra Offers AWS Training since 2013 and retained the positions of Top AWS Training Company in Bangalore and India.

    ReplyDelete
  46. IOT Training in Bangalore - Live Online & Classroom
    IOT Training course observes iot as the platform for networking of different devices on the internet and their inter related communication. Reading data through the sensors and processing it with applications sitting in the cloud and thereafter passing the processed data to generate different kind of output is the motive of the complete curricula. Students are made to understand the type of input devices and communications among the devices in a wireless media.

    ReplyDelete
  47. This was an nice and amazing and the given contents were very useful and the precision has given here is good.
    Bigdata Analytics Training in Bangalore

    ReplyDelete

  48. Best Solidworks training institute in noida

    SolidWorks is a solid modeling computer-aided design (CAD) and computer-aided engineering (CAE) computer program that runs on Microsoft Windows. SolidWorks is published by Dassault Systems. Solid Works: well, it is purely a product to design machines. But, of course, there are other applications, like aerospace, automobile, consumer products, etc. Much user friendly than the former one, in terms of modeling, editing designs, creating mechanisms, etc.
    Solid Works is a Middle level, Main stream software with focus on Product development & this software is aimed at Small scale & Middle level Companies whose interest is to have a reasonably priced CAD system which can support their product development needs and at the same time helps them get their product market faster.

    Company Address:
    WEBTRACKKER TECHNOLOGY (P) LTD.
    C-67,Sector-63,Noida,India.
    E-mail: info@webtracker.com
    Phone No: 0120-4330760 ,+91-880-282-0025

    webtrackker.com/solidworks-training-Course-institute-in-noida-delhi

    ReplyDelete
  49. 3D Animation Training in Noida

    Best institute for 3d Animation and Multimedia

    Best institute for 3d Animation Course training Classes in Noida- webtrackker Is providing the 3d Animation and Multimedia training in noida with 100% placement supports. for more call - 8802820025.

    3D Animation Training in Noida

    Company Address:

    Webtrackker Technology

    C- 67, Sector- 63, Noida

    Phone: 01204330760, 8802820025

    Email: info@webtrackker.com

    Website: http://webtrackker.com/Best-institute-3dAnimation-Multimedia-Course-training-Classes-in-Noida.php

    ReplyDelete
  50. Graphics designing training institute in Noida
    Best Graphics training institute in Noida, Graphic Designing Course, classes in Noida- webtrackker is providing the graphics training in Noida with 100% placement supports. If you are looking for the Best Graphics designing training institute in Noida For more call - 8802820025.

    Graphics designing training institute in Noida, Graphics designing training in Noida, Graphics designing course in Noida, Graphics designing training center in Noida

    Company address:
    Webtrackker Technology
    C- 67, Sector- 63, Noida
    Phone: 01204330760, 8802820025
    Email: info@webtrackker.com
    Website: http://webtrackker.com/Best-institute-for-Graphic-Designing-training-course-in-noida.php

    ReplyDelete
  51. You really touched some highly beneficial information here, for which I really appreciate you. Thank you for taking time to write this post here. Keep sharing.
    Website Design Company in Lucknow | Website Redesign Services

    ReplyDelete
  52. Existing without the answers to the difficulties you’ve sorted out through this guide is a critical case, as well as the kind which could have badly affected my entire career if I had not discovered your website.
    Digital Marketing online training

    full stack developer training in pune

    full stack developer training in annanagar

    full stack developer training in tambaram

    full stack developer training in velachery

    ReplyDelete
  53. This comment has been removed by the author.

    ReplyDelete
  54. Thank you for allowing me to read it, welcome to the next in a recent article. And thanks for sharing the nice article, keep posting or updating news article.
    Blueprism training in annanagar

    Blueprism training in velachery

    Blueprism training in marathahalli

    ReplyDelete
  55. I have been meaning to write something like this on my website and you have given me an idea. Cheers.
    java training in chennai | java training in bangalore


    java training in tambaram | java training in velachery

    ReplyDelete
  56. I am really happy with your blog because your article is very unique and powerful for new reader.
    Click here:
    Selenium Training in Chennai | Selenium Training in Bangalore | Selenium Training in Pune | Selenium online Training

    ReplyDelete
  57. Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.
    safety course in chennai

    ReplyDelete
  58. Well researched article and I appreciate this. The blog is subscribed and will see new topics soon.
    python training in tambaram | python training in annanagar | python training in jayanagar

    ReplyDelete
  59. Really very nice blog information for this one and more technical skills are improve,i like that kind of post.

    Online DevOps Certification Course - Gangboard
    Best Devops Training institute in Chennai

    ReplyDelete
  60. Fantastic work! This is the type of information that should follow collective approximately the web. Embarrassment captivating position Google for not positioning this transmit higher! Enlarge taking place greater than and visit my web situate
    Java training in Indira nagar | Java training in Rajaji nagar

    Java training in Marathahalli | Java training in Btm layout

    ReplyDelete
  61. This is most informative and also this post most user friendly and super navigation to all posts... Thank you so much for giving this information to me.. 

    best rpa training in chennai | rpa online training |
    rpa training in chennai |
    rpa training in bangalore
    rpa training in pune
    rpa training in marathahalli
    rpa training in btm

    ReplyDelete
  62. When I initially commented, I clicked the “Notify me when new comments are added” checkbox and now each time a comment is added I get several emails with the same comment. Is there any way you can remove people from that service? Thanks.

    AWS Interview Questions And Answers

    AWS Training in Pune | Best Amazon Web Services Training in Pune

    AWS Training in Chennai |Best Amazon Web Services Training in Chennai


    AWS Training in Chennai | Best AWS Training in Chennai


    Amazon Web Services Training in Pune | Best AWS Training in Pune

    ReplyDelete