Photoshop Question posted in
The official user guide can be found here.

Contour matchshapes in openCV – Photoshop

I am new to openCV(with Python) and image processing and tried to start a project of my own to learn how things work. I’m trying to select the teeth from an image of an open mouth. The best solution that I could think of is to match it with a template of a tooth and try to matchshapes based on the contours that I get from the mouth image. This might not be the best solution, but it does seem rather easy.

So far, my code looks like this:

img = cv2.imread('resources/1.jpg', cv2.IMREAD_COLOR)
compare = cv2.imread('resources/template.jpg', cv2.IMREAD_GRAYSCALE)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = cv2.bilateralFilter(gray, 11, 17, 17)
edges = cv2.Canny(gray, 20, 10)
compare = cv2.Canny(compare, 200, 200)

contours, _ = cv2.findContours(gray.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
compare_contours, _2 = cv2.findContours(compare.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

for c in contours:
    for c2 in compare_contours:
        ret = cv2.matchShapes(c, c2, 1, 0.0)
        if ret < 0.5:
            peri = cv2.arcLength(c, True)
            approx = cv2.approxPolyDP(c, 0.02 * peri, True)
            cv2.drawContours(img, [approx], -1, (0, 255, 0), 7)

img is the image of the mouth and compare is a tooth from the same image that I selected with photoshop, so I do expect to match at least that one. My current problem is that the only contour that matches seems to be the frame of the image.

Can someone please point out what am I doing wrong or suggest me a better approach of the project? Does the image size have something to do with the image size and if so, what is the best way to resize images?

Thank you for your time!

Tags:

2

Answers


  1. 0

    Shape descriptors are the way to go

    For matching an object in scene with little occlusion, I would recommend to try a keypoints extraction then matching, such as SIFT (and its friends ORB,SURF …)

    I tried a super-quick copy/pasting of code for SURF template matching and visualizing using this SO answer and your data, which is promising :

    SURF matching

    It shows 4 plausible matches already ! =)

    How it works

    You’ll need to do a bit of a deep dive to get the hang of the algorithms, but the basic idea remains the same under the hood :

    1. Given a template image, find points of interests (corners, recognizable points of interest with characteristic) based on mathematic sorcery. These are N lists of ‘M’ values. These values represent statistics of image around the point of interest (such as local shape histogram, edge information …)
    2. Given a scene (with the template included), compute all the points of interest of that image.
    3. Look for template-like keypoints in your scene by doing distance-calculations in M dimension : Each keypoint is like a point in 3D, and matching 2 keypoints as similar is nothing but calculating the distance of the points in 3D and saying “close enough “!.
    4. You now have 2 sets of keypoints, and a list of plausible keypoints matches. You are now trying to see “given a list of 2D vectors, find the most plausible displacement” (rotation,scaling,translation)

    Your code

    I tried your code against your data, here is what I get by visualizing edges and compare :

     cv2.imshow("Edges",edges)
 cv2.waitKey(0)

    Edges

     cv2.imshow("compare",compare)
 cv2.waitKey(0)

    Compare

    What it means

    As you can see your edges image has a significant number of irrelevant short edges.
    This is usually a side effect of canny parameter selection with too low parameters.

    This probably affects the shape matching in a big way, as OpenCV tries to match “edge for edge”, and an open edge map would probably give different results from the compare image.

    See this python recipe to visualize the impact of threshold selection and an automatic threshold selector . Very informative read in any case =)

    More generally

    Let’s think around the constraints you are working under and see if a solution for discerning teeth pops up :

    You are looking at images containing several features of interest (1 mouth = dozens of teeth), but are trying to find any match. There is no “wrong answer”, no tricky teethlike object around to discern from (unlike this really relevant SO question).

    • Mouth is delimited by the lips, sort of. You might use that ? Ok, tricky for people with little or no lips.
    • Teeth are generally in the same location on any face (citation needed). It is import since face detection is a very well researched problem, and several algorithms are implemented in OpenCV that might help you (this tutorial for instance). Once you have the mouth, you have a box around which you are certain to find your targets, reducing greatly the problem’s scope.

    Teeth :

    • have little to no texture information (no direct help from local histogram shape)
    • luminosity should be above average of mouth around picture
    • little to no saturation, contrary to lips that are red + high saturation (unless yellowing and other fun stuff)
    • delimited mainly by its edges with other teeth and of background (interior of mouth / gums)

    Your images:

    • have color !
      • Means you have access to lots more info via HSV color space
    • is targetted at one face only
    Login or Signup to reply.
  2. 0

    The return value of cv2.matchShapes is changed now. You should try to use ret > 5 to find the correct contour.

    I had the same problem, but when I used the biggest ret, my contour can match the correct contour.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search