detect a table part from entire image in python

You can use multi-scale template matching for the problem.

• 1. Apply Canny edge detector to see the corners.
• 2. Loop over the scales of each image in the given dir.
• 4. Print coordinates and if found, show it in the image.

The code:

import numpy as np
import imutils
import glob
import cv2

template = cv2.imread("apA8L.png")
template = cv2.cvtColor(template, cv2.COLOR_BGR2GRAY)
template = cv2.Canny(template, 50, 200)
(h, w) = template.shape[:2]

for imagePath in glob.glob("img2" + "/*.jpg"):
    image = cv2.imread(imagePath)
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    found = None

    for scale in np.linspace(0.2, 1.0, 20)[::-1]:
        resized = imutils.resize(gray, width=int(gray.shape[1] * scale))
        r = gray.shape[1] / float(resized.shape[1])

        if resized.shape[0] < h or resized.shape[1] < w:
            break

        edged = cv2.Canny(resized, 50, 200)
        result = cv2.matchTemplate(edged, template, cv2.TM_CCOEFF)
        (_, maxVal, _, maxLoc) = cv2.minMaxLoc(result)

        if found is None or maxVal > found[0]:
            found = (maxVal, maxLoc, r)

    (_, maxLoc, r) = found
    (startX, startY) = (int(maxLoc[0] * r), int(maxLoc[1] * r))
    (endX, endY) = (int((maxLoc[0] + w) * r), int((maxLoc[1] + h) * r))

    cv2.rectangle(image, (startX, startY), (endX, endY), (0, 0, 255), 2)
    cv2.imwrite("out.png", image)
    print("Table coordinates: ({}, {}, {}, {})".format(startX, startY, endX, endY))

Table Coordinates:

Table Coordinates: (352, 1915, 753, 2445)

Output:

First, We need to detect the table edges clearly..... Using the following preprocessing steps, we can get the edges of the table most of the time.

def get_edges(img):
  gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
  bitwised_img = cv2.bitwise_not(gray_img)
  guassian_img = cv2.GaussianBlur(bitwised_img, (5, 5), 0)
  bilateral_img = cv2.bilateralFilter(guassian_img, 11, 17, 17)
  t, thresh_bin = cv2.threshold(bilateral_img, 0, 255, cv2.THRESH_OTSU)
  canny = cv2.Canny(thresh_bin, 0.5 * t, t)
  dilated = cv2.dilate(canny,
                       cv2.getStructuringElement(cv2.MORPH_CROSS, (3, 3)))
  medianed = cv2.medianBlur(dilated, 3)
  # Edges will be discontinous so dialtion will be make them contionuous
  return medianed

We need to detect the contours and filter them using the validate_contour function where you need to define which contours you need to have

def validate_contour(img, cc):
  """Check if the contour is a good predictor of photo location. 
 Here you need to address some realtionship to find the table contour"""
  x, y, w, h = cv2.boundingRect(cc)
  if (170000<area <200000) and 500<h<600 and 300<w<400:
    return True
  return False
  
def get_contours(img):
  contours, hierarchy = cv2.findContours(edges, 1,
                                         2)
  # filter contours that are too large or small
  # print('not_filtered_contours_contours',contours)
  contours = [cc for cc in contours if validate_contour(img, cc)]
  return contours

Step 1: Your image seems like it is already black & white image, but it is not (i.e. there multiple graylevels in it), apply some simple threshold.

Step 2: Use findContours() function. It created list of contours from binary image. Among the rest there will be contours of cells of the table.

Step 3: For each contour calculate its area using contourArea function. Eliminate contours that are too small to be a cell in your table (to eliminate tiny contours created by noise)

Step 4: For each remaining contour use minAreaRect. It rinds a rotated rectangle of the minimum area enclosing given contour.

Step 5: For each contour compare its area and area of enclosing rectangle. It area/rectangleArea is close enough to 1, then its shape is close to rectangle, i.e. it has chance to be a cell in table. Eliminate all other contours

Step 6: Sort contours by their area

Step 6: For each contour compare it with contours of similar area. Check whether there multiple contours with similar width/height/orientation of their bounding rectangle (all cells in column has similar parameters). Create clusters of such contours, and eliminate outliers

Step 7: Look for columns in clusters (one cluster may contain more then one column if the cells are similar), according to distance between cells, ad their co-linearity

Step 8: Look for neighbor columns that will form the table

Step 9: Once you know the cels that belong to table - use minAreaRect on all points in all cells to find the boundaries of table

First plot this image using matplotlib.pyplot.subplot(). That will make it easier to find the coordinates of the table.

1. You can then define a new array with coordinates received using matplotlib like table = entire_image[x1:y1, x2:y2]
2. Or you can use the PIL.Image crop method by giving the coordinates of alternate corners. table = entire_image.crop((x1,y1,x2,y2))