Get the location of all contours present in image using opencv, but skipping text

Question

I want to retrieve all contours of the image below, but ignore text.

Image:

When I try to find the contours of the current image I get the following:

I have no idea how to go about this as I am new to using OpenCV and image processing. I want to get ignore the text, how can I achieve this? If ignoring is not possible but making a single bounding box surrounding the text is, than that would be good too.

Edit:

Criteria that I need to match:

• The contours may very in size and shape.
• The colors from the image may differ.
• The colors and size of the text inside the image may differ.

Answer 1

Here is one way to do that in Python/OpenCV.

• Read the input
• Convert to grayscale
• Get Canny edges
• Apply morphology close to ensure they are closed
• Get all contour hierarchy
• Filter contours to keep only those above threshold in perimeter
• Draw contours on input
• Draw each contour on a black background
• Save results

Input:

import numpy as np
import cv2

# read input
img = cv2.imread('short_title.png')

# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# get canny edges
edges = cv2.Canny(gray, 1, 50)

# apply morphology close to ensure they are closed
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3))
edges = cv2.morphologyEx(edges, cv2.MORPH_CLOSE, kernel)

# get contours
contours = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
contours = contours[0] if len(contours) == 2 else contours[1]

# filter contours to keep only large ones
result = img.copy()
i = 1
for c in contours:
    perimeter = cv2.arcLength(c, True)
    if perimeter > 500: 
        cv2.drawContours(result, c, -1, (0,0,255), 1)
        contour_img = np.zeros_like(img, dtype=np.uint8)
        cv2.drawContours(contour_img, c, -1, (0,0,255), 1)
        cv2.imwrite("short_title_contour_{0}.jpg".format(i),contour_img)
        i = i + 1

# save results
cv2.imwrite("short_title_gray.jpg", gray)
cv2.imwrite("short_title_edges.jpg", edges)
cv2.imwrite("short_title_contours.jpg", result)

# show images
cv2.imshow("gray", gray)
cv2.imshow("edges", edges)
cv2.imshow("result", result)
cv2.waitKey(0)

Grayscale:

Edges:

All contours on input:

Contour 1:

Contour 2:

Contour 3:

Contour 4:

Answer 2

Here are two options for erasing the text:

• Using pytesseract OCR.
• Finding white (and small) connected components.

Both solution build a mask, dilate the mask and use cv2.inpaint for erasing the text.

---

Using pytesseract:

• Find text boxes using pytesseract.image_to_boxes.
• Fill the boxes in the mask with 255.

Code sample:

import cv2
import numpy as np
from pytesseract import pytesseract, Output

# Tesseract path
pytesseract.tesseract_cmd = "C:\\Program Files\\Tesseract-OCR\\tesseract.exe"

img = cv2.imread('ShortAndInteresting.png')

# https://stackoverflow.com/questions/20831612/getting-the-bounding-box-of-the-recognized-words-using-python-tesseract
boxes = pytesseract.image_to_boxes(img, lang='eng', config=' --psm 6')  # Run tesseract, returning the bounding boxes

h, w, _ = img.shape # assumes color image
mask = np.zeros((h, w), np.uint8)

# Fill the bounding boxes on the image
for b in boxes.splitlines():
    b = b.split(' ')
    mask = cv2.rectangle(mask, (int(b[1]), h - int(b[2])), (int(b[3]), h - int(b[4])), 255, -1)

mask = cv2.dilate(mask, np.ones((5, 5), np.uint8))  # Dilate the boxes in the mask
   
clean_img = cv2.inpaint(img, mask, 2, cv2.INPAINT_NS)  # Remove the text using inpaint (replace the masked pixels with the neighbor pixels).

# Show mask and clean_img for testing
cv2.imshow('mask', mask)
cv2.imshow('clean_img', clean_img)
cv2.waitKey()
cv2.destroyAllWindows()

---

Mask:

---

Finding white (and small) connected components:

• Use mask = cv2.inRange(img, (230, 230, 230), (255, 255, 255)) for finding the text (assume the text is white).
• Finding connected components in the mask using cv2.connectedComponentsWithStats(mask, 4)
• Remove large components from the mask - fill components with large area with zeros.

Code sample:

import cv2
import numpy as np

img = cv2.imread('ShortAndInteresting.png')

mask = cv2.inRange(img, (230, 230, 230), (255, 255, 255))

nlabel, labels, stats, centroids = cv2.connectedComponentsWithStats(mask, 4)  # Finding connected components with statistics

# Remove large components from the mask (fill components with large area with zeros).
for i in range(1, nlabel):
    area = stats[i, cv2.CC_STAT_AREA]  # Get area
    if area > 1000:
        mask[labels == i] = 0  # Remove large connected components from the mask (fill with zero)

mask = cv2.dilate(mask, np.ones((5, 5), np.uint8))  # Dilate the text in the maks

cv2.imwrite('mask2.png', mask)

clean_img = cv2.inpaint(img, mask, 2, cv2.INPAINT_NS)  # Remove the text using inpaint (replace the masked pixels with the neighbor pixels).

# Show mask and clean_img for testing
cv2.imshow('mask', mask)
cv2.imshow('clean_img', clean_img)
cv2.waitKey()
cv2.destroyAllWindows()

---

Mask:

Clean image:

---

Note:

• My assumption is that you know how to split the image into contours, and the only issue is the present of the text.

Answer 3

I would recommend using flood fill, find the seed point for each color region, flood fill it to ignore the text values within. Hope that helps!

Refer to example of using floodfill here: https://www.programcreek.com/python/example/89425/cv2.floodFill

Example below copied from link above

def fillhole(input_image):
'''
input gray binary image  get the filled image by floodfill method
Note: only holes surrounded in the connected regions will be filled.
:param input_image:
:return:
'''
im_flood_fill = input_image.copy()
h, w = input_image.shape[:2]
mask = np.zeros((h + 2, w + 2), np.uint8)
im_flood_fill = im_flood_fill.astype("uint8")
cv.floodFill(im_flood_fill, mask, (0, 0), 255)
im_flood_fill_inv = cv.bitwise_not(im_flood_fill)
img_out = input_image | im_flood_fill_inv
return img_out