'Improving corner detection paintings OpenCV

I need to find the four corners of one (or more) paintings inside a picture. I got it working so far but only when the wall in the background is plain and there is only one painting in the picture.

This is the code I use currently:

def create_painting_mask(file_path):
def order(corners):
    # Order points in following order: top-left, top-right, bottom-left, bottom-right
    result = np.zeros((4, 2), dtype="float32")

    # Sum coordinate values, top left will have smallest sum,
    # top right will have biggest sum
    s = corners.sum(axis=1)
    result[0] = corners[np.argmin(s)]
    result[3] = corners[np.argmax(s)]

    # Subtract coordinate values, top right will be minimum,
    # bottom left will be maximum
    d = np.diff(corners, axis=1)
    result[1] = corners[np.argmin(d)]
    result[2] = corners[np.argmax(d)]

    return result

# reading image and shape
img = cv.imread(file_path)
h, w, c = img.shape

# converting to gray
grayscale = cv.cvtColor(img, cv.COLOR_BGR2GRAY)

# blurring
blurred = cv.medianBlur(grayscale, 25)

mask = np.zeros(img.shape[:2], dtype=np.uint8)

edges = cv.Canny(blurred, 5, 10)

kernel_dilate = cv.getStructuringElement(cv.MORPH_RECT, (10, 10))
dilate = cv.dilate(edges, kernel_dilate)

contours, hierarchy = cv.findContours(dilate, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)

result = np.zeros((h, w, 3), dtype=np.uint8)

for i, cnt in enumerate(contours):
    area = cv.contourArea(cnt)
    perimeter = cv.arcLength(cnt, closed=True)

    if area > 1000000:
        border = cv.approxPolyDP(curve=cnt, epsilon=0.05 * perimeter, closed=True)
        cv.drawContours(mask, [border], 0, (255), -1)
        if len(border) == 4:
            border = np.reshape(border, (4, 2))
            arranged_points = order(border)
            new_points = np.float32([[0, 0],
                                     [w, 0],
                                     [0, h],
                                     [w, h]])

            transformation = cv.getPerspectiveTransform(arranged_points, new_points)
            result = cv.warpPerspective(img, transformation, (w, h))

print_all(img, mask, result)

It needs to perform better, to reduce the errors like the ones below:

good result

bad result (background)

bad result (wires)

bad result (multiple frames)

If anyone has any help, for example tweaking some parameters or trying out new functions I don't know about, please reply! If you want to try and tweak a bit yourself a duplication of my code is available on github, including all the test images I use as well => https://github.com/rubendebbaudt/painting-corner-detection

THANKS IN ADVANCE!



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source