Remove green background screen from image using OpenCV/Python

Question

So I have this image below:

As you can see, there is some noisy green outline in the image. This is by far, my latest output with OpenCV with Python. And I'm still a beginner with this so I need your help.

Assuming I were to create a completely new script and feed this image and 'clean' the image, how will I do it?

EDIT: This is the original image:

Answer 1

Use Canny Edge Detection on your image. Then depending how it looks like use Morphological Dilation (cv.dilate()) to make the edges a bit thicker. Afterwards remove the edges from the green channel of the image or reduce the brightness of green channel. Or make the pixels transparent.

This post states the following approach:

1.) Convert green pixels to transparency. Basically uses a filtering rule in the HSV color space.

2.) On the hair and some boundary pixels colors are mixed with green. To reduce this problem these pixels are filtered and balanced to reduce their green proportion.

3.) Apply a gradient transparency to all boundary pixels.

Answer 2

Here is a rather simple approach.

Background:

Dominant colors like red, green, blue and yellow can be segmented pretty easily when analyzed in LAB color space. LAB space has 3 channels, out of which only 2 are color channels while 1 is a brightness channel:

• L-channel: represents the brightness
• A-channel: represents the amount of red/green color
• B-channel: represents the amount of blue/yellow color

Observing the plot above:

1. a-axis clearly shows red/green color on its extremes
2. b-axis shows blue/yellow color on its extremes

Applying a suitable threshold in the corresponding channels makes it easy for us to segment these colors.

Approach:

We will use the above info as a base to approach the problem at hand:

1. Perform basic masking:

-> Convert image to LAB space -> Threshold a-channel to isolate green background -> Masking original image with binary mask

img = cv2.imread('green_background.jpg')
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
a_channel = lab[:,:,1]
th = cv2.threshold(a_channel,127,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]
masked = cv2.bitwise_and(img, img, mask = th)    # contains dark background
m1 = masked.copy()
m1[th==0]=(255,255,255)                          # contains white background

2. Remove green shade along the border:

-> Convert masked image LAB space -> Normalize the a-channel mlab[:,:,1] to use the entire intensity range between [0-255] -> Inverse binary threshold to select area with green borders

mlab = cv2.cvtColor(masked, cv2.COLOR_BGR2LAB)
dst = cv2.normalize(mlab[:,:,1], dst=None, alpha=0, beta=255,norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8U)

Above image is the a-channel with dark pixels around the border.

threshold_value = 100
dst_th = cv2.threshold(dst, threshold_value, 255, cv2.THRESH_BINARY_INV)[1]

Above is the threshold image to segment green shaded regions.

In the a-channel, having green color represents the lower end of the range [0-255], while red color represents the higher end. We will set intensity value in the a-channel of the selected pixels to 127:

mlab2 = mlab.copy()
mlab[:,:,1][dst_th == 255] = 127

Now in the above image, we do not see the dark borders around the person.

Convert the image to BGR and the pixels that were dark (0) in the threshold image are set to white (255, 255, 255) in the color image

img2 = cv2.cvtColor(mlab, cv2.COLOR_LAB2BGR)
img2[th==0]=(255,255,255)

The resulting image looks like nothing much has changed, so here is a zoomed in comparison:

Before Step 2:

After Step 2:

Conclusion:

The noisy green outline around the border has improved to a certain extent. You can try varying threshold_value and experiment.

The green shade present within the person's face still remains. Sophisticated methods will be needed to get rid of those. Hope this approach helps.

Answer 3

Here is a variation on @Jeru Luke's answer using Python/OpenCV. It does the same thresholding in LAB colorspace, but I simply anti-alias using a blur and a stretch of half the input range of gray values on the A channel. It has the benefit of reducing some green and smoothing the result so that it will blend smoothly into any background image.

Adjust the blur sigma values and/or the input clip values in the rescale_intensity to adjust the smoothing and amount of green showing.

Input:

import cv2
import numpy as np
import skimage.exposure

# load image
img = cv2.imread('greenscreen.jpg')

# convert to LAB
lab = cv2.cvtColor(img,cv2.COLOR_BGR2LAB)

# extract A channel
A = lab[:,:,1]

# threshold A channel
thresh = cv2.threshold(A, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]

# blur threshold image
blur = cv2.GaussianBlur(thresh, (0,0), sigmaX=5, sigmaY=5, borderType = cv2.BORDER_DEFAULT)

# stretch so that 255 -> 255 and 127.5 -> 0
mask = skimage.exposure.rescale_intensity(blur, in_range=(127.5,255), out_range=(0,255)).astype(np.uint8)

# add mask to image as alpha channel
result = img.copy()
result = cv2.cvtColor(img,cv2.COLOR_BGR2BGRA)
result[:,:,3] = mask

# save output
cv2.imwrite('greenscreen_thresh.png', thresh)
cv2.imwrite('greenscreen_mask.png', mask)
cv2.imwrite('greenscreen_antialiased.png', result)

# Display various images to see the steps
cv2.imshow('A',A)
cv2.imshow('thresh', thresh)
cv2.imshow('blur', blur)
cv2.imshow('mask', mask)
cv2.imshow('result', result)

cv2.waitKey(0)
cv2.destroyAllWindows()

Threshold Image:

Mask Image (for alpha channel):

Result:

Answer 4

Try to make a custom threshould, like:

def color_filter(img, r, g, b):
    colors = [b, g, r]
    result = np.zeros(img.shape, dtype=np.uint8)
    for i in range(3):
        result[:, :, i] = np.where(img[:, :, i] < colors[i], 0, 255)
    return result.astype(np.uint8)

UPDATE: here another solution https://codereview.stackexchange.com/a/184059/15056

To threshould every color channel with a different value.

You can find the best configuration for you using

def test_colors(img):
    cv.imshow("test_colors", img)
    r = 100
    g = 100
    b = 100
    while True:
        k = chr(cv.waitKey(0))
        if k == 'a':
            r += 1
        elif k == 'q':
            r -= 1
        elif k == 's':
            g += 1
        elif k == 'w':
            g -= 1
        elif k == 'd':
            b += 1
        elif k == 'e':
            b -= 1
        elif k == 't':
            r += 1
            g += 1
            b += 1
        elif k == 'g':
            r -= 1
            g -= 1
            b -= 1
        elif k == 'r':
            r = 100
            g = 100
            b = 100
            cv.imshow("test_colors", img)
            continue
        elif k == 'x':
            cv.destroyAllWindows()   
            print("The RGB is ", (r, g, b))
            break
        else:
            continue
        cv.imshow("test_colors", color_filter(img, r, g, b))