Python: Fast way to remove horizontal black line in image

Question

I would like to remove horizontal black lines on an image:

To do this, I interpolate the RGB values ​​of each column of pixels.

The black line disappear but I think it is possible to optimize this function:

def fillMissingValue(img_in):
    img_out = np.copy(img_in)

    #Proceed column by column
    for i in tqdm(range(img_in.shape[1])):
        col = img_in[:,i]

        col_r = col[:,0]
        col_g = col[:,1]
        col_b = col[:,2]
        r = interpolate(col_r)
        g = interpolate(col_g)
        b = interpolate(col_b)
        img_out[:,i,0] = r
        img_out[:,i,1] = g
        img_out[:,i,2] = b

    return img_out

def interpolate(y):
    x = np.arange(len(y))
    idx = np.nonzero(y)
    interp = interp1d(x[idx],y[idx],fill_value="extrapolate" )

    return interp(x)

if __name__ == "__main__":
    img = cv2.imread("lena.png")
    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

    img = cv2.resize(img, (1024,1024))


    start = time.time()
    img2 = fillMissingValue(img)
    end = time.time()

    print("Process time: {}".format(np.round(end-start,3)))

Do you have any ideas ? I thought of doing a prepocessing step by identifying the position of the black lines. And thus only interpolated neighboring pixels. But I don't think it's faster

Current result:

Answer 1

interp1d is not very efficient.

As proposed by @ChristophRackwitz in the comments, you can detect the location of the lines and use the inpainting method provided by OpenCV:

img = cv2.imread('lena.jpg')

# Locate the relatively black lines
threshold = 25
lineIdx = np.where(np.mean(img, axis=(1,2)) < threshold)

# Perform inpainting on the located lines
mask = np.zeros(img.shape[:2], dtype=np.uint8)
mask[lineIdx] = 255

# Actual inpainting.
# Note: using 2 or 1 instead of 3 make the computation 
# respectively ~2 and ~4 time faster on my machine but 
# the result is not as beautiful with 3.
img2 = cv2.inpaint(img, mask, 3, cv2.INPAINT_NS)

The computational part takes 87 ms on my machine while your code takes 342 ms. Note that because of JPEG compression, the result is not so great. You can inpaint the neighbour lines (eg. lineIdx-1 and lineIdx+1) so to get a much better result at the expense of the slower computation (about 2.5 slower on my machine).

---

An alternative solution is to perform the interpolation yourself in Numpy:

%%time
# Locate the relatively black lines
threshold = 25
lineIdx = np.where(np.mean(img, axis=(1,2)) < threshold)[0]
lineIdxSet = set(lineIdx)

img2 = img.copy()
start, end = None, None
interpolate = False
for i in range(img.shape[0]+1):
    if i in lineIdxSet:
        if start is None:
            start = i
        end = i
    else:
        if not (start is None):
            assert not (end is None)
            # The first lines are black
            if start <= 0:
                i0, i1 = end+1, end+1
            # The last lines are black
            elif end >= img.shape[0]-1:
                i0, i1 = start-1, start-1
            # usual case
            else:
                i0, i1 = start-1, end+1
            # The full image is black
            if i0 < 0 or i1 >= img.shape[0]:
                continue
            end = min(end, img.shape[0]-1)
            # Actual linear interpolation (of a bloc of lines)
            coef = np.linspace(0, 1, end-start+3)[1:-1].reshape(-1, 1)
            l0 = img[i0].reshape(-1)
            l1 = img[i1].reshape(-1)
            img2[start:end+1] = (coef * l0 + (1.0-coef) * l1).reshape(-1, img.shape[1], 3)
        start, end = None, None

This code takes only 5 ms on my machine. The result should be similar to the one of your original code except that it works line-by-line and not column by column and that the detection is not independent for each colour channel. Note that the inpainting method give more beautiful results when large blocs of lines are black.