How to blur the image according to segmentation map

Question

Forgive me if I am unable to explain well because I am not native speaker.

I am working on blurring the part of image according to the white part of segmentation map. For example here is my segmentation image ( bmp image ). .

Now what I want is to blur the part of original image where the pixels are white in the segmentation map. I just wrote the following code to so.

mask = mask >= 0.5
mask = np.reshape(mask, (512, 512))

mh, mw = 512, 512
mask_n = np.ones((mh, mw, 3))

mask_n[:,:,0] *= mask
mask_n[:,:,1] *= mask
mask_n[:,:,2] *= mask

# discard padded area
ih, iw, _ = image_n.shape

delta_h = mh - ih
delta_w = mw - iw

top = delta_h // 2
bottom = mh - (delta_h - top)
left = delta_w // 2
right = mw - (delta_w - left)

mask_n = mask_n[top:bottom, left:right, :]


# addWeighted
image_n = image_n *1 +   cv2.blur(mask_n * 0.8, (800, 800))

Please help me, Thanks.

Answer 1

You can do it in the following steps:

1. Load original image and mask image.
2. Blur the whole original image and save it in a different variable.
3. Use np.where() method to select the pixels from the mask where you want blurred values and then replace it.

See the sample code below:

import cv2
import numpy as np

img = cv2.imread("./image.png")
blurred_img = cv2.GaussianBlur(img, (21, 21), 0)
mask = cv2.imread("./mask.png")

output = np.where(mask==np.array([255, 255, 255]), blurred_img, img)
cv2.imwrite("./output.png", output)

Answer 2

Here's an alternative to the solution proposed by @Chris Henri. It relies on scipy.ndimage.filters.gaussian_filter and NumPy's boolean indexing:

from skimage import io
import numpy as np
from scipy.ndimage.filters import gaussian_filter
import matplotlib.pyplot as plt

mask = io.imread('https://i.stack.imgur.com/qJiKf.png')
img = np.random.random(size=mask.shape[:2])
idx = mask.min(axis=-1) == 255
blurred = gaussian_filter(img, sigma=3)
blurred[~idx] = 0

fig, axs = plt.subplots(1, 3, figsize=(12, 4))
for ax, im in zip(axs, [img, mask, blurred]):
    ax.imshow(im, cmap='gray')
    ax.set_axis_off()
plt.show(fig)

Answer 3

Here is yet another alternative to do so, useful though when you have a 2D segmentation array indicating the segmented object class of pixel (mutually exclusive) for every index (i,j), and a 3D image on which you want to apply the blur.

def gaussian_blur(image: np.ndarray,
              segmentation: np.ndarray,
              classes_of_interest: list,
              gaussian_variance: float = 10) -> np.ndarray:
'''
Function that applies a gaussian filter to the image,
specifically to the pixels contained in the possible segmented classes.
Returns an image (np.ndarray) where the gaussian blur intensity is
regulated by the parameter gaussian_variance. 
'''
#Apply masking to select only the indices where the specific class is present
mask = np.isin(segmentation, classes_of_interest)
#Creating a 3D mask for all the channels and place it at channel axis
mask_3d = np.stack([mask,mask,mask], axis=2)
#Mask the image according to the 3D mask
img_masked = np.where(mask_3d, img, 0).astype(np.int8)
#Define gaussian blur noisy function
def noisy(image):
    row,col,ch= image.shape
    mean = 0
    var = gaussian_variance
    sigma = np.sqrt(var)
    gauss = np.random.normal(mean,sigma,(row,col,ch))
    gauss = gauss.reshape(row,col,ch)
    #Sums up gaussian noise to img
    noisy = image + gauss
    return noisy.astype(np.uint8)
#Blurs the masked segmentation
img_masked_noisy = noisy(img_masked)
#Puts the blurred part back in the original image as substitution
img[mask_3d] = img_masked_noisy[mask_3d]
return img

And here is a toy example:

import numpy as np
possible_classes = [1,2,3]
#Setting up a toy example with a small image,                               
#shape (N, N, 3)
img = np.floor(np.random.random(size=(8,8,3)) * 256).astype(np.uint8)
#Setting up a fake segmentation with 3 mutually exclusive possible classes, 
#shape (N, N)
segmentation = np.random.choice(possible_classes, size=(8,8))
new_img_blurred = gaussian_blur(img,
              segmentation= segmentation,
              classes_of_interest= possible_classes[:2])