How to speed up upscaling images using opencv?

Question

I try to upscale my images via Convolutional Neural Networks. The source image is B&W/(black and white) as below

Using 'super resolution' code:

import numpy as np
import cv2 as cv
from cv2 import dnn_superres

sr = dnn_superres.DnnSuperResImpl_create()
#read each image
img = cv.imread('img.jpg')

#read the model for resolution fixing 
sr.readModel('EDSR_x4.pb')
#Set the desired model and scale to get correct pre- and post-processing
sr.setModel("edsr", 4)

rimg = sr.upsample(img)

I got the model from github. Code is working, no errors no warnings. but so slow and battery consuming, the way that Running this code going to be a challenge for me. It freezes normal UI processes and draining the battery. the device also getting hot, till I have to restart it. What is going wrong?