OpenCV Window Freezing when trying to combine with Neural Network Image Classification

Question

I am using a NN to detect 4 types of objects (chassis, front-spoiler, hubcap, wheel) in the live feed of my webcam. When one is detected, I want to display an image with information about it (chassis.png, front-spoiler.png, hubcap.png, wheel.png). When I run my NN and hold one of the items in front of the webcam, the opencv windows freezes and doesnt display anything. What is the reason for that?

def displayImg(path):
    img = cv2.imread(path)
    cv2.namedWindow("window", cv2.WND_PROP_FULLSCREEN)
    cv2.setWindowProperty("window",cv2.WND_PROP_FULLSCREEN,cv2.WINDOW_FULLSCREEN)
    cv2.imshow("window", img)

# ----------------LIVE DETECTIONS ---------------
imagePath = "picture.jpg"
frontSpoilerImageOpen = False
chassisImageOpen = False
hubcapImageOpen = False
wheelImageOpen = False

model = torch.hub.load('ultralytics/yolov5', 'custom', path='yolov5/runs/train/exp5/weights/last.pt', force_reload=True)
cap = cv2.VideoCapture(0)
while cap.isOpened():
    ret, frame = cap.read()
    results = model(frame)

    try:
        detectedItem = results.pandas().xyxy[0].iloc[0, 6]
        if detectedItem == "front-spoiler" and not frontSpoilerImageOpen:

            frontSpoilerImageOpen = False
            chassisImageOpen = False
            hubcapImageOpen = False
            wheelImageOpen = False

            displayImg(os.path.join("imagesToDisplay", "front-spoiler.png"))
            frontSpoilerImageOpen = True

        elif detectedItem == "chassis" and not chassisImageOpen:

            frontSpoilerImageOpen = False
            chassisImageOpen = False
            hubcapImageOpen = False
            wheelImageOpen = False

            displayImg(os.path.join("imagesToDisplay", "chassis.png"))
            chassisImageOpen = True

        elif detectedItem == "hubcap" and not hubcapImageOpen:

            frontSpoilerImageOpen = False
            chassisImageOpen = False
            hubcapImageOpen = False
            wheelImageOpen = False

            displayImg(os.path.join("imagesToDisplay", "hubcap.png"))
            hubcapImageOpen = True


        elif detectedItem == "wheel" and not wheelImageOpen:

            frontSpoilerImageOpen = False
            chassisImageOpen = False
            hubcapImageOpen = False
            wheelImageOpen = False

            displayImg(os.path.join("imagesToDisplay", "wheel.png"))
            wheelImageOpen = True
    except Exception as e:
        print(e)

Answer 1

Your code contains no waitKey at all.

OpenCV GUI (imshow) requires waitKey to work.

This is described in all OpenCV documentation and tutorials.

• one basic tutorial: https://docs.opencv.org/4.x/db/deb/tutorial_display_image.html
• documentation for imshow mentions the requirement: https://docs.opencv.org/4.x/d7/dfc/group__highgui.html#ga453d42fe4cb60e5723281a89973ee563

waitKey isn't about delays or breaks. It runs the event loop that all GUI processing requires.

You can use waitKey(1) for the shortest non-zero delay of one millisecond (a little more in practice), or you can use pollKey(), which will not wait even that millisecond.

Answer 2

I had a similar issue. Adding cv2.waitKey after cv2.imshow helped in my case:

cv2.imshow("window", img)
cv2.waitKey(1)  # perform GUI housekeeping tasks