'Live Rectangle detection using Vision Framework

I'm trying to detect rectangle from live preview layer, but not able to detect all rectangles.

What I'm doing

To setup Vision Request

 func setupVision() {
    let rectanglesDetectionRequest = VNDetectRectanglesRequest(completionHandler: self.handleRectangles)
    rectanglesDetectionRequest.maximumObservations = 0
    rectanglesDetectionRequest.quadratureTolerance = 45.0
    rectanglesDetectionRequest.minimumAspectRatio = 0.64
    self.requests = [rectanglesDetectionRequest]
}

    
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
        return
    }
    
    let exifOrientation = self.exifOrientationFromDeviceOrientation()
    
    
    DispatchQueue.main.asyncAfter(deadline: .now() + 2) {
        var requestOptions: [VNImageOption : Any] = [:]
        
        if let cameraIntrinsicData = CMGetAttachment(sampleBuffer, kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, nil) {
            requestOptions = [.cameraIntrinsics: cameraIntrinsicData]
        }
        
        DispatchQueue.global(qos: .background).async {
            let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation:exifOrientation, options: requestOptions)
            do {
                try imageRequestHandler.perform(self.requests)
            } catch {
                print(error)
            }
        }
        

    }
    

    var arr = Array<VNTrackRectangleRequest>()
    for obs in self.rectanglesss{
            let trackRequest = VNTrackRectangleRequest(rectangleObservation: obs, completionHandler: self.handleSequenceRequestUpdate)
            trackRequest.trackingLevel = .accurate
            arr.append(trackRequest)
    }
    
    
    do {
        try self.sequenceHandler.perform(arr, on: pixelBuffer, orientation: exifOrientation)
    } catch {
        print(error)
    }
}

Can someone help me to figure out what I'm doing wrong ? When I try with Right angle sometimes it detect few of them, with Acute angle its detect only near by 2-3 rectangles. Here I try with SET cards, I added 2images of what I'm getting.

Result

Solution 1:^[1]

Try iphone to see them via bird view? And get a different table with non-white color

Solution 2:^[2]

I suggest you put your phone flat and shoot ,then setting minimumConfidence

Solution 3:^[3]

Use this...

https://developer.apple.com/documentation/vision/vndetectrectanglesrequest/2875373-maximumobservations

let request = VNDetectRectanglesRequest { (request, error) in
    // Your completion handler code
}

request.maximumObservations = 2

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	Meep
Solution 2
Solution 3	PAULMAX

'Live Rectangle detection using Vision Framework

Solution 1:[1]

Solution 2:[2]

Solution 3:[3]

Sources

Related Questions

Solution 1:^[1]

Solution 2:^[2]

Solution 3:^[3]