Apple Vision – Barcode Detection doesn't work for barcodes with different colours

Question

So, I have to scan different barcodes with various colours. For example, a yellow barcode on black background or yellow barcode on white background.

I don't have any issues with them being recognized by traditional linear and CCD barcode scanners. I have tried using Apple Vision framework but it doesn't work on them. They work perfectly fine on black barcodes with white background.

My barcodes are all Code 128 so I use this code for it:

var barcodeObservations: [String : VNBarcodeObservation] = [:]

for barcode in barcodes {

    if let detectedBarcode = barcode as? VNBarcodeObservation {

        if detectedBarcode.symbology == .code128 {
            barcodeObservations[detectedBarcode.payloadStringValue!] = detectedBarcode
        }
    }
}

And in 'captureOutput' function under AVCaptureVideoDataOutputSampleBufferDelegate, I use this to filter my live feed as black and white which helps in the recognition of the golden barcode on silver background (The first image):

let context = CIContext(options: nil)

let currentFilter = CIFilter(name: "CIPhotoEffectMono")
currentFilter!.setValue(CIImage(cvImageBuffer: pixelBuffer), forKey: kCIInputImageKey)
let output = currentFilter!.outputImage!

context.render(output, to: pixelBuffer)

How can I make the Vision Framework detect barcodes with invert colors?

The 'CIColorInvert' filter doesn't work.

Edit: These are the barcodes:

Answer 1

Theory

By default, Apple Vision, CoreML, ARKit and RealityKit frameworks are designed to detect barcodes that must be seen through a camera as a high-contrast black-and-white images (with a predictable weights for channels: r=30%, g=59%, b=11%). In your case a yellow barcode on a white background has the lowest contrast, so no one real barcode scanner can read it, including rgb camera feed for Vision.

Let's see what Best and Worst Colors for Barcode Labels article tells us:

One reason why barcodes can be hard to scan is color, or more specifically, the lack of contrast in colors. If there isn’t enough contrast between the background and bar colors, barcode scanners will have a hard time reading it.

Avoid the following colours' combination because they are in low-contrast greyscale spectre:

Practical Solution

(only if barcode was printed on planar surface with diffuse paint)

Although if you wanna successfully detect a chromatic barcode on a chromatic background using Vision, you definitely need to apply a grayscale filter to color CVPixelBuffer stream before Vision starts recognizing barcodes. For this use AVFoundation and CoreImage frameworks.

Please read these three posts to find out how you can do it:

• Applying Filters to a Capture Stream
• Convert Image to CVPixelBuffer for Machine Learning Swift
• Converting Color Images to Grayscale

P.S.

Metallic Paints

Barcodes printed with metallic paints (gold, silver, copper, etc) are the worst instances for barcode readers. This is due to the fact that metallic paint catches reflections and it has environment lights' speculars. So barcodes printed with metallic paints are barely read.