OpenCV Assertion failed ((unsigned)(i1 * DataType<_Tp>::channels) < (unsigned)(size.p[1] * channels())) in cv::Mat::at

Question

I'm trying to do the smoothing of an RGB image using OpenCV. I'm using spatial correlation formula:

Here's the code:

#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include <iostream>

using namespace cv;
using namespace std;

int main(int argc, char** argv) {
    if (argc < 2) {
        cout << "usage: " << argv[0] << " image_name" << endl;
        exit(0);
    }

    String imageName = argv[1];

    Mat image;

    image = imread(imageName, IMREAD_COLOR);

    if (image.empty()) {
        cout << "Could not open or find the image" << std::endl;
        return -1;
    }

    int padding = 2;
    int padding2 = padding / 2;

    Mat copy = Mat::zeros(image.rows + padding, image.cols + padding, image.type());

    image.copyTo(copy(Rect(padding2, padding2, image.cols, image.rows)));
    //image.copyTo(copy(Range(padding2,image.rows+padding2),Range(padding2,image.cols+padding2)));

    cout << "rows: " << image.rows << " cols: " << image.cols << endl;
    for (int i = 0; i < image.rows + 1; i++) {
        for (int j = 0; j < image.cols + 1; j++) {
            int sumB = 0;
            int sumG = 0;
            int sumR = 0;
            for (int r = -1; r <= 1; r++) {
                for (int c = -1; c <= 1; c++) {
                    sumB += image.at<Vec3b>(i + 1 + r, j + 1 + c)[0];
                    sumG += image.at<Vec3b>(i + 1 + r, j + 1 + c)[1];
                    sumR += image.at<Vec3b>(i + 1 + r, j + 1 + c)[2];
                }
            }
            cout << "i: " << i << " j: " << j << endl;
            copy.at<Vec3b>(i + 1, j + 1)[0] = sumB / 9;
            copy.at<Vec3b>(i + 1, j + 1)[1] = sumG / 9;
            copy.at<Vec3b>(i + 1, j + 1)[2] = sumR / 9;
        }
    }

    imshow("Original", image);
    imshow("Copy", copy);
    waitKey(0);
    return 0;
}

When I'm trying to execute it, I get this error:

OpenCV(3.4.9) Error: Assertion failed ((unsigned)(i1 * DataType<_Tp>::channels) < (unsigned)(size.p[1] * channels())) in cv::Mat::at, file C:\opencv\build\include\opencv2\core\mat.inl.hpp, line 1179 

I'm using OpenCV 3.4.9 with Visual Studio 2022 in Windows 11.

Answer 1

The cause for the assert:

Trying to access a cv::Mat out of its bound. In these lines:

sumB += image.at<Vec3b>(i + 1 + r, j + 1 + c)[0];
sumG += image.at<Vec3b>(i + 1 + r, j + 1 + c)[1];
sumR += image.at<Vec3b>(i + 1 + r, j + 1 + c)[2];

i can be up to image.rows, and j can be up to image.cols (see the for loops above). Then r and c can be up to 1. It means the .at method will be called with out-of-bound indices.

You could change your for loops to do 2 less iterations. But another related issue is that in each (i,j) iteration you actually handle the ((i+1),(j+1)) pixel which is a bit missleading.

I advise to change it to be more straightforward (see the code below).

Another issue is that calling .at is quite expensive (due to its implementation containing a lot of checks - one of them caused the assert). I reduced the number of calls by a factor of 3 by getting a reference to the pixel once, and then using this reference to access the 3 channels.

A few more comments:

1. Better to avoid using namespace std - see here Why is "using namespace std;" considered bad practice? I also believe using namespace cv should be avoided for similar (even if less strong) reasons.

2. This code can be further optimized, by using direct access to cv::Mat data using pointers. But I think this optimization can be subject of another stackoverflow entry.

Here is the code:

#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include <iostream>


int main(int argc, char** argv) {
    if (argc < 2) {
        std::cout << "usage: " << argv[0] << " image_name" << std::endl;
        exit(0);
    }

    cv::String imageName = argv[1];
    cv::Mat image = cv::imread(imageName, cv::IMREAD_COLOR);
    if (image.empty()) {
        std::cout << "Could not open or find the image" << std::endl;
        return -1;
    }

    int padding = 2;
    int padding2 = padding / 2;
    cv::Mat copy = cv::Mat::zeros(image.rows + padding, image.cols + padding, image.type());
    image.copyTo(copy(cv::Rect(padding2, padding2, image.cols, image.rows)));

    std::cout << "rows: " << image.rows << " cols: " << image.cols << std::endl;
    for (int i = 1; i < image.rows - 1; i++) {
        for (int j = 1; j < image.cols - 1; j++) {
            int sumB = 0;
            int sumG = 0;
            int sumR = 0;
            for (int r = -1; r <= 1; r++) {
                for (int c = -1; c <= 1; c++) {
                    auto const & srcPixel = image.at<cv::Vec3b>(i + r, j + c);
                    sumB += srcPixel[0];
                    sumG += srcPixel[1];
                    sumR += srcPixel[2];
                }
            }
            //cout << "i: " << i << " j: " << j << endl;
            auto & dstPixel = copy.at<cv::Vec3b>(i, j);
            dstPixel[0] = sumB / 9;
            dstPixel[1] = sumG / 9;
            dstPixel[2] = sumR / 9;
        }
    }

    cv::imshow("Original", image);
    cv::imshow("Copy", copy);
    cv::waitKey(0);
    return 0;
}