Subtract And Merging Images for Making Video Frames

Question

When you want create a video file,the size of it has important. I want to save the difference frames from each other , not whole of them,with JPEG format for decrease size of my new video file. I used Aforge.Net Framework and it produces the result of difference well (with Difference class). But when I want merge the frame with previous frame for generate current frame the result isn't that I need,see the example below :

This is the Background Frame

This is the Current Frame

This is the Difference Frame

the code for getting difference :

AForge.Imaging.Filters.Difference sb = new AForge.Imaging.Filters.Difference(bmpDest);
            Bitmap result = sb.Apply(bmpSrc);
            result.Save(string.Format(address, 10002), ImageFormat.Jpeg);

As you know, when I merge the difference frame with the background frame,the current frame must be generated,but this is generated frame :

And the source code is:

 AForge.Imaging.Filters.Merge mg = new AForge.Imaging.Filters.Merge(bmpSrc);
            Bitmap results = mg.Apply(new Bitmap(string.Format(address, 10002)));
            results.Save(string.Format(address, 1004), ImageFormat.Jpeg);

So,the problem is clear. These are screen captured but when I want use this operation to camera capture the result has many problem too.

I used openCV for this operation too, but in the instruction mentioned for getting difference, first I should convert frames to grayscale images and i have problem with merging image too, like AForge.

And at last i used my algorithm but the size of difference image be larger than original images!

How should I do this operation? Remember the size of Images is very important and I want to save the JPEG files (for the compression).

Answer 1

The problem is that the difference filter takes the absolute value. The result is that the mouse pointer is getting added back into the image rather than subtracting from the image since it should be negative in the difference image. I'm not sure how to fix this in AForge.net.

In opencv you resolve this by using an image type that supports signed numbers, like CV_16SC3 instead of the default CV_8UC3 for an RGB/BGR image. You shouldn't use CV_8SC3 because one of the 8 bits will be used for a sign, so pixel values outside +/-127 will be clamped.

void imageDelta(cv::Mat& A, cv::Mat& B, cv::Mat& deltaOut)
{
    //Calculate B - A, preserving sign
    A.convertTo(A, CV_16SC3);
    B.convertTo(B, CV_16SC3);
    cv::subtract(B, A, deltaOut); //deltaOut is a CV_16SC3
}

void reconstructB(cv::Mat& A, cv::Mat& delta, cv::Mat& BOut)
{
   //Calculate B = A + delta
   cv::add(A, delta, BOut);
   BOut.convertTo(BOut, CV_8UC3); //Safe to go back to CV_8UC3 now.
}