OpenCV resolvePnP without camera and distortion matrices

Question

I am trying to make a AR app for android in witch the user points the camera to a square marker and a 3d model should show on top of the marker.

I am using OpenCV to get the rotation and translation of the marker but..

To get these 2 matrices, I use solvePnP for which I have to provide a camera matrix and a distortion matrix which (from my understanding) are specific for each camera type. Unfortunately, this is a huge drawback for me since the app should be supported by most Android devices and I also can't ask the user to run the camera calibration procedure (functions provided by openCV).

So the question: is there a way to eliminate the camera and distortion matrices? Or, is there another way to calculate the marker 3D position relative to the device?

I tried QCAR and Unity AR but (since the 3D models are downloaded form a server and are changing constantly), I was forced to go with OpenCV.

Any help will be really appreciated.

Thanks.

Answer 1

Bad news for you. The answer is no.

You can't know anything about your image if you don't know anything about your camera. The camera is described by the two matrices: camera mat and distortions mat.

But.. along with the bad news, there are some good news. You can do something, maybe.

Distortion matrix can be ignored in many AR applications. You just don't care about the small distortions.

And the camera matrix can be constructed if you know camera's field of view. Camera matrix:

w/(2*tan(camera.fovX())), 0, w/2, 0,
0, h/(2*tan(camera.fovY())), h/2, 0,
0, 0,   1, 0

On Android there's an API for retrieving the aperture: getHorizontalAperture() and getVerticalAperture(). Aperture is field of view.

However, on many phones, the returned value isn't correct, because the manufacturers did not care to set it right. It might return bogus angles like 10 or 180 degrees. There's no good way to fix this apart from testing individual phone models.

Good luck!