Trasformation matrix for point cloud registration knowing the positions of the cameras

Question

I'm using Open3D in Python. I have 2 (or more) depth cameras in well known fixed positions. I want to align the generated pointclouds "manually" providing one trasformation matrix for each of them. I'd like to have something where I plug in position and orientation of the cameras and it gives me the transformation matrix to align them to a common reference. Then I'd use few iterations of ICP registration to fine tune the alignment if needed.

The problem with global registration is that it needs quite a bit of overlapping point to be useful, but in a setup where the cameras are, lets say, at opposite corners of a room, it is quite unreliable.

Is there a built-in method in Open3D to do so? Maybe I've been lazy with my research and didn't find any, but it seems to me this should be a quite common problem and there should be a general approach for these kind of situations. With my lack of knowledge in the field I've not been able to find an high level solution and instead I'm relying on calculating the rotation matrices "by hand" using trigonometry, but what I've come up with works only if they are all at the same distance and height from the reference point.

For reference, a testing setup I'm dealing with is the following: I have 2 depth cameras at opposite corners of a square room of 4 m sides, placed at 3 m of height and pointing down at the center of the floor. Using the center of the floor as the origin of the coordinate system the vector of the cameras are:

• camera_1: Pos (2, 3, 2), Up (0, 1, 0), LookAt (0, 0, 0)
• camera_2: Pos (-2, 3, -2), Up (0, 1, 0), LookAt (0, 0, 0)

In my case I know a-priori the positions and they are stored in a JSON file.

I'd love a way to deal with different LookAt vectors and Up vectors too, I think it should definitely be possibile.

Thank you so much to whoever is willing to help, really appreciate it!