Three.JS how to use ray casters with multiple cameras?

Question

I'm hoping someone here has done this before. I have a ThreeJS scene, which is essentially a cube. The cube has 6 images one on each of the scenes, basically rendering a 360 panoramic photo.

Within our 3D space, we have 6 cameras, one each pointing at each of the directions, forward, backward, left, right, up, and down. Basically we want to be able to project each of these views all at once.

So, to do this I've made 6 cameras, and added them to the scene. In the scene, we have clickable targets. I want to be able to use ray caster to register a click in any camera view. For example, this is what it looks like:

Within our scene we have clickable hot spots. I want to be able to click on a hotspot on ANY of the 6 cameras. But, I can't get the ray caster to work on even one. This is the code I am using:

if (this._my_cameras && this._my_cameras[0])
        {
            var mouse = {};
    
            mouse.x = ( this._nextMouseX / this._my_cameras[0].viewport.width ) * 2 - 1;
            mouse.y = - ( this._nextMouseY / this._my_cameras[0].viewport.height ) * 2 + 1;
    
            
            Object.assign(
                this._mouse,
                coords.normalizeScreenXY(mouse, this._my_cameras[0].viewport)
            );
    
            console.log("this._mouse: " + util.inspect(this._mouse));
    
            this._raycaster.setFromCamera(this._mouse, this._my_cameras[0]);
            
        }
        else
        {
            this._raycaster.setFromCamera(this._mouse, this._camera);
        }
        
        let intersections = this._raycaster.intersectObjects(
            this._scene.children,
            true
        );

this._nextMouseX / Y are the raw mouse coordinates on screen. My normalize function should normally the mouse coordinates to -1 to 1 as needed. This all works fine if I have one camera taking up the whole view. But with 6 cameras, I never get a ray caster intersection with my targets.

Does anyone have an idea on how to get raycasting or object picking working across multiple cameras?

Edit 1:

This is the code I am now trying to get to work using viewports for the cameras for the raycasting:

for (var i = 0; i < this._igloo_cameras.length; i++)
            {
                this._mouse.x = ( this._nextMouseX / this._igloo_cameras[i].viewport.w ) * 2 - 1;
                this._mouse.y = -1 * (this._nextMouseY  / this._igloo_cameras[i].viewport.z)  * 2 + 1;
    
                console.log("this._igloo_cameras[i].viewport: " + util.inspect(this._igloo_cameras[i].viewport));
                console.log("Igloo Camera #" + i + " this._mouse: " + util.inspect(this._mouse));
        
                this._raycaster.setFromCamera(this._mouse, this._igloo_cameras[i]);
    
                let intersections = this._raycaster.intersectObjects(
                    this._scene.children,
                    true
                );
                
                all_intersections.push(...intersections);
            }

with this I get mouse values outside of 1 to -1 and I still don't get accurate click locations/targets.

This is what I get in my console:

this._igloo_cameras[i].viewport: { x: 0, y: 685, z: 685, w: 685 }
Viewer3D.js:521 Igloo Camera #0 this._mouse: { x: -0.5883211678832116,
  y: -1.3649635036496353,
  rawX: 141,
  rawY: 810 }
Viewer3D.js:520 this._igloo_cameras[i].viewport: { x: 685, y: 685, z: 685, w: 685 }
Viewer3D.js:521 Igloo Camera #1 this._mouse: { x: -0.5883211678832116,
  y: -1.3649635036496353,
  rawX: 141,
  rawY: 810 }