QEMU loader device - load bare metal binary in same address space

Question

For reference, I'm running bare-metal QEMU-6.1.0 on aarch64 using the Xilinx fork.

I am loading a monolithic bare-metal binary into qemu-system-aarch64 using the generic loader device and am writing some peripheral drivers for it. The problem is that the binary is loaded in a separate address space from the one where custom devices are loaded.

Is there any way that I can specify the address space into which the device loader will load a bare-metal binary?

The method I used for loading the binary can be found in the Running a bare-metal application on Zynq Ultrascale+ MPSoC r5 section of the QEMU Xilinx docs.