How to share memory region allocated by mmap() with rights, with specific process only on Linux

Question

Scenario:

On Linux, Process A (privileged) maps physical memory region M to its virtual address space. A wants to share M with Process B (unprivileged) only (not even by the same user's other process). To accomplish this, A has to somehow pass M to B, allowing B to allocate M to B's virtual address space.

M has to be physically mapped to not-anonymous (= specific) physical address range.

How can I accomplish above scenario?

Use case:

Process A (privileged) as the userspace GPU memory manager allocates the GPU's PCIe BAR resource memory region (= M) to its virtual address space. A will distribute exclusive GPU memory chunks to each unprivileged process (= B). Each memory chunk will not be visible to other processes, even the ones from the same user, same as normal memory pages.