'Start a new process and share an object to it (shared memory)

Can a process #1 start a process #2 with Popen, and pass to it a reference to a dict D, so that process #2 can read its content?

# process1.py
import subprocess, time
D = {'foo': 'bar'}
subprocess.Popen(['python3', 'process2.py', str(id(D))])
time.sleep(3600)

# process2.py
import sys
ref = sys.argv[1]  # can we turn this address into an (at least read-only) object?
                   # and read foo/bar?

If not, I know sockets (and more generally messaging techniques involving networking), /dev/shm on Linux, and these IPC techniques to make 2 different processes communicate, but are there even simpler solutions by just sharing an in-memory object?
I guess, for security reasons, the processes should be started with a special option to authorize its memory to be shared with other processes.



Solution 1:[1]

After further research, it seems mmap is perfect for this (but note that it will not share an object but just bytes. pickle can be used for the serialization though).

To map anonymous memory, -1 should be passed as the fileno along with the length.

tagname, if specified and not None, is a string giving a tag name for the mapping. Windows allows you to have many different mappings against the same file. If you specify the name of an existing tag, that tag is opened, otherwise a new tag of this name is created.

Thus, this is a very simple shared-memory inter process communication without using a socket:

Server:

import mmap, time
mm = mmap.mmap(-1, 20, tagname="foo")
while True:
    mm.seek(0)
    mm.write(str(time.time()).encode())
    mm.flush()
    time.sleep(1)

Client:

import mmap, time
mm = mmap.mmap(-1, 20, tagname="foo")
while True:
    mm.seek(0)
    buf = mm.read(128)
    print(buf)
    time.sleep(1)

Note: this is only for Windows:

Avoiding the use of the tag parameter will assist in keeping your code portable between Unix and Windows.

Another example with video data: https://github.com/off99555/python-mmap-ipc

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Basj