'Why does multiprocessing raise `AttributeError` for monkey patching?
I'm confused about monkey patching in multiprocessing.Pool. The patching worked before pooling, but the method lost in the pool. I know it could be fixed by overriding or modifying TestClass.test = func to TestClass.func = func
from multiprocessing import Pool
class TestClass:
def __init__(self) -> None:
pass
def func(self, a):
print(a)
# monkey patching
TestClass.test = func
print('test monkey')
TestClass().test(True)
pool = Pool()
for i in range(2):
temp = TestClass()
print(f"monkey before pool has test [{bool(hasattr(temp, 'test'))}]")
pool.apply_async(temp.test, args=(i,))
pool.close()
pool.join()
The Traceback told me func lost?
❯ python monky_patch_multiprocessing.py
test monkey
True
monkey before pool has test [True]
monkey before pool has test [True]
Process ForkPoolWorker-1:
Process ForkPoolWorker-2:
Traceback (most recent call last):
File "/path/to/miniconda3/envs/seismic/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/path/to/miniconda3/envs/seismic/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/path/to/miniconda3/envs/seismic/lib/python3.10/multiprocessing/pool.py", line 114, in worker
task = get()
File "/path/to/miniconda3/envs/seismic/lib/python3.10/multiprocessing/queues.py", line 368, in get
return _ForkingPickler.loads(res)
AttributeError: 'TestClass' object has no attribute 'func'
Traceback (most recent call last):
File "/path/to/miniconda3/envs/seismic/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/path/to/miniconda3/envs/seismic/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/path/to/miniconda3/envs/seismic/lib/python3.10/multiprocessing/pool.py", line 114, in worker
task = get()
File "/path/to/miniconda3/envs/seismic/lib/python3.10/multiprocessing/queues.py", line 368, in get
return _ForkingPickler.loads(res)
AttributeError: 'TestClass' object has no attribute 'func'
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
