'Logging Type Error In Dask Compute Method
Here is the example python code, called dask_multithread_demo.py
import dask.bag as db
import logging
log = logging.getLogger(__name__)
def dask_function(input_list):
db.from_sequence(input_list) \
.map(lambda x: log.info(f"showing x: {x}")) \
.compute()
def main():
dask.config.set(scheduler='single-threaded')
dask_function(["abc"])
dask.config.set(scheduler='multiprocessing')
dask_function(["abc"])
When I run the main method and do the first call of dask_function, I do not get any Exceptions
When I get to the second call of dask_function, I get the following exception
File "dask_multithread_demo.py", line 18, in main
dask_function(["abc"])
File "dask_multithread_demo.py", line 10, in dask_function
.map(lambda x: log.info(f"showing x: {x}")) \
File "lib/python3.6/site-packages/dask/base.py", line 166, in compute
(result,) = compute(self, traverse=False, **kwargs)
File "lib/python3.6/site-packages/dask/base.py", line 437, in compute
results = schedule(dsk, keys, **kwargs)
File "lib/python3.6/site-packages/dask/multiprocessing.py", line 222, in get
**kwargs
File "lib/python3.6/site-packages/dask/local.py", line 489, in get_async
raise_exception(exc, tb)
File "lib/python3.6/site-packages/dask/local.py", line 318, in reraise
raise exc.with_traceback(tb)
File "lib/python3.6/site-packages/dask/local.py", line 224, in execute_task
task, data = loads(task_info)
TypeError: get_logger() missing 1 required positional argument: 'name'
get_logger() missing 1 required positional argument: 'name'
Here is my question: How do I do the logging in dask_function with scheduler set to "multiprocessing" without the Type Error Exception happening?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
