'Logging Type Error In Dask Compute Method

Here is the example python code, called dask_multithread_demo.py

import dask.bag as db
import logging

log = logging.getLogger(__name__)


def dask_function(input_list):
    db.from_sequence(input_list) \
        .map(lambda x: log.info(f"showing x: {x}")) \
        .compute()


def main():
    dask.config.set(scheduler='single-threaded')
    dask_function(["abc"])
    dask.config.set(scheduler='multiprocessing')
    dask_function(["abc"])

When I run the main method and do the first call of dask_function, I do not get any Exceptions

When I get to the second call of dask_function, I get the following exception

  File "dask_multithread_demo.py", line 18, in main
    dask_function(["abc"])
  File "dask_multithread_demo.py", line 10, in dask_function
    .map(lambda x: log.info(f"showing x: {x}")) \
  File "lib/python3.6/site-packages/dask/base.py", line 166, in compute
    (result,) = compute(self, traverse=False, **kwargs)
  File "lib/python3.6/site-packages/dask/base.py", line 437, in compute
    results = schedule(dsk, keys, **kwargs)
  File "lib/python3.6/site-packages/dask/multiprocessing.py", line 222, in get
    **kwargs
  File "lib/python3.6/site-packages/dask/local.py", line 489, in get_async
    raise_exception(exc, tb)
  File "lib/python3.6/site-packages/dask/local.py", line 318, in reraise
    raise exc.with_traceback(tb)
  File "lib/python3.6/site-packages/dask/local.py", line 224, in execute_task
    task, data = loads(task_info)
TypeError: get_logger() missing 1 required positional argument: 'name'
get_logger() missing 1 required positional argument: 'name'

Here is my question: How do I do the logging in dask_function with scheduler set to "multiprocessing" without the Type Error Exception happening?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source