'Problem in R: package reticulate does not work with doparallel

I recently bumped into a new problem with reticulate and doparallel that I never seen before.

The following code works well with %do% (which is regular for loop) BUT NOT with %dopar% (doParallel package)

library(doParallel)
library(reticulate)

cl <- makeCluster(4)
registerDoParallel(cl)

foreach(j=1:4,.combine=rbind,.packages = "reticulate") %dopar%
{
test <- "Testing to see if dopar can recognize reticulate"
py_run_string("for i in range(len(r.test)): print(i)")
}

Here is the error message:

"Error in { : 
  task 1 failed - "RuntimeError: Evaluation error: object 'test' not found."

By default, test is an R object and when using reticulate, r.test will be recognized as an object in python py_run_string. However, the dopar created different cluster and therefore the r.test is not recognized anymore. The strange thing in here is that I have test ran inside the foreach loop and reticulate loaded for each small cluster.

Any hint would help. Thanks in advance.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source