'R parallel: Using `source` output as input in a function within the main function

Apologies in advance, I will use pseudo-code. I tried to create a reprex but it didn't show the problem.

I am using the function parallel::parSapply to run a model and it's throwing an "object not found" error, even though the object is defined within the function. Below I present the structure of my "main function" and how I'm running the parallel.

#Intializing data
zp2 = data.frame() #Contains unique identifier "ID" and data inputs
list.of.packages = c() #contains all required librariers
ode_fun = function() #function of ordinary differential equations
times = seq(0,348) #times to run the ODE function

#Main function
mfun = function(k){
     dd = zp2[k,]$ID
     source("Script.R", local=TRUE) #takes the inputs from 'zp2' to quantify: 'inits', and 'new.inputs'. Saved in the same working directory

     out = euler(y = inits, times, func= ode_fun, params=new.inputs)

     return(out) 
}

#Parallel
cores <- parallel::detectCores()
cl <- parallel::makeCluster(cores-1, type ="PSOCK")

parallel::clusterExport(cl=cl, ls(globalenv()))
parallel::clusterEvalQ(cl, sapply(c(list.of.packages),require, character.only=T))

res.input = NULL
tictoc::tic()
res.input <- parallel::parSapply(1:350, FUN=function(x)mfun(x),
                               cl = cl)
tictoc::toc()

parallel::stopCluster(cl)

This creates the error:

Error in checkForRemoteErrors(val) : 
  3 nodes produced errors; first error: object 'inits' not found

After debugging, I realized that source was in fact creating and 'bringing back' "inits" and "new.inputs" -- I ran the function up to there and made it return both objects. But for some reason ode_fun was not recognizing those objects so it creates an error.

As a workaround, I split the function: 1. collect all the inputs, 2. run the ODE, but this is not very efficient and potentially confusing. Do you know what could be the problem and if there's a solution for it?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source