'R parallel: Using `source` output as input in a function within the main function
Apologies in advance, I will use pseudo-code. I tried to create a reprex but it didn't show the problem.
I am using the function parallel::parSapply to run a model and it's throwing an "object not found" error, even though the object is defined within the function. Below I present the structure of my "main function" and how I'm running the parallel.
#Intializing data
zp2 = data.frame() #Contains unique identifier "ID" and data inputs
list.of.packages = c() #contains all required librariers
ode_fun = function() #function of ordinary differential equations
times = seq(0,348) #times to run the ODE function
#Main function
mfun = function(k){
dd = zp2[k,]$ID
source("Script.R", local=TRUE) #takes the inputs from 'zp2' to quantify: 'inits', and 'new.inputs'. Saved in the same working directory
out = euler(y = inits, times, func= ode_fun, params=new.inputs)
return(out)
}
#Parallel
cores <- parallel::detectCores()
cl <- parallel::makeCluster(cores-1, type ="PSOCK")
parallel::clusterExport(cl=cl, ls(globalenv()))
parallel::clusterEvalQ(cl, sapply(c(list.of.packages),require, character.only=T))
res.input = NULL
tictoc::tic()
res.input <- parallel::parSapply(1:350, FUN=function(x)mfun(x),
cl = cl)
tictoc::toc()
parallel::stopCluster(cl)
This creates the error:
Error in checkForRemoteErrors(val) :
3 nodes produced errors; first error: object 'inits' not found
After debugging, I realized that source was in fact creating and 'bringing back' "inits" and "new.inputs" -- I ran the function up to there and made it return both objects. But for some reason ode_fun was not recognizing those objects so it creates an error.
As a workaround, I split the function: 1. collect all the inputs, 2. run the ODE, but this is not very efficient and potentially confusing. Do you know what could be the problem and if there's a solution for it?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
