'Transfer large objects through `socketConnection` in R fails on macOS
I am trying to transfer a large object from a session to another using a socketConnection. With the following I can establish the connection between the two session and see that it works for small messages:
socket <- serverSocket(11927)
r <- callr::r_session$new()
r$call(function() {
assign(
"conection",
socketConnection(port = 11927, open = "wb", blocking = TRUE, server = FALSE),
envir = .GlobalEnv
)
NULL
})
con <- socketAccept(socket, open = "wb", blocking = TRUE)
close(socket)
r$read()
serialize("hello world", con)
r$run(function() {
serialize(paste("hello from there: ", unserialize(conection)), conection)
})
unserialize(con)
Now if I try to serialize a large value, for example:
r$run(function() {
x <- runif(256*256*3)
serialize(x, conection)
TRUE
})
The serialization never finishes. It's worth noting that this works as expected on Linux. I didn't try on Windows.
I think this should work correctly because, using the parallel package, that also uses socket connections to transfer objects everything works as expected and I can transfer large objects pretty quickly. For example:
cl <- parallel::makePSOCKcluster(1)
parallel::clusterEvalQ(cl, {
get_batch <- function() runif(256*256*3)
})
out <- parallel::clusterCall(cl, "get_batch")
Any idea on what could be causing this behavior on macOS?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
