How can I get R to modify this matrix in place after a call to fastLmPure?

Question

I'm trying to run fastLmPure in a loop, avoiding copying a large matrix.

I've pre-allocated the matrix to the size I need, and it's only the last column I need to change values in. i.e. it doesn't grow.

I've boiled it down to the minimal cases below, which illustrate the problem I'm having.

I expect the modification of matrix to be done in-place, but instead I get a copy because ref counter has been incremented by the call out to Rcpp.

Rcpp doesn't modify X, so why does an extra refcount hang around, which causes R to make a copy when I next modify?
How can I get a modify-in-place after execution returns from fastLm?

(I'm using R 4.1.2, and running via console to avoid any RStudio env pane reference issues.)

Example 1:

library(RcppArmadillo)
minimal_fastLm <- function() {
  y <- c(1,2)

  X <- matrix(data = c(1,1,2,3), nrow = 2, ncol = 2)
  .Internal(inspect(X))

  model <- .Call("_RcppArmadillo_fastLm_impl", X, y, PACKAGE = "RcppArmadillo")
  .Internal(inspect(X))

  X[, 2] <- c(3,4)
  .Internal(inspect(X))
}

minimal_fastLm()

Gives output (edited for clarity):
Address unchanged through call to fastLm, but ref count increased, then copy made on modify. (See bold). @0x0000000017401560 14 REALSXP g0c3 [REF(1),ATT] (len=4, tl=0) 1,1,2,3 ... @0x0000000017401560 14 REALSXP g0c3 [REF(2),ATT] (len=4, tl=0) 1,1,2,3 ... @0x00000000174ad568 14 REALSXP g0c3 [REF(1),ATT] (len=4, tl=0) 1,1,3,4 ...

Example 2: Even more minimal (pure Rcpp)

library(Rcpp)
cppFunction('int rawSEXP(SEXP X) { return(1); }')
cppFunction('int asNumMat(NumericMatrix X) { return(1); }')

pureRcpp <- function() {
  X <- matrix(data = c(1,1,2,3), nrow = 2, ncol = 2)
  .Internal(inspect(X))

  X[, 2] <- c(4,5)    # Initially modifies in place because X has REF(1).
  .Internal(inspect(X))

  rawSEXP(X)          # Call Rcpp function with raw SEXP
  .Internal(inspect(X))

  X[, 2] <- c(6,7)    # Still modifies in place because X has REF(1).
  .Internal(inspect(X))

  asNumMat(X)         # Call Rcpp function that casts to NumericMatrix
  .Internal(inspect(X))

  X[, 2] <- c(8,9)    # Causes a copy because X has REF(3).
  .Internal(inspect(X))

}
pureRcpp()

X was never modified by Rcpp. Nor do I want it to be. I just want to read data.
So why was ref counter not decremented back to 1 when NumericMatrix went out of scope?

Answer 1

R uses copy-on-write and that doesn't change between the write coming from R or C(++) code -- an updated value is just an updated value.

So you if want mutable non-copied data you may have to manage the data yourself in a non R data structure. A simple Armadillo vector or matrix where you take care not to inherit from R data should do.

Allocate that once, modify values as you please, loop to your heart's content. That should suit your requirement, unless I misunderstood something.

Answer 2

After some digging, I found that there's some cleaning up that Rcpp could be doing, but isn't.

Raised as a bug, with solution identified. Will be fixed in the next release of Rcpp. https://github.com/RcppCore/Rcpp/pull/1205

So, the answer is:

• This modify-in-place should indeed be possible,
• And it will be after a future release of Rcpp.