Not Enough Columns Slots

Question

When try to create 12 lags for each variable in a data.table (108 variables) i get an error that says not enough column slots. This operation should create approx 1200 variables or columns.

Data_A = as.data.table (Datos_A)
Varnames = names(Datos_A)
Lagnumber= seq_len(12)

for(y in Varnames) { 
  for (z in Lagnumber)  set(Datos_GO, j = eval(paste0(y,"_lag_",z)), value = shift(Data_A [[y]],n = z, type = "lag"))
}

Error in set(Data_A, j = eval(paste0(y, "lag", z)), value = shift(Datos_A[[y]], : Internal logical error. DT passed to assign has not been allocated enough column slots. l=1132, tl=1132, adding 1

I tried to use the following, but it didn´t work.

alloc.col(Data_A,3500)

It displays now on the GLobal Enviroment that Data_A has 3132 variables, but when using the following code, I´ve got the same error

for(y in Varnames) {
  for (z in Lagnumber) set(Datos_GO, j = eval(paste0(y,"_lag_",z)), value = shift(Data_A [[y]],n = z, type = "lag"))
}

Error in set(Datos_GO, j = eval(paste0(y, "lag", z)), value = shift(Datos_GO[[y]], : Internal logical error. DT passed to assign has not been allocated enough column slots. l=6632, tl=6632, adding 1

That code worked properly with just about 80 variables in the original data.table but I don´t understand why this do not work with 30 more.

Sample data: https://ufile.io/3ka6g

ERROR and CODE on SAMPLE DATA

library(readxl)
> Data_A <- read_excel("C:/Example_data_source_108variables.xlsx")
> library(data.table)
data.table 1.10.4.3
  The fastest way to learn (by data.table authors): https://www.datacamp.com/courses/data-analysis-the-data-table-way
  Documentation: ?data.table, example(data.table) and browseVignettes("data.table")
  Release notes, videos and slides: http://r-datatable.com
> Data_A = as.data.table (Data_A)
> Varnames = names(Data_A)
> Lagnumber= seq_len(12)
> for(y in Varnames){for (z in Lagnumber) set(Data_A, j = eval(paste0(y,"_lag_",z)), value = shift(Data_A [[y]],n = z, type = "lag"))}
Error in set(Data_A, j = eval(paste0(y, "_lag_", z)), value = shift(Data_A[[y]],  : 
  Internal logical error. DT passed to assign has not been allocated enough column slots. l=1132, tl=1132, adding 1

Answer 1

Following the example here, this works for me:

Data_A = as.data.table (Datos_A)
Varnames = names(Datos_A)
Lagnumber= seq_len(12)

for(y in Varnames) { 
  for (z in Lagnumber){
    Datos_A[, c(paste0(y,"_lag_",z)):= shift(Data_A [[y]],n = z, type = "lag")]
  }
}