looping through and merging files in folder in R

Question

I am trying to loop through hundreds of weather data files (.nc) and merge them together.

I can load them, and merge them manually using:

library(raster)
library(ncdf4) 
library(ncdf4.helpers)
require(data.table)

#define input paths, load data, then merge

baseline_path_file <- "E:/input_data/HadOBS/tas/tas_hadukgrid_uk_1km_mon_202001-202012.nc"
baseline_path_file2 <- "E:/input_data/HadOBS/tas/tas_hadukgrid_uk_1km_mon_201901-201912.nc"

BASELINE <- setDT(as.data.frame(brick(baseline_path_file), xy = T))
BASELINE2 <- setDT(as.data.frame(brick(baseline_path_file2), xy = T))

combined <- merge(BASELINE, BASELINE2, by = c("x","y"))

but what I would like to do is define the list of files in a folder and merge them manually.

e.g.

library(fs)
files <- dir_ls("E:/input_data/HadOBS/tas") 


combined2 <- map(files, brick) %>% 
              as.data.frame %>%
              setDT %>%
  reduce(inner_join, by = c("x", "y"))

but that obviously isn't working... I can't seem to get the piping in the right order. Any ideas how to get this right? Many thanks indeed.

Answer 1

The problem appears to be that you are only using map() to apply brick to your list elements, and not also as.data.frame() and setDT().

Lacking the data, I didn't run this code, so it might not work, but you get the idea:

combined2 <- map(files, 
                 ~ .x %>% 
                   brick() %>%  
                   as.data.frame() %>%
                   setDT()
                 ) %>%
  reduce(inner_join, by = c("x", "y"))