'Rstudio open large csv (sep = '\u0007') in could not allocate memory and wrong format
I have a large csv file (14GB, 3.6M rows) and its delimiter is (sep = '\u0007').
df <- read.csv("test.csv", header = TRUE, sep='\u0007')
Error in scan(file = file, what = what, sep = sep, quote = quote, dec = dec, :
could not allocate memory (2048 Mb) in C function 'R_AllocStringBuffer'
I've already added "R_MAX_VSIZE=150Gb" in the .Renviron file. Can someone help to check?
And if I removed the sep='\u0007', the data can be loaded but the formate is wrong.
My laptop details:
macOS Catalina
Processor: 2.9 GHz 6-Core Intel Core i9
Memory: 32 GB 2400 MHz DDR4
—Memory Used: 15.13GB, Cached files: 2.92GB, Swap Used: 1.5 MB
Macintosh HD: 270.08GB available of 499.96GB
CPU: idle 94.72%
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
