Downloading data from using the skynet package in R

Question

I am trying to download all fields from the flight data from transtats.bts.gov using the skynet package in R for January 2019

First I created test data for the program selecting the files manually using the link

https://www.transtats.bts.gov/DL_SelectFields.asp?gnoyr_VQ=FGJ&QO_fu146_anzr=b0-gvzr

The resulting file this generates is https://transtats.bts.gov/PREZIP/On_Time_Reporting_Carrier_On_Time_Performance_1987_present_2019_2.zip

When I am using the skynet package from CRAN with the function download_ontime(2019,1) it only downloads a subset of the fields.

I would prefer to use the skynet package or a different package if this is not possible for downloading the flight data because it is likely that the download link will be fixed in the package in case it changes on the web page and it might crash the code later otherwise.

Downloading with skynet (not all fields included) works as follows:

require("skynet")

download_ontime(2019, 2, auto = TRUE)

I am not 100% sure if it downloads all fields or not because I cannot see where it stores the file.

• Check that all fields are downloaded
• Adjustable download directory

Answer 1

Without using skynet package we can download and import the file from above url by,

url = 'https://transtats.bts.gov/PREZIP/On_Time_Reporting_Carrier_On_Time_Performance_1987_present_2019_2.zip'

temp <- tempfile()
#download the file
download.file(url, temp)

#unzip in your working directory 
unzip(temp)
files <- list.files(pattern = ".csv", full.names = T)

#as its a large csv file we shall read it using fread
library(data.table)
df = read.csv(files)

#dimension
dim(df)
[1] 533175    110