Skip to content

Instantly share code, notes, and snippets.

@air-drummer
Last active August 29, 2015 14:08
Show Gist options
  • Save air-drummer/d766e66d27c2332781e3 to your computer and use it in GitHub Desktop.
Save air-drummer/d766e66d27c2332781e3 to your computer and use it in GitHub Desktop.
Getting csv data.frame from an "https" source in R (e.g. GitHub)
# if you are OK if a data.frame is produced and you don't want the function to install anything:
read.csv.url <- function(url, method = "curl", ...) {
tmpFile <- tempfile()
download.file(url, destfile = tmpFile, method = method)
url.data <- read.csv(tmpFile, ...)
return(url.data)
}
# Example:
# file_url <- "https://raw.githubusercontent.com/egorufimtsev/econ808/master/women_wages.csv"
# d <- read.csv.url(file_url)
#
# #the same result:
# d <- read.csv.url(file_url, header = T, sep = ",")
#
# #2 if you want the function be able to also return a data.table:
# install.packages("data.table")
# library(data.table)
# read.csv.url <- function(url, method = "curl", out = "data.frame", ...) {
# tmpFile <- tempfile()
# download.file(url, destfile = tmpFile, method = method)
# if (out == "data.frame") {
# url.data <- read.csv(tmpFile, ...)
# }
# if (out == "data.table") {
# url.data <- fread(tmpFile, ...)
# }
# return(url.data)
# }
#
# file_url <- "https://raw.githubusercontent.com/egorufimtsev/econ808/master/women_wages.csv"
# d1 <- read.csv.url(file_url)
# d2 <- read.csv.url(file_url, out = "data.table")
#
# #the same result:
# d1 <- read.csv.url(file_url, header = T, sep = ",")
# d2 <- read.csv.url(file_url, out = "data.table", header = T, sep = ",")
#
#
# #3 this fails
# d3 <- read.csv(url(file_url), header = T, sep = ",")
#
# #4 this fails
# install.packages("RCurl")
# library(RCurl)
# tmp <- getURL(file_url)
# d4 <- read.csv(text = tmp, header = T, sep = ",")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment