'How can I scrape older comments from a subreddit?

I am trying to scrape all the comments from r/europeanunion between 2010 and 2022 using the RedditExtractoR package in R. First step to do this is to extract all the URLs of the threads in this subreddit.

The code below works fine for extracting the URLs of the threads but it only brings the latest 950 threads, which only go back until January 2022.

library(RedditExtractoR)

# Extracting the thread urls from r/europeanunion 
urls <- find_thread_urls(sort_by = "new", period = "all", subreddit = "europeanunion")

I understand that there is perhaps a result limit of around 1000 for any kind of query from Reddit but is it possible to get around this limit by specifying a time period somewhere and running several queries?

r web-scraping reddit

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source

'How can I scrape older comments from a subreddit?

Sources

Related Questions