'How can I hydrate Tweet Ids from a directory of many .txt files using twarc and saving them as csv files (python)
I have been using the DocNow hydrator (https://github.com/DocNow/hydrator) to hydrate tweets from txt files containing Tweet IDs. Now, I have 100s of txt files in a folder and using the hydrator app is quite troublesome since I would have to do this file by file. Instead I want to write a python script to loop through all of the txt files, hydrate them and save each of them to a csv file.
Making one huge txt file would not be feasible since I am dealing with over a billion tweets, and I will not be able to open the resulting csv file with excel (although there might be methods I am unaware of) since it is too big.
Could someone help me write such a script using twarc (https://github.com/DocNow/twarc)? I am quite new to python, and am not sure how to do this. Thank you very much in advance!
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|