'Add Scrapy data to csv without the header row
We have a local website that tracks the number of people using a certain license. I have create a scraper with that should run every hour. The only issue I have it's creating data that looks like this.
active_users,date,time
35,22/03/2022,11:38:30.397745
active_users,date,time
36,22/03/2022,11:44:04.753589
the issue I find is that every time scrapy crawl users is ran it adds that header. I know scrapy has CsvItemExporter() that can remove the header but I'm not too sure how to use it.
I just need the output csv to look like
active_users,date,time
35,22/03/2022,11:38:30.397745
36,22/03/2022,11:44:04.753589
Solution 1:[1]
If you are using scrapy version 2.4 and above you can directly change this setting when defining the FEED such as below.
custom_settings = {"FEEDS": { "items.csv": {"format": "csv", "item_export_kwargs": {"include_headers_line": False}}}}
This will append to the contents of the file instead of adding a new header each time.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | msenior_ |