Scrapy - Unable to save the scraped data outside the project location

Question

I am a newbie to Python-Scrapy. I am unable to figure out how to save the scraped output file outside the project (Project is in Eclipse).

My Scrapy project structure is:

This is how I am running my scrapy project:

1. Changing "command line prompt" to project location folder i.e.,

cd D:/Y/Forum

2. Scrapy crawl Forum -o output.json

This is saving my scraped output file into the project location.

But, I want to save that output file in a user specified location, at run time, (which is not local to the project location)

Below is the command to save the output file in a different location,

Scrapy crawl Forum -o D:/output.json

But it is not working. My output file is still being saved in the project.

Can someone please help me achieve this.

Answer 1

Sorry, that I'm posting answer instead of comment - commenting requires more than 50 reputation.

This works just fine for me in Ubuntu (the actual spider is in very different directory and this path isn't in my project):

scrapy crawl spider_name -o /folder/file_name.csv

So I think there is something wrong with Windows path format in your case. Try D:\some_path or some slash escapers and so on.

Answer 2

It will give solution #for Windows10 and static folders

#settings.py

file = '\\' + 'D:\Test1\Test2' + '\\' + 'name_list.csv' #for Windows10 and static folders

FEEDS = {file: {'format': 'csv', 'fields': ['image_urls', 'image_names'],}}

output:
2022-05-05 13:10:12 [scrapy.extensions.feedexport] INFO: Stored csv feed (10 items) in: \D:\Test1\Test2\name_list.csv