Crontab for Script in venv

Question

My python environment and the script I want to run are located at:

/home/pi/webscrap/bin
/home/pi/webscrap/news/news.py

In the news directory I created a newscrap.sh file containing this:

/home/pi/webscrap/bin/python /home/pi/webscrap/news/news.py

I did this just with nano newscrap.sh.

then I created a crontab and added the line:

* * * * * /home/pi/webscrap/news/newscrap.sh >> /home/mypc/logs/cronlogs.log 2>&1

I checked this by looking at the size of some files that should change when I run the script. This does not happen, so what am I missing here?

Edit: I also did

sudo chmod a+x news.py

Answer 1

Is the destination /home/mypc/logs/cronlogs.log on the raspberry pi or is it on a different system? In other words, if you type ls /home/mypc from the command line on the raspberry pi, does that folder exist?

I'm going to answer as though your intent is to copy files from one system (the raspberry pi) to another (mypc), but you may be confusing filesystem paths with network locations. If that's the case, then there are several good options available to you. I'll give you a quick summary of two.

Solution 1: Use scp
Write to a file on the raspberry pi.
Use scp to copy the file to a location on mypc.
You will want to set up SSH keys so that it does not prompt you for a password each time.
This won't append to a file on the mypc, it will copy the whole thing, so you will have to come up with a solution to that problem. This solution feels a little hacky to me, and I don't love it.

Solution 2: Use a queueing system
This is a lot more involved, but much cleaner than scp, because it was designed for exactly this. You'll have to learn about networking and sockets, but you like that sort of thing or you wouldn't be here. I like ZMQ because it is easy to get started and is well supported. You will have to write a server on the raspberry pi and a client on mypc. This can be a bit tricky if there are firewalls, but there is help for that. I would start with client/server on one machine, and then figure out how to get it across the network.