How do I kill and restart a python script at regular intervals? Example python script included in description

Question

Here is the example:

from concurrent.futures import ProcessPoolExecutor
import time


elapsed = 0
start_time = time.time()
func_args = [i for i in range(100)]


def func(x):
    return x*x


if __name__ == '__main__':
    while elapsed < 600:
        with ProcessPoolExecutor() as executor:
            for item in executor.map(
                            func,
                            func_args
                            ):
                print(item)
        elapsed = time.time() - start_time

How can I kill and restart this script at regular intervals of 5 mins? I figure something using shell is possible but not sure how it can work when using parallel processes like in this script.

If you're curious why I want to kill and restart this script every 5 mins: In my actual/production code, func() is a function that leaks memory. It takes about 30 mins for it to cause any serious issues and I want to kill and restart the entire script before that. I'm also trying to resolve the memory leak so this is a temporary solution.

Answer 1

You could do this through crontab (see also man crontab). The script will be a simple bash script like e.g. the following:

#!/bin/bash

# kill running script
ps ax | grep bot.py | grep -v grep | awk '{print $1}' | xargs kill -9

# restart
/path/to/your_script.py & disown

The crontab entry (edit with crontab -e) should look like this:

*/5 * * * * /path/to/bash_script.sh

A much easier solution however IMHO would be to just set a hard resource limit with ulimit (see help ulimit in a bash shell) on the memory that can be used by the process, so that it will be killed whenever it exceeds the limit. Then, simply call the script in a loop with a bash script:

#!/bin/bash

# limit virtual memory to 500MB
ulimit -Sv 500000 -Hv 500000

while true; do
    /path/to/your_script.py
    sleep 1
done