Python APSCheduler throwing exception after removing job

Question

I am adding job in redis and on completion of job I have added an event handler. In eventhandler I am returning value based on which I am removing job id from jobstore. It is removed successfully but immediately it throws an exception.

Code

from datetime import datetime
from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.events import EVENT_JOB_EXECUTED
import logging
logging.basicConfig()

scheduler = BackgroundScheduler()
scheduler.add_jobstore('redis')
scheduler.start()

def tick():
    print('Tick! The time is: %s' % datetime.now())
    return 'success'

def removing_jobs(event):
    if event.retval == 'success':
        scheduler.remove_job(event.job_id)

scheduler.add_listener(removing_jobs, EVENT_JOB_EXECUTED)

try:
    count = 0 
    while True:
        count += 1
        time.sleep(10)
        job_ret = scheduler.add_job(tick, 'interval', id = str(count), seconds=10)
except (KeyboardInterrupt, SystemExit):
    scheduler.shutdown()

Exception

Exception in thread APScheduler:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/.virtualenvs/py3/lib/python3.5/site-packages/apscheduler/schedulers/blocking.py", line 30, in _main_loop
    wait_seconds = self._process_jobs()
  File "/.virtualenvs/py3/lib/python3.5/site-packages/apscheduler/schedulers/base.py", line 995, in _process_jobs
    jobstore.update_job(job)
  File "/.virtualenvs/py3/lib/python3.5/site-packages/apscheduler/jobstores/redis.py", line 91, in update_job
    raise JobLookupError(job.id)
apscheduler.jobstores.base.JobLookupError: 'No job by the id of 1 was found'

Answer 1

In short: you are removing the job while it is processed; so you should remove the job outside its execution.

That's because the scheduler doesn't know what the job's execution will do; so it launches tick and sends a job object to the redis jobstore thinking it will be executed again. Before that, the EVENT_JOB_LISTENER launches removing_jobs.

The problem is that when the redis' jobstore gets the job for update its status, it is already deleted so it raises the JobLookupError.