'Run Luigi scheduler every 15 minutes without using cron
i have to complete a task to run luigi wrapper without using cron since it is linux based. In this case i tried to use apschedule library. This is my test script:
from apscheduler.schedulers.blocking import BlockingScheduler
import luigi
import time
import pickle
sched = BlockingScheduler()
class HelloLuigi(luigi.Task):
def output(self):
return luigi.LocalTarget(path)
@sched.scheduled_job('interval', minutes=15)
def run(self):
# Read as bytes
with open(path, 'rb') as f:
data = f.read()
with open(self.output().path, "wb") as outfile:
pickle.dump(data, outfile)
sched.start()
how can l implement apscheduler or any other method that can make luigi run at a specific time? The command to run luigi from the terminal:
python3 -m luigi --module filename HelloLuigi --local-scheduler
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
