Django – Set Up A Scheduled Job?
I've been working on a web app using Django, and I'm curious if there is a way to schedule a job to run periodically. Basically I just want to run through the database and make some calculations/updates on an automatic, regular basis, but I can't seem to find any documentation on doing this. Does anyone know how to set this up? To clarify: I know I can set up a cron job to do this, but I'm curious if there is some feature in Django that provides this functionality. I'd like people to be able to deploy this app themselves without having to do much config (preferably zero). I've considered triggering these actions "retroactively" by simply checking if a job should have been run since the last time a request was sent to the site, but I'm hoping for something a bit cleaner.
One solution that I have employed is to do this:
1) Create a custom management command, e.g.
python manage.py my_cool_command
cron (on Linux) or
at (on Windows) to run my command at the required times.
This is a simple solution that doesn’t require installing a heavy AMQP stack. However there are nice advantages to using something like Celery, mentioned in the other answers. In particular, with Celery it is nice to not have to spread your application logic out into crontab files. However the cron solution works quite nicely for a small to medium sized application and where you don’t want a lot of external dependencies.
In later version of windows the
at command is deprecated for Windows 8, Server 2012 and above. You can use
schtasks.exe for same use.
**** UPDATE ****
This the new link of django doc for writing the custom management command
Celery is pretty easy to set up with django (docs), and periodic tasks will actually skip missed tasks in case of a downtime. Celery also has built-in retry mechanisms, in case a task fails.
We’ve open-sourced what I think is a structured app. that Brian’s solution above alludes too. We would love any / all feedback!
It comes with one management command:
That does the job. Each cron is modeled as a class (so its all OO) and each cron runs at a different frequency and we make sure the same cron type doesn’t run in parallel (in case crons themselves take longer time to run than their frequency!)
If you’re using a standard POSIX OS, you use cron.
If you’re using Windows, you use at.
Write a Django management command to
Figure out what platform they’re on.
Either execute the appropriate “AT” command for your users, or update the crontab for your users.
Interesting new pluggable Django app: django-chronograph
You only have to add one cron entry which acts as a timer, and you have a very nice Django admin interface into the scripts to run.
Look at Django Poor Man’s Cron which is a Django app that makes use of spambots, search engine indexing robots and alike to run scheduled tasks in approximately regular intervals
Brian Neal’s suggestion of running management commands via cron works well, but if you’re looking for something a little more robust (yet not as elaborate as Celery) I’d look into a library like Kronos:
# app/cron.py import kronos @kronos.register('0 * * * *') def task(): pass
RabbitMQ and Celery have more features and task handling capabilities than Cron. If task failure isn’t an issue, and you think you will handle broken tasks in the next call, then Cron is sufficient.
Celery & AMQP will let you handle the broken task, and it will get executed again by another worker (Celery workers listen for the next task to work on), until the task’s
max_retries attribute is reached. You can even invoke tasks on failure, like logging the failure, or sending an email to the admin once the
max_retries has been reached.
And you can distribute Celery and AMQP servers when you need to scale your application.
It makes scheduling jobs super simple, and keeps it independent for from request-based execution of some code. Following is a simple example.
from apscheduler.schedulers.background import BackgroundScheduler scheduler = BackgroundScheduler() job = None def tick(): print('One tick!')\ def start_job(): global job job = scheduler.add_job(tick, 'interval', seconds=3600) try: scheduler.start() except: pass
Hope this helps somebody!
Although not part of Django, Airflow is a more recent project (as of 2016) that is useful for task management.
Airflow is a workflow automation and scheduling system that can be used to author and manage data pipelines. A web-based UI provides the developer with a range of options for managing and viewing these pipelines.
Airflow is written in Python and is built using Flask.
Airflow was created by Maxime Beauchemin at Airbnb and open sourced in the spring of 2015. It joined the Apache Software Foundation’s incubation program in the winter of 2016. Here is the Git project page and some addition background information.
Put the following at the top of your cron.py file:
#!/usr/bin/python import os, sys sys.path.append('/path/to/') # the parent directory of the project sys.path.append('/path/to/project') # these lines only needed if not on path os.environ['DJANGO_SETTINGS_MODULE'] = 'myproj.settings' # imports and code below
I just thought about this rather simple solution:
- Define a view function do_work(req, param) like you would with any other view, with URL mapping, return a HttpResponse and so on.
- Set up a cron job with your timing preferences (or using AT or Scheduled Tasks in Windows) which runs curl http://localhost/your/mapped/url?param=value.
You can add parameters but just adding parameters to the URL.
Tell me what you guys think.
[Update] I’m now using runjob command from django-extensions instead of curl.
My cron looks something like this:
@hourly python /path/to/project/manage.py runjobs hourly
… and so on for daily, monthly, etc’. You can also set it up to run a specific job.
I find it more managable and a cleaner. Doesn’t require mapping a URL to a view. Just define your job class and crontab and you’re set.
after the part of code,I can write anything just like my views.py 🙂
####################################### import os,sys sys.path.append('/home/administrator/development/store') os.environ['DJANGO_SETTINGS_MODULE']='store.settings' from django.core.management impor setup_environ from store import settings setup_environ(settings) #######################################
- Database Administration Tutorials
- Programming Tutorials & IT News
- Linux & DevOps World
- Entertainment & General News
- Games & eSport