Cron and virtualenv
I am trying to run a Django management command from cron. I am using virtualenv to keep my project sandboxed. I have seen examples here and elsewhere that show running management commands from within virtualenv's like: 0 3 * * * source /home/user/project/env/bin/activate && /home/user/project/manage.py command arg However, even though syslog shows an entry when the task should have started, this task never actually runs (the log file for the script is empty). If I run the line manually from the shell, it works as expected. The only way I can currently get the command to run via cron, is to break the commands up and put them in a dumb bash wrapper script: #!/bin/sh source /home/user/project/env/bin/activate cd /home/user/project/ ./manage.py command arg EDIT: ars came up with a working combination of commands: 0 3 * * * cd /home/user/project && /home/user/project/env/bin/python /home/user/project/manage.py command arg At least in my case, invoking the activate script for the virtualenv did nothing. This works, so on with the show.
You should be able to do this by using the
python in your virtual environment:
/home/my/virtual/bin/python /home/my/project/manage.py command arg
EDIT: If your django project isn’t in the PYTHONPATH, then you’ll need to switch to the right directory:
cd /home/my/project && /home/my/virtual/bin/python ...
You can also try to log the failure from cron:
cd /home/my/project && /home/my/virtual/bin/python /home/my/project/manage.py > /tmp/cronlog.txt 2>&1
Another thing to try is to make the same change in your
manage.py script at the very top:
source from a cronfile won’t work as cron uses
/bin/sh as its default shell, which doesn’t support
source. You need to set the SHELL environment variable to be
SHELL=/bin/bash */10 * * * * root source /path/to/virtualenv/bin/activate && /path/to/build/manage.py some_command > /dev/null
It’s tricky to spot why this fails as
/var/log/syslog doesn’t log the error details. Best to alias yourself to root so you get emailed with any cron errors. Simply add yourself to
/etc/aliases and run
the link above is changed to:
Don’t look any further:
0 3 * * * /usr/bin/env bash -c 'cd /home/user/project && source /home/user/project/env/bin/activate && ./manage.py command arg' > /dev/null 2>&1
* * * * * /usr/bin/env bash -c 'YOUR_COMMAND_HERE' > /dev/null 2>&1
The beauty about this is you DO NOT need to change the
SHELL variable for crontab from
Rather than mucking around with virtualenv-specific shebangs, just prepend
PATH onto the crontab.
From an activated virtualenv, run these three commands and python scripts should just work:
$ echo "PATH=$PATH" > myserver.cron $ crontab -l >> myserver.cron $ crontab myserver.cron
The crontab’s first line should now look like this:
PATH=/home/me/virtualenv/bin:/usr/bin:/bin: # [etc...]
The only correct way to run python cron jobs when using a virtualenv is to activate the environment and then execute the environment’s python to run your code.
One way to do this is use virtualenv’s
activate_this in your python script, see: http://virtualenv.readthedocs.org/en/latest/userguide.html#using-virtualenv-without-bin-python
Another solution is echoing the complete command including activating the environment and piping it into
/bin/bash. Consider this for your
***** root echo 'source /env/bin/activate; python /your/script' | /bin/bash
The best solution for me was to both
- use the python binary in the venv bin/ directory
- set the python path
to include the venv modules directory.
man python mentions modifying the path in shell at
$PYTHONPATH or in python with
Other answers mention ideas for doing this using the shell. From python, adding the following lines to my script allows me to successfully run it directly from cron.
import sys sys.path.insert(0,'/path/to/venv/lib/python3.3/site-packages');
Here’s how it looks in an interactive session —
Python 3.3.2+ (default, Feb 28 2014, 00:52:16) [GCC 4.8.1] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', '/usr/lib/python3.3', '/usr/lib/python3.3/plat-x86_64-linux-gnu', '/usr/lib/python3.3/lib-dynload'] >>> import requests Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named 'requests' >>> sys.path.insert(0,'/path/to/venv/modules/'); >>> import requests >>>
I’d like to add this because I spent some time solving the issue and did not find an answer here for combination of variables usage in cron and virtualenv. So maybe it’ll help someone.
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin DIR_SMTH="cd /smth" VENV=". venv/bin/activate" CMD="some_python_bin do_something" # m h dom mon dow command 0 * * * * $DIR_SMTH && $VENV && $CMD -k2 some_target >> /tmp/crontest.log 2>&1
It did not work well when it was configured like
DIR_SMTH=”cd /smth && . venv/bin/activate”
Thanks @davidwinterbottom, @reed-sandberg and @mkb for giving the right direction. The accepted answer actually works fine until your python need to run a script which have to run another python binary from venv/bin directory.
- Database Administration Tutorials
- Programming Tutorials & IT News
- Linux & DevOps World
- Entertainment & General News
- Games & eSport