How to configure Celery¶
This article assumes that you are already familiar with the basics of using Celery with Django. If not, please see Celery’s documentation.
Request Celery for your project¶
Celery needs to be set up by our infrastructure team. See Celery on Divio Cloud for more information.
Your project will be set up with two new environment variables:
Test and Live servers will have different values for both.
Configure Celery in Cloud projects¶
Aldryn Celery’s aldryn_config.py file contains a number of settings that can be configured as environment variables.
Configure Celery for the local server¶
Set up Docker services¶
You will need to add some new services to your docker-compose.yml file.
Set up the RabbitMQ messaging service:
rabbitmq: image: rabbitmq:3.5-management hostname: rabbitmq ports: - "15672:15672" expose: - "15672" environment: RABBITMQ_ERLANG_COOKIE: <secret cookie value>
Set up the Celery worker service, with a
command option to start the
celeryworker: command: aldryn-celery worker
You will also need to add all of the other options already applied to the
web service in the same file, except the
links option, add:
In other words, the
celery service should now include something like:
celeryworker: command: aldryn-celery worker build: . links: - "db:postgres" - "rabbitmq:rabbitmq" volumes: - ".:/app:rw" - "./ data:/data:rw" env_file: .env-local
These are required because the
celeryworker needs access to the same Python
components and environment as the web applications.
The Celery beat scheduling service needs to be set up in much the same way.
celerybeat: command: aldryn-celery beat
Copy over the other options from
cam service captures events for monitoring purposes.
celerycam: command: aldryn-celery cam
followed once again by the other options from
Finally, to the
links option in
Set up local environment variables¶
RABBITMQ_ERLANG_COOKIE=<secret cookie value> BROKER_URL="amqp://guest:guest@rabbitmq:5672/"
Running the local project¶
docker-compose up or
divio project up will now also start Celery.
Environment variable changes¶
If you change environment variables locally, the containers will need to be stopped and restarted in order to pick up the changes.
When making project configuration changes, you will need to rebuild all the containers:
web container, Celery’s containers will not be reloaded on
Python code changes, so you will need to run
celeryworker manually when required (the other containers shouldn’t generally
need to be restarted).