Caching and CDN in Divio projects

Infrastructure-level caching

As well as the caching and CDN (Content Delivery Network) we provide, you can also apply your own.

Caching is generally provided by Cloudflare.

Caching for media storage is available on all projects and plans, and on Test as well as Live servers. It’s automatic and requires no additional action or configuration. Note that this only includes files served from our S3 media buckets (typically, images). Resources served by the application instances (HTML, static files) are not cached.

Caching is URL-based, and wholly content-unaware - caching does not detect when files have changed.

Options

However, other options for caching are available on request (on eligible projects only). This includes:

Media file caching

Typically, the bulk of a page web’s transfer load is accounted for by its media files, most of which will be images in the page.

All media files are handled by our dedicated storage and hosting providers, using S3 buckets. If your application uses the aldryn-media.io domain for S3 buckets, then the files are not cached.

Delivery of these files is handled by Cloudflare’s CDN, which also caches the files.

Controlling caching headers

Cloudflare will cache media files according to the Cache-Control header applied to the files. For example:

cache-control: max-age=3600

will set a TTL of one hour (3600 seconds).

Your application can set these headers when managing the file storage. Aldryn Django does this by default, also applying some sensible default values (see also DISABLE_S3_MEDIA_HEADERS_UPDATE).

Our Aldryn Django Filer addon applies a one-year TTL to its public thumbnail files using the MEDIA_HEADERS setting. In turn, our Aldryn Django addon applies the MEDIA_HEADERS values it discovers to the media storage class that configures the S3 bucket.

Any application that needs to control the behaviour of cached media will need either to make use of provided functionality (for example, such as in Aldryn Django), or configure the S3 bucket directly itself.

Applying your own caching/CDN

There is nothing to stop you applying your own caching and CDN.

For media files, if you are using our media domain (the default), your projects will automatically use the CDN we provide and this cannot be changed. However, it’s also possible to use your own domain (see for example How to configure media serving on a custom domain) for media, in which case you are free to use what you wish.

For the rest of the project, you can set up another CDN, for example using your own Cloudflare account. In such a case you should inform us so that instead of providing a certificate automatically ourselves, you can upload your own manually.

Application-level caching

Django applications

Caching in Divio Django applications will typically make use of Django’s own caching framework.

Caching in django CMS

The Aldryn django CMS addon applies caching rules by default, via the CMS_CACHE_DURATIONS setting.

Control over caching settings is exposed in the Divio Control Panel in the configuration options for Aldryn django CMS. Defaults are to cache content for 60 seconds and menus for one hour.

It is often convenient to disable caching while developing or working intensively on content. Adding:

import os
env = os.getenv
STAGE = env('STAGE', 'local').lower()
if STAGE in {'local', 'test'}:
    CMS_PAGE_CACHE = False
    CMS_PLACEHOLDER_CACHE = False
    CMS_CACHE_DURATIONS = {
        'menus': 0,
        'content': 0,
        'permissions': 0,
    }

to the project’s settings.py will disable all caching in the CMS in the local and Test environments.

Application caching options

What not to use in your code

Caching should rely on a shared store that persists for all containers. For example, caching that relies on a container’s local file-system or local memory should not be used, as only that container (and not a container running in parallel, or one instantiated later) will be able to access the items it stores.

In some cases, this can simply lead to inefficiency (not using cached data). In other cases, it could cause malfunction or even data-loss, if two instances are working with inconsistent data.

Database caching

Database caching is shared by all instances of an application server, making database caching suitable for many use-cases.

It’s our default cache backend for Django projects - all Divio Django projects are set up with Django’s database caching configured and ready to use.

This is a fast, scalable option, and is suited to most needs.

Third-party caching backends

Other backends, such as Redis (a popular open-source database) can be used as caching backends for Django.

If it suits your needs, you can procure a Redis or other caching instance from a provider and use it with your Divio project.