Add and configure media storage

Each cloud environment on Divio can have its own media storage running on a service such as S3 or MS Azure, and is provided with a DEFAULT_STORAGE_DSN variable containing details of how and where files are stored, and the URL from which they can be retrieved. Django can be configured to access the media storage of each environment by using the DEFAULT_STORAGE_DSN to determine Django's DEFAULT_FILE_STORAGE setting.

Create the media storage

For the cloud environments

In the same way that you did for the database earlier, in the application's Services view, add an Object storage instance. This will provide S3 storage for the application. Deploy the environment, or manually select Provision from the services options menu. The service will be provisioned for the application and the environment variable will be applied.

Locally

Locally, the most convenient way to work with media files is using local storage. You could use any directory in your local application, but since the Divio CLI will expect to find media files in /data, we will use that. We will configure the local set-up so that media files can be:

• stored in /data/media
• retrieved via the URL path /media

We can configure the storage system for this with the DEFAULT_STORAGE_DSN variable in .env-local. Edit .env-local, adding:

DEFAULT_STORAGE_DSN=file:///data/media/?url=%2Fmedia%2F

For convenience, we should expose the container's /data directory so you can see the files in it. In docker-compose.yml, add (make sure you're editing the web service, not db):

services:

  web:
    [...]
    volumes:
      - ".:/app:rw"
      - "./data:/data:rw"

And since media files should not be committed to the codebase, edit .gitignore:

# Divio
[...]
/data

Use the environment variable in our settings

The next task is to configure Django's DEFAULT_FILE_STORAGE setting. We need Django to parse the DEFAULT_STORAGE_DSN variable that contains the connection details and select the appropriate backend accordingly. For this, we'll use the django-storage-url library, which needs to be added to requirements.txt. We also need to install boto3, the Python storage backend that will handle files in the application's S3 cloud storage:

django-storage-url==0.11.1
boto3==1.29.6

Rebuild the image once more to include the new package.

Then in settings.py, add:

from django_storage_url import dsn_configured_storage_class

[...]

# Media files

# DEFAULT_FILE_STORAGE is configured using DEFAULT_STORAGE_DSN

# read the setting value from the environment variable
DEFAULT_STORAGE_DSN = os.environ.get('DEFAULT_STORAGE_DSN')

# dsn_configured_storage_class() requires the name of the setting
DefaultStorageClass = dsn_configured_storage_class('DEFAULT_STORAGE_DSN')

# Django's DEFAULT_FILE_STORAGE requires the class name
DEFAULT_FILE_STORAGE = 'myapp.settings.DefaultStorageClass'

In brief:

• We read the DEFAULT_STORAGE_DSN environment variable value into the setting DEFAULT_STORAGE_DSN.
• The DefaultStorageClass is defined using the setting.
• Finally that class is used in the DEFAULT_FILE_STORAGE setting.

Now when Django needs to handle media files, it can delegate the task to the appropriate backend, as defined by the class that DEFAULT_FILE_STORAGE refers to.

(Note that dsn_configured_storage_class() and DEFAULT_FILE_STORAGE both require the name of the value, rather than the value itself, which is why this looks a little long-winded.)

Configure local file storage serving

When using cloud storage, media files will be served directly by the external cloud storage service, at the URL provided in its DSN - nothing further needs to be configured. To serve files from local storage, we can use Django's own file serving functionality. It needs to be configured to recognise the media URL path (/media/) and to locate the media root directory of the files (/data/media).

First, set the Django settings MEDIA_URL and MEDIA_ROOT to match the values in the DEFAULT_STORAGE_DSN:

MEDIA_URL = 'media/'
MEDIA_ROOT = os.path.join('/data/media/')

And then add a new pattern to myapp/urls.py:

from django.conf import settings
from django.conf.urls.static import static

urlpatterns = [
    path('admin/', admin.site.urls),
]

if settings.DEBUG:
    urlpatterns.extend(static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT))

As the Django documentation notes, this is not for production use, but in any case, it will only work when Django is in debug mode, so it's ideal for local development.

Test file storage and serving configuration

This is a good point at which to test that your local and cloud file storage both work correctly. We'll create a very simple Django application in the application that saves uploaded files to storage.

Create the new application in the application:

docker-compose run web python manage.py startapp uploader

Add a new model to uploader/models.py:

from django.db import models

class UploadedFile(models.Model):
    file = models.FileField()

Wire it up to the admin in admin.py:

from django.contrib import admin

from uploader.models import UploadedFile

admin.site.register(UploadedFile)

Add it to INSTALLED_APPS in settings.py:

INSTALLED_APPS = [
    [...]
    'uploader',
]

And create and run migrations:

docker-compose run web python manage.py makemigrations uploader
docker-compose run web python manage.py migrate uploader

Test local media storage

Now when you start the application again with docker-compose up, you can go to the admin and try uploading a file .

Once you have saved it in the admin, you should be able to verify that it has been saved in the filesystem at /data/media, that Django shows its URL path in /media/ in the admin interface, and finally, that by selecting the link to the file in the admin it opens correctly in your browser.

Test cloud media storage

You can also check if this works with the cloud storage values and if it actually stores and serves files from the S3 object storage instance. To do this locally, follow the steps below.

Stop the application, and retrieve the value of the DEFAULT_STORAGE_DSN environment variable from the cloud test environment.

For more information on how to retrieve DEFAULT_STORAGE_DSN see How to retrieve built-in environment variables.

If you don't get a value, please check the Services view of the application and ensure that the respective service has been indeed provisioned. In your .env-local, temporarily apply this value as the DEFAULT_STORAGE_DSN, replacing the existing one. Launch the application once more, and run the test above again, uploading and saving a file. This time, you should find that the saved file is now served from the external media server.

The final test is to try it all in the cloud.

Revert the DEFAULT_STORAGE_DSN to its local value (file:///data/media/?url=%2Fmedia%2F). Now, commit all your code changes in the usual way and push them.

Finally deploy the changes and push your local media and database to the cloud:

divio app deploy
divio app push media
divio app push db

You should find all your media and database content in the cloud environment now, and you should be able to use the admin interface to upload new files too.