Add and configure media storage
Each cloud environment on Divio can have its own media storage running on a service such as
S3 or MS Azure, and is provided with a DEFAULT_STORAGE_DSN variable containing details of
how and where files are stored, and the URL from which they can be retrieved. Django can be configured to access the
media storage of each environment by using the DEFAULT_STORAGE_DSN
to determine Django's
DEFAULT_FILE_STORAGE
setting.
Create the media storage
For the cloud environments
In the same way that you did for the database earlier, in the application's Services view, add an Object storage instance. This will provide S3 storage for the application. Deploy the environment, or manually select Provision from the services options menu. The service will be provisioned for the application and the environment variable will be applied.
Locally
Locally, the most convenient way to work with media files is using local storage. You could use any directory in your
local application, but since the Divio CLI will expect to find media files in /data
, we will use that. We will
configure the local set-up so that media files can be:
- stored in
/data/media
- retrieved via the URL path
/media
We can configure the storage system for this with the DEFAULT_STORAGE_DSN
variable in .env-local
. Edit
.env-local
, adding:
DEFAULT_STORAGE_DSN=file:///data/media/?url=%2Fmedia%2F
For convenience, we should expose the container's /data
directory so you can see the files in it. In
docker-compose.yml
, add (make sure you're editing the web
service, not db
):
services:
web:
[...]
volumes:
- ".:/app:rw"
- "./data:/data:rw"
And since media files should not be committed to the codebase, edit .gitignore
:
# Divio
[...]
/data
Use the environment variable in our settings
The next task is to configure Django's DEFAULT_FILE_STORAGE
setting. We need Django to parse the
DEFAULT_STORAGE_DSN
variable that contains the connection details and select the appropriate backend accordingly.
For this, we'll use the django-storage-url
library, which needs to be added to requirements.txt
. We also need
to install boto3
, the Python storage backend that will handle files in the application's S3 cloud storage:
django-storage-url==0.11.1
boto3==1.29.6
Rebuild the image once more to include the new package.
Then in settings.py
, add:
from django_storage_url import dsn_configured_storage_class
[...]
# Media files
# DEFAULT_FILE_STORAGE is configured using DEFAULT_STORAGE_DSN
# read the setting value from the environment variable
DEFAULT_STORAGE_DSN = os.environ.get('DEFAULT_STORAGE_DSN')
# dsn_configured_storage_class() requires the name of the setting
DefaultStorageClass = dsn_configured_storage_class('DEFAULT_STORAGE_DSN')
# Django's DEFAULT_FILE_STORAGE requires the class name
DEFAULT_FILE_STORAGE = 'myapp.settings.DefaultStorageClass'
In brief:
- We read the
DEFAULT_STORAGE_DSN
environment variable value into the settingDEFAULT_STORAGE_DSN
. - The
DefaultStorageClass
is defined using the setting. - Finally that class is used in the
DEFAULT_FILE_STORAGE
setting.
Now when Django needs to handle media files, it can delegate the task to the appropriate backend, as defined by the
class that DEFAULT_FILE_STORAGE
refers to.
(Note that dsn_configured_storage_class()
and DEFAULT_FILE_STORAGE
both require the name of the value, rather
than the value itself, which is why this looks a little long-winded.)
Configure local file storage serving
When using cloud storage, media files will be served directly by the external cloud storage service, at the URL
provided in its DSN - nothing further needs to be configured. To serve files from local storage, we can use
Django's own file serving functionality. It needs to be
configured to recognise the media URL path (/media/
) and to locate the media root directory of the files
(/data/media
).
First, set the Django settings MEDIA_URL
and MEDIA_ROOT
to match the values in the DEFAULT_STORAGE_DSN
:
MEDIA_URL = 'media/'
MEDIA_ROOT = os.path.join('/data/media/')
And then add a new pattern to myapp/urls.py
:
from django.conf import settings
from django.conf.urls.static import static
urlpatterns = [
path('admin/', admin.site.urls),
]
if settings.DEBUG:
urlpatterns.extend(static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT))
As the Django documentation notes, this is not for production use, but in any case, it will only work when Django is in debug mode, so it's ideal for local development.
Test file storage and serving configuration
This is a good point at which to test that your local and cloud file storage both work correctly. We'll create a very simple Django application in the application that saves uploaded files to storage.
Create the new application in the application:
docker compose run web python manage.py startapp uploader
Add a new model to uploader/models.py
:
from django.db import models
class UploadedFile(models.Model):
file = models.FileField()
Wire it up to the admin in admin.py
:
from django.contrib import admin
from uploader.models import UploadedFile
admin.site.register(UploadedFile)
Add it to INSTALLED_APPS
in settings.py
:
INSTALLED_APPS = [
[...]
'uploader',
]
And create and run migrations:
docker compose run web python manage.py makemigrations uploader
docker compose run web python manage.py migrate uploader
Test local media storage
Now when you start the application again with docker compose up
, you can go to the admin and try uploading a file .
Once you have saved it in the admin, you should be able to verify that it has been saved in the filesystem at
/data/media
, that Django shows its URL path in /media/
in the admin interface, and finally, that by selecting
the link to the file in the admin it opens correctly in your browser.
Test cloud media storage
You can also check if this works with the cloud storage values and if it actually stores and serves files from the S3 object storage instance. To do this locally, follow the steps below.
Stop the application, and retrieve the value of the DEFAULT_STORAGE_DSN
environment variable from the cloud test environment.
For more information on how to retrieve DEFAULT_STORAGE_DSN
see
How to retrieve built-in environment variables.
If you don't get a value, please check the Services view of the application
and ensure that the respective service has been indeed provisioned. In your
.env-local
, temporarily apply this value as the DEFAULT_STORAGE_DSN
,
replacing the existing one. Launch the application once more, and run the test
above again, uploading and saving a file. This time, you should find that the
saved file is now served from the external media server.
The final test is to try it all in the cloud.
Revert the DEFAULT_STORAGE_DSN
to its local value (file:///data/media/?url=%2Fmedia%2F
). Now, commit all your
code changes in the usual way and push them.
Finally deploy the changes and push your local media and database to the cloud:
divio app deploy
divio app push media
divio app push db
You should find all your media and database content in the cloud environment now, and you should be able to use the admin interface to upload new files too.