How to deploy a Gatsby application on Divio

Gatsby is very popular React-based open-source framework for creating websites and apps.

This document will take you step-by-step through the tasks required to deploy a portable, vendor-neutral application, to Divio using Docker. The application architecture we adopt is in line with Twelve-factor design principles.

This guide will help you adapt an existing project for Docker or check that your existing Docker application will run on Divio. Note that exact steps you need to take may depend on details of your application.

Prerequisites

  • Your application needs to be managed in a Git repository, hosted on Divio or your preferred Git host.

  • You need to be familiar with the basics of the Divio platform and Docker, and have Docker and the Divio CLI installed on your machine. If not, please follow one of our tutorials.

  • You need to have at least a minimal working application ready to be deployed, whether it is Docker-ready already or not.

If you don’t already have a working Gatsby project

See How to deploy a Gatsby project with our quickstart repository. Or, if you already have a suitable version of Node installed, you can do:

npm install -g gatsby-cli
gatsby new hello-world https://github.com/gatsbyjs/gatsby-starter-hello-world

and then in the new directory:

gatsby develop -H 0.0.0.0 -p 8000

to see it running locally.

The Dockerfile - define how to build the application

The application needs a Dockerfile at the root of repository, that defines how to build the application. The Dockerfile starts by importing a base image.

For a Gatsby application, you can use:

FROM node:14.15.1-alpine

14.15.1-alpine is the name of the Docker base image. We cannot advise on what base image you should use; you’ll need to use one that is in-line with your application’s needs. It’s good practice to use a specific base image - for example node:14.15.1-alpine above (rather than say just node:14).

Install system-level dependencies

The Dockerfile needs to install any system dependencies required by the application. For example, if your chosen base image is Debian-based, you might run:

RUN apt-get update && apt-get install -y <list of packages>

In this case, we can use APK since we are using an Alpine Linux base image:

RUN apk add --no-cache \
  make g++ && \
  apk add vips-dev fftw-dev --update-cache \
  && rm -fR /var/cache/apk/*

WORKDIR

We recommend setting up a working directory early on in the Dockerfile before you need to write any files, for example:

# set the working directory
WORKDIR /app
# copy the repository files to it
COPY . /app

Install application-level dependencies

Run a command to install the application-level dependencies.

Assuming you’re using a package.json/package-lock.json:

RUN npm install
RUN npm install -g gatsby-cli

It’s important to pin dependencies as firmly as possible.

File-building operations

If the application needs to perform any build operations to generate files, they should be run in the Dockerfile so that they are built into the image. This could include compiling or collecting JavaScript or CSS, for example, and can make use of frameworks that do this work.

For Gatsby, add:

RUN gatsby build

EXPOSE and CMD

EXPOSE informs Docker that the container listens on the specified ports at runtime; typically, you’d need:

EXPOSE 80

Launch a server running on port 80 by including a CMD at the end of the Dockerfile.

A suitable command for Gatsby would be:

CMD gatsby serve --port 80 --host 0.0.0.0

However if your package.json already includes a suitable entry in its scripts section, you could also make use of that, so for example:

"scripts": {
  ...
  "deploy": "gatsby serve --port 80 --host 0.0.0.0"
},

and in the Dockerfile instead use:

CMD npm run deploy

Use CMD, not ENTRYPOINT, to start the server

Using CMD provides a default way to start the server, that can also be overridden. This is useful when working locally, where often we would use the docker-compose.yml to issue a startup command that is more suited to development purposes. It also allows our infrastructure to override the default, for example in order to launch containers without starting the server, when some other process needs to be executed.

An ENTRYPOINT that starts the server would not allow this.

If you are using a Docker entrypoint script, it’s good practice to conclude it with exec "$@", so that any commands passed to the container will be executed as expected.

Access to environment and services

Warning

During the build process, Docker has no access to to the application’s environment or services.

This means you cannot run database operations such as database migrations during the build process. Instead, these should be handled later as release commands.

Configuring your application

Your application will require configuration. You may be used to hard-coding such values in the application itself - though you can do this on Divio, we recommend not doing it. Instead, all application configuration (for access to services such as the database, security settings, etc) should be managed via environment variables.

Divio provides services such as database and media. To access them, your application will need the credentials. For each service in each environment, we provide an environment variable containing the values required to access it. This variable is in the form:

schema://<user name>:<password>@<address>:<port>/<name>

Your application should:

  • read the variables

  • parse them to obtain the various credentials they contain

  • configure its access to services using those credentials

We also provide environment variables for things like security configuration.

Helper modules

Rather than re-inventing the wheel when it comes to reading and applying configuration from environment variables, we advise to make use of existing helper modules that have solved this problem already.

Security settings

Typically, an application’s security settings will depend upon multiple variables. Some that are typically needed are provided by Divio’s cloud environments. For example, your application is likely to need information about:

Other variables specific to the application will need to be applied manually for each environment.

Database

Database credentials, if required, are provided in a DATABASE_URL environment variable. When a database (and therefore the environment variable) are not available (for example during the Docker build phase) the application should fall back safely, to a null database option or to an in-memory database (e.g. using sqlite://:memory:).

See the Django guide for a concrete example.

Static files

There are numerous options for static file serving. You can opt to serve them directly from the application, or to configure the web server/gateway server to handle them.

Media

If your application needs to handle generated or user-uploaded media, it should use a media object store.

Media credentials are provided in DEFAULT_STORAGE_DSN. See how to parse the storage DNS to obtain the credentials for the media object stores we provide. Your application needs to be able to use these credentials to configure its storage.

When working in the local environment, it’s convenient to use local file storage instead (which can be also be configured using a variable provided by the local environment).

Modern application frameworks such as Django make it straightforward to configure an application to use storage on multiple different backends, and are supported by mature libraries that will let you use a Divio-provided S3 or MS Azure storage service as well as local file storage.

See the Django guide for a concrete example.

Local file storage is not a suitable option

Your code may expect, by default, to be able to write and read files from local file storage (i.e. files in the same file-space as belonging to the application).

This will not work well on Divio or any similar platform. Our stateless containerised application model does not provide persistent file storage. Instead, your code should expect to use a dedicated file storage; we provide AWS S3 and MS Azure blob storage options.

Other values

You may need to make use of other variables for your application - take every opportunity to make use of the provided variables so that your codebase can contain as little as configuration as possible.

Local container orchestration with docker-compose.yml

What’s described above is fundamentally everything you need in order to deploy your application to Divio. You could deploy your application with that alone.

However, you would be missing out on a lot of value. Being able to build and then run the same application, in a very similar environment, locally on your own computer before deploying it to the cloud makes development and testing much more productive. This is what we’ll consider here.

docker-compose.yml is only used locally

Cloud deployments do not use Docker Compose. Nothing that you do here will affect the way your application runs in a cloud environment. See docker-compose.yml.

Create a docker-compose.yml file, for local development purposes. This will replicate the web image used in cloud deployments, allowing you to run the application in an environment as close to that of the cloud servers as possible. Amongst other things, it will allow the project to use a Postgres or MySQL database (choose the appropriate lines below) running in a local container, and provides convenient access to files inside the containerised application.

Take note of the highlighted lines below; some require you to make a choice.

version: "2.4"
services:
  web:
    # the application's web service (container) will use an image based on our Dockerfile
    build: "."
    # map the internal port 80 to port 8000 on the host
    ports:
      - "8000:80"
    # map the host directory to app (which allows us to see and edit files inside the container)
    volumes:
    # uncomment if you prefer to perform node install/gatsby build operations on the host
    # system - see the Divio Gatsby Quickstart how-to guide for an explanation
    #   - ".:/app:rw"
      - "./data:/data:rw"
    # the default command to run whenever the container is launched - this can be different from
    # the command built into the image
    command: gatsby develop --port 80 --host 0.0.0.0
    env_file: .env-local
    # database_default is the application's database service - but remove the entire links section
    # if not required
    # links:
    #   - "database_default"

  # # this entire section can be removed if you're not using a database
  #
  # database_default:
  #   # Select one of the following db configurations for the database
  #   image: postgres:9.6-alpine
  #   environment:
  #     POSTGRES_DB: "db"
  #     POSTGRES_HOST_AUTH_METHOD: "trust"
  #     SERVICE_MANAGER: "fsm-postgres"
  #   volumes:
  #     - ".:/app:rw"
  #
  #   image: mysql:5.7
  #   environment:
  #     MYSQL_DATABASE: "db"
  #     MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
  #     SERVICE_MANAGER: "fsm-mysql"
  #   volumes:
  #     - ".:/app:rw"
  #     - "./data/db:/var/lib/mysql"
  #   healthcheck:
  #       test: "/usr/bin/mysql --user=root -h 127.0.0.1 --execute \"SHOW DATABASES;\""
  #       interval: 2s
  #       timeout: 20s
  #       retries: 10

Local configuration using .env-local

As you will see above, the web service refers to an env_file containing the environment variables that will be used in the local development environment.

Divio cloud projects include a number of environment variables as standard. In addition, user-supplied variables may be applied per-environment.

If the application refers to its environment for variables to configure database, storage or other services, it will need to find those variables even when running locally. On the cloud, the variables will provide configuration details for our database clusters, or media storage services. Clearly, you don’t have a database cluster or S3 instance running on your own computer, but Docker Compose can provide a suitable database running locally, and you can use local file storage while developing.

Create a .env-local file. In this you need to provide some environment variables that are suitable for the local environment. The example below assumes that your application will be looking for environment variables to configure its access to a Postgres or MySQL database, and for local file storage:

# Select one of the following for the database
DATABASE_URL=postgres://postgres@database_default:5432/db
DATABASE_URL=mysql://root@database_default:3306/db

# Storage will use local file storage in the data directory
DEFAULT_STORAGE_DSN=file:///data/media/?url=%2Fmedia%2F

In cloud environments, we provide a number of useful variables. If your application needs to make use of them (see a Django example) you should provide them for local use too. For example:

With this, you have the basics for a Dockerised application that can equally effectively be deployed in a production environment or run locally, using environment variables for configuration in either case.

Building and running

Build with Docker

Now you can build the application containers locally:

docker-compose build

Building on the host as an option

If you uncomment the:

# - ".:/app:rw"

entry in the web:volumes section of docker-compose.yml, the entire /app directory will be overridden by the project files from the host. This can be useful for development. However, you will now need to run the commands npm install and gatsby build on the host as well in order to regenerate the files so that the container sees them.

Check the local site

You may need to perform additional steps such as migrating a database. To run a command manually inside the Dockerised environment, precede it with docker-compose run web. (For example, to run Django migrations: docker-compose run web python manage.py migrate.)

To start up the site locally to test it:

docker-compose up

Access the site at http://127.0.0.1:8000/. You can set a different port in the ports option of docker-compose.yml.

Git

Your code needs to be in a Git repository in order to be deployed on Divio.

You will probably want to exclude some files from the Git repository, so check your .gitignore and ensure that nothing will be committed that you don’t want included.

For a basic project you’ll probably need at least:

# used by the Divio CLI
.divio
/data.tar.gz

# for local file storage
/data

Deployment to Divio

Create a new project on Divio

The first step is to create a project on the Divio Control Panel, with your application repository.

You have two ways of doing this:

'New project options'
  • New project, in which you will push your local Git code to Divio’s Git server

  • New project from Git repository (Beta), in which your Divio project will fetch the code from a Git host

In the Divio Control Panel, add a New project. Select the Build your own option.

Once the project has been created, copy the project’s Git URL from its Repository view. Add the project’s Git repository as a remote, for example:

git remote add divio git@git.divio.com:my-divio-project.git

Commit and push your work.

Note

If you’re using the master branch, you will need to force push to overwrite the initial commits in a Divio project with your own.

You can use divio project commands such as divio project dashboard to interact directly with the Divio project.

In the Divio Control Panel, add a New project from Git repository. Once you have supplied the Git repository URL, you will need to use the public key provided to create a Deploy Key on the repository.

Beta status limitations

Some limitations apply to the current version of this functionality. In order to import a repository, at the time of import:

  • you will need to enable write access on the repository’s deploy key

  • the repository will need a master branch

Once imported, you can remove the write access and can delete the master branch if you don’t need it.

You should also add a webhook, so that when new commits are pushed to the repository, it will send a signal to update the Divio Control Panel.

In the Environments view, configure each environment to use the appropriate branch.

Connect your local application to the cloud project

You can connect a local application to a Divio project on the cloud. This is very convenient, allowing you to interact with the cloud project from your command-line.

Run:

divio project configure

and provide the slug (this creates a new file in the project at .divio/config.json).

The cloud project has a slug, based on the name you gave it when you created it. Run divio project list -g to get your project’s slug.

You can also read the slug from the Control Panel:

'Project slug'

You can now use commands such as:

divio project dashboard
divio project pull db  # also push
divio project pull media  # also push
divio project deploy

See some usage examples.

Add database and media services

The new Divio application does not include any additional services. If your application requires a database or media store, they must be added manually using the Divio Control Panel as required. Use the Services menu to add the services your application needs.

Add release commands

If your application needs to perform operations each time it is deployed, for example start-up health tests or database migrations, these should be applied as release commands.

Add additional environment variables

Your application may require additional environment variables in production. Apply any enviroment variables using the Divio Control Panel or CLI.

Push local database/media content

If you have local database or media content, push them to the Test environment:

divio project push db
divio project push media

Deploy the Test server

Deploy with:

divio project deploy

(or use the Deploy button in the Control Panel).

Once deployed, your project will be accessible via the Test server URL shown in the Control Panel.