Skip to main content

How to identify and resolve a dependency conflict

Occasionally when running a deployment on the Divio Cloud or building an application locally, the process will fail, with a message like:

ERROR: Service 'web' failed to build: The command '/bin/sh -c pip-reqs compile
&& pip-reqs resolve && pip install --no-index --no-deps --requirement
requirements.urls' returned a non-zero code: 1

This tells us that pip ran into problems while processing the application's requirements. Resolving this requires a little detective work. The good news is that the information you require is provided, and the process for working through it to find the answer is set out below.

Identify the conflict​

Looking a little further up the log, we'll see something like (this is just a representative example):

found candidate dj-redis-url==0.1.4 (constraint was <any>)
found candidate dj-static==0.0.6 (constraint was <any>)
Could not find a version that matches Django<1.11,<2.0,<2.1,<2.2,==1.8.18,>=1.11,
>=1.6,>=1.8
Tried: 1.1.3, 1.1.4, 1.2, 1.2.1, 1.2.2 [etc]

The highlighted line tells us what the problem is: pip could not find a version of Django that matched all the listed constraints - naturally, because it's impossible to have a version of Django that equals 1.8.18, is less than 1.11, and is also greater than or equal to 1.11.

So, between them, the packages being installed in the application have some mutually incompatible requirements. This can often be caused by unpinned dependencies, when a package is listed as a requirement without specifying a version.

In this example, we can see that the conflict is between Django <1.11 and ==1.8.18 on one hand, and >=1.11 on the other.

In your case, the packages and version numbers affected will be different, but the principle is the same.

But I didn't change anything in my application!

Because of the way pip works, even if you don't change anything at all in your application, simply rebuilding it can pull in new packages, if they were unpinned. Whenever the application is built, it will select the latest versions of unpinned packages, and those versions may introduce new, incompatible, requirements of their own.

Identify the problem requirement​

The question now is to ascertain which of these requirements we will accept and which we will change.

In example above, we have a strong clue. The most firmly-pinned of these requirements is ==1.8.18. All the others are more loosely pinned. That suggests that Django 1.8.18 has been specified for a good reason.

In this example, searching through the log for Django==1.8.18 will reveal:

adding Django==1.8.18
  from aldryn-django==1.8.18.1

which means that the requirement for Django 1.8.18 has come from the Aldryn Django addon. So, while that version of the addon is specified, it's the requirement for Django>=1.11 that is the problem.

There is a second clue in the log that indicates which requirement is the problem. As well as being incompatible with Django==1.8.18, the requirement for Django>=1.11 is also incompatible with another requirement: Django<1.11. A requirement that conflicts with multiple other requirements is most likely to be the one we should address.

So in this case, we now know that the Django>=1.11 requirement is the one to tackle.

Identify where the requirement comes from​

The next question is: where does the requirement for Django>=1.11 come from? A search in the log for the string >=1.11 will reveal this - for example (again, your own results will be different, but you will see something in this pattern):

adding django<2.2,>=1.11
  from djangocms-attributes-field==0.4.0

meaning that djangocms-attributes-field==0.4.0 wants to install a version of Django greater than or equal to 1.11 but less than 2.2.

We can quickly verify this by checking the setup.py in the 0.4.0 branch of the djangocms-attributes-field repository, where the incompatible requirement is introduced (it's not present in earlier versions).

Now we know that djangocms-attributes-field 0.4.0 has an incompatible Django requirement, so specifying a version lower than 0.4.0 (djangocms-attributes-field<0.4.0) should solve the problem.

Before doing that, it is wise to check what packages require djangocms-attributes-field, and what versions. So repeat the process above: search in the log for djangocms-attributes-field. You might find, for example:

adding djangocms-attributes-field>=0.1.1
  from djangocms-file==2.0.2
       djangocms-link==2.1.2
       djangocms-picture==2.0.5
       djangocms-style==2.0.2
       djangocms-video==2.0.3

meaning that all those packages have specified a version of djangocms-attributes-field greater than 0.1.1. In other words, there is nothing that seems to be incompatible with djangocms-attributes-field<0.4.0, so we can add:

djangocms-attributes-field<0.4.0

to the application's requirements.in file (outside the section that will be overwritten) and test it, by rebuilding the application with:

docker compose build web

If that completes without an error, you will know that you have successfully identified and addressed the dependency conflict.

Repeat the process​

Often you will need to repeat the process, as further dependency conflicts will be revealed after you have solved the first one. Each time you will need to pin the problem package in requirements.in and test the build with docker compose build web, until you have no further conflicts.

How to prevent this from happening again​

In general, the answer is to pin packages firmly, in each place that requirements are given.

An application's requirements can be specified:

  • by the addons system in the Control Panel
  • in its requirements.in (addons are automatically listed here too)
  • as dependencies of any addons, in their setup.py files
  • as any dependencies of dependencies

You have more control over some of these than others. The easiest way to do this is to pin requirements manually as necessary in requirements.in. However, if you want more thorough and precise control, please see Pin all dependencies.