How to identify and resolve a dependency conflict¶
Occasionally when running a deployment on the Cloud or building a project locally, the process will fail, with a message like:
ERROR: Service 'web' failed to build: The command '/bin/sh -c pip-reqs compile && pip-reqs resolve && pip install --no-index --no-deps --requirement requirements.urls' returned a non-zero code: 1
This tells us that pip ran into problems while processing the project’s requirements. Resolving this requires a little detective work. The good news is that the information you require is provided, and the process for working through it to find the answer is set out below.
Identify the conflict¶
Looking a little further up the log, we’ll see something like (this is just a representative example):
found candidate dj-redis-url==0.1.4 (constraint was <any>) found candidate dj-static==0.0.6 (constraint was <any>) Could not find a version that matches Django<1.11,<2.0,<2.1,<2.2,==1.8.18,>=1.11, >=1.6,>=1.8 Tried: 1.1.3, 1.1.4, 1.2, 1.2.1, 1.2.2 [etc]
The highlighted line tells us what the problem is: pip could not find a version of Django that matched all the listed constraints - naturally, because it’s impossible to have a version of Django that equals 1.8.18, is less than 1.11, and is also greater than or equal to 1.11.
So, between them, the packages being installed in the project have some mutually incompatible requirements. This can often be caused by unpinned dependencies, when a package is listed as a requirement without specifying a version.
In this example, we can see that the conflict is between Django
==1.8.18 on one
>=1.11 on the other.
In your case, the packages and version numbers affected will be different, but the principle is the same.
But I didn’t change anything in my project!
Because of the way pip works, even if you don’t change anything at all in your project, simply rebuilding it can pull in new packages, if they were unpinned. Whenever the project is built, it will select the latest versions of unpinned packages, and those versions may introduce new, incompatible, requirements of their own.
Identify the problem requirement¶
The question now is to ascertain which of these requirements we will accept and which we will change.
In example above, we have a strong clue. The most firmly-pinned of these requirements is
==1.8.18. All the others are more loosely pinned. That suggests that Django 1.8.18 has been
specified for a good reason.
In this example, searching through the log for
Django==1.8.18 will reveal:
adding Django==1.8.18 from aldryn-django==18.104.22.168
which means that the requirement for Django 1.8.18 has come from the Aldryn Django addon. So,
while that version of the addon is specified, it’s the requirement for
Django>=1.11 that is
There is a second clue in the log that indicates which requirement is the problem. As well as being
Django==1.8.18, the requirement for
Django>=1.11 is also incompatible
with another requirement:
Django<1.11. A requirement that conflicts with multiple other
requirements is most likely to be the one we should address.
So in this case, we now know that the
Django>=1.11 requirement is the one to tackle.
Identify where the requirement comes from¶
The next question is: where does the requirement for
Django>=1.11 come from? A search in the
log for the string
>=1.11 will reveal this - for example (again, your own results will be
different, but you will see something in this pattern):
adding django<2.2,>=1.11 from djangocms-attributes-field==0.4.0
djangocms-attributes-field==0.4.0 wants to install a version of Django greater
than or equal to 1.11 but less than 2.2.
We can quickly verify this by checking the setup.py in the 0.4.0 branch of the djangocms-attributes-field repository, where the incompatible requirement is introduced (it’s not present in earlier versions).
Now we know that djangocms-attributes-field 0.4.0 has an incompatible Django requirement, so
specifying a version lower than 0.4.0 (
djangocms-attributes-field<0.4.0) should solve the
Before doing that, it is wise to check what packages require djangocms-attributes-field, and what
versions. So repeat the process above: search in the log for
might find, for example:
adding djangocms-attributes-field>=0.1.1 from djangocms-file==2.0.2 djangocms-link==2.1.2 djangocms-picture==2.0.5 djangocms-style==2.0.2 djangocms-video==2.0.3
meaning that all those packages have specified a version of djangocms-attributes-field greater than
0.1.1. In other words, there is nothing that seems to be incompatible with
djangocms-attributes-field<0.4.0, so we can add:
to the project’s
requirements.in file (outside the section that will be overwritten) and
test it, by rebuilding the project with:
docker-compose build web
If that completes without an error, you will know that you have successfully identified and addressed the dependency conflict.
Repeat the process¶
Often you will need to repeat the process, as further dependency conflicts will be revealed after
you have solved the first one. Each time you will need to pin the problem package in
requirements.in and test the build with
docker-compose build web, until you have no
How to prevent this from happening again¶
In general, the answer is to pin packages firmly, in each place that requirements are given.
A project’s requirements can be specified:
- by the addons system in the Control Panel
- in its
requirements.in(addons are automatically listed here too)
- as dependencies of any addons, in their
- as any dependencies of dependencies
You have more control over some of these than others. The easiest way to do this is to pin
requirements manually as necessary in
requirements.in. However, if you want more thorough and
precise control, please see How to pin all of your project’s Python dependencies.