'Pip wheel collects 2 versions of a package then pip install gets a conflict
We use a pipeline that first uses pip wheel to collect all the packages that are needed in the project and then it creates a docker image that calls to pip install on the collected wheels.
The issue I am encountering is that when calling pip wheel, pip is collecting 2 different versions of a package. This has started occurring once a new version of the package is available.
The project has a requirement for an internal library ecs-deployer==10.1.2 and that library has in turn a requirement in the form of: elb-listener>=3.2.1+25,<4
The relevant output of pip wheel with the verbose option says:
Collecting elb-listener>=3.2.1+25,<4
Created temporary directory: /tmp/pip-unpack-zr930807
File was already downloaded /home/user/path/dist/elb_listener-3.2.2+26-py3-none-any.whl
Added elb-listener>=3.2.1+25,<4 from https://internal-repository.com/path/elb_listener/3.2.2%2B26/elb_listener-3.2.2%2B26-py3-none-any.whl#md5=foo (from ecs-deployer==10.1.2->service==1.0.0) to build tracker '/tmp/pip-req-tracker-1tz9t5ls'
Removed elb-listener>=3.2.1+25,<4 from https://internal-repository.com/path/elb_listener/3.2.2%2B26/elb_listener-3.2.2%2B26-py3-none-any.whl#md5=blabla (from ecs-deployer==10.1.2->service==1.0.0) to build tracker '/tmp/pip-req-tracker-1tz9t5ls'
And also:
Collecting elb-listener>=3.2.1+25,<4
Created temporary directory: /tmp/pip-unpack-yfnxim_u
File was already downloaded /home/user/path/dist/elb_listener-3.2.3+27-py3-none-any.whl
Added elb-listener>=3.2.1+25,<4 from https://internal-repository.com/path/elb_listener/3.2.3%2B27/elb_listener-3.2.3%2B27-py3-none-any.whl#md5=bar (from ecs-deployer==10.1.2->service==1.0.0) to build tracker '/tmp/pip-req-tracker-1tz9t5ls'
Then when the pip install is called I get this:
ERROR: Cannot install elb-listener 3.2.2+26 (from /opt/elb_listener-3.2.2+26-py3-none-any.whl) and cad-aws-elb-listener-target-group-builder 3.2.3+27 (from /opt/elb_listener-3.2.3+27-py3-none-any.whl) because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested elb-listener 3.2.2+26 (from /opt/elb_listener-3.2.2+26-py3-none-any.whl)
The user requested elb-listener 3.2.3+27 (from /opt/elb_listener-3.2.3+27-py3-none-any.whl)
We use pip 20.2.3 with the option --use-feature=2020-resolver
Is it normal that pip wheel collects several versions of the same package?
If so, can I indicate in any way to either pip wheel to only collect one of the versions or to pip install to only use the latest version?
If not, is there any way to solve this problem? I guess changing the requirement to elb-listener>=3.2.1+27,<4 would solve it, but we don't have direct access to that library and it would take a while for other team to change it.
Solution 1:[1]
As per @sinoroc comment, upgrading the python to 3.10 and pip version to 21.2.4 solved this particular issue.
Solution 2:[2]
As far as I understood, "local version identifiers" such as 3.2.1+25 are far from usual, apparently they are not meant to be used anywhere public (like PyPI), and that might be the reason for all the trouble here. I am really not sure how well they are supported by Python packaging tools and maybe they confuse the dependency resolution.
Local version identifiers SHOULD NOT be used when publishing upstream projects to a public index server, but MAY be used to identify private builds created directly from the project source. Local version identifiers SHOULD be used by downstream projects when releasing a version that is API compatible with the version of the upstream project identified by the public version identifier, but contains additional changes (such as bug fixes). As the Python Package Index is intended solely for indexing and hosting upstream projects, it MUST NOT allow the use of local version identifiers.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Dans Merino |
| Solution 2 | sinoroc |