Pip install -e packages don't appear in Docker

Question

I have a requirements.txt file containing, amongst others:

Flask-RQ==0.2
-e git+https://token:[email protected]/user/repo.git#egg=repo

When I try to build a Docker container using Docker Compose, it downloads both packages, and install them both, but when I do a pip freeze there is no sign of the -e package. When I try to run the app, it looks as if this package hasn't been installed. Here's the relevant output from the build:

Collecting Flask-RQ==0.2 (from -r requirements.txt (line 3))
  Downloading Flask-RQ-0.2.tar.gz
Obtaining repo from git+https://token:[email protected]/user/repo.git#egg=repo (from -r requirements.txt (line 4))
  Cloning https://token:[email protected]/user/repo.git to ./src/repo

And here's my Dockerfile:

FROM python:2.7

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

COPY requirements.txt /usr/src/app/
RUN pip install -r requirements.txt

COPY . /usr/src/app

I find this situation very strange and would appreciate any help.

Answer 1

I ran into a similar issue, and one possible way that the problem can appear is from:

WORKDIR /usr/src/app

being set before pip install. pip will create the src/ directory (where the package is installed) inside of the WORKDIR. Now all of this shouldn't be an issue since your app files, when copied over, should not overwrite the src/ directory.

However, you might be mounting a volume to /usr/src/app. When you do that, you'll overwrite the /usr/src/app/src directory and then your package will not be found.

So one fix is to move WORKDIR after the pip install. So your Dockerfile will look like:

FROM python:2.7

RUN mkdir -p /usr/src/app

COPY requirements.txt /usr/src/app/
RUN pip install -r /usr/src/app/requirements.txt

COPY . /usr/src/app
WORKDIR /usr/src/app

This fixed it for me. Hopefully it'll work for you.

Answer 2

@mikexstudios is correct, this happens because pip stores the package source in /usr/src/app/src, but you're mounting a local directory over top of it, meaning python can't find the package source.

Rather than changing the position of WORKDIR, I solved it by changing the pip command to:

pip install -r requirements.txt --src /usr/local/src

Either approach should work.

Answer 3

If you are recieving a similar error when installing a git repo from a requirements file under a dockerized container, you may have forgotten to install git.

Here is the error I recieved:

Downloading/unpacking CMRESHandler from 
git+git://github.com/zigius/python-elasticsearch-logger.git (from -r 
/home/ubuntu/requirements.txt (line 5))
Cloning git://github.com/zigius/python-elasticsearch-logger.git to 
/tmp/pip_build_root/CMRESHandler
Cleaning up...
Cannot find command 'git'
Storing debug log for failure in /root/.pip/pip.log
The command '/bin/sh -c useradd ubuntu -b /home && echo 
"ubuntu     ALL     = NOPASSWD: ALL" >> /etc/sudoers             && 
chown -R ubuntu:ubuntu /home/ubuntu && pip install -r /home/ubuntu/requirements.txt returned a non-zero code: 1

Here is an example Dockerfile that installs git and then installs all requirements:

FROM python:3.5-slim

RUN apt-get update && apt-get install -y --no-install-recommends git \

ADD . /code       

WORKDIR /code
RUN pip install --upgrade pip setuptools && pip install -r /home/ubuntu/requirements.txt

Now you can use git packages in your requirements file in a Dockerized environment