Installing python package from private gitlab repo in Dockerfile

Question

I'm currently trying to install python packages from a private gitlab repo. Unfortunately, I get problems with the credentials. Is there any way to install this package without writing my credentials into the Dockerfile or adding my personal ssh key into it?

Dockerfile:

FROM python:3.9.12-buster AS production

RUN apt-get update && apt-get install -y git

COPY ./requirements.txt /app/requirements.txt

RUN pip install -r /app/requirements.txt

requirements.txt:

fastapi
uvicorn
cycler~=0.10.0
networkx
python-multipart
git+https://gitlab.private.net/group/private-repo.git@commit_hash#egg=foo

Error message:

#10 3.760   Cloning https://gitlab.private.net/group/private-repo.git (to revision commit_hash) to /tmp/pip-install-q9wtmf_q/foo_commit_hash     
#10 3.769   Running command git clone --filter=blob:none --quiet https://gitlab.private.net/group/private-repo.git /tmp/pip-install-q9wtmf_q/foo_commit_hash
#10 4.039   fatal: could not read Username for 'https://gitlab.private.net/group/private-repo.git': No such device or address
#10 4.060   error: subprocess-exited-with-error

Answer 1

Generally speaking, you can use multi-stage docker builds to make sure your credentials don't stay in the image.

In your case, you might do something like this:

FROM python:3.9.12-buster as download
RUN apt-get update && apt-get install -y git
RUN pip install --upgrade pip wheel
ARG GIT_USERNAME
ARG GIT_PASSWORD

WORKDIR /build
COPY requirements.txt .
# add password to requirements file
RUN sed -i -E "s|gitlab.private.net|$GIT_USERNAME:[email protected]|" requirements.txt

# download dependencies and build wheels to /build/dist
RUN python -m pip wheel -w /build/dist -r requirements.txt

FROM python:3.9.12-buster as production
WORKDIR /app
COPY --from=download /build/dist /wheelhouse
# install dependencies from the wheels created in previous build stage
RUN pip install --no-index /wheelhouse/*.whl

COPY . .
# ... the rest of your dockerfile

In GitLab CI, you might use the build command like this:

script:
  # ...
  - docker build --build-arg GIT_USERNAME=gitlab-ci-token --build-arg GIT_PASSWORD=$CI_JOB_TOKEN -t $CI_REGISTRY_IMAGE .

Then your image will be built and the final image won't contain your credentials. It will also be smaller since you don't have to install git :)

As a side note, you can simplify this somewhat by using the GitLab PyPI package registry.