'Keeping dependencies in one place only when building a Docker container for CI/CD pipelines

Helllo there,

I am building a small Python package to a large extend because I want to learn about the packaging process itself and how to setup a proper CI/CD Pipeline. For the latter I use GitLab.

Here is my current problem that I could not fix to my satisfaction:

My project depends on numpy and matplotlib. I have added these in my setup.cfg under [options] as

install_requires =
    numpy >=1.21, <2
    matplotlib >= 3.5, <4

For my CI Pipeline I need a Docker container that also satisfies these requirements. Specifically I want to lint my source code with pylint that will complain about the imported packages not beeing installed otherwise. Of course many container images on dockerhub would fit for such a small and common set of requirements. But as I said earlier I want to implement a comparatively general approach and not rely on existing containers.

So I set up a Dockerfile. But I did not want to write down the dependencies explicitly again. At least not the ones that are already mentioned in setup.cfg. My current solution is ... ugly ... to say the least:

FROM python:3.8-alpine

RUN pip install --upgrade pip
RUN pip install pylint==2.12.2 pytest==6.2.1 flake8==4.0.1

RUN apk add g++ jpeg-dev zlib-dev libjpeg make
COPY setup.cfg .

RUN python -c "import setuptools; import subprocess; CONFIG = setuptools.config.read_configuration('setup.cfg'); PACKAGES = CONFIG['options']['install_requires']; subprocess.check_call(['pip', 'install', '-v', *PACKAGES]);"

COPY install_requires.py .

I could have put the python code that reads the setup.cfg in a separate script and also copy that into the container but I didn't want to overload my repo with this kind of stuff.

Ok ... finally the now obvious questions: Do you think this is a proper solution? What could I have done differently?

Ty all, Franz



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source