Reduce docker image size with Python and R

Question

Hi have created dockerfile(multi-stage) for python and R. But the size was nearly 1.95GB. Initially it was 2.45GB, after using multi-stage it is reduces to 1.95GB. But still I'm looking for optimization.When I checked, the python/site-package size was nearly 985MB.

Dockerfile:

FROM tiangolo/uvicorn-gunicorn-fastapi:python3.8-slim-2021-06-09 As stage1

COPY key_gnupg.gpg /app/key_gnupg.gpg
RUN echo "Acquire::Check-Valid-Until \"false\";\nAcquire::Check-Date \"false\";" | cat > /etc/apt/apt.conf.d/10no--check-valid-until
RUN apt-get update && apt-get install -y gnupg2 && \
    echo "deb http://cloud.r-project.org/bin/linux/debian buster-cran40/" > /etc/apt/sources.list.d/cran.list && \
    apt-key add /app/key_gnupg.gpg && \
    apt-get update && \
    apt-get install --no-install-recommends -y pkg-config liblapack-dev gfortran libssl-dev libcurl4-openssl-dev libnlopt-dev r-base curl postgresql libpq-dev libblas-dev libpcre2-8-0 libgomp1 build-essential && \
    rm -rf /var/lib/apt/lists/* && \
    apt-get purge --auto-remove && \
    apt-get clean

FROM stage1 AS stage2
COPY ./ /app
COPY requirements*.txt ./    
COPY r-scripts /app/r-scripts
WORKDIR /app/r-scripts
RUN R -e " install.packages('remotes')"
RUN R -e "remotes::install_local('.', dependencies=T, Ncpus=3)"
WORKDIR /app/
RUN pip install -U pip && \ 
    pip install --no-cache-dir  --user -r requirements-frozen-prod.txt

FROM stage1
COPY --from=stage2 /usr/local/lib/R/site-library /usr/local/lib/R/site-library
COPY --from=stage2 /root/.local /root/.local
COPY --from=stage2 /usr/local/bin /usr/local/bin
WORKDIR /app/
ENV PYTHONPATH="${PYTHONPATH}:/usr/local/lib/python3.8/site-packages"
ENV PATH=/root/.local/bin:$PATH
ENV R_HOME="/usr/lib/R"
RUN rm -rf /app/*
CMD ["/bin/bash"]

python requirement.txt

alembic==1.7.1
anyio==3.3.0
astroid==2.7.3
attrs==21.2.0
autoflake==1.4
autopep8==1.5.7
backports.zoneinfo==0.2.1
black==21.8b0
cachetools==4.2.2
certifi==2021.5.30
cffi==1.14.6
charset-normalizer==2.0.4
click==7.1.2
coverage==5.5
dependency-check==0.6.0
et-xmlfile==1.1.0
fastapi==0.65.2
flake8==3.9.2
flake8-formatter-junit-xml==0.0.6
future==0.18.2
google-api-core==2.0.1
google-auth==2.0.2
google-cloud-bigquery==2.26.0
google-cloud-core==2.0.0
google-crc32c==1.2.0
google-resumable-media==2.0.3
googleapis-common-protos==1.53.0
grpcio==1.39.0
gunicorn==20.0.4
h11==0.11.0
httpcore==0.13.6
httptools==0.1.1
httpx==0.19.0
idna==3.2
importlib-resources==5.2.2
iniconfig==1.1.1
isort==5.9.3
Jinja2==3.0.1
junit-xml==1.8
kaleido==0.2.1
lazy-object-proxy==1.6.0
Mako==1.1.5
MarkupSafe==2.0.1
mccabe==0.6.1
mypy==0.910
mypy-extensions==0.4.3
numpy==1.21.2
openpyxl==3.0.7
openpyxl-image-loader==1.0.5
packaging==21.0
pandas==1.3.2
pathspec==0.9.0
patsy==0.5.1
Pillow==8.3.2
platformdirs==2.3.0
plotly==5.3.1
pluggy==1.0.0
proto-plus==1.19.0
protobuf==3.17.3
psycopg2-binary==2.9.1
py==1.10.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybigquery==0.10.2
pycodestyle==2.7.0
pycparser==2.20
pydantic==1.8.2
pyflakes==2.3.1
pylint==2.10.2
pylint-junit==0.3.2
pyparsing==2.4.7
pytest==6.2.5
pytest-asyncio==0.15.1
pytest-cov==2.12.1
python-dateutil==2.8.2
python-dotenv==0.15.0
pytz==2021.1
PyYAML==5.3.1
regex==2021.8.28
requests==2.26.0
rfc3986==1.5.0
rpy2==3.4.5
rsa==4.7.2
scipy==1.7.1
six==1.16.0
sniffio==1.2.0
SQLAlchemy==1.3.24
starlette==0.14.2
statsmodels==0.12.2
tenacity==8.0.1
toml==0.10.2
tomli==1.2.1
typing-extensions==3.10.0.0
tzlocal==3.0
urllib3==1.26.6
uvicorn==0.13.1
uvloop==0.14.0
watchgod==0.6
websockets==8.1
wrapt==1.12.1
xlrd==2.0.1
XlsxWriter==3.0.1
zipp==3.5.0

Answer 1

This technique is used in the context of reducing the size of AWS lambda functions but also applies to Docker images.

The idea is to remove all the .py files under site-packages and use the .pyc files only since these libraries will not (I'm assuming) be changed.

Note that for same cases the .py may still be required but that seems to be rare.