Docker-Compose script not running

Question

I'm pretty new to Docker and especially to docker-compose and I'm running into an issue I can't seem to fix.

I have a docker-compose.yml file that looks like

version: '3.7'

services:
  backup:
    build: 
      context: .
      dockerfile: Dockerfile
    command: sh -c "while :;do sleep 5; done"
    tty: true
    stdin_open: true
    volumes:
      - ./data:/app/data

and I have a file called start.sh that looks simple like

python3 -u ./upload_to_s3.py > log/upload_to_s3.f9beb4d9.out 2>&1 &

When I run docker-compose exec backup /bin/sh I can get onto the docker image and I can run ./start.sh and it will run my processes which I can verify through a simple ps aux. However, when I run

docker-compose exec backup sh start.sh

it doesn't seem to run at all.

I try to verify by getting back onto the image and running ps aux and, in fact, the python script is not running.

What's going on? Why can't I seem to run my start.sh file using docker-compose?

EDIT: I've also tried to run this using docker-compose run --rm --detach --entrypoint="sh" backup -c "/app/start.sh"and I get the exact same issue

Answer 1

The script you show starts a background process. But if that's run in the context of a docker exec debugging shell, as soon as the docker exec command completes, any background processes that are still running will get terminated.

I might run this in a temporary container instead of a docker exec session. The important thing is to run this as a foreground process instead of launching a background job. For example:

docker-compose run backup \
  ./upload_to_s3.py

docker-compose run will inherit many of the settings from the backup container, like its image: and volumes: mounts, but you get to specify the command: to run at the command line. This also saves you the trouble of keeping a meaningless container alive so that you can docker exec into it later; just run a new container for these one-off tasks.

(Note, the specific invocation I've shown here assumes that the Python script is marked executable, with chmod +x if required; that it begins with a "shebang" line like #!/usr/bin/env python3; and that the image sets an environment variable ENV PYTHONUNBUFFERED=1.)