How to separate docker build and publish scripts into separate stages in Gitlab CI

Question

I would like to build and publish docker images in two separate stages:

• build
• publish

Unfortunately, Gitlab runs those stages inside different workers - ci-runner-worker11 and ci-runner-worker15. Therefore images built in the first stage are not visible in the second stage. Do you know a clean solution to have this separation and be able to pass those artifacts between stages?

I've tried this solution:

container_images:
  stage: build
  artifacts:
    untracked: true
  image: ...
...

push_images:
  stage: publish
  dependencies: 
- container_images
...

Unfortunately, I get an error:

untracked: found 170 files                         
ERROR: Uploading artifacts as "archive" to coordinator... too large archive  id=15085670 responseStatus=413 Request Entity Too Large status=413
FATAL: too large  

Answer 1

Why do you need this? Maybe there is a better solution.

untracked artifacts are relative to the project root folder, and the docker build command will store the image layers elsewhere.

If you need to test the container before setting a final tag, you can:

• Use a temp tag
• Push it to the repository
• Test the image (with the temp tag)
• If all tests pass, change the tag and push it again to the repository
• If the test fails, remove the test image.

You can use a fixed temp tag, but you need to be sure that you only run one pipeline at a time. Or, you can use a temp tag based on the CI_PIPELINE_ID environment variable.