Is there a way to make docker layer cache more lenient?

Question

I have some Rust Cargo compilations in my dockerfile and I know that the files generated in the layer will be the same as long as its RUN instruction remains unchanged.

However, the cache will become invalidated if any of the instructions from above it are touched. This uses up a lot of time re-compiling these things.

So I am wondering if there is any feature that Docker can make available to let me mark a RUN as "robust" to reordering, so that it can apply the file changes from cache all the way until I touch that RUN instruction itself?

I guess without anything built-in, I could perhaps do some sort of manual work and turn it into a COPY somehow (but this is not very portable or streamlined). Maybe the layer itself could somehow be manually manipulated?

Answer 1

No, the layer caching process is not configurable with the Dockerfile model. If a previous layer has changed, then the RUN step will use the cache. Perhaps a custom buildkit frontend syntax could be created to allow this, I know there have been suggestions to allow copying a layer from another stage (rather than copying a directory from all the of the layers reassembled).

The options left include using a multi-stage build, perform any complex processes in separate stages, and use COPY --from to import the results from each of those stages into your final build. Or you can create your own tooling to build an image.

Realize there are various risks with blindly assembling layers. Some files are modified, rather than created, e.g. package managers that keep track of the list of installed packages. Each added layer will replace those files rather than performing some kind of intelligent merging. You may also find the sum of your layers is significantly larger if each separate RUN command is installing the same prerequisites. I've seen security projects that tout the ability to quickly swap out vulnerable layers with patched dependencies, but if they don't rebuild the application from those updated dependencies, they still have the vulnerability and now they have a false sense of security.