'make: disable parallel execution of some targets

I have a compile job where linking is taking a lot of IO work. We have around a dozen of cores so we run make -j13, but when it comes to linking the 6 targets, I'd like those to be done in a round robin way. I thought about making one depend on the next but I think this would break the individual targets. Any ideas how to solve this small issue?

makefileparallel-processing


Solution 1:[1]

I've stumbled upon a hacky solution:

For each recipe it runs, Make does two things: it expands variables/functions in the recipe, and then runs the shell commands.

Since the first step can read/write the global variables, it seems to be done synchronously.

So if you run all your shell commands during the first step (using $(shell )), no other recipe will be able to start while they're running.

E.g. consider this makefile:

all: a b

a:
        sleep 1

b:
        sleep 1

time make -j2 reports 1 second.

But if you rewrite it to this:

# A string of all single-letter Make flags, without spaces.
override single_letter_makeflags = $(filter-out -%,$(firstword $(MAKEFLAGS)))

ifneq ($(findstring n,$(single_letter_makeflags)),)
# See below.
override safe_shell = $(info Would run shell command: $1)
else ifeq ($(filter --trace,$(MAKEFLAGS)),)
# Same as `$(shell ...)`, but triggers a error on failure.
override safe_shell = $(shell $1)$(if $(filter-out 0,$(.SHELLSTATUS)),$(error Unable to execute `$1`, exit code $(.SHELLSTATUS)))
else
# Same functions but with logging.
override safe_shell = $(info Shell command: $1)$(shell $1)$(if $(filter-out 0,$(.SHELLSTATUS)),$(error Unable to execute `$1`, exit code $(>
endif

# Same as `safe_shell`, but discards the output and expands to nothing.
override safe_shell_exec = $(call,$(call safe_shell,$1))


all: a b

a:
        $(call safe_shell_exec,sleep 1)
        @true

b:
        $(call safe_shell_exec,sleep 1)
        @true

time make -j2 now reports 2 seconds.

Here, @true does nothing, and suppresses Nothing to be done for ?? output.

There are some problems with this approach though. One is that all output is discarded unless redirected to file or stderr...

Solution 2:[2]

It won't break individual targets. You can create any number of (:) rules for a target, as long as only one of them has an actual recipe for building it. This appears to be a good use case for that.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 HolyBlackCat
Solution 2 reinierpost

Related Questions