Metaprogramming is the ability of a program generate other programs, or even modify itself while running. This is the fourth in a series of posts exploring how to use metaprogramming in GNU make makefiles, discussing secondary expansion.

Secondary Expansion

In previous posts we’ve discussed the concepts of evaluation and expansion and recursive expansion. In this post we’ll examine secondary expansion.

When discussing constructed variable names we pointed out that this commonly-attempted make construct:

foo_OBJECTS = baz.o biz.o
bar_OBJECTS = baz.o boz.o
LIBS = libwidget.a

foo bar: $@.o $($@_OBJECTS) $(LIBS)

cannot work, because the prerequisite list of a rule is always evaluated in immediate context; at that time the value of $@ is not set. One way to allow this to work is by enabling secondary expansion.

Secondary expansion simply means that the prerequisite list of a target which has enabled secondary expansion is evaluated twice: the first time in immediate context while the makefile is parsed, as usual, but then also a second time after the rule has been chosen and (most of) the automatic variables are set. Because automatic variables are set during the second round of expansion, they are able to be used while constructing the list of prerequisites, either directly or as part of a constructed variable name.

Enabling Secondary Expansion

Secondary expansion is not enabled by default. In order to use it, you must define the pseudo-target .SECONDEXPANSION before the first target that you want to use it with. The prerequisites of the .SECONDEXPANSION target are ignored (at least for current versions of GNU make). It’s wise to not declare any prerequisites, in case this changes in the future. For example:

.SECONDEXPANSION:

Using Secondary Expansion

Once the feature is enabled, you can then create secondary expansion references in your prerequisite lists. How do you distinguish between references that you want to be expanded immediately versus those you want to be expanded during the second expansion? Very simple: you escape the references which you would like to defer to the second round of expansion. Remember that you must escape all the references you would like deferred: if you use constructed variable names you must escape the “inner” references as well.

For our example above, we could rewrite it like so:

foo_OBJECTS = baz.o biz.o
bar_OBJECTS = baz.o boz.o
LIBS = libwidget.a

.SECONDEXPANSION
foo bar: $$@.o $$($$@_OBJECTS) $(LIBS)

Note how both the inner dollar sign ($@) and outer dollar sign ($($@_OBJECTS)) must be escaped.

As a result, after reading the makefile the prerequisite list for the targets foo and bar will contain the literal strings $@ and $($@_OBJECTS), and the secondary expansion flag will be set for those targets.

When make decides that it wants to build either of these targets, it will note that the secondary expansion flag has been set, and proceed to first set up the automatic variables then perform a second round of expansion on the prerequisites. This will cause the prerequisite list to be filled in with the appropriate files: foo.o, baz.o, and biz.o for foo and bar.o, baz.o, and boz.o for bar.

Second Expansion and Pattern Rules

One of the more interesting capabilities of secondary expansion is that it also works for pattern rules, not just explicit rules:

%.o : %.c $$($$*_DEPS)

When you consider that GNU make cannot decide whether a pattern can be used to build a given target without first processing its prerequisites to see if they can be built, you’ll realize that the above simple-looking statement actually implies quite a bit of work and complexity for GNU make.

Why Is It Off by Default?

Some may wonder why we have to declare .SECONDEXPANSION explicitly: why doesn’t GNU make simply always assume that escaped references in the prerequisite list should be expanded again?

There are two reasons for this: first, it would be a backward-compatibility break. Some makefiles may have prerequisites that contain dollar signs, and be escaping them using $$. If secondary expansion were enabled by default those prerequisites would break: you’d need to replace them with four dollar signs: $$$$ to escape the dollar sign during both the first and second expansions.

The other reason traditionally was performance: performing a secondary expansion on every prerequisite list has an impact on the speed of make, most especially for pattern rules where the same prerequisite list is expanded many times during a single invocation of make.

However, newer versions of GNU make will check to see if there is any possible way that second expansion could modify the prerequisite list (in particular, if any dollar sign appears in the prerequisite list) when the rule is defined. Thus, the performance impact is only felt for rules which contain possible secondary expansions, not every rule, meaning performance is not a major factor any longer.

It may be that in some future version of GNU make, secondary expansion will be enabled by default.

Next …

Secondary expansion is undeniably useful, but it’s fairly limited: it only allows for modifying the prerequisite lists of existing targets. What if we want to create entirely new rules? GNU make has long had a powerful method for this: constructed include files.