VPATH+: GNU make Enhanced VPATH Patch

Paul D. Smith

VPATH+ is a patch to GNU make (3.63 and above) which modifies the functionality of VPATH/vpath. The shipped functionality is consistent with AT&T make's VPATH, so Roland is reluctant to change it for Rev 3.xx although he has promised to address the problem in Rev 4.

Until that time, I will try to maintain this patch for each new 3.xx revision of GNU make so that I (we) can have a VPATH/vpath which "does what you want", and is thus more useful (just because it's AT&T compatible doesn't mean it makes sense! :-)

You can obtain the patches for all the above versions of GNU make, along with a copy of this README, from ftp://ftp.wellfleet.com/netman/psmith/gmake/. You can also get them from this patch list.

A brief description of the problem with the shipped VPATH/vpath functionality follows, then a general discussion of my proposed solution, and finally details on how I implemented the solution in the enclosed patch.

If you use VPATH+, please send me email and I'll put you on my ANNOUNCE mailing list. This list is not accessible by the general public; only I can email to it (it's actually stored in BBDB :), so don't worry about excess traffic.

The only items I email to the list are bug announcements (hasn't been one since Jan, 1992) and new versions of VPATH+.

The Problem

A concise (albeit somewhat obtuse) statement of the problem might be:
Once GNU make finds a target file through VPATH/vpath it changes the target path to the VPATH/vpath name immediately, causing all dependencies of the target to be searched for in the VPATH directory to the exclusion of local files which might be newer.

For example, suppose you have a source tree with stable code built in it. You want to create a local directory, modify one .c file, and build a local copy of the binary to test without modifying anything in the tree. Although this seems like a perfect job for VPATH, this is exactly where the problem arises. To take a very simple example:

Suppose we have a stable tree:

      '' /bar.c
      '' /bar.o
      '' /foo
      '' /foo.c
      '' /foo.o

Where Makefile says:

    VPATH = /tree

    foo: foo.o bar.o

This will build just fine in /tree. Now suppose we want to test a change to bar.c. We go into our working directory and modify a copy of bar.c, so we have:

      '' /bar.c

Now it's fairly obvious what we want to happen when we run make: we want a local copy of bar.o and foo built, but we don't want to have to re-compile foo.c needlessly; it hasn't changed. So after the make what we want is:

      '' /bar.c
      '' /bar.o
      '' /foo

But this isn't what happens. What happens is this:

  1. make examines target foo. It doesn't find it, so it searches VPATH and locates "/tree/foo". make then changes the name of the target to "/tree/foo".

    It's the nature of VPATH that it is only used on relative target pathnames; i.e., ones that don't begin with "/". Thus, we are now hosed :) ... watch:

  2. make then examines the dependencies of foo. It sees foo.o and bar.o. Because the target name is now "/tree/foo", the dependencies are looked for in /tree as well.

  3. /tree/foo.o and /tree/bar.o exist, and are newer than /tree/foo.c and /tree/bar.c, so there's no need to remake anything, and bingo! make says your product is up-to-date and quits.

Step 1 is an example of the problem I'm trying to solve. Because make expands the target to be the VPATH name *before* examining the dependencies, it looks for all further dependencies, and their dependencies, etc. in the same directory the target was found in, so it doesn't notice that there is a local copy of bar.c which is newer than /tree/bar.o and it doesn't even try to rebuild foo.

My Solution

My idea for the proper functioning of VPATH is this:

  1. If the target does not exist locally then it is looked for on VPATH.

  2. If the target is found on VPATH then it is marked as found and its mod time is obtained. The VPATH pathname of the file is stored, but the target name is *not* modified.

  3. All dependencies of the target are examined using this method recursively.

  4. make decides if it needs to rebuild the target:

    1. If the target does not have to be rebuilt, then the name of the target is modified to be the VPATH pathname. This modification is of course reflected in all dependency lists containing this target as well, so that all targets dependent on this file will use the VPATH pathname if they should need to be rebuilt.

    2. If the target must be rebuilt, then the VPATH pathname is thrown away and the target is rebuilt using the local name.

Simply put, if a target does not need to be rebuilt the VPATH version of the target will be used. If it does need to be rebuilt then it will be built locally (in the current working directory), using whatever dependencies were found (local or VPATH).

This seems to be exactly what I want. RMS, Roland and I have batted this around and RMS suggested one additional step which I have not implemented here because I don't need it. Briefly, RMS suggested a way in which to mark, on a per-directory basis, files which should be created *in the tree* rather than locally. I.e., all files which need to be rebuilt and which are found in a marked directory will be built in the marked directory, rather than in the current working directory. If you feel you have a need for this, let me know and we'll talk: so far I don't need it but I have a few ideas on how to implement such a thing if someone else needs it.

If you find problems with the above description please let me know.

Implementation Details

For those who care, this is how I modified GNU make to do the above:

That's it! It has worked just fine for me in my development environment and in a number of tests I whipped up for different things. If you have any problems whatsoever with behavior which you feel is unexpected or nonintuitive W.R.T. VPATH+, please let me know...

Thanks, and happy making!

Paul D. Smith <paul@mad-scientist.net>
Last modified: Mon Jun 19 00:49:45 EDT 2000

Copyright © 1997 Paul D. Smith --- Verbatim copying and distribution is permitted in any medium, provided this notice is preserved.