5f47cb8c328b76c41a9a21599477bfacf4d91a37
braney
  Sun Sep 6 16:57:16 2026 -0700
makefiles: let the compiler write the header dependencies

The tree had 72 hand-written "foo.o: bar.h" lines across 15 makefiles, so
almost every object was rebuilt only when its own .c file changed.  Editing a
header left every other object that included it holding the old layout, and the
crash landed somewhere the change never touched.

The %.o: %.c rule in inc/common.mk now passes -MMD -MP.  The compiler writes
foo.d beside foo.o listing the headers that compile really read, and an
-include reads them back.  Nine makefiles keep a compile rule of their own,
because they add -DGBROWSE, -DGFSERVER_HUGE or -DCGI_BIN=; each got ${DEPGEN}
too.  lib and hg/lib are the only two that build objects into a subdirectory,
and each reads its own subdirectory .d files at the foot of its own file.

Touching hg/hgTracks/wigCommon.h used to rebuild 2 objects.  It now rebuilds
11, which is every .c file in that directory that includes the header.

A .d file holds rules, and make takes its default goal from the first rule it
reads, included files and all.  common.mk is read before a makefile's own
rules, so the include has to save $(.DEFAULT_GOAL) and set it back afterwards;
without that, make in lib built adjacency.o and stopped.

Fifteen link rules in directories the build enters named a library on the
command line without depending on it.  Each now lists it.  hg/hgPhyloPlace,
hg/visiGene/hgVisiGene and hg/orthoMap were the three whose target is a real
file and could go stale.

make clean still leaves the .d files.  A shared clean:: rule in common.mk would
first mean converting about 320 single-colon clean: rules, since make refuses
to mix the two forms on one target.  A leftover .d cannot break a build: -MP
writes an empty target for each header, so a deleted or renamed one does not
leave make asking for a file no rule can build.

refs #36621

diff --git src/inc/common.mk src/inc/common.mk
index 0a78054e177..28f1e85b702 100644
--- src/inc/common.mk
+++ src/inc/common.mk
@@ -522,31 +522,61 @@
     endif
 endif
 
 #ENCODE COMMON VARIABLES
 CONFIG_FILES = \
 	fields.ra \
 	labs.ra
 CV = cv.ra
 CVDIR=${HOME}/kent/src/hg/makeDb/trackDb/cv/alpha
 PIPELINE_PATH=/hive/groups/encode/dcc/pipeline
 CONFIG_DIR = ${PIPELINE_PATH}/${PIPELINE_DIR}/config
 ENCODEDCC_DIR = ${PIPELINE_PATH}/downloads/encodeDCC
 
 CC_PROG_OPTS = ${COPT} ${CFLAGS} ${HG_DEFS} ${LOWELAB_DEFS} ${HG_WARN} ${HG_INC} ${XINC}
 %.o: %.c
-	${CC} ${CC_PROG_OPTS} -o $@ -c $<
+	${CC} ${CC_PROG_OPTS} ${DEPGEN} -o $@ -c $<
+
+# Generated header dependencies.  Every makefile in the tree includes this file,
+# so the two lines below give the whole tree the rebuild rule that hand-written
+# "foo.o: bar.h" lines only ever covered a few objects of.  refs #36621
+#
+# -MMD writes foo.d next to foo.o, listing every header that compile actually
+# read, and the -include below feeds those back to make.  -MP adds an empty
+# target for each of those headers, so deleting or renaming a header does not
+# leave make asking for a file no rule can build.  System headers are left out
+# (-MMD rather than -MD) because they do not change between builds here.
+#
+# Objects are built next to their source, so the current directory is the whole
+# of it here.  The two makefiles that put objects in a subdirectory, lib and
+# hg/lib, pick those up themselves at the foot of their own file.  ${wildcard}
+# is evaluated when the makefile is read, which is the right time: a .d written
+# during this run belongs to an object this run just compiled from scratch, so
+# there is nothing stale to catch.
+#
+# Turn it off for one build with "make DEPGEN=".
+#
+# The save and restore around the include is not decoration.  A .d file holds
+# rules, and make takes its default goal from the first rule it sees, included
+# files and all.  Without this, "make" in lib/ built adjacency.o and stopped,
+# because that was the first line of the first .d file.  Setting .DEFAULT_GOAL
+# back to what it was (usually nothing, since common.mk is read before the
+# makefile's own rules) hands the choice back to the makefile.
+DEPGEN = -MMD -MP
+kentSavedGoal := $(.DEFAULT_GOAL)
+-include $(wildcard *.d)
+.DEFAULT_GOAL := $(kentSavedGoal)
 
 # autodetect UCSC installation of node.js:
 ifeq (${NODEBIN},)
     NODEBIN = /cluster/software/src/node-v22.19.0-linux-x64/bin
     ifeq ($(wildcard ${NODEBIN}),)
 	NODEBIN=
     endif
 endif
 
 # node.js tools: jshint, jsx, jsxhint, uglifyjs, ...
 ifeq (${JSHINT},)
     JSHINT=${NODEBIN}/jshint
     ifeq ($(wildcard ${JSHINT}),)
 	    JSHINT=true
     endif