67efa330d2830c30741f9524abfd72b59f8791f8
braney
  Fri Sep 4 11:17:46 2026 -0700
docent: share the test rules, add tests/regress, and stop an empty suite passing

Moves the test rules out of tests/makefile into tests/docentTest.mk so a second
directory of Docent tests runs the same code rather than a copy. tests/makefile
and the new tests/regress/makefile each set DOCENT and include it.

tests/regress/ is for one script per already-fixed bug, asserting the behavior
its ticket says is correct, to be run nightly against genome-test (#38252). It
is kept out of tests/ so that suite stays short enough to run before a commit:
measured, its eleven scripts take 56 seconds.

Two fixes to the rules while they moved:

- make derive strips "trackDb: N tracks for DB from .../hubApi" from both the
run and the baseline. docent.js caches the trackDb listing for a day and
prints that line only on a cold fetch, so the first make derive of any day
reported CHANGED against a baseline captured warm. Verified cold and warm,
and make derive-accept reproduces the three committed baselines byte for byte.
- make test with no *.docent.yaml, and make derive with no baseline, now fail
instead of printing "docent tests passed". An empty tests/regress reported a
pass before this.

refs #37892 #38252

diff --git src/hg/utils/docent/tests/regress/README.txt src/hg/utils/docent/tests/regress/README.txt
new file mode 100644
index 00000000000..ef9b0f72abf
--- /dev/null
+++ src/hg/utils/docent/tests/regress/README.txt
@@ -0,0 +1,36 @@
+Docent regression tests
+-----------------------
+
+One script per fixed bug, named for its ticket. Run by hand:
+
+    make test               # every *.docent.yaml here
+    make test T=rm36382     # just one
+
+The candidate list this directory is being built from, with the recipe and assertion
+worked out for each ticket, is at
+
+    /hive/groups/browser/redmineNotes/37892/claude/2026-09-04_1100_regression_candidates.md
+
+What these are, and what they are not
+-------------------------------------
+
+Each script asserts the behavior the ticket says is correct, on genome-test. None of them
+was run against a build that still had the bug, so none has been seen to fail for the
+reason it exists. That is a deliberate choice about cost, and it puts the whole weight on
+how tight the assertion is:
+
+  * name the error string the ticket quoted in `noText:`, not a generic "Error"
+  * prefer `rows: [...] exact: true` and `noRows:` over a bare `rows:`
+  * a test that only checks a row is PRESENT usually passes on the buggy build too,
+    because the bug was an extra row, a wrong label, or a bad tooltip
+
+Two things will rot these tests
+-------------------------------
+
+Most recipes start from the saved session named in the ticket, because that is the
+cheapest way to reach the exact state. A session that is deleted does not fail loudly:
+hgTracks serves a page saying it could not find it, and every `noText:` check on that
+page passes. So a session-based test also asserts something that is only true when the
+session really loaded.
+
+Six recipes need a test hub on a colleague's public_html. Same problem, same remedy.