5011b739391b6a1b6129d473b6c050010baf1f6f braney Sat Sep 19 17:57:37 2026 -0700 testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391 diff --git src/utils/testRegistry/tests/input/bad.tsv src/utils/testRegistry/tests/input/bad.tsv new file mode 100644 index 00000000000..7caed7d444e --- /dev/null +++ src/utils/testRegistry/tests/input/bad.tsv @@ -0,0 +1,16 @@ +# Every complaint check can make, one per row, so a change to check that stops making one +# of them shows up as a diff rather than as silence. +#ticket release test docent why evidence note +37263 504 lib/tests/pathSimplifyTest.c - library assertion-only fine +37263 504 lib/tests/pathSimplifyTest.c - library assertion-only the same row twice +38000 504 lib/tests/noSuchTest.c - library assertion-only the file is not in the tree +38001 v504 lib/tests/pathSimplifyTest.c - library assertion-only the release column carries the v +38002 504 lib/tests/pathSimplifyTest.c - library watched-it no such evidence level +38003 504 lib/tests/pathSimplifyTest.c - slow assertion-only no such reason +38004 504 lib/tests/pathSimplifyTest.c - library assertion-only +38005 504 - - perf assertion-only no test, but a claim of evidence anyway +38006 504 lib/tests/pathSimplifyTest.c - library assertion-only this one says covered +38006 504 - - library - while this one says waiting +38000 504 lib/tests/pathSimplifyTest.c - library assertion-only out of order, so the sort check fires +38007 504 lib/tests/pathSimplifyTest.c rm38007.docent.yaml library assertion-only names a docent script that is not there +38008 504 lib/tests/pathSimplifyTest.c rm38303.docent.yaml library assertion-only names another ticket's docent script