eccdddfb22cf35d4e5698f72a4b12707aa43b349 braney Sun Sep 20 14:27:27 2026 -0700 testRegistry: check the docent column's "-" as well as its names The docent column says which browser test watches the same ticket, and a named script was already checked: it has to exist and has to belong to that ticket. A "-" was checked by nothing, and that was the wrong half to leave out. `unwatched` -- the list of tickets where nothing anywhere would go red if the bug came back -- is built entirely out of those "-" values. They were entered by hand, by reading the docent directory once, so a script written the next day left the ticket sitting on that queue with nothing to notice. For the four tickets that are on it because their code lives inside a CGI, somebody writing the docent script is the expected outcome, which made it the likeliest kind of row to go stale. So a row claiming no docent script is now checked against the tree the same way a named one is: if rm.*docent.yaml turns up, the check fails and asks for the row to be filled in. The failure is somebody doing the right thing and the table not having heard about it. Watched to fail and then pass: blanking the docent column of #36212, whose script is in the tree, turns the real table red with the name of the script it found. refs #38391 diff --git src/utils/testRegistry/registry.tsv src/utils/testRegistry/registry.tsv index 139c1af3ddb..5a2661d97a1 100644 --- src/utils/testRegistry/registry.tsv +++ src/utils/testRegistry/registry.tsv @@ -7,31 +7,34 @@ # # Unit tests only. The browser-page regression tests are the docent suite, refs #38252, # where the script is already named for its ticket. A ticket whose fix only changes what a # page says is that suite's job and is not in here at all. # # Seven tab separated columns, sorted by ticket then by test: # # ticket the Redmine number, digits only # release the version the fix ships in, digits only, from the ticket's Target version. # The table starts at v504. A fix on master with no target version yet takes # "-" until the ticket says. # test the file to open, as a path from kent/src, or "-" for a ticket that needs a # test and does not have one. A suite whose cases are make targets adds # ::target, e.g. tests/makefile::relPath # docent the browser test that watches the same ticket, from the docent suite in -# hg/utils/docent/tests/regress, or "-" when there is none. Not a second copy +# hg/utils/docent/tests/regress, or "-" when there is none. BOTH values are +# checked: a named script must exist and must belong to this ticket, and a "-" +# must still be true, so a script written later for a ticket whose row says "-" +# fails rather than passing unnoticed. Not a second copy # of that suite: it is here so "nothing is watching this ticket at all" is a # question the tool can answer, which is the question worth acting on. A # docent script does not make a unit test unnecessary where the why is # invisible; rm38309 cannot see a read past the end of an array, it asserts # something next to it. # why why this ticket needs a unit test rather than a browser test: # invisible the fix changes nothing on screen. No browser test can see # it, so a unit test is the only test there can be. # perf the fix is about speed or memory. The test has to catch # backsliding, so it measures work done -- queries, passes, # allocations, bytes -- and never wall-clock seconds. # library the fix is in library code that a browser reaches only # through a page, where much else can go wrong first. # evidence what has been seen, weakest first: unrecorded, assertion-only, sandbox-ab, # release-ab, caught-regression. "-" on a row with no test. The vocabulary is