bbe654a9dc89bc7b75d40d1fe197d5376d756580
braney
  Thu Sep 17 17:08:10 2026 -0700
docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252

One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969,
rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248,
rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303
and rm38309.  No theme -- they run from a menu-bar color to a SIGSEGV in a
quickLift view.  Three fixtures came with them, under
~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the
field count on the type line), rm38283 (one description page full of dollar
variables) and rm38184 (a one-line session settings file).

Every one was then run against v503, which is what the new proof level is for.
v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since
three of the fixes live in hg/js or htdocs/style -- and twenty-one of the
twenty-four fix commits landed after that branch was cut, so the release has the
bugs.  Eighteen scripts failed there on their own check and carry a release-ab
line quoting the failure.

release-ab sits between sandbox-ab and server-flip.  It is stronger than
sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree
lives in one person's home directory and does not survive a rebase.

The four that v503 does not cover are each said in their own header: rm38171's
fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws
nothing there, so the run never reaches the check; rm38257's login: dies against
a park on any baseline; and rm38309 PASSES, which is exactly what its header
claims, since that fix changes no byte of any page.

rm38251 had to be rewritten because of what the sweep said.  It PASSED on a build
carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px
left but was still inside the bar, and the clear: check named the links, where
the icon fell in a gap between two of them.  Naming the menu list instead has no
gaps.  Without the sweep it would have sat here looking green.

make proof: 89 scripts, 33 watched to fail for the reason they exist and then
pass, up from 13.

diff --git src/hg/utils/docent/tests/proof.js src/hg/utils/docent/tests/proof.js
index b998d0cfef8..5d60c820d8c 100755
--- src/hg/utils/docent/tests/proof.js
+++ src/hg/utils/docent/tests/proof.js
@@ -5,30 +5,31 @@
  * bug it was written for, and tallies it.
  *
  * The problem it solves. A regression test written after the fix asserts the behavior
  * the ticket says is correct, but nobody has seen it fail for the reason it exists. A
  * loose assertion in that state is indistinguishable from no test at all, and a reader
  * a year from now cannot tell which scripts have been proven and which are hopeful.
  * That evidence was being recorded in prose in each script's header, where counting it
  * meant a grep and reading, so the claim "these are regression tests" could not be made
  * out loud with a number behind it.
  *
  * So every script carries a top-level `proof:` key, a list of one line per piece of
  * evidence, each beginning with a level token from LEVELS below and a date:
  *
  *     proof:
  *       - "sandbox-ab 2026-09-09 -- ticket sandbox on 48099, patched and control builds"
+ *       - "release-ab 2026-09-17 -- fails on v503_branch 707b184e329, passes on genome-test"
  *       - "server-flip 2026-09-10 -- 68f831e1209 reached genome-test, the xfail passed"
  *
  * QUOTE every line. Nearly all of them name a ticket, and a bare `#` in an unquoted YAML
  * scalar starts a comment, so the note is silently cut off at the ticket number and the
  * evidence disappears from the middle of the sentence. That is checked for below rather
  * than left as a trap.
  *
  * docent.js reads only the keys it names off the parsed document, so `proof:` costs it
  * nothing and is never executed. Nothing here drives a browser either: this reads the
  * yaml and prints, so it is instant and can go in a commit message.
  *
  * Exit 1 if any proof line is malformed or names a level that is not in the vocabulary,
  * because a vocabulary nobody enforces drifts into free text within a month, and free
  * text cannot be tallied. A script with no `proof:` key at all is NOT an error -- the
  * tests next door are not regression tests and have no bug to have proven -- but it is
@@ -36,30 +37,32 @@
  */
 'use strict';
 const fs = require('fs');
 const path = require('path');
 const yaml = require('js-yaml');
 
 // Weakest first. The order is the point: it is what `make proof` sorts and sums by, and
 // what tells you whether the directory is getting stronger.
 const LEVELS = [
   ['assertion-only',
    'asserts the fixed behavior; never seen to fail for the reason it exists'],
   ['xfail',
    'seen failing right now for the reason it exists; the fix has not shipped'],
   ['sandbox-ab',
    'seen failing on a build with the bug and passing on a build with the fix, both built by hand'],
+  ['release-ab',
+   'seen failing on a RELEASED version that predates the fix and passing on one that carries it'],
   ['server-flip',
    'seen failing then passing on a real server as a real build arrived'],
   ['caught-regression',
    'went red for a regression that was then filed and fixed'],
 ];
 const RANK = new Map(LEVELS.map(([n], i) => [n, i]));
 
 // Same shape as preflight.js: DIR first, then optional bare script names, because the
 // nightly hands both of them the same committed list.
 const DIR = process.argv[2] || '.';
 const named = process.argv.slice(3).map(s => s.replace(/\.docent\.yaml$/, ''));
 
 const all = fs.readdirSync(DIR).filter(f => f.endsWith('.docent.yaml')).sort();
 let files = all;
 if (named.length) {