e8fd1a37d1959a9e73943e946088d1b86d2b1bb4 braney Fri Sep 4 11:17:35 2026 -0700 docent: correct the claim that expect: is the only verb that can fail a run An audit of tests/README.txt against the code found the claim wrong in both READMEs. docent.js turns any step's throw into "step N (verb) failed" and exit 1, so mouseover:, loadSession:, drag:, convert: and go: all fail a run when they cannot do what they were told. Verified by running a script with no expect: step in it at all: it exited 1 from loadSession:. The conclusion drawn from the claim still holds and is kept: expect: is the only verb that CHECKS anything, and track: does not even check its own input, since a name no assembly has is sent as name=mode and the run exits 0 (also verified). Also documents why make derive strips one line of trackDb provenance: the listing is cached for a day, a cold fetch prints a line a warm run does not, and the first run of any day otherwise fails against a baseline captured warm. refs #37892 diff --git src/hg/utils/docent/tests/README.txt src/hg/utils/docent/tests/README.txt index 8d0a5f71c62..bf7ca335cad 100644 --- src/hg/utils/docent/tests/README.txt +++ src/hg/utils/docent/tests/README.txt @@ -1,90 +1,103 @@ Docent tests ------------ Run by hand, not by the kent tree's `make test`: make test # every *.docent.yaml here make test T=composite # just one make parity # one script FAST and slow, and twice over make derive # the derivation alone, against expected/ (no browser) make derive-accept # rewrite those baselines, then read `git diff expected/` Most tests drive a real browser against a real server, so they need the network and the shared Playwright install (/hive/groups/browser/uiTest/pw; see ../README.md). That is why none of this is part of the tree-wide test target: a broken network would fail the build. A test is an ordinary Docent script that asserts with `expect:`. It passes by exiting -0. `expect:` is the only verb that fails a run, so a test with no `expect:` step in it -tests nothing. +0. `expect:` is the only verb that CHECKS anything, so a test with no `expect:` step in +it tests nothing: `track:` accepts a name no assembly has and still exits 0. + +Other verbs do fail a run, so do not read the line above as "nothing else can stop it". +A verb throws when it cannot do what it was told -- `mouseover:` cannot find the item, +`loadSession:` cannot find the file, `drag:`, `convert:` and `go:` likewise -- and +docent.js turns any step's throw into `step N (verb) failed` and exit 1. None of them +looks at whether the page came out right, which is the part only `expect:` does. A script named *.xfail.docent.yaml is expected to FAIL, and the run fails if it passes. That is how a trap gets pinned rather than merely written down. `make derive` is the cheap half: DOCENT_DERIVE=1 resolves the `track:` steps against trackDb and prints the cart variables without opening a browser, in about a second. It is where Docent's own decisions live, and the baselines in expected/ are what catch a change to visVars() or tdbHideTargets() that a rendered page would hide. +The trackDb listing is cached in $TMPDIR for a day (docent.js, TDB_TTL), and a cold +fetch prints one provenance line that a warm run does not. That line would make the +first `make derive` of the day differ from a baseline captured warm, for a reason that +has nothing to do with trackDb changing, so the makefile strips it from both the run and +the baseline. Everything else trackDb says about itself is kept, including the two lines +that report a hub genome or an unreachable hubApi. + What is covered --------------- selftest session: -> expect: -> loadSession:, on hg38 at SHH. Saves the cart, changes the view, restores it from the local file, checks rows both times. composite clinvar with clinvarCnv hidden: the two-request split (#37953). One request would leave clinvarCnv_sel=1 and the CNV row drawn. views hideKids on the VIEW that holds the subtrack, with the sibling views hidden by name. Also covers the `_sel` checkbox, since the subtrack is `parent wgEncodeRegDnaseSignal off`, and pins the superTrack side effect below. views.xfail the same thing aimed at the COMPOSITE instead, which loses the row. Expected to fail. supertrack varsInPubs hideKids + one member: `exact: true`, because a test that only checked the member was present would pass with all six drawn. urllen {cCREs: hideKids} must not become the 1701-variable, 42,020-character GET that Apache answered with 414. Checks `noText: "Too Long"`, since a 414 renders as a perfectly good page; the derive baseline pins it at 3. customtrack addCustomTrack: with inline BED, tabs and newlines surviving the trip. scale a 3x run draws the same rows as a 1x one. ordered `ordered: true` on rows:, and the fact that a row which was not drawn is reported by rows: alone rather than failing the order check as well. ordered.xfail the same two rows named the wrong way round. Expected to fail -- a flag that cannot fail is not a check, it is a second copy of the set test. expectfail an assertion that is plainly false. Expected to fail -- if it ever passes, .xfail `expect:` has stopped throwing and every other test here means nothing. make parity FAST vs slow, and a rerun, on composite. FAST drops the dwells and the recording and must not change what the page ends up showing; the rerun catches state left behind in the cart. Two things these tests found ---------------------------- Worth knowing before writing more: * Turning on anything under a superTrack sends `<superTrack>=show`, and every OTHER member then comes up at its own trackDb visibility -- so `hide: all` is undone for them. views asserts wgEncodeRegMarkH3k27ac comes back, rather than working around it. Whether Docent should be cleverer here is an open question, not a settled one. * `hideKids` on a VIEW has to enumerate its leaves (a view holds no sub-containers to stop at), so views sends 188 variables in a 6,986-character request. That is under Apache's 8,190 limit with less room than is comfortable. Its `noText: "Too Long"` is what turns a future overflow into a clear failure instead of a strange one. Still to write -------------- mouseover: by item: on stacked items, and the timing case where a neighbour's tooltip is still up on arrival pinShot: several tooltips in one figure, cursors drawn convert: quickLift onto a GenArk haplotype, hideDefaults re-checked -- note a session taken after it cannot be checked in, see #38046 drag: each of then: zoom / highlight / cancel addHub:, addPublicHub: the two hub attach paths (a stable hub URL is the hard part) montage: panel order, lettering, a named shot that was never taken goShow: the suggestion menu, including a `pick:` that matches nothing loadSession: the three remote forms -- only the local-file form is covered the YAML lint `{item:name}` with no space warns and drops the argument. This needs a test that reads stderr, which the harness does not do yet. A test that needs a stable server-side fixture (a hub, a custom track) should carry it in the script rather than assume something on disk.