e8fd1a37d1959a9e73943e946088d1b86d2b1bb4 braney Fri Sep 4 11:17:35 2026 -0700 docent: correct the claim that expect: is the only verb that can fail a run An audit of tests/README.txt against the code found the claim wrong in both READMEs. docent.js turns any step's throw into "step N (verb) failed" and exit 1, so mouseover:, loadSession:, drag:, convert: and go: all fail a run when they cannot do what they were told. Verified by running a script with no expect: step in it at all: it exited 1 from loadSession:. The conclusion drawn from the claim still holds and is kept: expect: is the only verb that CHECKS anything, and track: does not even check its own input, since a name no assembly has is sent as name=mode and the run exits 0 (also verified). Also documents why make derive strips one line of trackDb provenance: the listing is cached for a day, a cold fetch prints a line a warm run does not, and the first run of any day otherwise fails against a baseline captured warm. refs #37892 diff --git src/hg/utils/docent/tests/README.txt src/hg/utils/docent/tests/README.txt index 8d0a5f71c62..bf7ca335cad 100644 --- src/hg/utils/docent/tests/README.txt +++ src/hg/utils/docent/tests/README.txt @@ -2,41 +2,54 @@ ------------ Run by hand, not by the kent tree's `make test`: make test # every *.docent.yaml here make test T=composite # just one make parity # one script FAST and slow, and twice over make derive # the derivation alone, against expected/ (no browser) make derive-accept # rewrite those baselines, then read `git diff expected/` Most tests drive a real browser against a real server, so they need the network and the shared Playwright install (/hive/groups/browser/uiTest/pw; see ../README.md). That is why none of this is part of the tree-wide test target: a broken network would fail the build. A test is an ordinary Docent script that asserts with `expect:`. It passes by exiting -0. `expect:` is the only verb that fails a run, so a test with no `expect:` step in it -tests nothing. +0. `expect:` is the only verb that CHECKS anything, so a test with no `expect:` step in +it tests nothing: `track:` accepts a name no assembly has and still exits 0. + +Other verbs do fail a run, so do not read the line above as "nothing else can stop it". +A verb throws when it cannot do what it was told -- `mouseover:` cannot find the item, +`loadSession:` cannot find the file, `drag:`, `convert:` and `go:` likewise -- and +docent.js turns any step's throw into `step N (verb) failed` and exit 1. None of them +looks at whether the page came out right, which is the part only `expect:` does. A script named *.xfail.docent.yaml is expected to FAIL, and the run fails if it passes. That is how a trap gets pinned rather than merely written down. `make derive` is the cheap half: DOCENT_DERIVE=1 resolves the `track:` steps against trackDb and prints the cart variables without opening a browser, in about a second. It is where Docent's own decisions live, and the baselines in expected/ are what catch a change to visVars() or tdbHideTargets() that a rendered page would hide. +The trackDb listing is cached in $TMPDIR for a day (docent.js, TDB_TTL), and a cold +fetch prints one provenance line that a warm run does not. That line would make the +first `make derive` of the day differ from a baseline captured warm, for a reason that +has nothing to do with trackDb changing, so the makefile strips it from both the run and +the baseline. Everything else trackDb says about itself is kept, including the two lines +that report a hub genome or an unreachable hubApi. + What is covered --------------- selftest session: -> expect: -> loadSession:, on hg38 at SHH. Saves the cart, changes the view, restores it from the local file, checks rows both times. composite clinvar with clinvarCnv hidden: the two-request split (#37953). One request would leave clinvarCnv_sel=1 and the CNV row drawn. views hideKids on the VIEW that holds the subtrack, with the sibling views hidden by name. Also covers the `_sel` checkbox, since the subtrack is `parent wgEncodeRegDnaseSignal off`, and pins the superTrack side effect below. views.xfail the same thing aimed at the COMPOSITE instead, which loses the row. Expected to fail. supertrack varsInPubs hideKids + one member: `exact: true`, because a test that only checked the member was present would pass with all six drawn.