cad0600bd259bae35de7b39e49c2f95af8b624ca braney Mon Sep 14 16:14:40 2026 -0700 docent: point a whole test directory at another server with TARGET, refs #37892 A script says where it runs with `target:`, and until now that was the only way to say it, so trying a suite against a branch build meant editing every script in the directory. DOCENT_TARGET overrides `target:` for the run, and docentTest.mk turns a TARGET= variable into it for preflight, test, parity and derive: make test TARGET=hgwdev-demo9 make preflight TARGET=hgwdev-demo9 It takes the same values `target:` does -- a shorthand, a bare hgwdev-<name> sandbox or demo, or a full .../cgi-bin URL -- so a ts park works too. preflight.js reads the same variable, so the fixture check names the server that will actually be driven rather than the one the scripts name. The trackDb cache already keys on the resolved server, so a redirected run cannot read back a listing fetched from somewhere else. The committed scripts keep their own `target:`. Redirecting is for trying a suite elsewhere, not for moving it: the nightly reads what is in the file. Both READMEs say so, and say to read a redirected failure with the other server's trackDb in mind, since a script asserts what its own server draws. Verified against the ten methbase scripts: all ten pass with TARGET=hgwdev-demo9, and mbMouse still passes with no TARGET, against the RR. diff --git src/hg/utils/docent/tests/README.txt src/hg/utils/docent/tests/README.txt index e6d356abb28..c3d117deba9 100644 --- src/hg/utils/docent/tests/README.txt +++ src/hg/utils/docent/tests/README.txt @@ -1,26 +1,43 @@ Docent tests ------------ Run by hand, not by the kent tree's `make test`: make test # every *.docent.yaml here make test T=composite # just one make parity # one script FAST and slow, and twice over make derive # the derivation alone, against expected/ (no browser) make derive-accept # rewrite those baselines, then read `git diff expected/` + make test TARGET=hgwdev-demo9 # the same scripts, against another server + +TARGET overrides the `target:` each script carries, for every target above, and takes +the same values it does: a shorthand (rr, genome-test, hgwdev, hgwbeta), a bare +hgwdev-<name> sandbox or demo, or a full .../cgi-bin URL. It is how you try a suite +against a branch build -- a sandbox, a ticket park from `ts`, a demo browser -- without +editing the scripts. `make preflight TARGET=...` checks that server rather than the one +the scripts name, so the fixture check and the run agree. + +Read a redirected run's failures with the server in mind. A script asserts what its OWN +server draws, so a red one somewhere else can be the other machine's trackDb rather than +a bug: a demo sandbox that carries only one assembly fails every script on the others, +and a sandbox trackDb with a track the RR has not released changes what `exact: true` +counts. Redirecting is for trying a suite elsewhere, not for moving it: the committed +scripts stay pointed at the server they were written against, which is the one the +nightly reads. + Most tests drive a real browser against a real server, so they need the network and the shared Playwright install (/hive/groups/browser/uiTest/pw; see ../README.md). That is why none of this is part of the tree-wide test target: a broken network would fail the build. A test is an ordinary Docent script that asserts with `expect:`. It passes by exiting 0. `expect:` is the only verb that CHECKS anything, so a test with no `expect:` step in it tests nothing: `track:` accepts a name no assembly has and still exits 0. Other verbs do fail a run, so do not read the line above as "nothing else can stop it". A verb throws when it cannot do what it was told -- `mouseover:` cannot find the item, `loadSession:` cannot find the file, `drag:`, `convert:` and `go:` likewise -- and docent.js turns any step's throw into `step N (verb) failed` and exit 1. None of them looks at whether the page came out right, which is the part only `expect:` does. A script named *.xfail.docent.yaml is expected to FAIL, and the run fails if it passes.