6f3f3596f3dc61a8a738717b9701e69a86b5bb0b
braney
  Mon Sep 21 17:24:33 2026 -0700
docent: wait: {gone:} for a selector to leave, and rm38257 stops racing itself

rm38257 went red in the 2026-09-21 nightly with the #38257 fix still live on
genome-test.  The script was waiting on the wrong half of the click.

hgHubConnect.js switches the tab synchronously inside the click dispatch, while
topLinks.js closes the Account popup from a setTimeout(..., 0).  So the tab is
the half that settles FIRST, and waiting for it returned a tick early: the
expect after it read a page that still had the popup on it.  Measured with a
MutationObserver over the click, the tab goes active at t=43ms and the popup
goes on the next task.

wait: now also takes {gone: <selector>}, which waits for a selector to leave the
DOM, so a script can wait on the vanishing half of an answer.  rm38257 uses it.

Six runs each way on genome-test, same build, only the script differing: 6 of 6
green with the new wait:, 3 of 6 red with the old one, failing at the same step
with the same message the nightly printed.  The whole directory is green, 98 of
98.

refs #37892, refs #38252

diff --git src/hg/utils/docent/README.md src/hg/utils/docent/README.md
index b741b51f634..14e5d93455f 100644
--- src/hg/utils/docent/README.md
+++ src/hg/utils/docent/README.md
@@ -164,31 +164,39 @@
 | `hub: https://example.org/hub.txt` | **Quick, silent** attach of a track hub by URL (`hgTracks?hubUrl=...`): connects the hub so its tracks are available at their hub-declared visibility. Follow with `track:` to turn specific ones on. Map form `hub: {url: ..., db: hg38, position: chr7:...}` overrides the db/position (default: current `db` + last position). |
 | `addHub: https://example.org/hub.txt` | **Demonstrates the attach through the UI** (for the figure/video): opens My Data → Track Hubs, clicks the **Connected Hubs** tab, types the URL into the box, and clicks **Add Hub** — cursor glides and the URL is typed on screen. Then, on the "Hub Connect Successful" page, it **clicks the `Open:` link for `db`** so the demo ends on the browser with the hub loaded. Map form `addHub: {url: ..., db: hg38, shot: loaded}` sets which assembly to open and captures the still on that tracks view. Use `hub:` instead when you just need the hub attached without showing the steps. |
 | `addCustomTrack: <text-or-url>` | **Demonstrates loading a custom track via the UI**: opens My Data → Custom Tracks, types the track data (or a data URL) into the paste box, clicks **Submit**, then clicks through to the browser (**Go to first annotation**). Bare string is the data or URL; map form `addCustomTrack: {data: "track ...\nchr7 ...", db: hg38, goto: first, shot: loaded}` (use `url:` for a data URL, `goto: current` to land on **Return to current position** instead). Data is inserted literally (tabs/newlines preserved). In YAML, a multi-line track uses a block scalar: `addCustomTrack: |` then the indented lines. |
 | `addPublicHub: GTEx` | **Demonstrates connecting a PUBLIC hub via the UI**: opens My Data → Track Hubs, the **Public Hubs** tab, types the search terms, clicks **Search Public Hubs**, then clicks **Connect** on the matching hub row, and finally **clicks the `Open:` link for `db`** to land on the browser. Bare string is the search term (also used to match the row). A search usually returns several hubs, so use the map form `addPublicHub: {search: "GTEx", match: "GTEx Analysis Hub", db: hg38, shot: loaded}` to pick the exact hub by a substring of its row text (`match:` defaults to `search:`) and the assembly to open. If no row matches it **won't connect** (warns and stops) rather than pick the wrong hub. |
 | `drag: chr7:155,805,900-155,806,950` | Emulates the **Shift+drag-select** gesture: the cursor sweeps across the selection (a visible selection box is drawn) and the browser's own drag-select dialog is raised, then a button is clicked. The argument is one genomic region, `chrom:start-end`; a bare range **zooms**. For any other action, or to pass other keys, put the region under `range:` and quote it — unquoted commas split a `{..}` flow map: `drag: {range: "chr7:155,805,900-155,806,950", shot: dragselect, then: highlight}`. Endpoints that are not genomic coordinates are given as a fraction (`fromFrac:`/`toFrac:`) or raw px (`fromX:`/`toX:`) instead. Optional `track:` picks the row the box is drawn over; default is the top of the image. `shot: dragselect` captures the open dialog. `then:` = `zoom` (default → **Zoom In**) \| `highlight` (→ Single Highlight) \| `cancel` (Escape, view unchanged). (A real button-held drag would just pan, so the dialog is driven directly.) |
 | `open: lift` | Click the returned coordinate link → the lifted view. |
 | `zoom: out` / `zoom: in` | One zoom step (2×). |
 | `montage: {name: figure1, shots: [source, lifted]}` | Compose stills already written this run into **one multi-panel PNG**, which is what a journal wants for a figure with parts (A), (B), and so on. Panels are stacked in order and lettered automatically; `labels: [Before, After]` overrides the letters, `labels: false` drops them, `direction: horizontal` puts them side by side, and `gap:` / `labelSize:` tune the spacing and lettering. Composed at deviceScaleFactor 1 with every panel at its **natural pixel size**, so the composite is pixel-for-pixel its inputs: a `make hires` montage is print resolution because the panels were, not because anything was upscaled. Panels narrower than the widest are left-aligned and padded, never stretched. A named shot that was never taken is warned about and skipped. Put it last, after the `shot:`/`pinShot:` steps it names. |
 | `login` | **Sign in through hgLogin**, for the pages that refuse a visitor who is not logged in -- hgCollection above all (`hgCollection.c` doMiddle: *You must be logged in to edit collections*), and the saving half of hgSession. The login cookie is validated against a salted hash (`login.cookieSalt`, `hg/lib/wikiLink.c`), so there is no cookie to hand the browser: a script that needs one of those pages has to sign in the way a person does. **The step takes no credentials and cannot be given any.** They are read from `~/.docentLogin` (override with `DOCENT_LOGIN_FILE`), **one section per hgcentral database** -- an account is a row in `gbMembers` in one of them, so that is the key, not the server and not the sandbox:<br><br>`[hgcentraltest]`<br>`user=docentTest`<br>`password=...`<br><br>genome-test, hgwdev, every `hgwdev-<name>` sandbox and every ticket park read hgcentraltest, so one account covers all of them; hgwbeta reads hgcentralbeta and the RR reads hgcentral. Which central a server reads is **read from its hg.conf**, not assumed from the host, because a sandbox can say so for itself -- 45 of the personal confs on hgwdev set `central.db=hgcentraltest` and two do not. A server whose conf is on another machine falls back to a small table (the RR, the two mirrors, hgwbeta), and `[default]` catches the rest. A run redirected with `DOCENT_TARGET` looks up the server it is really driving. The file is refused unless it is mode 0600 -- the rule `hg/lib/hgConfig.c` applies to `hg.conf`. `DOCENT_LOGIN_USER` + `DOCENT_LOGIN_PASSWORD` override the file for one run. Nothing prints a password. A wrong password fails the step rather than carrying on logged out, because hgLogin answers one by drawing the same form again, which is a perfectly good page. Map form `login: {shot: signed_in}`. `make preflight` reports the account and the central it resolved, so a missing password is caught before the browser starts. |
 | `loadSession: https://example.org/settings.txt` | Start from a **saved state** instead of a clean cart, so one tour can begin where another ended and a bug report that arrives as a session link becomes a starting position. Four forms: a **settings file by URL** (as above), a **share link** (`loadSession: https://genome.ucsc.edu/s/Braney/hg38`), a **named session** (`loadSession: {user: Braney, name: hg38}`), or a **local file** written by an earlier `session:` (`loadSession: {file: saved}` → `sessions/<base>/saved.txt`, sent up through hgSession's own upload form, so the project's sessions need not be published at all). Quick and silent, like `hub:` — this is setup, not something the tour demonstrates; add `shot:` to capture where it lands. Whatever the form, the load is issued against `target:` — see **Sessions** for why a share link is not simply followed. |
 | `expect: {rows: [ruler, mane]}` | **The one verb that can fail a run.** Everything else renders happily whatever it is handed, so a wrong figure is written over a right one and only an eye catches it. State the expectation instead and the run stops, non-zero, at the step that broke it. Checks, any combination: `rows:` (these were drawn — plain names, matched by suffix so a lifted `hub_<n>_mane` counts), `exact: true` (…and nothing else), `ordered: true` (…and in that order, top to bottom), `noRows:` (these were not), `height: 2000` (the still is no taller than that in pixels; `"<1200"`, `">=300"` for another comparison), `tip: "mismatch A->C"` (the tooltip now up says this, as a SUBSTRING -- an item's tooltip is markup and its tail can render differently from the title it came from), `noTip:` (...and does not say this, one string or a list. It is the delimiter `tip:` has no other way to carry: a wiggle's tooltip is a bare number, so `tip: "1"` is also satisfied by "1.5" and by "13", and naming the other digits and the decimal point leaves one number), `text:` / `noText:` (the page does / does not contain this, one string or a **list** of them — `noText: "Too Long"` catches the Apache 414 that renders as a perfectly good page), `url:` / `noUrl:` (the current address does / does not contain this — which CGI a click reached, or what a form put in the query string; `noUrl: "%E2%80%8B"` is the only way to see that a search term's zero-width space was stripped, since it is invisible in the page), `color:` (the color a track's row is actually **drawn** in -- `{track: crm4, is: "0,0,255"}`, or `not:` for one it must not be; `part: label` asks about the center label instead of the items, `at:`/`frac:`/`x:` about one item instead of the whole row, and a **list** states several rows in one step. The only check that reads the IMAGE, for a bug that leaves the page identical -- same rows, same height, same names, same tooltips), `has:` / `noHas:` (a CSS selector matches / matches nothing — for a bug whose whole signature is where something sits in the page's TREE, like a center label attached to the wrong row: same rows, same height, same pixels. Reach for these last, since an assertion on hgTracks' own ids breaks easily for reasons that are not bugs), `box:` (where an element sits on the SCREEN — `{sel: "#topRightLinks", inside: "#main-menu-whole"}`, plus `clear:` for what it must not overlap and `height:`/`width:` for its own size. The only check that reads a bounding box, for a bug that leaves every selector matching and every word of the page in place). A failure names every check that failed **and the rows actually drawn**. `warn: true` downgrades it to a warning for a check worth logging but not worth stopping a build over. |
 | `session: source` | Write `sessions/<base>/<name>.txt`: the **whole cart at this step**, in the format hgSession's "save settings to a local file" produces, so anyone can load it and get this exact view. Every track's visibility, the attached hubs, the custom tracks, the window. Off the video and off the page — it is fetched over the tour's own cookies, so the tour is not disturbed and nothing appears in the mp4. With `sessionUrlBase:` set at the top of the file, the run also prints the ready-made load URL. See **Sessions**. |
 | `shot: source` | Write `<name>.png` **and** pause the video here. On a tracks page the still is the track image (`#imgTbl`), plus any open tooltip/dialog. On any other page (an hgc detail page, an external page a link led to) it is the **viewport only — the top of the page**, never the whole scrolling document. |
 
 Escape hatches for anything the verbs don't cover: `goto: <url>`, `click: <sel>`,
-`hover: <sel>`, `wait: <sel>`, `sleep: <ms>`.
+`hover: <sel>`, `wait: <sel>`, `wait: {gone: <sel>}`, `sleep: <ms>`.
+
+`wait:` goes both ways on purpose. A script that asserts what a click did has to wait on
+the half of the answer that settles **last**, and that is not always the half that
+appears -- a handler can do its visible work inside the click dispatch and defer the rest
+to a `setTimeout(..., 0)`. Waiting for the wrong half returns a tick early and the assert
+reads a page that is still mid-answer, which arrives as a script that passes about half
+the time. `{gone:}` waits for a selector to leave the DOM, so the vanishing half can be
+the one waited on. tests/regress/rm38257.docent.yaml is the worked example.
 
 ## Sessions
 
 A `shot:` gives a picture of the view. A `session:` gives the view itself:
 
 ```yaml
 sessionUrlBase: https://hgwdev-you.gi.ucsc.edu/~you/docent/sessions
 steps:
   - hide: all
   - track: {clinvar: pack}
   - session: clinvar_view
   - shot: clinvar_view
 ```
 
 That writes `sessions/<base>/clinvar_view.txt` and prints the URL that loads it: