7aba31f14aed7f2620e4746b54f9569a5c93e1ad braney Sat Sep 12 15:07:34 2026 -0700 docent: ten regression tests for quickLift, on hgTracks and on hgc, refs #38252 Fourteen scripts here already lift something -- they are the ones that call `convert: {quicklift: true}` -- so these take the parts of the lift that had no test. Five read the lifted image and five read a details page: rm38032 the target keeps the source's track order. First use of `ordered:`, which was added to expect: for this bug rm38042 a ClinVar CNV running past the chains quickLift loads is clipped rather than dropped, so the spanned-item merge still has it rm37646 a lolly composite subtrack lifts, and its map boxes still carry its own track name -- the string the stale pop pointer clobbered rm36048 the spanned-item merge still works on a lifted DECIPHER track rm37815 "Hide all default tracks on the target" hides all six of hs1's own tracks and keeps the lifted one rm36059 a lifted GENCODE Versions item gives the real details page, in destination coordinates, with no "Can't start query" rm36370 a lifted knownGene click renders GeneReviews and Methods, the two sections the ticket says were missing rm36125 a lifted RefSeq item's page, and its Predicted Protein link returning SHH's peptide instead of a blank page rm36942 the Alignment Differences description, reached from a difference item: the four colors and the figure rm38146 the same page with a GenArk assembly as the SOURCE, down to the base alignment that reads query bases out of a two bit file All ten are assertion-only: every fix shipped long ago. make test is 57 of 57 green in 10m18s, up from 7m34s -- each script costs a convert, about 17 seconds, because no URL builds a quickLift hub. README.txt gains what the batch cost. A lifted row and map box carry a per-run hub__ prefix, so rows: matches by suffix and a has: selector must use a substring. Never assert a count an otto reload can move: rm38042 and rm36048 both read the merged-item box and leave its count (45 for ClinVar today) to a comment. And a details page prints the track's own labels whether or not it worked, so each hgc assertion names something only the fixed page has. Three candidates were rejected: #38033's "(N items could not be lifted)" label is only in the drawn image and the page JSON, where no expect: check reaches it; #37970 needs a broadPeak track and hg38 has none; #37974's center-label drag is pixels. diff --git src/hg/utils/docent/tests/regress/README.txt src/hg/utils/docent/tests/regress/README.txt index 5ad966e8844..df58a6e4934 100644 --- src/hg/utils/docent/tests/regress/README.txt +++ src/hg/utils/docent/tests/regress/README.txt @@ -1,151 +1,185 @@ Docent regression tests ----------------------- One script per fixed bug, named for its ticket. Run by hand: make test # every *.docent.yaml here make test T=rm36382 # just one make proof # what evidence each script has, and the tally The candidate list this directory is being built from, with the recipe and assertion worked out for each ticket, is at /hive/groups/browser/redmineNotes/37892/claude/2026-09-04_1100_regression_candidates.md What these are, and what they are not ------------------------------------- Each script asserts the behavior the ticket says is correct, on genome-test. Most were written after the fix had already shipped, so most have never been seen to fail for the reason they exist. `make proof` says exactly how many have and which ones, reading a `proof:` key that every script carries; as of 2026-09-10 it is 4 of 37. That is a deliberate choice about cost, and for the other 33 it puts the whole weight on how tight the assertion is: * name the error string the ticket quoted in `noText:`, not a generic "Error" * prefer `rows: [...] exact: true` and `noRows:` over a bare `rows:` * a test that only checks a row is PRESENT usually passes on the buggy build too, because the bug was an extra row, a wrong label, or a bad tooltip Three things will rot these tests --------------------------------- Most recipes start from the saved session named in the ticket, because that is the cheapest way to reach the exact state. A session that is deleted does not fail loudly: hgTracks serves a page saying it could not find it, and every `noText:` check on that page passes. So a session-based test also asserts something that is only true when the session really loaded. Six recipes need a test hub on a colleague's public_html. Same problem, same remedy. Fixtures we own live in ~/public_html/docentFixtures/, and `make preflight` checks that every hub a script here names still answers. Copy a reporter's hub in there rather than loading theirs, so nothing outside this repository can change what a test measures. A fixture hub must never name a track anything the assembly might also call it. A track name resolves to `img_data_` first and only then to a hub row's `hub__`, so an exact native id wins: the hub row is on the page, and every `track:`, `mouseover:` and `rows:` in the script reads the NATIVE row instead. Nothing warns. rm35920's fixture called its track `ultras`, hg38 has its own `ultras`, and that script asserted a tooltip off the native data for as long as it existed -- it looked green and tested nothing. Prefix a fixture's track names with the ticket number. Proof: which scripts have been watched to fail for their own reason ------------------------------------------------------------------- Every script carries a top-level `proof:` key, one quoted line per piece of evidence, each ` -- `. docent.js reads only the keys it names, so the key costs a run nothing. `make proof` tallies it and fails on a line that is malformed or names a level outside the vocabulary, which is what keeps it countable. The levels, weakest first: assertion-only asserts the fixed behavior; never seen to fail for its own reason xfail seen failing right now for its own reason; the fix has not shipped sandbox-ab seen failing on a build with the bug and passing on a build with the fix, both built by hand server-flip seen failing then passing on a real server as a real build arrived caught-regression went red for a regression that was then filed and fixed Two ways to earn the middle levels. sandbox-ab is the one you can choose to do: build the fix into a ticket sandbox, point a copy of the script at that port with `target: http://127.0.0.1:PORT/cgi-bin`, and record which checks flipped. It costs one build and it settles what a tight assertion can only argue. server-flip is the one this directory gets for free, and it is better evidence, because nothing about the server changed except the build. Commit a script for an unshipped fix as an .xfail. `make test` fails when an xfail PASSES, so the morning the fix reaches genome-test the nightly goes red and says so. nightly.sh appends that to /hive/users/braney/docentNightly/flips.log one line per script ever, outside the checkout because --update resets the tree. Then drop the .xfail from the name and add the server-flip line to the script's proof: key. rm38272, rm36212 and rm38310 all arrived that way. rm36212 is still the one to read before writing another -------------------------------------------------------- It is the worked example of both routes: sandbox-ab on 2026-09-09 against parked #36212 on port 48099, then server-flip the same morning when cbb406cd96e reached genome-test. It is also the first script to assert a COLOR, using `expect: {color: ...}`, because it is the first bug here that leaves the page identical -- same rows, same height, same item names, same tooltips. When rows:, height:, text: and has: are all blind to a bug, the pixels are what is left. See tests/colorchecks.docent.yaml for the check itself. rm38310 is the second color check, and the second script watched both ways --------------------------------------------------------------------------- Same recipe as rm36212, and worth reading for the reason it needs pixels, which is different. Its bug does not draw the wrong color; it replaces the row with the bigWarn bar, 240,240,180 (undefinedYellowColor, hg/hgTracks/simpleTracks.c), and paints an error message INSIDE the png. So the row is still drawn, still the same name, and every text check on the page passes -- the message is in the image, where noText: cannot reach it. That is also why the ticket was filed saying there was no warning at all. Two things fall out of it that apply to any script here: * `rows:` cannot express "this track drew its items". The broken build draws the row. `color:` with `is:` on the item color and `not: "240,240,180"` can, and a failure prints what each row really came out. * A drawn item that cannot be clicked through is half a bug. rm38310 clicks its item and asserts the item's POSITION on the hgc page, because the aborted hgc page carries the track's longLabel twice in its own header and a text: check on that alone passes on it. Measured both ways on 2026-09-09: the whole directory was run against the #38310 ticket sandbox twice, once with the patched hgTracks and hgc and once with unpatched controls built from the same tree. Thirty-seven scripts, identical verdicts, except this one. Ten scripts for multi-region view, and the two traps they hit -------------------------------------------------------------- rm22144, rm23922, rm26772, rm27855, rm29452, rm29787, rm30833, rm34250, rm35472 and rm37175 are one batch, written 2026-09-12, and between them they cover the four modes (exon, custom regions, alt haplotype, exit), the dialog, the custom-region BED reader, hideEmptySubtracks across windows and highlights in both directions across the mode change. Before them the only script here that entered multi-region at all was rm35580, which uses singleAltHaplo to reach a different bug. Two things learned writing them, both of which cost a red run first: **Never assert on a `title` attribute.** hgTracks' own tooltip code moves a title into `data-tooltip` once the page's JavaScript has run, so `area[title="chr1:10001-11000"]` matches nothing in the live DOM even though the server sent exactly that. The server writes both attributes on a map box; assert `data-tooltip`. The same applies to the buttons, where the title changes with the mode and would otherwise be a second, free assertion -- it is not available. **Multi-region is reachable from the URL, and the dialog is not.** `virtModeType=`, `multiRegionsBedInput=` (the textarea's own cart variable, newlines as %0A), `singleAltHaploId=`, `virtWinFull=on` and `.hideEmptySubtracks=on` all work on a `goto:`, which is how nine of the ten set their state -- Docent has no verb that types into an arbitrary field, so the textarea and the alt-haplotype input cannot be filled. What still needs the real dialog is anything the page's JavaScript decides: rm29452's disabled radio and its status line are invisible to curl, because the server sends the same HTML on a build with the bug and a build without it. `virtWinFull=on` is worth knowing for a third reason: without it a region change lands zoomed in on one region, so a second region is off screen and a script cannot tell a region that failed to resolve from one that is merely not in view. + +Ten more for quickLift, five on hgTracks and five on hgc +---------------------------------------------------------- + +rm36048, rm36059, rm36125, rm36370, rm36942, rm37646, rm37815, rm38032, rm38042 and +rm38146 are one batch, written 2026-09-12. Fourteen scripts here already lifted something +(they are the ones that call `convert: {quicklift: true}`); these add the parts of the +lift that had no test: the order tracks come out in, an item bigger than the chains +quickLift loads, the spanned-item merge, a lolly subtrack, the hide-target-defaults +checkbox, and five details pages -- GENCODE archive, hgGene, NCBI RefSeq, the Alignment +Differences description, and the same page with a GenArk assembly as the SOURCE. + +Each one costs a convert, which is about 17 seconds: hgConvert plus a hub build plus the +click through to the browser. Budget for that before adding more. + +Three things worth reusing from them: + +**The lift is set up through the UI and read from the map.** There is no URL that makes a +quickLift hub, so every script here does `convert:` then `open: lift`. What comes back +carries a per-run `hub__` prefix on every row id and every map box, so `rows:` matches +by suffix and a `has:` selector has to use a substring (`area[href*="clinvarSubLolly"]`), +never an exact id. + +**Do not assert a count that a data update can move.** rm38042 and rm36048 both read the +spanned-item merge box, and the tooltip on it counts the items merged -- 45 for ClinVar on +2026-09-12. That number is reloaded by an otto cron every month. Both scripts assert that +the box is THERE (`area[data-tooltip^="Merged "]`) and leave the count to a comment, so a +red morning is news about quickLift rather than about ClinVar. + +**A details page carries the track's own labels, so name something else.** rm36125 asserts +SHH's N-terminal peptide, rm36059 a UniProtKB section, rm36370 two section headings that +were missing, rm38146 the query sequence read out of a two bit file. Each of those is +absent from the page the ticket was filed about and present on the fixed one; the track +name and longLabel are on both.