31dd19d6395d10ee6632bb2afcf1947f33eff3dd braney Sat Sep 12 16:03:40 2026 -0700 docent: ten regression tests for hgTrackUi, refs #38252 Four scripts here already touched hgTrackUi in passing (rm37389, rm37489, rm38126, rm38272). These are about the page itself: the superTrack configuration page, composite and subtrack configuration, filters, the color override, the parent link, and two bad-input paths. rm20460 the color override is offered on a genePred track and not on a chain track, which is the type restriction 6d78a8e2d72 added rm32263 a composite child's page names its container, and the link reaches the container's own configuration page rm34651 the density-graph options div is densGraphOptions<track> and the old shared id is gone rm35906 Clear filters survives a submit: the cart comes back on All, which is what the button only appeared to do before rm36484 the filterComposite select has no stray <br> inside it rm36668 both the rearrangement and density-graph checkboxes on, submitted together, does not crash hgTracks rm36917 the superTrack page's Hide all / Show all / Apply to all controls, and Show all leaving no child hidden rm37130 under noParentConfig both filters are on the jaspar child page and neither is on the parent rm37282 Hide all greys the superTrack's own dropdown, and a child's Hide does not force the container back to show rm37743 a dup_1_refGene request with nothing in the cart errAborts with a message naming it, instead of taking a SIGSEGV All ten are assertion-only. make test is 67 of 67 green: the suite went from 57 scripts to 67 for 14 extra seconds, 10m32s in total, because hgTrackUi draws no image and most of these never leave it. README.txt gains the four things the batch settled. No track image means rows: is unavailable and a positive text: is mandatory, since a crash hands the browser an empty document where every noText: passes. Most hgTrackUi bugs ARE the markup, so naming the id or class the commit changed is the right check here, unlike on hgTracks. A cart round trip is the only way to tell a control that works from one that looks right. And Docent cannot pick an option from a select, so a visibility goes in through the URL and a button is clicked wherever one exists. Two candidates were rejected: #38192's missing-track message is live but the ticket is still Reviewing, and #37840's "add db= to links" is not true of every link on the page today, so a blanket assertion would fail for a reason that is not a bug. diff --git src/hg/utils/docent/tests/regress/README.txt src/hg/utils/docent/tests/regress/README.txt index df58a6e4934..68bf0cced2c 100644 --- src/hg/utils/docent/tests/regress/README.txt +++ src/hg/utils/docent/tests/regress/README.txt @@ -1,185 +1,218 @@ Docent regression tests ----------------------- One script per fixed bug, named for its ticket. Run by hand: make test # every *.docent.yaml here make test T=rm36382 # just one make proof # what evidence each script has, and the tally The candidate list this directory is being built from, with the recipe and assertion worked out for each ticket, is at /hive/groups/browser/redmineNotes/37892/claude/2026-09-04_1100_regression_candidates.md What these are, and what they are not ------------------------------------- Each script asserts the behavior the ticket says is correct, on genome-test. Most were written after the fix had already shipped, so most have never been seen to fail for the reason they exist. `make proof` says exactly how many have and which ones, reading a `proof:` key that every script carries; as of 2026-09-10 it is 4 of 37. That is a deliberate choice about cost, and for the other 33 it puts the whole weight on how tight the assertion is: * name the error string the ticket quoted in `noText:`, not a generic "Error" * prefer `rows: [...] exact: true` and `noRows:` over a bare `rows:` * a test that only checks a row is PRESENT usually passes on the buggy build too, because the bug was an extra row, a wrong label, or a bad tooltip Three things will rot these tests --------------------------------- Most recipes start from the saved session named in the ticket, because that is the cheapest way to reach the exact state. A session that is deleted does not fail loudly: hgTracks serves a page saying it could not find it, and every `noText:` check on that page passes. So a session-based test also asserts something that is only true when the session really loaded. Six recipes need a test hub on a colleague's public_html. Same problem, same remedy. Fixtures we own live in ~/public_html/docentFixtures/, and `make preflight` checks that every hub a script here names still answers. Copy a reporter's hub in there rather than loading theirs, so nothing outside this repository can change what a test measures. A fixture hub must never name a track anything the assembly might also call it. A track name resolves to `img_data_<name>` first and only then to a hub row's `hub_<n>_<name>`, so an exact native id wins: the hub row is on the page, and every `track:`, `mouseover:` and `rows:` in the script reads the NATIVE row instead. Nothing warns. rm35920's fixture called its track `ultras`, hg38 has its own `ultras`, and that script asserted a tooltip off the native data for as long as it existed -- it looked green and tested nothing. Prefix a fixture's track names with the ticket number. Proof: which scripts have been watched to fail for their own reason ------------------------------------------------------------------- Every script carries a top-level `proof:` key, one quoted line per piece of evidence, each `<level> <YYYY-MM-DD> -- <what was seen>`. docent.js reads only the keys it names, so the key costs a run nothing. `make proof` tallies it and fails on a line that is malformed or names a level outside the vocabulary, which is what keeps it countable. The levels, weakest first: assertion-only asserts the fixed behavior; never seen to fail for its own reason xfail seen failing right now for its own reason; the fix has not shipped sandbox-ab seen failing on a build with the bug and passing on a build with the fix, both built by hand server-flip seen failing then passing on a real server as a real build arrived caught-regression went red for a regression that was then filed and fixed Two ways to earn the middle levels. sandbox-ab is the one you can choose to do: build the fix into a ticket sandbox, point a copy of the script at that port with `target: http://127.0.0.1:PORT/cgi-bin`, and record which checks flipped. It costs one build and it settles what a tight assertion can only argue. server-flip is the one this directory gets for free, and it is better evidence, because nothing about the server changed except the build. Commit a script for an unshipped fix as an .xfail. `make test` fails when an xfail PASSES, so the morning the fix reaches genome-test the nightly goes red and says so. nightly.sh appends that to /hive/users/braney/docentNightly/flips.log one line per script ever, outside the checkout because --update resets the tree. Then drop the .xfail from the name and add the server-flip line to the script's proof: key. rm38272, rm36212 and rm38310 all arrived that way. rm36212 is still the one to read before writing another -------------------------------------------------------- It is the worked example of both routes: sandbox-ab on 2026-09-09 against parked #36212 on port 48099, then server-flip the same morning when cbb406cd96e reached genome-test. It is also the first script to assert a COLOR, using `expect: {color: ...}`, because it is the first bug here that leaves the page identical -- same rows, same height, same item names, same tooltips. When rows:, height:, text: and has: are all blind to a bug, the pixels are what is left. See tests/colorchecks.docent.yaml for the check itself. rm38310 is the second color check, and the second script watched both ways --------------------------------------------------------------------------- Same recipe as rm36212, and worth reading for the reason it needs pixels, which is different. Its bug does not draw the wrong color; it replaces the row with the bigWarn bar, 240,240,180 (undefinedYellowColor, hg/hgTracks/simpleTracks.c), and paints an error message INSIDE the png. So the row is still drawn, still the same name, and every text check on the page passes -- the message is in the image, where noText: cannot reach it. That is also why the ticket was filed saying there was no warning at all. Two things fall out of it that apply to any script here: * `rows:` cannot express "this track drew its items". The broken build draws the row. `color:` with `is:` on the item color and `not: "240,240,180"` can, and a failure prints what each row really came out. * A drawn item that cannot be clicked through is half a bug. rm38310 clicks its item and asserts the item's POSITION on the hgc page, because the aborted hgc page carries the track's longLabel twice in its own header and a text: check on that alone passes on it. Measured both ways on 2026-09-09: the whole directory was run against the #38310 ticket sandbox twice, once with the patched hgTracks and hgc and once with unpatched controls built from the same tree. Thirty-seven scripts, identical verdicts, except this one. Ten scripts for multi-region view, and the two traps they hit -------------------------------------------------------------- rm22144, rm23922, rm26772, rm27855, rm29452, rm29787, rm30833, rm34250, rm35472 and rm37175 are one batch, written 2026-09-12, and between them they cover the four modes (exon, custom regions, alt haplotype, exit), the dialog, the custom-region BED reader, hideEmptySubtracks across windows and highlights in both directions across the mode change. Before them the only script here that entered multi-region at all was rm35580, which uses singleAltHaplo to reach a different bug. Two things learned writing them, both of which cost a red run first: **Never assert on a `title` attribute.** hgTracks' own tooltip code moves a title into `data-tooltip` once the page's JavaScript has run, so `area[title="chr1:10001-11000"]` matches nothing in the live DOM even though the server sent exactly that. The server writes both attributes on a map box; assert `data-tooltip`. The same applies to the buttons, where the title changes with the mode and would otherwise be a second, free assertion -- it is not available. **Multi-region is reachable from the URL, and the dialog is not.** `virtModeType=`, `multiRegionsBedInput=` (the textarea's own cart variable, newlines as %0A), `singleAltHaploId=`, `virtWinFull=on` and `<composite>.hideEmptySubtracks=on` all work on a `goto:`, which is how nine of the ten set their state -- Docent has no verb that types into an arbitrary field, so the textarea and the alt-haplotype input cannot be filled. What still needs the real dialog is anything the page's JavaScript decides: rm29452's disabled radio and its status line are invisible to curl, because the server sends the same HTML on a build with the bug and a build without it. `virtWinFull=on` is worth knowing for a third reason: without it a region change lands zoomed in on one region, so a second region is off screen and a script cannot tell a region that failed to resolve from one that is merely not in view. Ten more for quickLift, five on hgTracks and five on hgc ---------------------------------------------------------- rm36048, rm36059, rm36125, rm36370, rm36942, rm37646, rm37815, rm38032, rm38042 and rm38146 are one batch, written 2026-09-12. Fourteen scripts here already lifted something (they are the ones that call `convert: {quicklift: true}`); these add the parts of the lift that had no test: the order tracks come out in, an item bigger than the chains quickLift loads, the spanned-item merge, a lolly subtrack, the hide-target-defaults checkbox, and five details pages -- GENCODE archive, hgGene, NCBI RefSeq, the Alignment Differences description, and the same page with a GenArk assembly as the SOURCE. Each one costs a convert, which is about 17 seconds: hgConvert plus a hub build plus the click through to the browser. Budget for that before adding more. Three things worth reusing from them: **The lift is set up through the UI and read from the map.** There is no URL that makes a quickLift hub, so every script here does `convert:` then `open: lift`. What comes back carries a per-run `hub_<n>_` prefix on every row id and every map box, so `rows:` matches by suffix and a `has:` selector has to use a substring (`area[href*="clinvarSubLolly"]`), never an exact id. **Do not assert a count that a data update can move.** rm38042 and rm36048 both read the spanned-item merge box, and the tooltip on it counts the items merged -- 45 for ClinVar on 2026-09-12. That number is reloaded by an otto cron every month. Both scripts assert that the box is THERE (`area[data-tooltip^="Merged "]`) and leave the count to a comment, so a red morning is news about quickLift rather than about ClinVar. **A details page carries the track's own labels, so name something else.** rm36125 asserts SHH's N-terminal peptide, rm36059 a UniProtKB section, rm36370 two section headings that were missing, rm38146 the query sequence read out of a two bit file. Each of those is absent from the page the ticket was filed about and present on the fixed one; the track name and longLabel are on both. + +Ten for hgTrackUi, and what makes that page testable +------------------------------------------------------ + +rm20460, rm32263, rm34651, rm35906, rm36484, rm36668, rm36917, rm37130, rm37282 and +rm37743 are one batch, written 2026-09-12. Four scripts here already touched hgTrackUi in +passing (rm37389, rm37489, rm38126, rm38272); these are about the page itself: the +superTrack configuration page, composite and subtrack configuration, filters, the color +override, the parent link, and two bad-input paths. + +They are also the cheapest scripts in the directory -- one to three seconds each, because +hgTrackUi draws no image and most of them never leave it. + +**There is no track image, so `rows:` is not available and a positive `text:` is +mandatory.** A crash gives the browser an empty document, where every `noText:` and every +`noHas:` passes. Every script here names something the real page says. + +**Most of what hgTrackUi does is in ids, names and classes, so `has:`/`noHas:` carries +these tests.** README says to reach for a selector last, and that is still right for +hgTracks, where rows, height, text and color can usually say it instead. On a settings +page the bug often IS the markup: a shared id that should be per-track (rm34651), a stray +tag inside a select (rm36484), a control that should not be offered for this track type +(rm20460), a class that greys a dropdown (rm37282). Name the id or class the commit +changed, and say in the header which one it is. + +**A cart round trip is what tells a control that works from one that only looks right.** +rm35906's clear-filters button set every dropdown to All on screen on the buggy build too; +only submitting and coming back shows whether anything was saved. rm36668 does the same in +reverse, checking after the fact that the two checkboxes it clicked really are on. + +**A dropdown cannot be driven.** Docent has no verb that picks an option from a select, so +a visibility is set on the way in through the URL (rm36668) and a button is clicked +instead where one exists (rm36917, rm37282).