Commits for braney
switch to files view, user index
v503_base to v504_preview (2026-08-31 to 2026-09-07) v504
Show details
4037b582757f86eed1c5559eca9ccc7af85c1496 Fri Aug 21 10:00:49 2026 -0700
- lib: add htmlSanitize, an allowlist filter for HTML written elsewhere, refs #38126
htmlSanitize() takes a piece of HTML and returns a copy holding only the
elements, attributes and style properties on its lists. An element on the
keep list survives with its allowed attributes. A short list of elements
that carry nothing for a reader, script and style and form among them, is
dropped along with its contents. Every other element loses its tag and
keeps its text, so a whole document that somebody saved and pasted in comes
out as the article it was meant to be.
The lists come from a survey of all 5390 description pages reachable from
the public hub list, so they are sized to what hubs actually write. The
style attribute is filtered a property at a time, and href and src are
checked for a scheme we do not print, after decoding entities and padding.
An iframe is kept only when it plays a video from one of a few hosts, and
then with a sandbox attribute.
htmlSanitizeReport() returns the same copy plus a list of one-line messages
naming what came out, for hubCheck to show a hub author.
The tokenizer is hand written and forgiving. It never aborts and always
returns something, because the HTML it will be handed is often broken.
lib/htmlPage.c cannot be reused for this: its parser aborts on bad input.
- src/inc/htmlSanitize.h - lines changed 25, context: html, text, full: html, text
- src/lib/htmlSanitize.c - lines changed 766, context: html, text, full: html, text
- src/lib/tests/expected/htmlSanitizeTest - lines changed 53, context: html, text, full: html, text
- src/lib/tests/htmlSanitizeTest.c - lines changed 63, context: html, text, full: html, text
f759bf663debc22e6015c6c38c22f922bfbdf45b Fri Aug 21 10:01:02 2026 -0700
- lib, cgilib: sanitize description HTML where it comes in, refs #38126
A track hub's track description, a custom track's documentation, and an
assembly hub's genome description are all written by somebody else and were
stored and printed as they arrived. Run each through htmlSanitize() at the
point it is read, so that everything downstream sees text we are willing to
print. Sanitizing on the way in is the only practical place: printTrackHtml()
alone has more than a hundred callers, and hgTrackUi, hui.c, hgGene,
hgGtexTrackSettings and hgTables all print tdb->html directly.
The five places are trackHubAddOneDescription(), customTrack.c where it used
to call jsStripJavascript(), the htmlUrl fetch in customFactory.c,
hgPositionsHelpHtmlCart() and hAssemblyDescription(). In the last two, only
the branch that fetches over the network is touched. The branch that reads a
local file is left as it is, because those are our own description.html files.
trackHubAddOneDescription() keeps its old shape. The fetch moves into a
helper so that trackHubDescriptionRemovals() can hand hubCheck the list of
things the filter takes out.
- src/hg/cgilib/cartJson.c - lines changed 5, context: html, text, full: html, text
- src/hg/lib/customFactory.c - lines changed 5, context: html, text, full: html, text
- src/hg/lib/customTrack.c - lines changed 2, context: html, text, full: html, text
717a40d56723cebacd7cbb8c240f9c2e72a4160a Fri Aug 21 10:01:12 2026 -0700
- hubCheck: name the parts of a description page we do not print, refs #38126
Now that a hub description is filtered on the way in, a hub author could
find out about it from a page that comes out looking wrong. Warn instead,
one line per thing removed, up to ten, then a count of the rest.
Only things a reader would notice are named: an element dropped with its
contents, an event attribute, a link with a scheme we do not print, a style
value holding url() or expression. A class attribute or an unlisted style
property goes quietly. Over the 5390 pages of the public hubs, 118 draw at
least one of these warnings.
- src/hg/utils/hubCheck/hubCheck.c - lines changed 17, context: html, text, full: html, text
f83fd4fd350727eb3a42a709849bab0c4658ab84 Fri Aug 21 10:17:16 2026 -0700
- htmlSanitize: three fixes from a review of it, refs #38126
Read the scheme of an href or src the way a browser arrives at it. A browser
turns a numeric character reference into its character before it decides what
the scheme is, and it accepts one written with any number of leading zeros and
with no closing semicolon at all. Decode those the same way, then insist that
what stands in front of the first slash is either a plain scheme we allow or a
plain path. A named entity, a backslash or a control character in that part of
the value means we do not print the link, because those are the ways the check
gets walked around. This keeps the 33 encoded mailto links two hubs write, and
they were the only links in the public hubs the plainer rule would have lost.
Treat a trailing slash on a kept element as the nothing that HTML says it is.
Otherwise <div/> came out as an open div and the rest of our own page sat
inside it.
Remember when the search for a closing tag has run off the end of the input.
Every later search for that same tag runs off the end too, so a page made of
two hundred thousand unclosed tags no longer costs one pass over the page each.
The unit test grows a case for each of the three.
- src/lib/htmlSanitize.c - lines changed 118, context: html, text, full: html, text
- src/lib/tests/expected/htmlSanitizeTest - lines changed 19, context: html, text, full: html, text
- src/lib/tests/htmlSanitizeTest.c - lines changed 9, context: html, text, full: html, text
8983998334c36e7dfefe608b9a4effe3a10ae9ab Fri Aug 21 10:21:02 2026 -0700
- hubCheck: give chopByChar the number of slots, not the size of the array, no redmine
splitMessages holds sixteen pointers, and the call passed sizeof(splitMessages),
which is the byte count. A track that gathers more than sixteen warning lines
therefore wrote past the end of it. Twenty filterByRange settings with no
matching filter setting is enough to do it, and hgHubConnect runs hubCheck with
-htmlOut on any hub address somebody types in. Pass ArraySize instead.
The mistake dates from 702afb4c465 in 2020.
- src/hg/utils/hubCheck/hubCheck.c - lines changed 1, context: html, text, full: html, text
71366eb92d89bbb2cf87e2a9293ea019b63981a4 Fri Aug 21 10:22:02 2026 -0700
- htmlSanitize: keep odd characters out of the messages we hand back, refs #38126
An attribute name in HTML can hold almost anything, and the name goes into a
message that hubCheck prints to a terminal. Name the attribute only when it
reads like a name.
- src/lib/htmlSanitize.c - lines changed 13, context: html, text, full: html, text
fcf433a736a8670c9da0e2af3ac54db24b0aaa3b Mon Aug 24 14:47:47 2026 -0700
- htmlSanitize: rename the ids that come in with the HTML, refs #38126
The description HTML is printed inside a page of ours, and it is our own
JavaScript that looks ids up. A page that carries an id we already use puts
two elements of that name in one document, and getElementById returns whichever
comes first. Two ids in the public hub pages collide with ours today, one of
them "content". An id also becomes a property of that name on window, which
reaches the guards we write as "typeof X !== 'undefined' && X".
Every id, and every name on an anchor, now gets the prefix descPage- . The
hyphen means the window property it makes can never be spelled as a JavaScript
name, so it cannot stand in for one of our globals either. A link to a name on
the same page, href="#x", is rewritten with the same prefix and keeps working.
A link that names another page is left alone: it leaves our page for the hub's
own file, where the names are unchanged.
Measured over the 5390 description pages of the public hubs: 1643 ids and
anchor names renamed, no id left without the prefix, and the number of same
page links with nothing to point at is 22 before and 22 after, the same ones.
No page loses reader-visible text and the hubCheck warnings are unchanged at
118 pages, since a reader sees nothing of this.
- src/lib/htmlSanitize.c - lines changed 30, context: html, text, full: html, text
- src/lib/tests/expected/htmlSanitizeTest - lines changed 5, context: html, text, full: html, text
- src/lib/tests/htmlSanitizeTest.c - lines changed 4, context: html, text, full: html, text
9f5a3871b95f595b5a218f0d05395a8c72d58844 Tue Aug 25 12:57:10 2026 -0700
- remove the bigBedOnePath fallback, leaving a single bigBed load path, refs #36940
bigBedOnePath has defaulted to on since v492 and the setting has been removed
from the hg.conf of the RR, Euro, Asia and hgwbeta. Drop the check and the code
it guarded at all four sites: the field-count default in
bigBedAddLinkedFeaturesFromExt, the method setup in commonBigBedMethods, the
bedSize default in bigBedClick, and the bigBed case in hgc. The two branches in
the hgc case differed only in a minimum the surviving path does not want.
complexBedMethods lost its only isBigBed=TRUE caller with that branch, so drop
the parameter.
Retire the setting from the hg.conf catalog and its gate backlog, and repair the
two hgc.c citations whose line numbers this commit shifted, refs #37925.
Verified pixel-identical: 14 scenarios rendered before and after, covering the
four QA sessions on the ticket plus mm10 knownGene, dbSnp155, rmsk,
encRegTfbsClustered, clinvar and tandemDups. All AE=0.
- src/hg/hgTracks/bedTrack.c - lines changed 5, context: html, text, full: html, text
- src/hg/hgTracks/bigBedTrack.c - lines changed 50, context: html, text, full: html, text
- src/hg/hgTracks/hgTracks.h - lines changed 1, context: html, text, full: html, text
- src/hg/hgTracks/simpleTracks.c - lines changed 4, context: html, text, full: html, text
- src/hg/hgc/bigBedClick.c - lines changed 2, context: html, text, full: html, text
- src/hg/utils/hgConfCatalog/hgConfCatalog.py - lines changed 7, context: html, text, full: html, text
- src/hg/utils/hgConfCatalog/hgConfGateBacklog.txt - lines changed 1, context: html, text, full: html, text
f001e39d1561c86eb6832c556888e73dc200f8d0 Thu Aug 27 14:10:09 2026 -0700
- quickLift: let a second lift update a track already in the hub, refs #38198
The hub writer skipped any track whose name was already in the hub file. The
first lift writes a container together with all of its children, including the
ones switched off, so those names were taken from then on. A later lift skipped
the container and everything inside it: a subtrack switched on after the first
lift never reached the target, and a visibility change on a track that had
already been lifted never reached it either.
walkTree no longer takes the set of names already in the file and no longer
skips a track for being there. It writes every visible track into a dyString,
and the new writeMergedHubFile merges that against the file: a generated stanza
replaces the old stanza of the same track, in the slot the old one held, so
parents stay ahead of their children; a track in the file that was not generated
this time is kept as it was, so tracks still accumulate across lifts in one
session. The skip could not simply be dropped because a duplicate track name
makes hub loading abort, and replacing in place keeps names unique.
The stanza parser inside quickLiftHubRemoveTrack is now readStanzas, shared by
both paths. A container is many stanzas there, one per track line, which is
what makes the per-track replacement work.
The file is written to a temporary name and renamed into place. Every lift now
rewrites the whole file and the target assembly reads that same file, so an
in-place rewrite could hand a reader a truncated hub. outTrack also frees the
dyString it gets from trackDbString.
- src/hg/lib/trackHub.c - lines changed 169, context: html, text, full: html, text
a6a7e658e255ff09b7128f43b84de3a5ad1d74d2 Tue Aug 25 09:28:08 2026 -0700
- cart.c: resolve a GenArk db= again after a session load, refs #38184
A URL with both db=GCF_... and a session load failed with an unknown
database error. fixUpDb turns db=GCF_... into genome= plus hubUrl=, but
the session load empties the cart and then restores the raw CGI
variables, so the bare accession reached hDbConnect.
Split the GenArk part of fixUpDb into resolveGenarkDb and call that once
more after the session-load block. Only the rewrite runs there, not the
bad-database errAbort, so a session that carries a retired db fails the
way it does now. Covers hgS_doLoadUrl and hgS_doOtherUser.
b0f8840863d5673f41d45932502249141a9e3241 Mon Aug 31 10:00:49 2026 -0700
- cart.c: keep the session position when db= resolves to the same assembly, refs #38184
resolveGenarkDb takes db out of the cart, so the block that runs after
hubConnectLoadHubs saw an empty db and called every GenArk db= a database
change. It then set the old db to "none", which getDbAndGenome reads as a
switch of assembly: the position was replaced with the assembly default and
virtMode, virtModeType and nonVirtPosition were dropped. A session loaded
with db=GCF_... came up at the default position instead of the one it saved.
Only call it a database change when the value the cart had before the CGI
variables were applied names a different database. A session that really is
on another assembly still gets the default position.
d44a9daabcca7a322db6907add7ccac336394c85 Mon Aug 31 12:21:36 2026 -0700
- Merge branch 'onePath36940': remove the bigBedOnePath fallback, refs #36940
Brings in the single bigBed load path. bigBedOnePath has defaulted to on
since v492, so this drops the check and the code it guarded at four sites:
bigBedAddLinkedFeaturesFromExt, commonBigBedMethods, bigBedClick and the
bigBed case in hgc. complexBedMethods lost its only isBigBed=TRUE caller,
so the parameter is gone.
Conflict in hgConfCatalog.py, resolved in favour of master. Master has since
changed every citation to name the file without a line number, so the branch's
two hgc.c line-number repairs are obsolete and are dropped. What survives from
the branch is the part that still applies: the bigBedOnePath row leaves the
catalog and the gate backlog, refs #37925.
Verified after the merge: hgTracks and hgc both build clean under -Wall -Werror,
and hgConfCatalog.py --check reports 0 problems.
- src/hg/utils/hgConfCatalog/hgConfCatalog.py - lines changed 57, context: html, text, full: html, text
0e4e0c0af65eea70f64edbc68348ce0972c4bbf4 Fri Aug 28 16:45:00 2026 -0700
- Correct the track types listed for trackDb settings, and the hub settings list
The "For Types" list in the trackDb docs was wrong for about sixty settings, so
the docs named the wrong track types for settings that have always worked. Most
named only the older type and left out its big* counterpart. The clearest case
is the multiple-alignment family: a bigMaf track is drawn and configured by the
same code as a wigMaf track, but only speciesOrder said so, while irows,
itemFirstCharCase, speciesGroups, speciesCodonDefault, speciesDefaultOff,
treeImage, pairwiseHeight and speciesUseFile all claimed wigMaf alone. The
hapCluster settings said vcf and not vcfTabix. noScoreFilter said bed while its
own example uses type bigBed 6 +. Six settings said "all" for something that
only works on item tracks.
Two documented settings do not exist. pslSequence describes a variable that was
replaced by the baseColor family long before the setting was listed, and nothing
has read either spelling since; it is removed. noStems is renamed to
lollyNoStems, which is what the Browser actually reads. That one mattered:
hubCheck builds its list of valid settings from trackDbHub.html, so it accepted
the spelling that does nothing and rejected the one that works.
Fourteen settings that work in hubs had no entry in the hub spec, so hubCheck
reported them as unrecognized. They are listed now: chainColor,
chainNormScoreAvailable, pairwiseHeight, barChartMatrixUrl, mouseOverFunction,
intronGap, filterBy, baseColorTickColor, speciesGroups, speciesDefaultOff,
speciesCodonDefault, itemFirstCharCase, irows, and canPack with
configureByPopup and origAssembly. The last three, along with filterBy and
baseColorTickColor, were marked "NOT FOR HUBS", which was wrong: the Browser
reads them from a hub's trackDb the same way it reads them from ours.
The type setting on the hub page listed every type the Browser knows, including
ones that only work for tracks loaded into our own databases. It now shows only
the types a hub can use. A hub-specific blurb for this already existed and had
never been referenced.
Three settings had no blurb at all, so the generated trackDbSettings.json never
saw them: metadata, noInherit and useScore. Written, and the five hand-written
copies in trackDbDoc.html that had drifted from the library are brought back
into line.
The library's header told the reader to always check their work in
trackDbTestBlurbs.html, which was deleted in November 2025. It now points at
"make settings" instead. That target regenerates trackDbSettings.yaml and .json,
which are updated here, and its name map gains an entry so the hub-specific type
blurb is still keyed as "type".
refs #37908
- src/hg/htdocs/goldenPath/help/trackDb/changes.html - lines changed 99, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbDoc.html - lines changed 30, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbHub.v3.html - lines changed 60, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml - lines changed 169, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.json - lines changed 448, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.yaml - lines changed 530, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbSettingsGen.py - lines changed 1, context: html, text, full: html, text
0ca1bd9ec1196e15b2ea7cef508bd7f382bb5add Mon Aug 31 13:30:10 2026 -0700
- hgc: reject a bigBed type line that declares fewer than three fields, refs #36940
A bigBed always has chrom, chromStart and chromEnd. bigBedClick trusted the
field count from the type line, so a track hub could declare "type bigBed 1",
"type bigBed 2" or a negative count and reach the item detail page with that
value. Two things then went wrong. The variable length array at the top of the
interval loop was sized from it, which is undefined behaviour for a negative
count. Then restBedFields went negative, so extraFields pointed before the
start of restFields and getExtraFields read off the front of that stack array.
The existing field count check sits twelve lines further down and never got the
chance to stop either one.
Check the count once, right after the zero sentinel has been resolved from the
file, and abort with a message that names the track and the count it declared.
The old path aborted too, just later and with wording that blamed a
disagreement rather than the bad type line.
The check has to come after the zero case is resolved. Zero means "take the
count from the file", which is why the similar minimum in hgc.c was wrong and
was removed earlier in this ticket.
Tested with hubs declaring -5, 1, 2, 3, 6 and no count. The first three now
stop with the new message; the last three are unchanged. Output for the valid
counts is byte identical to a control build from the same tree, and a real
"bigBed 3" track still renders.
- src/hg/hgc/bigBedClick.c - lines changed 7, context: html, text, full: html, text
7007b23dcc178ac0d6a64642ecd5a1060c6f6a40 Thu Aug 27 13:47:54 2026 -0700
- hgTracks: measure how long the track image takes to reach the reader, refs #38109
We cannot choose a png compression level without knowing the reader's
connection speed. Our own logs will not tell us: apache stops timing once the
kernel has the bytes, so its duration field stays near two milliseconds whether
the image is 20 KB or 500 KB.
The reader's browser does know. It keeps a timing record for every image it
loads, holding the bytes taken off the wire and the time waited. This reads
that record for the track image and reports the numbers on the query string of
DOT.gif, a 43 byte image that already sits in htdocs/images. No process
starts, nothing touches the cart, and the client address, the time and the
numbers all land in the same access log we already archive.
New hg.conf setting pngTimingSampleRate. It is the N in one page load in N.
Zero, or the setting left out, turns the whole thing off, which is the default.
The report carries three numbers. ts is the bytes off the wire, d is the whole
fetch, and x is only the time the bytes were arriving. x is the denominator for
throughput, d is what the reader actually waited.
- src/hg/hgTracks/hgTracks.c - lines changed 6, context: html, text, full: html, text
f01c7dc7ea48d00510033d7c9448b20bd9c68e19 Mon Aug 31 13:44:24 2026 -0700
- hgTracks: keep the fast png deliveries in the timing sample, refs #38109
Two fixes from a review of the timing beacon.
The first threw away a real measurement. A download whose responseEnd and
responseStart fall in the same clock tick was treated the same as an image that
never crossed the wire. Firefox rounds resource timing to a millisecond, so a
small track image on a fast link lands there. Dropping those samples would
leave only the slow connections in the record, which is the wrong population to
pick a compression level from. The beacon now sends them and reports x=0.
Take d as the denominator when x is zero.
The second was the beacon image itself. An Image with no reference to it can be
collected before the request goes out. It now lives in a variable that outlives
the function.
c01de718b2fb8e7706a0fede0e5ea4149ac6f378 Mon Aug 31 13:48:13 2026 -0700
- htmlSanitize: six fixes from a second review of it, refs #38126
Filter the style attribute on the text a browser will see. A browser turns a
character reference into the character it names before the CSS parser runs, so
a value spelled url( reached the page as url( and walked past the check
that is there to stop it. The value is decoded before the check now, and the
author is told which property lost its value.
Say something when we drop the rest of the page. A tag that never ends, most
often an attribute value whose quote is never closed, threw away everything
after it and reported nothing, so hubCheck stayed quiet about it. The depth
cap did the same.
Write a less than sign that starts no tag as <. A browser reads text like
"</ " as a comment and swallows markup of ours up to the next '>', which could
eat a closing tag we added.
Write an attribute once. A browser keeps the first of a repeated attribute and
drops the rest. We checked the first and then printed them all, which was only
correct because of that rule.
Do not stack a second descPage- prefix on an id we already renamed, and do not
add a second noopener noreferrer to a rel that already has one. hgCustom hands
the text we returned back to us when a custom track is edited and saved again,
so both of these grew a little more on every save.
The test now re-runs the filter over its own output and prints any case that
changes, so a transform that compounds shows up in the diff.
- src/lib/htmlSanitize.c - lines changed 70, context: html, text, full: html, text
- src/lib/tests/expected/htmlSanitizeTest - lines changed 21, context: html, text, full: html, text
- src/lib/tests/htmlSanitizeTest.c - lines changed 26, context: html, text, full: html, text
1258d7f65e7058408390246f0b592ae00e1b4d26 Mon Aug 31 13:48:21 2026 -0700
- trackHub: keep a description a track inherited from its parent, refs #38126
trackHubAddOneDescription used to return early when a track had no html setting
of its own, which left tdb->html alone. Sanitizing where the text comes in made
it assign every time, so it cleared a description that trackHubAddDescription
had copied down from an ancestor. No caller loses a page over this today,
because each one re-fetches afterward, but the function should not be
destructive.
2c66cf38bdd66fd2c5cb9a5a23de69ca2c24bec5 Mon Aug 31 13:14:15 2026 -0700
- Add the track types that a code read confirms for nine trackDb settings
This is the first part of the Tier B pass on the "For Types" lists. Tier B
holds the settings that real tracks use on a type the docs never mention. Usage
alone is not proof, because a setting can be set on a track and do nothing
there, so each row here was confirmed by finding the code that reads the
setting and showing that it serves the added type.
The barChart family, barChartBars, barChartLabel, barChartMetric and
barChartUnit, listed bigBarChart alone. cfgTypeFromTdb sends both barChart and
bigBarChart to cfgBarChart, and barChartUi.c serves the pair, so barChart is
added to all four.
indelDoubleInsert listed bam. One blurb covers it and indelQueryInsert and
indelPolyA. indelEnabled (hui.c:1586) takes a trackDb and never looks at the
type, and linkedFeaturesDrawAt calls it at simpleTracks.c:4391, so the whole
linked-features family reaches it and not only bamTrack.c. psl and bigPsl are
added: they are the two types with real usage, 3494 and 2338 tracks, and the
hub spec has listed all three settings under bigPsl for years, so the hub page
and the library have disagreed about this.
mouseOver and mouseOverField gain bigLolly, which lollyTrack.c reads at lines
384 and 377. motifPwmTable gains bigBed, read on the bigBed details path at
hgc/bigBedClick.c:584. logoMaf gains wig, read off tg->tdb with no type gate at
wigTrack.c:2029, in a file that serves plain wig as well as bigWig.
trackDbSettings.yaml and .json are regenerated.
refs #37908
- src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml - lines changed 9, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.json - lines changed 12, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.yaml - lines changed 9, context: html, text, full: html, text
744c3f21361148e959397572e346516a81ea52ed Mon Aug 31 13:21:01 2026 -0700
- Replace the dead snp track type with bed in thirteen trackDb settings
Thirteen settings in the library declared the type snp and nothing else. There
is no such track any more. hg38 and hg19 both have zero tracks of that type,
and nothing in the track engine dispatches on it: the only startsWith("snp",
...) left, at cgilib/snp125.c:318, tests the track name rather than the type.
The dbSNP tracks are bigDbSnp and "bed 6 +". So snp is not a missing type here,
it is a type that went away and took thirteen type lists with it.
Every one of them is used on bed today, between 2 and 4047 tracks each, and
each has a reader that a bed track reaches: snp125Ui.c for the ortho tables,
hapmapTrack.c for hapmapPhase, variation.c for defaultMaxWeight, and hgc.c for
the rest.
defaultGeneTracks gets bed and bigDbSnp. Its 16 bigDbSnp tracks are real, not
convention: snp153OfferGeneTracksForFunction reads the setting at hui.c:4923
and bigDbSnpCfgUi calls that function at hui.c:4999. None of the other twelve
is read on a bigDbSnp path.
Two of the thirteen were missing from the earlier survey of this family.
chimpMacaqueOrthoTable is a separate setting from chimpOrangMacOrthoTable and
is read at snp125Ui.c:16. codingAnnoLabel_<table> looked unused because hgc.c
builds the setting name at run time, at hgc.c:20153 inside
printSnpAlleleAndOrthos.
The two remaining snp spans are left alone on purpose. Neither is a setting:
one documents the snp track type itself and already says "type bed 6 +", and
the other is the example track stanza beneath it.
None of these thirteen has a row in the hub spec, which is correct, since they
all name a SQL table or a SQL-backed gene track. So the generated
trackDbSettings.yaml and .json do not change.
refs #37908
- src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml - lines changed 13, context: html, text, full: html, text
92af4b0363b70062f69a5730648b20bd8e52a317 Mon Aug 31 13:34:03 2026 -0700
- Drop bed from searchIndex and searchTrix, and deprecate pslSequence instead of deleting it
This is the reverse of the earlier passes. Instead of asking what a type list
is missing, it asks which declared type nothing supports.
searchIndex and searchTrix both declared bed. Neither works on a SQL bed table.
The trackDb read for searchIndex is at bigBedFind.c:270 and it then needs a bbi
file, from bigDataUrl or from the table's fileName, which a plain bed table does
not have. searchTrix has two readers: bigBedFind.c:312 reads it from trackDb on
the bigBed path, while hgFind.c:2393 reads it from hgFindSpec, which is how a
SQL track gets trix search. Native usage agrees, with zero bed tracks for
either setting. searchIndex also gains bigPsl, bigGenePred and bigBarChart, and
searchTrix gains bigGenePred, all of which have real usage.
pslSequence was deleted outright in 0e4e0c0af65. It is obsolete, but deleting
the row was the wrong way to say so. 118 psl and 6 bigPsl native tracks set it,
and so do three public-hub bigPsl tracks. hubCheck takes its vocabulary from
this page, so with no row those hub authors would be told the setting "is not
recognized. Check for typos", with a spelling suggestion, which is the wrong
advice for a setting they took from our own documentation. The row is back at
level-deprecated, which makes hubCheck say "is deprecated" instead, the way it
already does for canPack, useScore, metadata and noInherit. Verified by running
hubCheck against both spellings of the page. The library blurb is restored too,
rewritten to say the setting is obsolete and to map its three values onto
baseColorDefault, which replaced it in 8d32ba75938 in 2006.
chainMinScore is deliberately left alone. It was proposed for deletion on the
grounds that nothing reads it. Nothing does, but our own automation writes it:
asmHubChainNetTrackDb.pl:101 and chainNetCompositeTrackDb.pl:186 emit
"chainMinScore 5000" into the chainNet composite stanza, and findScores.pl
converts -minScore into it. 40,920 native bed stanzas and 16 chain tracks carry
it today. It records the minScore the chains were built with. Removing its
documentation would leave those stanzas with an undocumented setting.
changes.html gains a row for the type lists corrected across all three of these
commits, and its existing pslSequence row is rewritten: it said the setting was
removed, which is no longer what happened. Since the earlier commit has not
shipped to the RR, no reader ever saw the "Removed" wording.
trackDbSettings.yaml and .json are regenerated, and now hold 263 settings.
refs #37908
- src/hg/htdocs/goldenPath/help/trackDb/changes.html - lines changed 35, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbHub.v3.html - lines changed 3, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml - lines changed 19, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.json - lines changed 21, context: html, text, full: html, text
- src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.yaml - lines changed 25, context: html, text, full: html, text
646595ff1c14e1e575a832fc17ec8e238d7ba2e5 Mon Aug 31 14:01:32 2026 -0700
- Merge branch 'hubSanitize': filter the description HTML a hub sends us, refs #38126
A track hub, a custom track and an assembly description page all hand us HTML
that we print inside a page of ours. htmlSanitize reduces that HTML to an
allowlist of elements, attributes and style properties before we print it. An
element that is not on the keep list loses its tag but keeps its text, so a
whole pasted document still comes out as the article it was meant to be.
Script, style, form and their kin go away with everything inside them.
Every id, and every name on an anchor, is renamed with a fixed prefix, so a
name the outside HTML chose cannot be one our own JavaScript looks up. A link
to a name on the same page is renamed to match.
hubCheck now names the parts of a description page that the Browser will not
print, so a hub author hears it from us rather than from a page that comes out
wrong.
992880730e0f4acdc4ba941906059b0dc2e3ce1d Mon Aug 31 14:50:10 2026 -0700
- hgSession: take the session description back out of the cart, refs #38205
The Sessions page has a Description box. The text stayed in the cart after
the page was done with it, so every session the user saved from then on kept
a copy, including sessions they never described. On hgwdev 41 saved sessions
carry a description that belongs to a different session.
cleanHgSessionFromCart() now removes hgsNewSessionDescription and
hgsNewSessionName. That function already runs in the two places that matter:
at the end of hgSession(), so the user's own cart is clean, and inside
saveCartAsSession() just before the cart is encoded, so no saved session holds
them. Adding the names to excludeVars[] instead would not work, because
doSessionChange() reads the description back out of the cart in the same
request that posts it.
hgsNewSessionShare has the same problem but is left alone. It is also the
only memory of the user's "Allow this session to be loaded by others" choice,
and showSavingOptions() defaults that box to checked, so removing it would
quietly re-check the box for someone who keeps their sessions private.
doNewSession() and doReSaveSession() now clone the session name before use.
Both read it straight out of the cart's hash, and the new cartRemove inside
saveCartAsSession() frees that copy while they still hold it.
- src/hg/hgSession/hgSession.c - lines changed 15, context: html, text, full: html, text
437e6ad4150619a44529edaad85db32ba498ee3a Mon Aug 31 14:52:55 2026 -0700
- Merge branch 'secondLift38198': let a second lift update a track already in the hub, refs #38198
4917e0d56156dc93c4d950c89afc857e2e7fe8f4 Mon Aug 31 15:26:36 2026 -0700
- Merge branch 'sessDesc38205': stop leaving the session description in the cart, refs #38205
cd0d053cdf8a7f53ee89a920eab46353797a90ed Mon Aug 31 15:33:37 2026 -0700
- ts: drop the cookie domain so a parked instance can hold a cart, refs #37867
The shared config sets central.domain=.ucsc.edu, and cartWriteCookie puts that on
the cart cookie. A parked instance answers on loopback, so the browser dropped
the cookie and every request got a fresh cart. Each track then came up at its
trackDb default instead of the setting the run had asked for.
Nothing errored, which is what made it worth fixing rather than documenting. The
page still rendered, so a scripted check measured the defaults and reported a
clean pass. Clicking through a park by hand hid it completely, because hgTracks
puts the hgsid into the links on its own pages; only a run that navigates by
absolute URL has nothing to carry.
setCookieDomain writes an empty central.domain into the frozen hg.conf, which
leaves the domain attribute off the cookie and makes it host-only. That works
whether the instance is reached as localhost or as 127.0.0.1, so neither the
tunnel line nor any existing script has to change. HTTPHOST would not do: it
uses the request's own host, and an IP address has a dot in it, so 127.0.0.1
becomes a real Domain attribute and the browser rejects that too. The login
cookies follow the same setting through getCookieDomainString in wikiLink.c.
It is written the way setUdcDir is, idempotent on a marker comment and called
from both freeze and conf, so "ts conf NNNNN" retrofits an instance frozen
before this existed.
03583f9df6c2c04a7e1072b021bbdd73d5072926 Mon Aug 31 15:36:58 2026 -0700
- cheapcgi: skip an empty pair in a query string, refs #38185
cgiParseInputAbort and cgiParseNext end a value at the first separator after
it, so an empty pair left the next variable named "&name". Nothing looks
that name up, so the variable was lost with no warning. This is what broke
the link in #38145: the URL had "&&" in front of hgS_doLoadUrl, so the saved
session never loaded and the reporter saw a browser without the tracks he
expected. The same empty pair at the end of a query string has no '=' after
it, and aborted the CGI instead, so the behavior differed by position.
Both parsers now skip the separators of an empty pair and carry on.
cgiEncode escapes everything but alphanumerics, '.' and '_', so no encoded
name can begin with a separator and none is ever eaten.
Checked against the unpatched library over every string of "& ; = a b % +"
up to length five, 19608 inputs per parser. Nothing that parsed before now
aborts, no variable is ever lost, and every difference is the intended one.
484 inputs to cgiParseInputAbort and 62 to cgiParseNext used to abort and
now parse.
558006de7cb71ab41a81a62dc346456c933a169c Mon Aug 31 15:37:04 2026 -0700
- Merge branch 'cgiEmptyPair38185': skip an empty CGI pair instead of eating the next variable, refs #38185
65bfad661897f4f834dd07b89c507bae0a19bdde Tue Sep 1 00:41:08 2026 -0700
- trackDb: clear MAKEFLAGS for the metaDb sub-make, refs #35489
checkMetaDb runs a plain make in the per-database metaDb directory. The
recipe that calls it is not a recursive make, so under -j the parent has
already closed the jobserver pipe and the sub-make prints
make[2]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
makeStrictBeta.csh greps the log for "error|warn" and exits 1 on any hit,
so those warnings would stop the build even though the trackDb work
succeeded. The beta path has three of these sub-makes (hg18, hg19, mm9),
so three hits.
Clearing MAKEFLAGS for that one command silences it. The sub-make only
needs DB and TABLE, and both are passed on the command line.
Tested with the real script in the same three-level shape the build uses:
before, serial gives zero hits on that grep and -O -j 8 gives three; after,
both give zero, and the metaDb recipe still runs in all four cases.
- src/hg/makeDb/trackDb/checkMetaDb - lines changed 3, context: html, text, full: html, text
cab003c85820ef181c537ad402aad0044e4e3878 Tue Sep 1 09:06:56 2026 -0700
- registryPages: draw two pages from all four configuration catalogs, refs #37838 #37923 #37925 #37623
Each of the four catalogs answers for its own surface and none of them knows
the others exist. This reads all four and draws the two pages that need them
at once: a Venn of the registries, and an index of every name they describe
with its description on hover.
The reason it is a program and not two files: every count on both pages is
computed from the catalogs, so a page cannot drift from the tree. A hand-built
version of the Venn had the track registry 12 names short and one region count
wrong, and neither error was visible on the page.
The one judgement is which names two registries describe as the same variable.
That is matched on the cart variable a row names, not on the suffix the track
catalog stores, because comparing bare names both misses the four track-scoped
names and invents six overlaps that are not there. KNOWN_SHARED records the
eleven confirmed by reading the call sites, and --check fails when the computed
set no longer matches it, the same contract the sibling catalogs' --reconcile
has.
--check writes nothing and is the mode for a cron. --audit adds the saved
session counts from sessionCartAudit and needs the database.
- src/hg/utils/registryPages/index.css - lines changed 133, context: html, text, full: html, text
- src/hg/utils/registryPages/index.js - lines changed 133, context: html, text, full: html, text
- src/hg/utils/registryPages/registryData.py - lines changed 473, context: html, text, full: html, text
- src/hg/utils/registryPages/registryPages.py - lines changed 716, context: html, text, full: html, text
- src/hg/utils/registryPages/venn.css - lines changed 125, context: html, text, full: html, text
3c814b674f49f9a30d4b8d227e0fe7061a18766a Tue Sep 1 09:21:28 2026 -0700
- registryPages: correlate the trackDb settings docs with the cart, refs #37908 #37838
Two files in the tree say which track types a setting applies to, and they were
written from different evidence. trackDbLibrary.shtml carries a hand-written
types list per setting, which is what #37908 has been correcting.
cartTrackVarCatalog files each cart variable under the config function that
reads it and records the trackDb types that function serves, which came from
reading hui.c and the per-type Ui functions.
Where a trackDb setting and a cart variable are the same knob, the two are
answering the same question, so they can be compared. 60 of the 261 documented
settings have a runtime override. 46 of those pairs are comparable, 12 agree
exactly, and 28 have a type the config code serves that the docs do not list.
The output is a candidate list, not a verdict, and the page says so. Two known
false-positive shapes are called out on it: a variable read by two config
functions collects the types of both, and a pair joined by tdbDefault rather
than by name is weaker evidence, so those are reported separately.
Also factors the palette and the shared reset out of venn.css and index.css
into tokens.css, since a third page now needs them.
- src/hg/utils/registryPages/correlate.css - lines changed 91, context: html, text, full: html, text
- src/hg/utils/registryPages/index.css - lines changed 27, context: html, text, full: html, text
- src/hg/utils/registryPages/registryPages.py - lines changed 264, context: html, text, full: html, text
- src/hg/utils/registryPages/tokens.css - lines changed 52, context: html, text, full: html, text
- src/hg/utils/registryPages/trackDbData.py - lines changed 238, context: html, text, full: html, text
- src/hg/utils/registryPages/venn.css - lines changed 31, context: html, text, full: html, text
0c504951ad80a920183bcb6406ab5f6f1264cc53 Tue Sep 1 10:24:09 2026 -0700
- trackDbConditions: work out what has to be true before a setting does anything, refs #37908
The types list in trackDbLibrary.shtml is flat, so it cannot say "only in
coverage mode", "only when the track is in pack", "only when another setting is
on". Those conditions are real and there are a lot of them. This finds them by
reading the C.
harvestConditions.py records, for every read of a track setting, the conditions
that enclose it, using offset ranges rather than brace depth so that
`if (x) return;` is handled as exactly as a braced block, and an early return
contributes its negation to the rest of the function. That alone finds little,
because the read is usually plain and the test sits at the caller, so it also
builds a call graph and computes the conditions that hold on EVERY path into
each function. Only those are reported, which keeps every claim a necessary
condition rather than a guess. A function whose address is taken can be a track
method and is reached from outside the scan, so it reports nothing rather than
something false.
Render and config are scanned as separate call graphs. What has to be true for
a setting to change the picture is a different question from what has to be true
for its control to appear, and mixing them empties every intersection.
trackDbConditions.py sorts the conditions into kinds and reports them against
the documented settings. --check is the cron mode and fails when a documented
setting gains its first condition or loses its last, since either way the
documentation and the code have parted company.
Where it stands: 325 settings read across 779 sites. In the render scope 96
have a condition that holds at every read, 43 of them documented, and 27 of
those turn on something other than the track type. Known limits, all in the
module docstrings: a name built with safef is invisible, a condition carried in
a variable to a later use is not followed, and the boundary between drawing and
the configuration popup inside hgTracks.c is not clean, which is why
configurable and filterBy come back with popup plumbing in their lists.
- src/hg/utils/trackDbConditions/conditionBaseline.txt - lines changed 45, context: html, text, full: html, text
- src/hg/utils/trackDbConditions/harvestConditions.py - lines changed 656, context: html, text, full: html, text
- src/hg/utils/trackDbConditions/trackDbConditions.py - lines changed 328, context: html, text, full: html, text
fd2771d51dfa6526d51778a1a3e2e553aa75290c Tue Sep 1 10:37:16 2026 -0700
- trackDbConditions: follow a setting to where its value is used, refs #37908
The first pass only saw a condition when it enclosed the read. That misses the
commonest shape in the drawing code: the setting is read plainly at the top of a
loader, carried into a struct, and used far below behind a test of a different
setting. bamColorTag came back unconditional even though it does nothing unless
bamColorMode is tag.
So follow the value. A name that holds exactly one setting across a file is
taken to carry it, including into a struct field of the same name, which is how
the value usually travels. Then find where the value is used and intersect the
conditions guarding those uses.
Two distinctions do the work. A mention at paren depth zero is one side of an
assignment or an element of an initializer list, which only moves the value
somewhere else, so it is not a use; inside a call it is an argument and it is.
Counting the struct initializer as a use put an unguarded site in the set and
emptied every intersection. And matching drops the field prefix, so the test
written sameString(colorMode, ...) in the loader is the same condition as
sameString(btd->colorMode, ...) in the drawer.
These are reported as "when used" and kept apart from "always". The every-path
conditions are necessary by construction; a use-site condition is only as good
as the set of uses found, so it is a strong hint rather than a claim, and it
keeps the strict key rather than the loose one for that reason.
43 settings in the render scope are read plainly and used only under a
condition, 21 of them documented. Among them: bamColorTag needs
bamColorMode=tag, pairSearchRange needs pairEndsByName, speciesCodonDefault
needs mafChain and frames, and speciesOrder, speciesGroups and speciesDefaultOff
turn out to gate each other.
- src/hg/utils/trackDbConditions/conditionBaseline.txt - lines changed 14, context: html, text, full: html, text
- src/hg/utils/trackDbConditions/harvestConditions.py - lines changed 81, context: html, text, full: html, text
- src/hg/utils/trackDbConditions/trackDbConditions.py - lines changed 25, context: html, text, full: html, text
0549a7574450f649cf2444a62876a2db1cca47fe Tue Sep 1 12:18:12 2026 -0700
- redmineCli: add a sql subcommand for read-only reporting queries
Questions that span many tickets took hundreds of API calls. They are one
join in SQL. The subcommand runs a query against the Redmine database and
prints an aligned table, or TSV, or JSON.
redmineCli sql --tables
redmineCli sql --describe issues
redmineCli sql "select id, subject from issues limit 5"
It sends read-only statements only, and multi-statement support is off so a
semicolon cannot smuggle in a second statement. Credential columns are never
printed. --timeout sets max_statement_time, default 30 seconds, so a bad join
does not sit on the server. --limit caps printed rows at 500.
Credentials come from ~/.hg.conf. read_api_key now shares one config reader
with the new code instead of parsing the file itself. No RM.
b712b918c9fdf2aac65b5f6ceb076e4af584231c Tue Sep 1 12:29:36 2026 -0700
- trackDbConditions: scan the whole library, and check the scanner against known cases, refs #37908
Four refinements, one of which fixes wrong output rather than noisy output.
The scanned file list was hand-kept, and it had silently missed netCart.c,
chainCart.c, pgSnp.c, hgMaf.c and a dozen more that read track settings on the
drawing path. An unscanned read site is worse than an unclassified one: the
every-path analysis was claiming a condition holds at every read of a setting
while never having seen one of the reads. Three settings were carrying false
claims because of it, barChartBars, barChartCategoryUrl and bigDataUrl. So scan
hg/lib and hg/cgilib whole and let reachability decide which side of the browser
each read belongs to. Coverage goes from 325 settings to 347.
An early return is treated as a precondition only near the top of a function.
The same shape four hundred lines down is sound but says nothing about the
setting, and it was how the jsonp output check at the tail of doTrackForm came
to look like a condition on filterBy.
A negated disjunction is a conjunction, so NOT (A || B) now splits into NOT A
and NOT B. The squishyPack guard was one unsplittable string that was neither a
visibility condition nor a coverage one; it is now correctly both.
--self-test checks the harvest against ten cases read out of the C by hand.
Every one of them broke at least once while this was being built, usually
silently, so they are checked rather than trusted, and --check runs them first
and refuses to report anything if the scanner itself has moved. It earned its
place immediately by catching two misclassifications in the same commit that
added it.
Also caches the harvest in a temp file keyed on the newest source mtime, since
scanning takes forty seconds and reading the output takes several runs. Warm
runs are now instant.
- src/hg/utils/trackDbConditions/conditionBaseline.txt - lines changed 14, context: html, text, full: html, text
- src/hg/utils/trackDbConditions/harvestConditions.py - lines changed 60, context: html, text, full: html, text
- src/hg/utils/trackDbConditions/trackDbConditions.py - lines changed 129, context: html, text, full: html, text
fd380f3ae9c678f8e92b4728d5f614a71764fecd Tue Sep 1 12:56:18 2026 -0700
- trackDbConditions: four fixes found by walking the results against the code, refs #37908
Walking the worklist row by row is what found these; none of them was visible
from the summary counts.
A read wrapped in another call was invisible to the value map, because it
stopped at the first call name. cloneString(trackDbSetting(...)) and
atoi(cartOrTdbString(...)) are both common. This is why
hideEmptySubtracksSourcesUrl came back as merely composite-only when it in fact
also needs hideEmptySubtracks and hideEmptySubtracksMultiBedUrl.
A macro in a condition was read as an unknown word, so
cartVarExistsAnyLevel(cart, tdb, FALSE, MAF_CHAIN_VAR) did not resolve. That
hid the fact that irows is consulted only when mafChain is absent from the cart.
Same indirection trap as the chained defines, in a different place.
The variable-to-setting map kept only the last assignment.
wigFetchMinMaxYWithCart assigns defaultViewLimits from defaultViewLimits and
then, if that came back NULL, from viewLimits, so the test in between looked
like viewLimits testing itself. It is now resolved at the position of the test,
which turns an artifact into the real finding: viewLimits is read only when
defaultViewLimits is absent.
Two classes of noise removed. "Read the trackDb value when the cart has none"
is how every setting with a default resolves, and it was a third of the
worklist. A guard on the trackDb type line having words is a sanity check that
is true of every track, and it was the whole of what the scan had to say about
chainNormScoreAvailable, lollyMaxSize and lollyNoStems.
- src/hg/utils/trackDbConditions/conditionBaseline.txt - lines changed 9, context: html, text, full: html, text
- src/hg/utils/trackDbConditions/harvestConditions.py - lines changed 47, context: html, text, full: html, text
- src/hg/utils/trackDbConditions/trackDbConditions.py - lines changed 10, context: html, text, full: html, text
d2a2afcd6c5c19257f8ed2f746c3f94474f4ab78 Tue Sep 1 13:37:34 2026 -0700
- hgTracks: let hg.conf set how big a step malloc takes, refs #38225
hgTracks loads tracks in parallel threads, and each thread grows its own
memory pool in the 128 kB steps glibc uses by default. A heavy render asks
the kernel to enlarge a pool about 133,000 times, work that does nothing for
the reader.
The new hg.conf setting mallocTopPad is a number of bytes. When it is set,
cfgSetMallocTopPad() passes it to mallopt(M_TOP_PAD) before anything else in
main() has a chance to allocate, so the heap grows in that size step instead.
Absent or zero, glibc is left alone and nothing changes.
Measured with the #38094 harness over eight sessions and two positions, at
32 MB: renders take 0.926 of the time they did, against a noise floor of
1.006. With the setting absent the same build measures 1.004, inside that
noise floor. Through apache on a sandbox it measures 0.906. The cost is
about 65 MB more resident memory on a heavy session and 8 to 20 MB on an
ordinary view, where it also saves no measurable time; the saving and the
cost are the same effect and appear together. The rendered image is byte
identical either way.
Off by default, so a server opts in.
- src/hg/hgTracks/mainMain.c - lines changed 4, context: html, text, full: html, text
- src/hg/hgTracks/renderMain.c - lines changed 5, context: html, text, full: html, text
a5cefd59e34e6a831de80a7b0a9c774423580c1d Tue Sep 1 13:43:25 2026 -0700
- Merge branch 'perfTopPad': hg.conf setting for the malloc heap step, refs #38225
cec595b267a0dae66dab498417df29b3bb8b60d0 Wed Sep 2 10:21:29 2026 -0700
- trackDb: rsync to the destination string as it stands, refs #35489 #38211
Code review noted that "${dest%%:*}:${dest#*:}/" is hard to read. It is also
a no-op. It takes the machine:path string apart at the first colon and puts
it back together with a colon, which is the same string as "$dest/".
- src/hg/makeDb/trackDb/buildTrix - lines changed 1, context: html, text, full: html, text
811d6fef1fb93efa979d039f4effb7ebe7d98780 Wed Sep 2 10:21:29 2026 -0700
- trackDbCacheCleaner: parse -n with getopts, and allow a cache at the top level, refs #37551 #38211
Code review asked why the option parsing compared $1 to "-n" by hand instead
of using getopts, and pointed out that the path depth rule refused a cache
directory a mirror put at the top level, such as /mirrorTrash.
Use getopts for -n, and move the usage text into a function so that an unknown
option prints it too.
Drop the depth rule. It also did not do what its comment claimed, since
/dev/shm is two levels deep and passed it. Refuse only the root directory,
after stripping any trailing slash so that "/" and "//" are both caught. The
check on name.txt below it is what decides whether a directory really is a
trackDb cache.
- src/product/scripts/trackDbCacheCleaner.sh - lines changed 32, context: html, text, full: html, text
6d28e748f8aee8c252386600fc93d7692aa6929d Wed Sep 2 10:39:13 2026 -0700
- hgTracks: don't escape the Track Search title twice, refs #38172
The Track Search page built its blue bar title with htmlEncode() on the
organism and the freeze name, then handed it to
webStartWrapperDetailedNoArgs. That call prints the title through
htmlTextOut, which already escapes & < > and the double quote, so the
ampersand starting each entity htmlEncode() wrote was escaped a second
time and the page printed the entity instead of the character. Every
assembly whose dbDb description carries a '/' showed it: hg38 read
"Search for Tracks in the Human Dec. 2013 (GRCh38/hg38) Assembly".
The title lands in element text between two divs rather than in an
attribute, so htmlTextOut covers it on its own and the encode here is
not needed. Checked against the assembly hub case that prompted the
encode in the first place: a hub whose genomes.txt organism and
description hold markup still comes back as text in the blue bar.
Found by Max in the v503 Preview II code review.
- src/hg/hgTracks/searchTracks.c - lines changed 6, context: html, text, full: html, text
be830879c66213de693050b5e8697f5826f51ad1 Wed Sep 2 10:56:33 2026 -0700
- ts: serve each ticket sandbox over https as well as http, refs #37867
A parked instance answered only over plain http, so anything a CGI decides from
the request scheme could not be exercised in one at all. Apache sets HTTPS=on
for a TLS request and cgiServerHttpsIsOn() reads it, so a CGI that branches on
it, such as one deciding whether to mark a cookie Secure, always took the same
branch in a park no matter what was being tested.
Each instance now listens twice: plain http on its registered port, as before,
and https on that port plus 1000, from a self-signed certificate generated once
and shared by every park on the account. Both listeners serve the same frozen
code, so hitting the pair is the comparison.
Only the http port is in the registry and the https port is derived from it, so
nothing about the existing layout changes and "ts conf NNNNN" adds https to an
instance frozen before this. http ports are now kept below the start of the
https range so the two cannot overlap. "ts list" prints both, "ts tunnel"
forwards both, and "ts port NNNNN ssl" gives the https one on its own, which is
how the laptop wrapper asks, rather than repeating the offset in a second file.
7b4952c3b7ec30772836c67d40db442a13b83476 Wed Sep 2 11:00:09 2026 -0700
- pngTimingReport: read the track image timing beacons out of an apache log, refs #38109
hgTracks reports how long its track image took to reach the reader on the
query string of a 43 byte image, so the apache log line is the whole record.
This turns those lines into a throughput distribution, split by continent,
which is the input the png compression level decision needs.
Reads plain logs and .gz archives, and defaults to the live hgwdev log.
Country comes from geoIpCountry6, the table geoMirror.c uses, read once and
bisected here. A delivery that finished inside one clock tick (x=0) is
counted in its own column and kept out of the percentiles, because it cannot
give a throughput. Our own headless test runs are dropped by default.
- src/hg/utils/pngTiming/pngTimingReport.py - lines changed 502, context: html, text, full: html, text
9ce14583f270ff891278d2cf4f0c6432bf79fc61 Wed Sep 2 11:20:27 2026 -0700
- hubConnect: hubEncode covers custom tracks as well as hub tracks, refs #38172
hubEncode() keyed off isHubTrack() alone, so the thirty-odd call sites that
use it treated a custom track's data fields as our own. A custom track's
data comes from outside the same way a hub's does, and customFactory.c only
looks at the track and label lines, not the data fields, which are what
hubEncode() is called on. Add isCustomTrack() to the test.
Raised in the v503 Preview II code review.
- src/hg/lib/hubConnect.c - lines changed 9, context: html, text, full: html, text
44d8050c280e78571f46ea34b1ebffedca9758ce Wed Sep 2 11:20:27 2026 -0700
- vcfClick: print only the columns a tabular INFO value actually has, refs #38172
printTabularData() dropped the return value of chopByChar and then looped to
the column count taken from the header row. chopByChar fills only the slots
it uses, so a value carrying fewer pipe separated fields than its Description
declares took the loop past the end of what was filled. looksTabular()
requires just one of an element's values to match the header, so the rest of
a multi-value element can be shorter. Keep the count and print an empty cell
past it.
Both functions also sized a stack array off VCF text with char copy[len+1],
one from a record's INFO value and one from the header Description. Use the
heap for those.
Raised in the v503 Preview II code review.
f247b79b61ca172b3146284594b97c848db4c8bf Wed Sep 2 11:20:43 2026 -0700
- hgSession, hgGenome: tighten handling of an uploaded file's name, refs #38172
Both CGIs print the name that arrives with an upload straight into the page.
hgSession prints it in four of its load messages, so encode it once where it
is read; the sibling URL message three lines above already did the same for
the URL. hgGenome puts it in a quoted attribute, which wants
attributeEncode.
Raised in the v503 Preview II code review.
- src/hg/hgGenome/upload.c - lines changed 3, context: html, text, full: html, text
- src/hg/hgSession/hgSession.c - lines changed 4, context: html, text, full: html, text
a1b16efb1479af46ea0ed5d6464f084479ce8490 Wed Sep 2 11:20:43 2026 -0700
- wikiLink: returnUrlSchemeIsSafe should refuse a scheme-relative URL, refs #38172
The function allows an http URL, an https URL or a relative one. It found
the scheme by looking for a colon, so "//host/path" fell through the first
test and was accepted as relative, though it names another host. Refuse it,
and say so in the comment.
Raised in the v503 Preview II code review.
95807c7178fd38d52564f06df9fd5b90ceca1d24 Wed Sep 2 11:20:44 2026 -0700
- hgTracks, hgc, hgPhyloPlace: small fixes from the v503 Preview II review, refs #38172
hgTracks.c: skipBeyondDelimit returns NULL when the delimiter is absent, and
the caller decremented and printed it without checking. The noYearDbs list
keeps every assembly that reaches it carrying a '(' today, so nothing shows,
but printf used to print "(null)" where htmlEncode now walks off the end.
Fall back to the whole freeze name.
gtexTracks.c: the guard before the in place truncation was two bytes stricter
than the buffer needs, so a description just over the budget printed in full.
Say what the buffer requirement actually is. vcfTrack.c has the same shape a
byte the other way and is already right.
hgPhyloPlace.c: initialize size, as the two sibling call sites do.
cgiMemBlobFind always sets it when it returns a block, so this is for
consistency.
bigBedClick.c: hubEncode was called twice on the same string in one
statement.
- src/hg/hgPhyloPlace/hgPhyloPlace.c - lines changed 1, context: html, text, full: html, text
- src/hg/hgTracks/gtexTracks.c - lines changed 4, context: html, text, full: html, text
- src/hg/hgTracks/hgTracks.c - lines changed 4, context: html, text, full: html, text
- src/hg/hgc/bigBedClick.c - lines changed 2, context: html, text, full: html, text
8cb00e70bc6a533345d30c0456cd7f43e53a6f40 Wed Sep 2 11:34:55 2026 -0700
- weeklybld: default the authorEmail.pl fallback to the current buildmeister
Fall back to braney rather than hiram when a login is not found in the
git-reports authors.html.
- src/utils/qa/weeklybld/authorEmail.pl - lines changed 1, context: html, text, full: html, text
4407e86d09bfd82cce247c0e8f209e3720713343 Wed Sep 2 11:50:53 2026 -0700
- Merge branch 'cr38172': code review findings from v503 Preview II, refs #38172
Findings 2 through 5 from the review on #38172. Finding 1 went in separately
as 6d28e748f8a with build patch #38231.
hubConnect hubEncode covers custom tracks as well as hub tracks
vcfClick print only the columns a tabular INFO value actually has
hgSession encode an uploaded file's name once where it is read
hgGenome the same name lands in an attribute, so attributeEncode
wikiLink returnUrlSchemeIsSafe refuses a scheme-relative URL
hgTracks skipBeyondDelimit can return NULL; fall back to the freeze name
gtexTracks say what the truncation buffer requirement actually is
hgPhyloPlace initialize size like the sibling call sites
bigBedClick one hubEncode call instead of two on the same string
6d3ac828268e706ba9dbc4baa01d444839c9aec2 Wed Sep 2 13:05:49 2026 -0700
- trackDb: write the gbdb file list to $TMPDIR, not the scanned directory
The parallel beta make (0b36c1276f7) aborted the v503 final build and both
attempts at the #38231 build patch:
/cluster/bin/x86_64/tdbQuery -check -release=beta -strict \
'select count(*) from mm10' -root=.../trackDb
No such file or directory
stat failed in listDirX: .../trackDb/musFur1.gbdbList.txt
make[1]: *** [makefile:281: mm10_beta] Error 255
Note the two different databases -- that is the signature. The %_beta recipe
writes $*.gbdbList.txt into the trackDb directory, uses it, then removes it.
loadTracks runs tdbQuery -check with -root set to that same directory, and
listDirX stats every entry it finds there, so at -j 8 one database unlinks its
list file between another database's readdir and stat.
Naming each file after its database, which is what the original commit relied
on, prevents two recipes from writing the same name but not one recipe from
seeing another's file. Put the list in $TMPDIR instead, where nothing scans
it, in all three recipes that build one (beta, publicTest, public). Fixing
listDirX to tolerate ENOENT would not have helped: the recipe runs the
installed /cluster/bin/x86_64/tdbQuery, not the binary the build compiles.
This also keeps stray list files out of the working directory, where they have
occasionally been picked up by a commit.
The header comment claimed per-database naming was sufficient for -j; correct
it and say where scratch files belong. Verified by expansion with make -n; a
real parallel run has to wait for the next beta make. refs #35489
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- src/hg/makeDb/trackDb/makefile - lines changed 28, context: html, text, full: html, text
aeb00b82ee56facd622b8f5347ebbf85309c9403 Wed Sep 2 14:02:01 2026 -0700
- Ask the database once per track for RefSeq status, not once per gene, refs #38233
refGeneColor is the per-item color callback for the RefSeq gene tracks. Each call
opened a database connection, checked that the status table existed, asked for one
gene's status, and closed the connection again. A view with two RefSeq tracks in
pack mode draws a few thousand genes, so the browser made a few thousand round
trips to the database while it was drawing the image.
refSeqStatusHashLoad now builds the whole name-to-status map with one batched query
when the track loads, and leaves it on tg->customPt. refGeneColorByStatus reads
that map, so drawing asks the database nothing. Both are static; nothing outside
this file used either.
The map is built after limitVisibility() and only for a track that will draw. A
track limitVisibility hides needs no colors, and the tracks it hides are the ones
with the most items, so they are exactly the ones whose query would be largest.
The existence test goes through a connection rather than hTableExists(), because
refSeqStatusTable carries its database (normally hgFixed.refSeqStatus) and
hTableExists() looks a name up in one database's own list of tables. It answers
FALSE for any name with a database prefix, which silently drops the shading for
plain refGene and xenoRefGene.
This also removes a latent errAbort: the old gate tested refSeqStatus OR
ncbiRefSeqLink and then queried whichever table the track type wanted, so a
database with only ncbiRefSeqLink sent a refGene track to query a table that was
not there.
Rendering is unchanged. Eight scenarios pixel-identical to genome-test, including
whole chr1 and the plain refGene and xenoRefGene tracks. Across the seven clinical
Recommended Track Sets on hg38, 14% faster on a 2.5 Mb view and 20% on a 25 Mb view,
with peak memory unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- src/hg/hgTracks/simpleTracks.c - lines changed 93, context: html, text, full: html, text
07ba37612b12dad969ed452c55780c20b7d65712 Wed Sep 2 17:03:38 2026 -0700
- hgTracks: do not crash when a quickLift chain has no aligned block in the window
chainLoadIdRangeHub returns every chain whose span overlaps the window, but
only loads the link rows that fall inside it. A chain can therefore reach
across the window with no aligned block in it. Its blockList is empty,
chainSubsetOnT has nothing to subset, and it returns NULL. bigWigLoadPreDraw
dereferenced that NULL and hgTracks died with SIGSEGV, so the user got a blank
page.
This is what happens on the smaller of the two regions hgConvert offers for a
quickLift from hg38 to hs1. Three of the four chains whose span covers
hs1 chr7:59,324,765-59,325,193 have no link there. Widen the view past 100 kb
and those chains pick up blocks again, which is why only the narrow views
crashed.
The rest of the quickLift code already skips a chain with an empty blockList.
Do the same here.
refs #38236
- src/hg/hgTracks/bigWigTrack.c - lines changed 8, context: html, text, full: html, text
5dd36916814777c4c330de1b41b33c33fbcf758d Wed Sep 2 17:42:43 2026 -0700
- hgTracks: place a quickLift bigWig block by the window, not by the chain
bigWigLoadPreDraw scaled each aligned block by the span of its own chain, but
the preDraw buffer covers the window. A chain that only partly covers the view
was therefore stretched across the whole image, and when several chains were in
the window each stretched independently and overwrote the others.
Scale by (winEnd - winStart) instead. chainSubsetOnT clips the blocks to the
window, so a block cannot reach past the end of the buffer, but summary[] is
exactly summarySize long with no slack, so clamp anyway.
Measured on a quickLift from hg38 to hs1, at hs1 chr7:59,324,000-59,326,000.
One chain covers 429 of those 2,000 bases, which is x 516 to 717 of the image.
Before, the signal was painted from x 156 to 1098, so 625 columns carried a
value where nothing is aligned. After, it is painted from x 516 to 717 and no
column outside an aligned block carries a value. Same result on a 100 kb view
with four chains in it: 24 such columns before, none after. A window whose
chain reaches both edges, which is the ordinary case, renders pixel for pixel
as it did.
The second half of the ticket, the bigWig zoom level being chosen from query
coordinates while summarySizeBlock comes from target coordinates, is not a bug.
A cBlock has the same length on both sides by construction, so the two agree.
refs #37621
- src/hg/hgTracks/bigWigTrack.c - lines changed 26, context: html, text, full: html, text
520b1f069967ad1a0a0d39eb3a30acc0ab99fb42 Thu Sep 3 02:30:35 2026 -0700
- hgConfCatalog: register snapshot.ttlDays, refs #37925
Written by nightlyRegister.sh, which records the settings the tree
reads that the catalog was missing. Only facts copied off the call
site are filled in. No classification is guessed: a new boolean gets no
role=, because calling a release gate a knob would hide it from the
sunset report for good, and every row lands in the 'Awaiting review'
section until somebody reads the call site.
snapshot.ttlDays hg/utils/snapshotReaper/snapshotReaper.c:46
wrote 1 row to the 'Awaiting review' section of hgConfCatalog.py; classify it and move it out
- src/hg/utils/hgConfCatalog/hgConfCatalog.py - lines changed 7, context: html, text, full: html, text
f8ce38bb1a25f654f4052f0e44b8328decd8f8ab Thu Sep 3 17:39:21 2026 -0700
- docker: add an OCI description label pointing at the docker help page
Sets org.opencontainers.image.description on the base Dockerfile, so it
shows up in `docker inspect` for every image we publish: the release
amd64 base and arm64 images from buildReleaseDocker.sh, the amd64 beta
overlay that FROMs the base, and the kent:tip QA image.
The Docker Hub repo page has its own description fields, which are not
read from this label; those are set through the Hub API and now carry
the same text.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- src/product/installer/docker/Dockerfile - lines changed 1, context: html, text, full: html, text
e8fd1a37d1959a9e73943e946088d1b86d2b1bb4 Fri Sep 4 11:17:35 2026 -0700
- docent: correct the claim that expect: is the only verb that can fail a run
An audit of tests/README.txt against the code found the claim wrong in both
READMEs. docent.js turns any step's throw into "step N (verb) failed" and exit
1, so mouseover:, loadSession:, drag:, convert: and go: all fail a run when they
cannot do what they were told. Verified by running a script with no expect: step
in it at all: it exited 1 from loadSession:.
The conclusion drawn from the claim still holds and is kept: expect: is the only
verb that CHECKS anything, and track: does not even check its own input, since a
name no assembly has is sent as name=mode and the run exits 0 (also verified).
Also documents why make derive strips one line of trackDb provenance: the
listing is cached for a day, a cold fetch prints a line a warm run does not, and
the first run of any day otherwise fails against a baseline captured warm.
refs #37892
- src/hg/utils/docent/README.md - lines changed 8, context: html, text, full: html, text
- src/hg/utils/docent/tests/README.txt - lines changed 15, context: html, text, full: html, text
67efa330d2830c30741f9524abfd72b59f8791f8 Fri Sep 4 11:17:46 2026 -0700
- docent: share the test rules, add tests/regress, and stop an empty suite passing
Moves the test rules out of tests/makefile into tests/docentTest.mk so a second
directory of Docent tests runs the same code rather than a copy. tests/makefile
and the new tests/regress/makefile each set DOCENT and include it.
tests/regress/ is for one script per already-fixed bug, asserting the behavior
its ticket says is correct, to be run nightly against genome-test (#38252). It
is kept out of tests/ so that suite stays short enough to run before a commit:
measured, its eleven scripts take 56 seconds.
Two fixes to the rules while they moved:
- make derive strips "trackDb: N tracks for DB from .../hubApi" from both the
run and the baseline. docent.js caches the trackDb listing for a day and
prints that line only on a cold fetch, so the first make derive of any day
reported CHANGED against a baseline captured warm. Verified cold and warm,
and make derive-accept reproduces the three committed baselines byte for byte.
- make test with no *.docent.yaml, and make derive with no baseline, now fail
instead of printing "docent tests passed". An empty tests/regress reported a
pass before this.
refs #37892 #38252
- src/hg/utils/docent/tests/docentTest.mk - lines changed 116, context: html, text, full: html, text
- src/hg/utils/docent/tests/makefile - lines changed 85, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/README.txt - lines changed 36, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/makefile - lines changed 27, context: html, text, full: html, text
b46fbfda9130992e290b222840f1d3b7bba15c6c Fri Sep 4 11:15:50 2026 -0700
- hgTracks: build item coverage from feature runs, not one counter per base
A track drawn as a coverage graph asked for one unsigned counter per base of
the window, added one to every base of every feature, then read the whole array
back to produce about 1,200 pixels. On a 25 Mb view that is a 100 MB
allocation per track and two passes over it, and the cost grows with the width
of the view rather than with the amount of data. It was 38% of the render on
one Recommended Track Set page and 29% on another, inside the drawing step
where the reader waits for it.
Coverage only changes where a feature starts or ends, so it is now held as a
step function: two events per feature, sorted, swept into runs, and sampled
with a forward cursor.
The arithmetic at the two fractional bases of each pixel is deliberately
unchanged, including its habit of squaring the weighted value rather than
weighting the square, so the drawn image is identical. Whole bases fold a run
at a time as cov*n, which is the same value the per-base loop reached, because
those quantities are integers held in doubles.
Verified with 96 pixel comparisons and no difference: the 80 case sweep is
identical on every cell that renders deterministically, plus 18 comparisons on
views narrower than the image, which is the only path the sweep does not cover.
Geometric mean 0.81 over the eight page benchmark set, 0.51 on the worst cell,
and about 100 MB off peak memory on a wide view.
refs #38253
- src/hg/hgTracks/simpleTracks.c - lines changed 224, context: html, text, full: html, text
fb5899cd6117d7e5143b1318767b766d4a8fa4ca Fri Sep 4 11:32:53 2026 -0700
- Merge branch 'countOverlaps': build item coverage from feature runs, refs #38253
6af92621b7dba437be69c05f3ff49079ba637383 Fri Sep 4 11:43:39 2026 -0700
- docent: let convert: take a bare string, like every other verb
convert: hs1 left the argument as a String, so o.to was undefined and o.search
picked up String.prototype.search. The run then reported
convert: "function search() { [native code] }" matched nothing
which says nothing about what is wrong with the script. A bare string is now
the target assembly, matching hub:, addCustomTrack:, loadSession: and the rest.
quicklift: is still never implied, so convert: hs1 is a plain coordinate
convert; the README row says so now.
Found while writing the first of the regression tests for #38252, fourteen of
which open with convert:.
refs #37892
- src/hg/utils/docent/README.md - lines changed 1, context: html, text, full: html, text
- src/hg/utils/docent/docent.js - lines changed 6, context: html, text, full: html, text
e128682fd48d974c0eaa74366087d2be47080bf5 Fri Sep 4 11:43:52 2026 -0700
- docent: add make preflight, which checks the fixtures a test suite does not own
A Docent test that loads a saved session or attaches a hub depends on something
outside the tree, and the failure when that thing goes away is silent rather
than loud. A session that has been renamed or deleted is not an error: hgTracks
answers HTTP 200 with a page titled "Very Early Error" whose body reads "Could
not find session NAME for user USER", the page carries no track image, and every
noText: assertion on it passes. The run goes green having tested nothing.
Verified both ways: a bogus session name passes a test whose only check is
noText:, and fails once the script also asserts noText: "Could not find
session".
preflight.js reads the fixtures out of the scripts themselves -- loadSession:,
hub:, addHub:, addCustomTrack: url:, and hubUrl= inside a goto: -- so the list
cannot drift from what the scripts actually use. It needs no browser, runs in a
couple of seconds, and exits non-zero if anything is unreachable, which is what
lets a nightly run tell "the fixtures are gone" from "a bug came back".
Checking it against the sessions cited by the tickets in #38252 found four that
do not exist on genome-test because they were saved on the RR or on beta, and
corrected one I had wrongly called missing: session names store a dash as %2D,
so a MySQL LIKE with a literal dash misses them. The HTTP check has no such
problem, which is a reason to prefer it over a query against namedSessionDb.
refs #37892 #38252
- src/hg/utils/docent/tests/docentTest.mk - lines changed 12, context: html, text, full: html, text
- src/hg/utils/docent/tests/makefile - lines changed 2, context: html, text, full: html, text
- src/hg/utils/docent/tests/preflight.js - lines changed 175, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/makefile - lines changed 2, context: html, text, full: html, text
443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
- docent: first eleven regression tests, one per fixed bug, refs #38252
Ten assert the behavior their ticket says is correct; the eleventh is an xfail.
All eleven pass against genome-test, 40s for the set.
rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT
rm35920 wrong bigBed type in a hub, no crash and no garbage item label
rm36029 a MAF displays after several phyloP tracks are on
rm36331 quickLift of the GENCODE Archive container, no "Unknown database"
rm36514 a chromosome search after a quickLift hop
rm36702 quickLift hg19 to hs1, no "Unknown database"
rm36798 the two OMIM tracks survive a configure submit
rm37388 hgc on a quickLifted hub item, no connect to the source assembly
rm37520 lifted tracks survive a hop to another genome and back
rm37906 Neandertal tracks draw data in a narrow window on hg18
rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the
symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test
(v503), measured with the reporter's own hub URL as well as with our copy of it:
genome= instead of db= for a hub-backed assembly still reaches a query against a
chromInfo table that does not exist. There is no fixed behavior to assert, so it
is pinned as an xfail and the run fails if it ever starts passing.
Six of these build their own state rather than loading the session their ticket
names, which is deliberate: a session on someone's account can be renamed or
deleted, and hgTracks answers a missing session with a 200 and an early-error
page that every noText: assertion passes on. Where a session is genuinely the
cheapest way to a state (rm37388), the script also asserts noText: "Could not
find session" so that a deleted session fails loudly instead of quietly.
Four traps cost a run each and are written into the scripts that hit them, since
the next twenty-nine will hit them too: a container never gets an img_data_ row
(the rows carry its children's names); a leaf track with no features at the
ticket's position has no row either, so a lifted view is better checked by its
own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an
ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view
of a composite turns its sibling views on as well.
- src/hg/utils/docent/tests/regress/rm35333.docent.yaml - lines changed 33, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm35920.docent.yaml - lines changed 39, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36029.docent.yaml - lines changed 41, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36331.docent.yaml - lines changed 37, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36514.docent.yaml - lines changed 31, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36540.xfail.docent.yaml - lines changed 32, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36702.docent.yaml - lines changed 32, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36798.docent.yaml - lines changed 27, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37388.docent.yaml - lines changed 30, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37520.docent.yaml - lines changed 32, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37906.docent.yaml - lines changed 37, context: html, text, full: html, text
2201c5c416938253128eebcac00c0a240574a889 Fri Sep 4 12:47:16 2026 -0700
- docent: nightly cron wrapper for the regression suite, refs #38252
Mails a report every night whether anything failed or not, matching the
catalogNightly job: no mail means the cron has stopped, not that the browser is
fine. The script mails on its own and always exits 0, so cron adds nothing.
Two things it does deliberately:
- It runs the COMMITTED tests, listed with git ls-files rather than by globbing
the directory. The same directory holds scripts written against a ticket whose
recipe is not right yet, and those must not mail a failure every night.
A newly committed test is picked up with no edit.
- It reports preflight separately from the tests, and preflight is now given an
explicit script list (preflight.js takes names after the directory) so a dead
fixture belonging to a work-in-progress script is not reported against a run
that never included it. A missing session and a returning bug are different
news and should not arrive as the same red.
Verified: the pass path, the fail path (broke one committed assertion, restored
it from git), and a run under `env -i` with only HOME and SHELL set, which is
what cron will give it.
Installed as `10 4 * * *` in braney's crontab, reading the tests out of the
worktree because they are on this branch and not yet on master. It should move
to a checkout of its own when the branch lands, so that a day's editing cannot
change what the cron measures.
- src/hg/utils/docent/tests/preflight.js - lines changed 10, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/nightly.sh - lines changed 95, context: html, text, full: html, text
4c20412187794d30583e3b68a102b7a8612d29a1 Fri Sep 4 13:47:28 2026 -0700
- hgTracks: settle a composite subtrack's visibility before the parallel loaders start
The same request could return two different images. A composite child's
visibility is written lazily, the first time anything asks for it, by
limitedVisFromComposite(). The load phase starts the parallel loader threads
and only waits for them later, and in between the main thread runs
compositeLoad() over every subtrack of the composite, including the ones a
worker is loading right now. The loop condition there is what performs the
write. So the main thread wrote the field while a worker read it, and whichever
got there first decided what the track looked like.
findLeavesForParallelLoad() already computes the same value, with the same test,
for exactly the subtracks a worker will load, and then discards it. Keep it:
write the visibility there, before pthread_create, and skip nothing else. The
value and the guards are the ones limitedVisFromComposite() uses, so the write
only moves earlier.
Caught on clinvarSubLolly, whose loader divides its 128 pixel default height by
1.5 for pack mode, so the band came out 128 or 85 pixels. The effect is wider
than one band: on BRCA1_BRCA2_ENIGMA_hg19 at chr17:41196312-41277500 in
Helvetica, the race hid clinvarMain and clinvarSubLolly and collapsed the two
gnomAD variant rows from about 15000 pixels each to 31, an image of 1259 rows
instead of 31967. barChartTrack.c, chainTrack.c, gtexTracks.c and
halSnakeTrack.c read the same field in their loaders, and three of them use it
to decide what to load rather than how tall to draw.
Verified with the 80 cell pixel sweep from #38094. Before: 33 of 80 cells
render differently at one thread than at twenty, and the sweep reports
differences when master is compared against itself. After: 0 of 80, and the
control passes.
refs #38254
- src/hg/hgTracks/hgTracks.c - lines changed 13, context: html, text, full: html, text
39d5f23118d973ad0ab79820918617c91fa581f1 Fri Sep 4 13:47:45 2026 -0700
- Merge branch 'subtrackVis38254': settle composite subtrack visibility before the parallel loaders start, refs #38254
90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
- Pass element counts, not byte sizes, to the chop routines
chopByWhite, chopString and chopByChar take their last argument as a
count of elements in the output array. Fifteen call sites passed
sizeof(array) instead. For an array of pointers that is eight times
the real capacity on a 64-bit build, so the chop could write well past
the end of the array.
Switched each to ArraySize(). Behaviour is unchanged for any input
that already fitted in the array.
Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck,
utils/bedScore, parasol/lib, parasol/parasol. Three directories do not
build, but they fail the same way without this change: checkExp is
missing htslib link flags, and cgapSageFind and affySplice have stale
prototypes in files this does not touch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- src/hg/altSplice/affySplice/altProbes.c - lines changed 1, context: html, text, full: html, text
- src/hg/hgTracks/bedTrack.c - lines changed 1, context: html, text, full: html, text
- src/hg/hgTracks/pubsTracks.c - lines changed 1, context: html, text, full: html, text
- src/hg/lib/hubSearchText.c - lines changed 2, context: html, text, full: html, text
- src/hg/makeDb/outside/cgapSage/cgapSageFind/uninteresting.c - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/outside/hgGtex/hgGtex.c - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDbRaFormat/trackDbRaFormat.c - lines changed 1, context: html, text, full: html, text
- src/hg/pslPseudo/checkExp/checkExp.c - lines changed 1, context: html, text, full: html, text
- src/hg/utils/hubCheck/hubCheck.c - lines changed 1, context: html, text, full: html, text
- src/parasol/lib/paraMessage.c - lines changed 1, context: html, text, full: html, text
- src/parasol/parasol/parasol.c - lines changed 1, context: html, text, full: html, text
- src/utils/bedScore/bedScore.c - lines changed 1, context: html, text, full: html, text
870b28e103151338d1a2f1ad58099f2a086225c8 Fri Sep 4 17:30:27 2026 -0700
- Remove src/CLAUDE.md from the tree. The file is down to a pointer at two skills, edit-kent-code and make-track, and both of those live only in the private genecats repo. The kent tree is mirrored publicly on GitHub, so anyone reading our source there is told to load two files they cannot get. The C and SQL rules that used to be in this file moved into edit-kent-code in c9093b16dc8. refs #37358
752e20a9a1dc08793c5b529bbeac4ce5836a6be4 Fri Sep 4 17:13:55 2026 -0700
- hgc: stop five details-page handlers from aborting over a missing piece.
chromSeqFileExists() opened its connection with sqlConnect() and only then
asked whether the database existed, so it aborted before it could answer.
The otherDb of a chain or net track is often not a local database at all --
a GenArk hub assembly, or one long retired -- so use sqlMayConnect() and
return FALSE. This is 112 of the failures the TrackCheck robot reports, most
of them hg38 net tracks against HPRC assemblies. Also moved the disconnect
out of the if, where it leaked a connection whenever a database had no
chromInfo table, and dropped the now-redundant sqlDatabaseExists() call,
which was itself a second connection.
mgcCloneInfoLoad() aborted when a clone carried no MGC: id in
hgFixed.mrnaClone. Nothing on the page reads that field, and a clone can
legitimately have only an IMAGE: id, so the check went away rather than the
page. hDbOrganism() aborted for an assembly that has left dbDb but is still
named by a maf component, which hg16.evofold does via mm3; it now falls back
on the database name.
The pgSnp SIFT and Polyphen prediction tables are loaded separately from the
tracks that name them, so a machine can have the track and not the table, as
hgwbeta and the RR do for hg18. Check with hTableExists first and say the
predictions are unavailable instead of letting the query take the page down.
Same treatment for the RNA fold diagram: a non-zero ghostscript exit now
drops only the diagram and keeps the rest of the page, including the
PseudoViewer link.
Two of these report an unavailable piece through warn(), which still marks
the page for the robot. That is deliberate -- the missing hg18 tables and
the RNA fold diagram are real defects, and the log should keep naming them
until they are fixed. On the RNA fold diagram in particular: RNAplot
truncates the sequence id it is given to 42 characters, and the trash path
we build is already 41, so it has never written the file ghostscript is
asked to convert. That is worth its own fix.
refs #37424
- src/hg/cgilib/pgPolyphenPred.c - lines changed 10, context: html, text, full: html, text
- src/hg/cgilib/pgSiftPred.c - lines changed 10, context: html, text, full: html, text
- src/hg/hgc/rnaFoldClick.c - lines changed 20, context: html, text, full: html, text
- src/hg/lib/chromInfo.c - lines changed 12, context: html, text, full: html, text
9a4d8c8cdd36457a18219cfb445abf9ce4e3f843 Fri Sep 4 17:28:09 2026 -0700
- hgc: make the RNA fold diagram actually appear.
RNAplot takes the name of its output file from the sequence id on the fasta
header it is given: it keeps the first 42 characters of that id and appends
"_ss.ps". We were handing it "../trash/<table>/<table>_<name>.ps" and then
asking ghostscript to read back exactly that path, which is wrong twice over.
The "_ss.ps" means the name never matched even for a short item, and the path
prefix alone is 41 characters, so RNAplot kept a single letter of the item
name and every item in a track landed on the same file. Ghostscript was then
pointed at something that had never been written, returned 1, and until the
previous commit that killed the whole details page.
There is no output-file option in RNAplot, and no way to hand our pipeline a
working directory, so the id has to carry the path and still fit. The item
name cannot be part of it: names in wuhCor1.rnaStructRangan are themselves 42
characters. So the id is now a short trash directory plus a 20 character hash
of the track and item name, which is always 37 characters, and ghostscript
reads the "_ss.ps" file RNAplot really wrote.
All 112 items of wuhCor1.rnaStructRangan, the only rnaStruct track we have,
now draw their own diagram: 112 distinct images where before there were none.
Note for whoever looks at this next: RNAplot is not part of this tree and is
not installed by it. It sits in the shared cgi-bin and in cgi-bin-beta, and
it is absent from cgi-bin-$USER, so this feature cannot be exercised from a
developer sandbox without setting rnaPlotPath in hg.conf. Worth confirming the
binary is present in the RR's cgi-bin. Where it is missing the page now says
the diagram could not be made rather than failing outright.
refs #37424
- src/hg/hgc/rnaFoldClick.c - lines changed 37, context: html, text, full: html, text
c430b31cb48da35050fd294fd29934e6e6fa033e Sat Sep 5 02:30:49 2026 -0700
- hgConfCatalog: register 3 settings the tree reads, refs #37925
Written by nightlyRegister.sh, which records the settings the tree
reads that the catalog was missing. Only facts copied off the call
site are filled in. No classification is guessed: a new boolean gets no
role=, because calling a release gate a knob would hide it from the
sunset report for good, and every row lands in the 'Awaiting review'
section until somebody reads the call site.
blatNewFormNewsUrl hg/hgBlat/hgBlat.c:848 (37996)
blatNewFormSwitchDate hg/hgBlat/hgBlat.c:843 (37996)
blatOnlyLatestCheckbox hg/hgBlat/hgBlat.c:2815 (36292)
wrote 3 rows to the 'Awaiting review' section of hgConfCatalog.py; classify them and move them out
- src/hg/utils/hgConfCatalog/hgConfCatalog.py - lines changed 23, context: html, text, full: html, text
25458068678366409e5e4923a382da4ab1344844 Sat Sep 5 07:28:59 2026 -0700
- hgConfCatalog: classify the six settings waiting for review.
--auto-register had recorded six reads that nobody had classified, and
--reconcile counted every one of them as a problem, so the nightly cron
mailed a finding every morning. All six call sites are now read and the
rows have moved into the section each one belongs in with verified=True.
mallocTopPad Limits, a limit. mallopt(M_TOP_PAD) at CGI
startup, hgConfig.c:396. refs #38225
snapshot.ttlDays Limits, a limit. How long an anonymous
Share-a-link snapshot lives before
snapshotCleaner deletes it.
pngTimingSampleRate Logging, a debug setting. One page load in
this many reports how long the track image
took to reach the reader. refs #38109
blatNewFormSwitchDate Branding, site text. The date the classic
BLAT banner names. refs #37996
blatNewFormNewsUrl Branding, a url. Where that banner's news
link points. refs #37996
blatOnlyLatestCheckbox Mirror knobs, internal. Whether the BLAT
form offers the "Keep only last search"
checkbox. refs #36292
None of the six is a boolean flag, so none of them needed a gate or knob
call and none enters the sunset arithmetic.
Two things came out of reading the call sites rather than the harvest.
blatOnlyLatestCheckbox has a second call site the auto-registered row did
not name, hgc.c:27616, which is where the tracks are actually deleted; the
row names both, since the two have to agree. And it is a string compared
with sameString against "on", so only that exact value turns the feature
on: true, 1 and yes do not, which is not how a flag read through
cfgOptionBooleanDefault behaves. The row says so.
refs #37925
- src/hg/utils/hgConfCatalog/hgConfCatalog.py - lines changed 124, context: html, text, full: html, text
008d638cf77546cee712cda8412c13ed2782c838 Sat Sep 5 07:29:12 2026 -0700
- urlCommandCatalog: describe the recovery-email link and the Share dialog.
--reconcile was reporting six names the tree reads that were in neither
the catalog nor the baseline. Four are URL parameters and get rows; two
are cart reads and go in the baseline, by the rule that a cgi* read is a
URL parameter and a cart* read is not.
recovEmail hgLogin.c:1391, cgiUsualString. The recovery
address a confirmation link is claiming, on the
hgLogin.do.confirmRecovEmail path. Same shape as
newEmail and authorized by the same sig and exp.
In hgLogin's excludeVars, so it does not persist.
hgS_failIfExists hgSession.c:1096, cgiBoolean.
hgS_snapshotType hgSession.c:1099, cgiOptionalString.
hgS_doAnonName one more of the hgS_do* commands, so it is listed
as a member of the hgS_* family rather than given
a row of its own.
hgLogin_newRecovEmail1, hgLogin_newRecovEmail2 cartUsualString reads
at hgLogin.c:1532 and 1533, the two inputs on the
change-recovery-address form. Baselined next to
hgLogin_newEmail1 and 2, which are the same thing.
While writing those, hgS_shareAnon turned out to be misfiled. It sat in
urlNamesNotCataloged.txt as internal state, but hgSession.c:1095 reads it
with cgiBoolean, which makes it a request parameter like the two that
arrived beside it. It now has a row with them. All three are
cartRemove'd in the handler, at hgSession.c:1107 to 1109, so none of them
leaks into the saved session.
The baseline also moves s and u from hgBlat.c to hgc.c. Neither name
changed; hgc.c gained its own reads of the two old BLAT share parameters,
and the annotation records one site.
refs #37923
- src/hg/utils/urlCommandCatalog/urlCommandCatalog.py - lines changed 44, context: html, text, full: html, text
- src/hg/utils/urlCommandCatalog/urlNamesNotCataloged.txt - lines changed 5, context: html, text, full: html, text
ff4d234bc327996bad7dfe2d7a26a4a579c20444 Sat Sep 5 07:29:21 2026 -0700
- cartTrackVarCatalog: baseline two filenames the scan reads as cart vars.
Both are the false-positive class the harvester's own docstring names
beside .bai and .tbi: a filename built with safef, where the literal in
the format string looks like a track-scoped suffix.
ss.ps hgc/rnaFoldClick.c:426, "%s_ss.ps", the PostScript file
RNAplot writes for the RNA fold diagram.
tmp lib/trackHub.c:2197, "%s.tmp", the temporary file
writeMergedHubFile renames into place so a reader on the
target assembly never sees a half-written quickLift hub.
Neither is a cart variable, so they go in cartVarsNotCataloged.txt and
the nightly reconcile goes quiet again.
refs #37838
- src/hg/utils/cartTrackVarCatalog/cartVarsNotCataloged.txt - lines changed 2, context: html, text, full: html, text
754f5637634694624fd359811c60513966f3c1a9 Sat Sep 5 07:58:46 2026 -0700
- cartTrackVarCatalog: read a filename as a filename, not as a cart variable.
The harvester looks for a track-scoped name built as safef(buf, size,
"%s.%s", track, SUFFIX). Code that builds "%s.tmp" from a filename has
exactly that shape, so every such site arrived as a name a person had to
write down in cartVarsNotCataloged.txt as not-a-cart-variable. There were
15 of them, and they were arriving at a rate of one every few weeks: .tmp
came in on 2026-08-27 with writeMergedHubFile, _ss.ps on 2026-09-04 with
the RNA fold fix. The docstring predicted the class from the start and
still asked for it to be thrown away by hand.
harvestCartVars.py now answers the question with fileNameLike(), which
asks whether the name's trailing dot-separated component is a file
extension. FILE_SUFFIXES holds only the extensions the tree builds today
plus tbi beside bai: each entry is a name nobody classifies again, so a
guessed one adds a way to lose a real cart variable and buys nothing.
Two cleverer tests were tried and rejected, and the reasons are in the
docstring: the destination buffer's declaration does not decide it, since
the .bai and .link.bb sites format into a plain buf and buffer while
psName and tmpName are char[PATH_LEN]; and neither does the argument being
formatted, which is a filename at some sites, a url at others and a table
name at a third set.
The records still carry these names, because a harvest that hides what it
saw cannot be checked. What changed is that --reconcile no longer asks a
person about them, and --update-baseline no longer writes them back. They
are listed under --reconcile --verbose, and harvestCartVars.py --filenames
prints the rule's claims with their call sites.
Two things guard against the rule going wrong in the direction that would
matter. --reconcile tests cataloged() before the filename rule, so a name
the catalog describes can never be suppressed by it. And --check now fails
if any cataloged name would be read as a filename, which is the case where
the two halves of this file disagree about what a name is; verified with a
planted row that it reports rather than passing.
Baseline 70 names to 55.
refs #37838
- src/hg/utils/cartTrackVarCatalog/cartTrackVarCatalog.py - lines changed 63, context: html, text, full: html, text
- src/hg/utils/cartTrackVarCatalog/cartVarsNotCataloged.txt - lines changed 23, context: html, text, full: html, text
- src/hg/utils/cartTrackVarCatalog/harvestCartVars.py - lines changed 58, context: html, text, full: html, text
06e8572fc3c80c01b547e5616191347674d971cc Sat Sep 5 08:03:52 2026 -0700
- hgConfCatalog: register excludeDbs and slow-db, the failover profile.
hg/lib/jksql.c:1325 builds "<profile>.excludeDbs" with safef from a failover
profile's name and reads it with cfgOption. So it is an hg.conf setting,
and this catalog did not have it: not in the profile suffix list, not
anywhere else. The harvester cannot see it, because the name never appears
as a literal, which is the same blind spot login.oauth.<provider>.<field>
sits in. It is not obscure. product/ex.hg.conf:36 documents
slow-db.excludeDbs and goldenPath/help/gbib.html explains it twice. It is
a comma-separated list of databases that exist only on the local server, so
the failover connection is never opened for them.
It is the fifteenth profile suffix, and the only one that is not read
through cfgOption2 and the only one that belongs to a failover profile
rather than to any profile. The section text now says both, since the
suffix list alone would claim central.excludeDbs works.
slow-db joins knownProfiles while I am here. It is a real profile, named
by failoverProfPrefix at jksql.c:106 as "slow-" plus the main profile's
name, and ex.hg.conf documents slow-db.host, .user and .password.
Where it came from: excludeDbs had been written down in the CART variable
baseline, one registry over, as a name that is not a track-scoped cart
variable. True, but it left the setting described nowhere and suppressed
in the wrong place. The companion commit teaches harvestCartVars.py to
tell the two apart, so it is no longer in that file either.
refs #37925
- src/hg/utils/hgConfCatalog/hgConfCatalog.py - lines changed 16, context: html, text, full: html, text
2f60825bdf72a8d0c99b81552266c9537e9ee2dc Sat Sep 5 08:03:52 2026 -0700
- cartTrackVarCatalog: tell an hg.conf name from a cart name.
An hg.conf setting and a track-scoped cart variable are built the same way.
jksql.c:1325 does safef(cfgName, sizeof cfgName, "%s.excludeDbs",
failoverProf->name) and reads the result with cfgOption on the next line,
which the scan cannot distinguish from safef(buf, size, "%s.heightPer",
track). So excludeDbs sat in cartVarsNotCataloged.txt: correctly, in that
it is not a cart variable, but that left the setting described in no
registry at all and suppressed in the wrong one.
The accessor that reads the buffer back is what tells them apart, so
hgConfRead() looks for a cfg* call taking the same identifier within six
lines of the safef. Six because the read is normally the next line and a
buffer reused later in the function for something else must not excuse an
unrelated name; the destination has to be a plain identifier, or we do not
know what was filled in. It claims exactly one name in the tree today, and
harvestCartVars.py --hgconf prints it with its call site.
--reconcile no longer asks about such a name and --update-baseline no
longer writes it back, so the baseline is 55 names to 54. The reverse case
is now an error rather than a silence: if this catalog ever describes a name
the tree reads with a cfg* accessor, --reconcile says so and exits 1,
because that is one of the two registries being wrong about what the name
is rather than a matter of taste. Verified by planting excludeDbs in the
catalog and confirming the report.
hgConfCatalog gained the row in the companion commit.
refs #37838 #37925
- src/hg/utils/cartTrackVarCatalog/cartTrackVarCatalog.py - lines changed 42, context: html, text, full: html, text
- src/hg/utils/cartTrackVarCatalog/cartVarsNotCataloged.txt - lines changed 1, context: html, text, full: html, text
- src/hg/utils/cartTrackVarCatalog/harvestCartVars.py - lines changed 78, context: html, text, full: html, text
287583e8d72b4abc90ac33694147f604e1238e93 Sat Sep 5 08:17:39 2026 -0700
- cartTrackVarCatalog: walk the 31 names left in the baseline, catalog 23.
The baseline's own header said that a name in it was not evidence anybody
had looked at it, because the first version was accepted wholesale. Every
remaining name has now been read at its call site. Twenty-three turned out
to be real cart variables and get rows; the other 31 are not cart variables
at all, and the header now says which five things they are instead.
The 23:
My Variants edit form, 18 rows, a new group in OTHER_CGIS. The hgc
details page prints a form named <track>_<field>, the form posts so the
values land in the cart like any request variable, and hgTracks reads them
out of the cart on the next render, writes the row to SQL and removes
them. Cart variables by mechanism, one-shot commands by intent, which is
why _id is the trigger and nothing has a default.
<field>FilterLabel, <field>FilterValuesDefault, <field>HighlightType.
Each of these settings is read through cartOrTdbString in three spellings:
filterLabel.<field>, <field>.FilterLabel, <field>FilterLabel. The catalog
had the first and, for four sibling settings, the third; for these three it
had neither, which is the only reason they showed up as unknown while
<field>FilterType did not.
<container>.defaults, the track UI reset button, read with cartUsualInt for
a composite or a superTrack and then used to clear that track's cart
variables and its children's.
<track>.minAc, the VCF minimum allele count, so a reader can hide
singletons with 2 instead of a frequency cutoff.
What the other 31 are: gvfItemName's item labels (_unk, _dnovo and six more
appended to an item's name for display, which is not a variable at all),
HTML element ids, table and db.table names, one submit button read with
cgiOptionalString rather than from the cart, and one printf artifact.
projectSelect is an id, not a name: the <select> that writes it carries no
name attribute, so nothing is ever submitted.
One cost, measured rather than guessed, and written into the new group's
note. The My Variants suffixes are BED field names, so they collide with
hgTables' per-field cart variables, whose last component is also a column
name. In the 6,620 saved sessions that moves 24 names out of
sessionCartAudit's "matched only by a catch-all" bucket and into the known
one. The real cause is in that audit: peel() only tries suffixes beginning
after a separator, so a left-anchored row such as
hgta_fs.check.<db>.<table>.<field> can never match the name it describes,
which is why 4,299 hgta_ names are in that bucket already. Fixing it would
make the longer match win and take the 24 back. Not fixed here; it is the
audit's own bug and its numbers are published.
Catalog 341 rows to 364, baseline 54 names to 31.
refs #37838 #37979
- src/hg/utils/cartTrackVarCatalog/cartTrackVarCatalog.py - lines changed 145, context: html, text, full: html, text
- src/hg/utils/cartTrackVarCatalog/cartVarsNotCataloged.txt - lines changed 40, context: html, text, full: html, text
a88795fb31a7f51bef649eb6eedb626b771a8893 Sat Sep 5 08:20:34 2026 -0700
- urlCommandCatalog: the SNP "Set defaults" button persists in the session.
Fallout from walking the cart baseline: snp125Defaults_coloring is read with
cgiOptionalString at hgTrackUi.c:563, deliberately so, because only a click
in this request should clear the SNP colour variables. Nothing then keeps
the click out of the session. It is in one saved session and four live
carts.
Harmless in itself, since the reset only fires when the value arrives on the
request and a stored copy does nothing, but a one-shot command that persists
is the same defect whatever its blast radius, so it gets a row with
leaks=True.
Two things the row records because they are not obvious from the name. It
looks track-scoped and is not: the prefix is the fixed string
snp125Defaults. And listing it in hgTrackUi's excludeVars would not help as
that array stands, because hgTrackUi.c:4718 has a stray NULL in the middle
of it, so "ajax" and anything added after it are unreachable. That is
latent rather than live: hgTrackUi reads ajax from the cart and removes it
explicitly, and no cart in the three tables holds it. I sorted through
every excludeVars[] in the tree and it is the only one with a NULL before
the end. Not fixed here.
refs #37923 #37979
- src/hg/utils/urlCommandCatalog/urlCommandCatalog.py - lines changed 19, context: html, text, full: html, text
ce8cbfc3d2ef6853f94bf80b0f8b5f389c60025b Sat Sep 5 09:21:32 2026 -0700
- docent: two more expect: checks, a positional click:, and three fixes found writing tests
Everything here was needed by a regression script that could not otherwise be
written, or by one that failed for a reason that was not a bug.
expect: {url:, noUrl:} is a substring check on the current address. Some things
are visible nowhere else: which CGI a click reached, and what a form put in a
query string. #36387's fix strips zero-width characters out of a search term
before the position box submits it, and the character is invisible in the
rendered page, so whether %E2%80%8B survives into the URL is the only evidence
either way.
expect: {has:, noHas:} takes a CSS selector, for a bug whose whole signature is
WHERE something sits. #37785 attached a squishyPack track's center label to the
wrong row: same rows drawn, same image height, same pixels, and only the row the
label's image map hangs off changed, so rows:, height: and text: are all blind
to it. Documented as a last resort, since an assertion on hgTracks' own ids
breaks easily for reasons that are not bugs.
click: now takes the positional forms mouseover: already had (at:/frac:/x:) and
follows the item box nearest that point. Some tracks have no item that can be
named at all: every GIAB Problematic Regions subtrack is type bigBed 3, so
hgTracks writes an EMPTY i= into the hgc href and gives every box the title
"Start of Exon (1/1)". Neither item: nor title: can pick one, and a raw mouse
click on the data area is swallowed by the drag-select handler.
The "item not found" message now picks the row's map boxes by MAP NAME instead of
by a y-band, reports how many boxes are in the row, and falls back to a box's
title when it has no name. On a quickLift target the band test dropped every item
box, so the message said the GIAB row held three things when it held twenty-six,
and the three it named were a density control and two exon arrows.
goShow now scrolls to the top and takes the Login/Share links out of the way
before clicking Search. Those links sit in an absolutely-positioned container at
the top right of the header bar and on a wide page land on top of #goButton;
Playwright then retries for the full 30s and fails with "<a id=loginLink ...>
subtree intercepts pointer events", which reads like a broken Search button
rather than a covered one. Pressing Enter in the position box is not a
substitute: by the time a 30s click timeout has been caught, the navigation wait
armed before it has already expired.
tests/pagechecks and tests/pagechecks.xfail cover all of it, per the rule in
tests/README.txt that a verb we touch and find untested belongs on its list. The
xfail aims all four new checks the wrong way at once, because a check that cannot
fail is not a check.
refs #38252
- src/hg/utils/docent/README.md - lines changed 2, context: html, text, full: html, text
- src/hg/utils/docent/docent.js - lines changed 156, context: html, text, full: html, text
- src/hg/utils/docent/tests/README.txt - lines changed 9, context: html, text, full: html, text
- src/hg/utils/docent/tests/pagechecks.docent.yaml - lines changed 56, context: html, text, full: html, text
- src/hg/utils/docent/tests/pagechecks.xfail.docent.yaml - lines changed 30, context: html, text, full: html, text
e040e51d90b1b7af95a56c14199e4238e6912b48 Sat Sep 5 09:21:54 2026 -0700
- docent: the four regression scripts that were failing, now green
These were written yesterday, failed, and were left out of the last commit so
the suite stayed green. None of the four was a browser bug. Each was the test
being wrong, and each was wrong in a way worth writing into the script.
rm36387 had lost the bug. The term the ticket quotes carries a ZERO-WIDTH SPACE
(U+200B) between "):" and "n.46G>A", which is how it arrived from a clinical
report; hgTracks passed it to encodeURIComponent unstripped, hgSuggest put it in
a LIKE, and MySQL died on a collation mismatch. Without that character there is
no bug at all and the test passes on any build ever shipped, so the term is now
written as a escape. The fixed behavior is also not what the test assumed:
the position box submits to hgSearch, which reports no results for an HGVS term.
The old script waited for #imgTbl and timed out.
rm37805 sat at a position where the track cannot draw. gnomadGenomesVariantsV3_1_1
carries filterValuesDefault.annot pLoF,missense,synonymous, and all 258 v3.1.1
variants in chr17:43044295-43045295 are annotated "other", so the row does not
exist. Moved to chr17:43091000-43092000, inside BRCA1 exon 11, which holds 80
missense, 13 pLoF and 29 synonymous. gnomadVariants is also a superTrack, so
hideKids is needed to keep its coverage siblings off.
rm37326 hovered into a gap. at: sets x from the coordinate but forces y to the
MIDDLE of the row, and in pack mode that track is 324px of stacked guides. It
now hovers by title:, which lands on the item's own row and waits for that item's
tooltip.
rm37553 asserted label text that does not exist. "GFF example" is not a label:
the file says `track name=GFF example description=` with the name unquoted, so
hgCustom takes the name as "GFF". Nor is the description on the page, since
hgTracks puts a longLabel in the control's title attribute, which innerText does
not see. All 46 tracks were loading the whole time.
refs #38252
- src/hg/utils/docent/tests/regress/rm36387.docent.yaml - lines changed 47, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37326.docent.yaml - lines changed 34, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37553.docent.yaml - lines changed 47, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37805.docent.yaml - lines changed 42, context: html, text, full: html, text
e497eccfb431be6af38d22c5bc2bb0e745c89b2d Sat Sep 5 09:22:17 2026 -0700
- docent: eight more regression scripts, one per fixed bug
rm35580 "Show <alt> placed on its chromosome" errored on the window
rm36061 DECIPHER SNVs details missing after a quickLift
rm36335 a quickLifted GIAB item click printed a garbled "can't port"
rm36340 a search-result click carried the source coordinate
rm37491 hgConvert named subtracks the user never showed as not liftable
rm37562 a stale lastPosition banner on the lifted search-results page
rm37615 a quickLifted bigBed tooltip reported pre-lift coordinates
rm37785 a squishyPack track stole a neighbouring track's center label
Each asserts the behavior its ticket says is correct, on genome-test, and names
in noText:/noUrl: the exact string the buggy build produced rather than a generic
"Error". Where the last page has no track image the noText: is paired with a
positive text: or url: check, because a bare noText: passes just as happily on a
blank page or an early error.
Four things found writing these, all of them recorded in the scripts:
A ticket's stated position and its stated item can disagree. #36061 says to go to
chr7:156,982,676-156,996,015 and click item 517898; that item is at
chr7:155,803,191, and decipherSnvs draws nothing at all in the ticket's window.
hideKids cannot express "the user deselected this subtrack". It writes a
<track>=hide cart variable, and isSubtrackVisible() (hg/lib/trackHub.c) begins
with overrideComposite = (NULL != cartOptionalString(cart, tdb->track)) and then
forces enabled = TRUE, so the sibling counts as visible to hgConvert and lands
back in the "failed to lift" warning. A first rm37491 failed on exactly that and
looked like the bug being back. A deselected subtrack is <track>_sel=0 with no
visibility variable, which is what hgTrackUi's checkbox writes and what the goto:
in that script sets.
#37615's fix is not the one the plan for this suite described. The first commit
appended a note saying the coordinates came from the source assembly; the second
replaced it with the real fix, which puts the TARGET coordinates in the tooltip.
A test for the note would fail.
decipherSnvs (SNVs) and decipher (CNVs) are two different members of the
superTrack decipherContainer, and the row that turns up when the SNVs track draws
nothing is the CNVs one, which reads like the same track under a shorter id.
refs #38252
- src/hg/utils/docent/tests/regress/rm35580.docent.yaml - lines changed 46, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36061.docent.yaml - lines changed 46, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36335.docent.yaml - lines changed 55, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36340.docent.yaml - lines changed 50, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37491.docent.yaml - lines changed 60, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37562.docent.yaml - lines changed 46, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37615.docent.yaml - lines changed 45, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37785.docent.yaml - lines changed 47, context: html, text, full: html, text
4be6477ebcb6be9290d5d439cdb48cb714e744b5 Sat Sep 5 11:07:29 2026 -0700
- hgc: let a quickLifted track fall back to the source hub's description again
getHtmlFromSelfOrParent() reaches for the source assembly's description page only
when the lifted track has none of its own, and it tested that with
tdb->html == NULL. The lifted stanza in a quickLift hub carries the source hub's
own relative html path, which cannot resolve from the hub's home in trash, so that
fetch fails and the test used to hold.
1258d7f65e7 made trackHubAddOneDescription return early instead of assigning a
failed fetch, so tdb->html keeps whatever it held rather than becoming NULL. The
test then never holds, getTrackHtml is never called, and a track lifted from a
GenArk assembly loses the description that #37389 added. Ask isEmpty() instead,
which is the question this line meant: an empty description is the same as none
here, and 1258d7f65e7 keeps what it was for.
Bisected against genome-test: the description is drawn with trackHub.c at
v503_base, at f759bf663de and at f001e39d156, and gone at 1258d7f65e7. Not
shipped - v502 and v503 both draw it - so this is a fix before v504 rather than a
patch.
refs #38275
c6b4a4bd0e3024efed0f59f21dc01b5aa3c351f1 Sat Sep 5 09:22:30 2026 -0700
- docent: pin two Closed tickets whose symptom is still live, as xfail
Both were measured on genome-test (v503) on 2026-09-05, both are Closed and out
on the RR, and both are pinned rather than deleted so the suite reports it when
they start passing. Same treatment as rm36540.
rm38272 is NOT a second copy of rm37388. That script covers hgc, where the fix
landed and works. This one follows the same lifted track's own settings link, and
hgTrackUi still does what #37388 described:
Couldn't set connection database to GCA_018466835.2
mySQL error 1049: Unknown database 'GCA_018466835.2'
#37388's fix was per-CGI. 921a40c472e guarded the one call in hgc (hgc.c:4984,
!trackHubDatabase(liftDb) && !isGenArk(liftDb) before hAllocConnTrack) and the
branch in getTrackHtml. hgTrackUi has a path of its own that was never guarded:
specificUi() (hgTrackUi.c:3432) reassigns db to the quickLiftDb setting and hands
it to cfgByCfgType(), labelCfgUi() and extraUiLinks(), which connect to it. Where
the page stops fits -- specificUi() is called at :4202, after the "Remove from
QuickLift" link at :4118, and that link is the last thing the 451-character page
contains.
It reproduces from scratch with no saved session and no hub of ours, and it is
live on genome.ucsc.edu, hgwbeta and genome-test alike: open GenArk
GCA_018466835.2 at CM089257.1:80,200,000-80,360,000, leave the default tracks
alone, QuickLift to hg38, click the gear on RefSeq mRNAs. Filed as #38272. The
script uses braney/crash1 instead because that is the cheap way to the same state
and rm37388 already depends on that session.
rm37389 asserts the track description block that is still absent from a
quickLifted hgc page whose source is a GenArk. Four things were checked rather
than assumed and none of them explains it away: GCA_018466835.2 is in the genark
table, the hub declares its xenoRefGene html file, that file is served 200 from
hgdownload, and the fix (5aed2d465f1) is still in hg/lib/hui.c. hgc prints no
warning of its own, which points at getHtmlFromSelfOrParent (hgc.c:3715) never
calling getTrackHtml, since it only does so `if (liftDb && ...)`. Not chased
further. This is a different path from the hgTrackUi one above, not the same bug
seen twice. The verification session on the ticket does not settle it either:
the only lifted track with clickable items at its position is a bigWig, whose
hgc page has no description section in any build.
refs #38252
- src/hg/utils/docent/tests/regress/rm37389.xfail.docent.yaml - lines changed 60, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm38272.xfail.docent.yaml - lines changed 59, context: html, text, full: html, text
12d4ad442f780318c6a9a7bd93ae64da5019b689 Sat Sep 5 11:22:24 2026 -0700
- docent: rm37389 is no longer an xfail, the bug it pins is fixed
#38275 was the second cause of #37389's symptom: 1258d7f65e7 made
trackHubAddOneDescription return early rather than assign a failed fetch, so
tdb->html kept a non-NULL value and hgc's `tdb->html == NULL` test stopped
reaching the quickLift fallback. Fixed on master in 4be6477ebcb, pushed, and
verified on genome-test once it was rebuilt: the lifted details page is back to
3047 characters, the same as the RR.
So the script started passing and the .xfail had to go, which is the signal the
makefile was built to give. Renamed, and the comment now records both causes -
the v498 fix that works and the commit that defeated it - because a reader who
sees only one of them will not understand why the script is worded as it is.
The assertion is tighter than it was. "Description" alone is the heading
printTrackHtml writes; the script now also asks for a phrase out of the hub's own
description file, which is the only thing that cannot be on the page unless the
file was really fetched from the GenArk hub.
This is the one script in the suite that has watched its behaviour break and come
back, rather than only asserting a fix it never saw fail.
refs #38252
- src/hg/utils/docent/tests/regress/rm37389.docent.yaml - lines changed 0, context: html, text, full: html, text (a binary file or whitespace-only change or file-permission change shows no diff)
b984f60db782dcba2b421556088de0d01295fb51 Sat Sep 5 11:29:58 2026 -0700
- Merge branch 'docentTests37892': a Docent regression test per fixed browser bug, refs #38252
Twenty-five scripts in hg/utils/docent/tests/regress, one per browser bug that has
already been fixed, each asserting the behavior its ticket says is correct against
genome-test. A shared docentTest.mk drives both test directories, and `make
preflight` checks the saved sessions and hub URLs the scripts depend on over HTTP
with no browser, so a fixture that has gone away is told apart from a bug that has
come back. nightly.sh mails a report every night whether or not anything failed,
and runs the committed scripts rather than whatever is in the directory.
Brian's decision, and it shapes every script: assert the fixed behavior only. None
of these was watched to fail on a build that still had its bug, so the whole weight
is on how tight the assertion is - the error string the ticket actually quoted,
`exact:` and `noRows:` over a bare `rows:`.
The renderer gained what the scripts could not be written without. expect: takes
url:/noUrl:, since #36387's whole signature is that a zero-width space does not
survive into a search URL and the character is invisible in the page; and
has:/noHas:, since #37785 drew the same rows at the same height with the same pixels
and only the row its center label hung off changed. click: takes the positional
forms mouseover: already had, because a type-bigBed-3 track gets an empty i= in its
hgc href and the same title on every box, so #36335's item cannot be named at all.
The "item not found" message now picks a row's boxes by map name rather than by a
y-band, which on a quickLift target had been dropping every one of them.
Two scripts are .xfail, pinning Closed tickets whose symptom is live: #36540, and
#38272, which was filed off the back of this work. A third, rm37389, was an xfail
for a few hours until #38275 was fixed, and is now an ordinary test - the only one
here that has watched its behavior break and come back.
Four of the forty tickets this suite was planned around turn out never to have been
fixed: the candidate pool was Bug tickets *closed* in GB, and Closed includes
Hibernating and Rejected. The target is 36.
cdb348c551227ac24dc0fab9f18516459b47b962 Sat Sep 5 11:33:32 2026 -0700
- hgTrackUi: don't try MySQL for a GenArk quickLift source assembly, refs #38272
The track settings page swaps in the track's quickLiftDb for the database it
passes down to the cfg routines. For a GenArk source that is a bare accession
with no MySQL database behind it, and asForDb() connected anyway, so the page
stopped after the display-mode row. trackHubDatabase() does not catch this
because the source hub is not loaded in that request; hgc got the matching
isGenArk() guard in #37388.
Two filter blocks in the same file guarded with isHubTrack(), a plain hub_
prefix test that a bare accession also slips past. Same guard added there.
2a24179e43152309beca3dd0311504e77cb35be6 Sat Sep 5 11:35:47 2026 -0700
- Merge branch 'quickLiftUi38272': hgTrackUi settings page for a quickLifted GenArk track, refs #38272
7f68cb2a6a108e5b09074ce5a08c3b6a78509fe9 Sun Sep 6 07:31:02 2026 -0700
- docent: rm38272 is no longer an xfail, the bug it pins is fixed
The nightly run mailed FAIL this morning with 24 ok and rm38272.xfail passing,
which is the signal the makefile was built to give. cdb348c5512 added the
isGenArk() guard to asForDb() and to two trackDbFilter blocks in hui.c, so
hgTrackUi no longer tries a MySQL connect to a quickLifted GenArk source
assembly. genome-test picked it up overnight. Renamed, and the comment now
records the fix rather than the symptom.
The assertions are tighter than they were, and for the same reason rm37389's
were. "RefSeq mRNAs Track Settings" was the only positive check, and the
451-character broken page carries that heading too, so it never told the two
apart -- the noText on the MySQL warning was doing all the work. The checks are
now on "Color track by codons" and "Data schema/format description and
download", which come from cfgByCfgType() and extraUiLinks(), two of the three
routines that were handed the bad db, and neither is in the broken page.
Both pages were measured on 2026-09-06, not inferred: hgwbeta still returned the
451-character page with the warning while genome-test returned 2800 characters
with none. The fix ships in v504.
refs #38252
- src/hg/utils/docent/tests/regress/rm38272.docent.yaml - lines changed 54, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm38272.xfail.docent.yaml - lines changed 59, context: html, text, full: html, text
7f5ea857be45e43cece2cfb37093c6c51b23f79d Sun Sep 6 08:23:15 2026 -0700
- docent: run the nightly regression out of a checkout of its own
The header of this script has said since it was written that once the tests were
on master the cron should read them out of a clone of its own, the way
catalogNightly has one under /hive/users/braney, so that an ordinary day's
editing in ~/kent cannot change what the cron measures. The suite landed on
master yesterday, so this does it. The clone is
/hive/users/braney/docentNightly/kent, beside the log directory the job already
writes to. Nothing here is built, so it needs no submodules and no make.
--update brings that clone to origin/master and then re-runs this script from
the result. The re-exec is why this is a flag here rather than a separate driver
script beside the crontab: the whole job stays in the tree where it can be
reviewed and committed, and a change to this file takes effect the same night as
a change to a test instead of a night later.
--update will only ever touch a checkout that is a pristine mirror of
origin/master. A working tree with uncommitted edits, or one holding a commit
that has not been pushed, is left alone and reported rather than reset, because
`reset --hard` in ~/kent would throw away work. That guard is what makes it safe
for the flag to live in a script that also sits in a working tree. Both refusals
mail with the same subject shape as a finished run, since a night when nothing
ran must not read as a quiet night.
refs #38252
- src/hg/utils/docent/tests/regress/nightly.sh - lines changed 61, context: html, text, full: html, text
62132f503d586f5172a397772c6ca6af3e8eccfb Sun Sep 6 13:16:05 2026 -0700
- docent: four more regression scripts, three of them watched to fail
29 scripts now, 28 of the 36 tickets that can have one. All green against
genome-test.
The suite's standing weakness is that almost nothing in it was ever seen to fail
on a build that still had its bug, so the whole weight sits on assertion
tightness. Three of these four do not have that problem. hgwbeta is running
v503 and two of these fixes ship in v504, so the broken behavior was still there
to be read this morning and each assertion was checked against it:
rm38185 empty CGI pair. hgwbeta lands on chr7:155,799,529-155,812,871,
hg38's default, because the position= after the && is silently lost;
the same pair at the end returns "Mangled CGI input string &". The
assertion is the position itself, since losing a variable produces no
error to look for.
rm38126 hub description filtering. The fixture hub is ours and deliberately
malformed. Every structural check flips: on hgwbeta the chosen id,
the script element and the form input are all still in the DOM, and
on genome-test the id carries the descPage- prefix and the other two
are gone. has:/noHas: rather than text:, because a script element's
contents are not in innerText on either build -- a noText: there would
have passed everywhere and asserted nothing. The two text checks
deliberately do not flip: they are what stops the run passing on a
page where the description simply failed to load.
rm36836 a GenArk assembly and an assembly hub opened by URL. This one was
only ever broken on hgwdev, so genome-test is the only server it could
have been caught on. It was fixed by reverting #36835, so what the
script guards is a second attempt at that work.
rm38108 the caller-supplied upload address. No server left to watch it fail
on: the fix is in v503_branch so hgwbeta has it, and the RR is still
on v502 and so still vulnerable -- deliberately not driven there,
since the reproducer's whole effect is to kill the CGI serving it.
hgSession shares the fix but answers the crafted GET with the sign-in
page, so it is not in the script.
New fixture ~braney/docentFixtures/descFilterHub, for rm38126 only. hubCheck
reports its script and form on purpose: that reporting is the other half of
#38126, and the hub is not to be tidied up.
refs #38252
- src/hg/utils/docent/tests/regress/rm36836.docent.yaml - lines changed 40, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm38108.docent.yaml - lines changed 57, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm38126.docent.yaml - lines changed 61, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm38185.docent.yaml - lines changed 48, context: html, text, full: html, text
2b8d9275548bcc9bd161b68f24848b545e1069ef Sun Sep 6 14:00:28 2026 -0700
- docent: click raw:, and the first two scripts that do anything twice
Every one of the 29 scripts in the suite was a straight line: fresh cart, a few
steps, assert once. A bug that only exists on the repeat was invisible to that
shape, and the closed-bug pool holds several. These are the first two that
repeat a gesture.
rm36805 click a TOGA item, dismiss the pop-up, click the same item again.
One click passes on the broken build as happily as on the fixed one,
so the second click is the test.
rm27113 three clicks on the centre of the base-position ruler. A single
click zooms 3x about the base under the cursor, and the bug drifted
that centre one base left each time.
Both needed a gesture the language could not express. click: {track, item}
follows the item's own map-box href, which is right when the assertion is about
the hgc PAGE, but hgTracks answers a real click with an ajax DIALOG
(popUpHgcOrHgGene.hgc) and following the href never opens one. raw: true
presses the mouse where a user presses it and lets the page answer -- a
navigation, a dialog, or a new image in place, whichever arrives, waited for
rather than slept through. With no item name it is a bare point on the row,
which is the only way to click the ruler at all: the ruler carries no hgc map
boxes for areaXY to snap to.
Two things measured while writing these, both in the script comments. jQuery UI
HIDES a dialog on close rather than removing it, so the assertion has to be
has: "#hgcDialog:visible" -- without :visible the second half of rm36805 passes
whether or not the pop-up ever comes back. And one base is invisible at the
13kb windows the rest of the suite uses, where a pixel is fourteen bases, so
rm27113 starts at 100 bases and its three expected windows are arithmetic rather
than three observed strings: every one of them is centred on base 155,806,200.
#37014 is the other repeat-click bug in the pool and is deliberately NOT written.
Its reproducer no longer reaches the code it was about: 9e9ee32a4c8 (#37878)
later excluded crossTissue* tracks from the pop-up path altogether, and the
session's track is crossTissueMapsTissueCellType, so the click now navigates and
no dialog is involved. Writing it against a different bar chart track would be
pinning the ticket to something it was never about.
refs #38252
- src/hg/utils/docent/README.md - lines changed 1, context: html, text, full: html, text
- src/hg/utils/docent/docent.js - lines changed 26, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm27113.docent.yaml - lines changed 56, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36805.docent.yaml - lines changed 48, context: html, text, full: html, text
79aa96ab96eaf59b3d4779e60b9e5a4f012884f3 Sun Sep 6 15:04:38 2026 -0700
- docent: three scripts that assert a value rather than the shape of a page
34 scripts, all green.
I told Brian this class had no coverage. That was wrong, and the correction
matters more than the three scripts: rm36061 already asserts five fields off a
Decipher hgc page, rm37615 asserts an exact lifted coordinate and names the
source coordinate in noText:, and rm37326 asserts a CRISPR score triple. What is
thin is not value assertions, it is value assertions anywhere other than an hgc
page reached by clicking an item. These three are each somewhere else.
rm37489 a lifted track printed the TEMPLATE of its dataVersion setting,
"/gbdb/$D/bbi/clinvar/version.txt", where a release date belongs.
The positive check is the prefix "ClinVar Release:" only -- the date
after it changes every month and asserting it would fail on the next
ClinVar update, which is not a regression.
rm36810 trackDb's `urls` statement stopped turning data fields into links.
The assertion is the HREF, a value the browser BUILT from the data
rather than text it copied, and it asks for /clinvar/RCV specifically:
every ClinVar hgc page carries NCBI links in its boilerplate, so a
bare ncbi.nlm.nih.gov would have passed on the broken page. Measured:
exactly one RCV link on the page, and it is the substituted one.
rm35865 hgGene's Microarray Expression Data section was empty. The heading is
on the page either way, so the two data set names under it are the
check. First script here to open hgGene at all.
Three things learned, all in the script comments. A lifted session cannot be
reused: Gerardo's RM_37489 comes back as plain hg38 with no hub_ prefix on
anything, because a lift dies with the trash it points at, so a script that needs
one has to make it. A lifted composite has no hgTrackUi link for its subtrack --
every settings link points at the container, and there are four of them. And
hgGene writes its sections with style='display:none', so expect: text cannot see
the contents until the + is clicked; a has: written to dodge that would have
passed on the empty page the ticket is about.
Nothing in these three names a transcript accession or a variant accession.
Those are data, and the next knownGene or ClinVar build can retire them. Items
are taken by gene symbol or by position instead.
refs #38252
- src/hg/utils/docent/tests/regress/rm35865.docent.yaml - lines changed 50, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm36810.docent.yaml - lines changed 44, context: html, text, full: html, text
- src/hg/utils/docent/tests/regress/rm37489.docent.yaml - lines changed 50, context: html, text, full: html, text
a8459150ae249f002ecf6d4213ca0ef8fd6a620e Mon Sep 7 10:01:14 2026 -0700
- v504 preview1 (automated)
- src/utils/qa/weeklybld/buildEnv.csh - lines changed 2, context: html, text, full: html, text
switch to files view, user index