All File Changes

v503_base to v504_preview (2026-08-31 to 2026-09-07) v504

• confs/hgwdev.hg.conf
  • lines changed 15, context: html, text, full: html, text
    ea3205d8275e1096ebd1c4c6e25c79d79533770e Sun Sep 6 01:11:15 2026 -0700
    Installing updated hg.conf files from UCSC servers
• docs/tutorials/customTrackTutorial.md
  • lines changed 1, context: html, text, full: html, text
    caa05e37c4fe0649d3b26e69712244d90291b1b5 Tue Sep 1 14:52:44 2026 -0700
    Fixing broken links that I found, no Redmine
• docs/tutorials/gb101.md
  • lines changed 1, context: html, text, full: html, text
    caa05e37c4fe0649d3b26e69712244d90291b1b5 Tue Sep 1 14:52:44 2026 -0700
    Fixing broken links that I found, no Redmine
• src/CLAUDE.md
  • lines changed 9, context: html, text, full: html, text
    870b28e103151338d1a2f1ad58099f2a086225c8 Fri Sep 4 17:30:27 2026 -0700
    Remove src/CLAUDE.md from the tree. The file is down to a pointer at two skills, edit-kent-code and make-track, and both of those live only in the private genecats repo. The kent tree is mirrored publicly on GitHub, so anyone reading our source there is told to load two files they cannot get. The C and SQL rules that used to be in this file moved into edit-kent-code in c9093b16dc8. refs #37358
• src/blat/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/altSplice/affySplice/altProbes.c
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/cgilib/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/cgilib/cartJson.c
  • lines changed 5, context: html, text, full: html, text
    f759bf663debc22e6015c6c38c22f922bfbdf45b Fri Aug 21 10:01:02 2026 -0700
    lib, cgilib: sanitize description HTML where it comes in, refs #38126 A track hub's track description, a custom track's documentation, and an assembly hub's genome description are all written by somebody else and were stored and printed as they arrived. Run each through htmlSanitize() at the point it is read, so that everything downstream sees text we are willing to print. Sanitizing on the way in is the only practical place: printTrackHtml() alone has more than a hundred callers, and hgTrackUi, hui.c, hgGene, hgGtexTrackSettings and hgTables all print tdb->html directly. The five places are trackHubAddOneDescription(), customTrack.c where it used to call jsStripJavascript(), the htmlUrl fetch in customFactory.c, hgPositionsHelpHtmlCart() and hAssemblyDescription(). In the last two, only the branch that fetches over the network is touched. The branch that reads a local file is left as it is, because those are our own description.html files. trackHubAddOneDescription() keeps its old shape. The fetch moves into a helper so that trackHubDescriptionRemovals() can hand hubCheck the list of things the filter takes out.
• src/hg/cgilib/pgPolyphenPred.c
  • lines changed 10, context: html, text, full: html, text
    752e20a9a1dc08793c5b529bbeac4ce5836a6be4 Fri Sep 4 17:13:55 2026 -0700
    hgc: stop five details-page handlers from aborting over a missing piece. chromSeqFileExists() opened its connection with sqlConnect() and only then asked whether the database existed, so it aborted before it could answer. The otherDb of a chain or net track is often not a local database at all -- a GenArk hub assembly, or one long retired -- so use sqlMayConnect() and return FALSE. This is 112 of the failures the TrackCheck robot reports, most of them hg38 net tracks against HPRC assemblies. Also moved the disconnect out of the if, where it leaked a connection whenever a database had no chromInfo table, and dropped the now-redundant sqlDatabaseExists() call, which was itself a second connection. mgcCloneInfoLoad() aborted when a clone carried no MGC: id in hgFixed.mrnaClone. Nothing on the page reads that field, and a clone can legitimately have only an IMAGE: id, so the check went away rather than the page. hDbOrganism() aborted for an assembly that has left dbDb but is still named by a maf component, which hg16.evofold does via mm3; it now falls back on the database name. The pgSnp SIFT and Polyphen prediction tables are loaded separately from the tracks that name them, so a machine can have the track and not the table, as hgwbeta and the RR do for hg18. Check with hTableExists first and say the predictions are unavailable instead of letting the query take the page down. Same treatment for the RNA fold diagram: a non-zero ghostscript exit now drops only the diagram and keeps the rest of the page, including the PseudoViewer link. Two of these report an unavailable piece through warn(), which still marks the page for the robot. That is deliberate -- the missing hg18 tables and the RNA fold diagram are real defects, and the log should keep naming them until they are fixed. On the RNA fold diagram in particular: RNAplot truncates the sequence id it is given to 42 characters, and the trash path we build is already 41, so it has never written the file ghostscript is asked to convert. That is worth its own fix. refs #37424
• src/hg/cgilib/pgSiftPred.c
  • lines changed 10, context: html, text, full: html, text
    752e20a9a1dc08793c5b529bbeac4ce5836a6be4 Fri Sep 4 17:13:55 2026 -0700
    hgc: stop five details-page handlers from aborting over a missing piece. chromSeqFileExists() opened its connection with sqlConnect() and only then asked whether the database existed, so it aborted before it could answer. The otherDb of a chain or net track is often not a local database at all -- a GenArk hub assembly, or one long retired -- so use sqlMayConnect() and return FALSE. This is 112 of the failures the TrackCheck robot reports, most of them hg38 net tracks against HPRC assemblies. Also moved the disconnect out of the if, where it leaked a connection whenever a database had no chromInfo table, and dropped the now-redundant sqlDatabaseExists() call, which was itself a second connection. mgcCloneInfoLoad() aborted when a clone carried no MGC: id in hgFixed.mrnaClone. Nothing on the page reads that field, and a clone can legitimately have only an IMAGE: id, so the check went away rather than the page. hDbOrganism() aborted for an assembly that has left dbDb but is still named by a maf component, which hg16.evofold does via mm3; it now falls back on the database name. The pgSnp SIFT and Polyphen prediction tables are loaded separately from the tracks that name them, so a machine can have the track and not the table, as hgwbeta and the RR do for hg18. Check with hTableExists first and say the predictions are unavailable instead of letting the query take the page down. Same treatment for the RNA fold diagram: a non-zero ghostscript exit now drops only the diagram and keeps the rest of the page, including the PseudoViewer link. Two of these report an unavailable piece through warn(), which still marks the page for the robot. That is deliberate -- the missing hg18 tables and the RNA fold diagram are real defects, and the log should keep naming them until they are fixed. On the RNA fold diagram in particular: RNAplot truncates the sequence id it is given to 42 characters, and the trash path we build is already 41, so it has never written the file ghostscript is asked to convert. That is worth its own fix. refs #37424
• src/hg/encode/getTrackReferences/getTrackReferences
  • lines changed 5, context: html, text, full: html, text
    52432895f426bc722014061bc7a79eb1ca8b025d Sun Sep 6 06:55:18 2026 -0700
    getTrackReferences: abort with an error on stderr when the NCBI full-text link cannot be fetched, instead of printing the error onto stdout where it ends up in the middle of the generated references. refs #38265
  • lines changed 85, context: html, text, full: html, text
    e3962943daee77f7327e2a49ba894f157646ec68 Sun Sep 6 07:02:29 2026 -0700
    getTrackReferences: get the full text link from the eutils elink API. Scraping the full-text-links box out of the PubMed web page stopped working: NCBI now answers non-browser clients with a Javascript cookie challenge, so the page never contains any links and every citation fell back to the PubMed URL. Use elink/prlinks instead, resolve the doi.org links it returns to the publisher page, and retry on the eutils 429 rate limit. refs #38265
• src/hg/hgApi/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgBlat/hgBlat.c
  • lines changed 6, context: html, text, full: html, text
    42b33234bee213325c2b6eede35edbb4fb2e566c Thu Sep 3 06:40:30 2026 -0700
    hgBlat: spinner while a search runs, shorter example, left-justified result columns Four items from Lou's note-37 on #38086. The search is a plain form POST, so nothing on screen changed between Submit and the arrival of the results - on an assembly served by a dynamic BLAT server that is a ten second wait with no sign that anything is happening. A busy overlay now covers the form: a generic .gbBusyBg/.gbSpinner in gbModern.css (CSS-drawn, since the page is already navigating away when it appears), put up by blatFormBusyWire() 350 ms after submit so a sub-second hg38 search does not flash a scrim, and swapping its text after 8 s to say that an index is probably being loaded on demand. Skipped for Clear and for an empty query; a pageshow handler takes it down again after a Back navigation. The example sequence goes from 13.8 kb of the SOD1 locus to 2.5 kb of PTP4A3, hg38 chr8:141,424,501-141,427,000. That window is carried by both chr8_KI270819v1_alt and chr8_MU273363v1_fix, so the example now demonstrates the alt and fix rows and their FAQ icons instead of returning one plain hit, and it covers two PTP4A3 exons so the result track is named after the gene. Every column of the results table is left-justified; the numeric ones keep tabular figures, and the score value gets a fixed width so its bars stay in a straight line. The alt/fix explanation icon is red rather than faint grey, and is drawn as the browser's own info-icon SVG rather than the U+24D8 glyph, which some system fonts do not have. refs #38086
  • lines changed 7, context: html, text, full: html, text
    994528edb7d89068f371fef6d8a5fc72566f7334 Fri Sep 4 13:35:16 2026 -0700
    Adding an opt-in "Keep only last search" checkbox to the new BLAT search form. refs #36292 Behind a new hg.conf setting, blatOnlyLatestCheckbox (default off, documented in ex.hg.conf). When checked, each new BLAT search removes the user's earlier BLAT result tracks in hgc's buildBigPsl (cart variable blatOnlyLatest, remembered per browser in localStorage); unchecked keeps the current accumulate behavior, so site defaults are unchanged. Deletion requires both the hg.conf setting and the cart variable, so a stale cart cannot delete tracks where the feature is off. Independent of, and coexists with, the blatOldTracks hg.conf setting.
  • lines changed 9, context: html, text, full: html, text
    cc23b76a1f2c7dc6a29c227661bfb0ce4d20eb54 Fri Sep 4 13:43:48 2026 -0700
    BLAT page polish from QA: Load example button, corrected input limits, wording fixes. refs #37996 Load example is a chip-style button with a mouseover instead of a hyperlink. Removed the Show input limits modal; corrected the limits in hgTracksHelp.html, stale since 2013 (DNA is 75,000 bases per sequence and 187,500 combined, protein and translated 10,000 and 25,000), added a #blatLimits anchor and pointed the two over-limit BLAT warnings and the form at it. Genome selector: section title Target assembly, type-to-search hint in the dropdown, Shortcuts: label, input and Query type select sized to match. Similar tools sidebar uses house-style hyphens and corrects the findMotif name and description. Share and rename text now states link lifetimes accurately (snapshot links last years, only sessions are permanent). Alignment page buttons match .gbPill, summary strip spacing fixed, New BLAT search button demoted from primary. Documents blatResultsGroup in ex.hg.conf.
  • lines changed 15, context: html, text, full: html, text
    7a69a5535f1f66f23f5b1329ecbe508983ea01fa Fri Sep 4 14:44:10 2026 -0700
    BLAT classic form: announce the new-page switch date via hg.conf, fix stale input limits. refs #37996 New hg.conf setting blatNewFormSwitchDate (documented in ex.hg.conf): when set, the classic form's banner announces "We will be updating this BLAT page on <date>" with an opt-in link, a feedback mailto, and a pointer to the news announcement; unset keeps the previous "We are testing" banner. The date lives in hg.conf so a schedule slip is a config change, not a build patch. Also corrects the classic form's stated input limits (25,000/50,000 bases, stale since 2013) to the enforced 75,000 per sequence and 187,500 combined.
  • lines changed 7, context: html, text, full: html, text
    a9c71c94309db1f28fc29cbb2d9c062e79c7201f Fri Sep 4 14:58:18 2026 -0700
    BLAT dated banner: make the news announcement URL an hg.conf setting. refs #37996 blatNewFormNewsUrl, defaulting to the news archive page, so the banner can point at the announcement's anchor once it is published without a CGI rebuild. Documented in ex.hg.conf.
• src/hg/hgCollection/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgFileSearch/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgFileUi/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgGateway/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgGene/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgGenome/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgGenome/upload.c
  • lines changed 3, context: html, text, full: html, text
    f247b79b61ca172b3146284594b97c848db4c8bf Wed Sep 2 11:20:43 2026 -0700
    hgSession, hgGenome: tighten handling of an uploaded file's name, refs #38172 Both CGIs print the name that arrives with an upload straight into the page. hgSession prints it in four of its load messages, so encode it once where it is read; the sibling URL message three lines above already did the same for the URL. hgGenome puts it in a quoted attribute, which wants attributeEncode. Raised in the v503 Preview II code review.
• src/hg/hgIntegrator/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgLiftOver/hgLiftOver.c
  • lines changed 1, context: html, text, full: html, text
    29179bd78a99f965fd88c46606d9d9af98aa9431 Tue Sep 1 15:24:26 2026 -0700
    Delete stray formMethod value in hgLiftover form, refs #38223
• src/hg/hgLogin/hgLogin.c
  • lines changed 213, context: html, text, full: html, text
    1d079d218c274f09b61ec390619782e0c505d157 Tue Sep 1 06:44:56 2026 -0700
    hgLogin: confirm the recovery email address by mail, refs #38197 The optional recovery address given at signup is now mailed a signed confirmation link, and counts for sign-in and password recovery only once that link has been opened. The link is signed with login.cookieSalt, expires after a week and works exactly once. Adds gbMembers.recovEmailVerified through the usual auto-upgrade. Existing rows default to 'Y', so recovery addresses set up before this keep working and nobody has to re-confirm one. An install that sends no mail, or that has no login.cookieSalt to sign a link with, stores the address as it does today.
  • lines changed 241, context: html, text, full: html, text
    d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
    hgLogin: let users set or change their recovery email address, refs #38197 Adds a page where a signed-in user can set or change the recovery address on their account, offered in the account menu next to "Change email" -- both in the top right blue bar popup and on the session page. It is off by default: set login.recovEmailChange=on in hg.conf to offer it. The page also needs login.cookieSalt, working outbound mail and the recovEmailVerified column, and stays hidden where any of those is missing. The new address is confirmed by mail before it takes effect, so whatever is on the account keeps working until the link is opened and a typo costs the user nothing. An account that has a password must supply it, since a confirmed recovery address can sign in. Once the address does change, the account's main address is told, the same notice that a change of the main address already sends. One signature now covers both the address given at signup and a later change, so there is a single confirmation path rather than two.
  • lines changed 14, context: html, text, full: html, text
    0b638d6d33466fc848c63d261eab155a22741bdf Wed Sep 2 16:58:32 2026 -0700
    Changing hgLogin's "choose a username" page back to the original generic text for ORCID and CILogon sign-ins, per code review feedback on #38213 (the provider-name check was unreliable and hardcoded the mirror name), refs #37929
  • lines changed 3, context: html, text, full: html, text
    3eb8de69a3d713dbf37a1b1f056cefe73de785e1 Wed Sep 2 17:11:05 2026 -0700
    Updating hgLogin's Change password, Change email, and Sign up using email headings to sentence case, matching the rest of the file, per code review feedback on #38213, refs #37929
• src/hg/hgMenubar/hgMenubar.c
  • lines changed 5, context: html, text, full: html, text
    d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
    hgLogin: let users set or change their recovery email address, refs #38197 Adds a page where a signed-in user can set or change the recovery address on their account, offered in the account menu next to "Change email" -- both in the top right blue bar popup and on the session page. It is off by default: set login.recovEmailChange=on in hg.conf to offer it. The page also needs login.cookieSalt, working outbound mail and the recovEmailVerified column, and stays hidden where any of those is missing. The new address is confirmed by mail before it takes effect, so whatever is on the account keeps working until the link is opened and a typo costs the user nothing. An account that has a password must supply it, since a confirmed recovery address can sign in. Once the address does change, the account's main address is told, the same notice that a change of the main address already sends. One signature now covers both the address given at signup and a later change, so there is a single confirmation path rather than two.
• src/hg/hgPal/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgPcr/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgPhyloPlace/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgPhyloPlace/hgPhyloPlace.c
  • lines changed 1, context: html, text, full: html, text
    95807c7178fd38d52564f06df9fd5b90ceca1d24 Wed Sep 2 11:20:44 2026 -0700
    hgTracks, hgc, hgPhyloPlace: small fixes from the v503 Preview II review, refs #38172 hgTracks.c: skipBeyondDelimit returns NULL when the delimiter is absent, and the caller decremented and printed it without checking. The noYearDbs list keeps every assembly that reaches it carrying a '(' today, so nothing shows, but printf used to print "(null)" where htmlEncode now walks off the end. Fall back to the whole freeze name. gtexTracks.c: the guard before the in place truncation was two bytes stricter than the buffer needs, so a description just over the budget printed in full. Say what the buffer requirement actually is. vcfTrack.c has the same shape a byte the other way and is already right. hgPhyloPlace.c: initialize size, as the two sibling call sites do. cgiMemBlobFind always sets it when it returns a block, so this is for consistency. bigBedClick.c: hubEncode was called twice on the same string in one statement.
• src/hg/hgPublicSessions/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgSession/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgSession/hgSession.c
  • lines changed 15, context: html, text, full: html, text
    992880730e0f4acdc4ba941906059b0dc2e3ce1d Mon Aug 31 14:50:10 2026 -0700
    hgSession: take the session description back out of the cart, refs #38205 The Sessions page has a Description box. The text stayed in the cart after the page was done with it, so every session the user saved from then on kept a copy, including sessions they never described. On hgwdev 41 saved sessions carry a description that belongs to a different session. cleanHgSessionFromCart() now removes hgsNewSessionDescription and hgsNewSessionName. That function already runs in the two places that matter: at the end of hgSession(), so the user's own cart is clean, and inside saveCartAsSession() just before the cart is encoded, so no saved session holds them. Adding the names to excludeVars[] instead would not work, because doSessionChange() reads the description back out of the cart in the same request that posts it. hgsNewSessionShare has the same problem but is left alone. It is also the only memory of the user's "Allow this session to be loaded by others" choice, and showSavingOptions() defaults that box to checked, so removing it would quietly re-check the box for someone who keeps their sessions private. doNewSession() and doReSaveSession() now clone the session name before use. Both read it straight out of the cart's hash, and the new cartRemove inside saveCartAsSession() frees that copy while they still hold it.
  • lines changed 95, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
  • lines changed 3, context: html, text, full: html, text
    d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
    hgLogin: let users set or change their recovery email address, refs #38197 Adds a page where a signed-in user can set or change the recovery address on their account, offered in the account menu next to "Change email" -- both in the top right blue bar popup and on the session page. It is off by default: set login.recovEmailChange=on in hg.conf to offer it. The page also needs login.cookieSalt, working outbound mail and the recovEmailVerified column, and stays hidden where any of those is missing. The new address is confirmed by mail before it takes effect, so whatever is on the account keeps working until the link is opened and a typo costs the user nothing. An account that has a password must supply it, since a confirmed recovery address can sign in. Once the address does change, the account's main address is told, the same notice that a change of the main address already sends. One signature now covers both the address given at signup and a later change, so there is a single confirmation path rather than two.
  • lines changed 93, context: html, text, full: html, text
    8a648c2ffc0e2c8a671a9074e00a0a3d56575589 Wed Sep 2 04:48:57 2026 -0700
    Merge remote-tracking branch 'origin/master' # Conflicts: # src/hg/hgSession/hgSession.c
  • lines changed 4, context: html, text, full: html, text
    f247b79b61ca172b3146284594b97c848db4c8bf Wed Sep 2 11:20:43 2026 -0700
    hgSession, hgGenome: tighten handling of an uploaded file's name, refs #38172 Both CGIs print the name that arrives with an upload straight into the page. hgSession prints it in four of its load messages, so encode it once where it is read; the sibling URL message three lines above already did the same for the URL. hgGenome puts it in a quoted attribute, which wants attributeEncode. Raised in the v503 Preview II code review.
  • lines changed 2, context: html, text, full: html, text
    89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
    snapshotSession: rename the snapshot "reaper" to "cleaner" Renames the snapshotReaper utility to snapshotCleaner and drops the word "reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and the doc/usage text now say clean/cleaned/cleaner). No behavior change. refs #38197
  • lines changed 9, context: html, text, full: html, text
    ef8d1004e3d984933a79686a7369d74825886c48 Fri Sep 4 11:35:31 2026 -0700
    snapshotSession: reject a snapshot missing its required var A "blat" snapshot is a dead link without blatLastBigBed (the pinned bigPsl). That variable is set by an async hgc buildBigPsl call, so a share clicked before the build finished - or after it failed - would have minted a link that reopens to nothing. A snapshotType may now name a requiredVar, and doSaveSessionJson refuses the save (asking the caller to retry) instead of handing out a broken link. Found in code review of the BLAT share wiring. refs #38197
• src/hg/hgSession/hgSession.h
  • lines changed 10, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
• src/hg/hgSuggest/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgTables/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgTrackUi/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgTracks/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgTracks/bedTrack.c
  • lines changed 5, context: html, text, full: html, text
    9f5a3871b95f595b5a218f0d05395a8c72d58844 Tue Aug 25 12:57:10 2026 -0700
    remove the bigBedOnePath fallback, leaving a single bigBed load path, refs #36940 bigBedOnePath has defaulted to on since v492 and the setting has been removed from the hg.conf of the RR, Euro, Asia and hgwbeta. Drop the check and the code it guarded at all four sites: the field-count default in bigBedAddLinkedFeaturesFromExt, the method setup in commonBigBedMethods, the bedSize default in bigBedClick, and the bigBed case in hgc. The two branches in the hgc case differed only in a minimum the surviving path does not want. complexBedMethods lost its only isBigBed=TRUE caller with that branch, so drop the parameter. Retire the setting from the hg.conf catalog and its gate backlog, and repair the two hgc.c citations whose line numbers this commit shifted, refs #37925. Verified pixel-identical: 14 scenarios rendered before and after, covering the four QA sessions on the ticket plus mm10 knownGene, dbSnp155, rmsk, encRegTfbsClustered, clinvar and tandemDups. All AE=0.
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/hgTracks/bigBedTrack.c
  • lines changed 50, context: html, text, full: html, text
    9f5a3871b95f595b5a218f0d05395a8c72d58844 Tue Aug 25 12:57:10 2026 -0700
    remove the bigBedOnePath fallback, leaving a single bigBed load path, refs #36940 bigBedOnePath has defaulted to on since v492 and the setting has been removed from the hg.conf of the RR, Euro, Asia and hgwbeta. Drop the check and the code it guarded at all four sites: the field-count default in bigBedAddLinkedFeaturesFromExt, the method setup in commonBigBedMethods, the bedSize default in bigBedClick, and the bigBed case in hgc. The two branches in the hgc case differed only in a minimum the surviving path does not want. complexBedMethods lost its only isBigBed=TRUE caller with that branch, so drop the parameter. Retire the setting from the hg.conf catalog and its gate backlog, and repair the two hgc.c citations whose line numbers this commit shifted, refs #37925. Verified pixel-identical: 14 scenarios rendered before and after, covering the four QA sessions on the ticket plus mm10 knownGene, dbSnp155, rmsk, encRegTfbsClustered, clinvar and tandemDups. All AE=0.
• src/hg/hgTracks/bigWigTrack.c
  • lines changed 8, context: html, text, full: html, text
    07ba37612b12dad969ed452c55780c20b7d65712 Wed Sep 2 17:03:38 2026 -0700
    hgTracks: do not crash when a quickLift chain has no aligned block in the window chainLoadIdRangeHub returns every chain whose span overlaps the window, but only loads the link rows that fall inside it. A chain can therefore reach across the window with no aligned block in it. Its blockList is empty, chainSubsetOnT has nothing to subset, and it returns NULL. bigWigLoadPreDraw dereferenced that NULL and hgTracks died with SIGSEGV, so the user got a blank page. This is what happens on the smaller of the two regions hgConvert offers for a quickLift from hg38 to hs1. Three of the four chains whose span covers hs1 chr7:59,324,765-59,325,193 have no link there. Widen the view past 100 kb and those chains pick up blocks again, which is why only the narrow views crashed. The rest of the quickLift code already skips a chain with an empty blockList. Do the same here. refs #38236
  • lines changed 26, context: html, text, full: html, text
    5dd36916814777c4c330de1b41b33c33fbcf758d Wed Sep 2 17:42:43 2026 -0700
    hgTracks: place a quickLift bigWig block by the window, not by the chain bigWigLoadPreDraw scaled each aligned block by the span of its own chain, but the preDraw buffer covers the window. A chain that only partly covers the view was therefore stretched across the whole image, and when several chains were in the window each stretched independently and overwrote the others. Scale by (winEnd - winStart) instead. chainSubsetOnT clips the blocks to the window, so a block cannot reach past the end of the buffer, but summary[] is exactly summarySize long with no slack, so clamp anyway. Measured on a quickLift from hg38 to hs1, at hs1 chr7:59,324,000-59,326,000. One chain covers 429 of those 2,000 bases, which is x 516 to 717 of the image. Before, the signal was painted from x 156 to 1098, so 625 columns carried a value where nothing is aligned. After, it is painted from x 516 to 717 and no column outside an aligned block carries a value. Same result on a 100 kb view with four chains in it: 24 such columns before, none after. A window whose chain reaches both edges, which is the ordinary case, renders pixel for pixel as it did. The second half of the ticket, the bigWig zoom level being chosen from query coordinates while summarySizeBlock comes from target coordinates, is not a bug. A cBlock has the same length on both sides by construction, so the two agree. refs #37621
• src/hg/hgTracks/gtexTracks.c
  • lines changed 4, context: html, text, full: html, text
    95807c7178fd38d52564f06df9fd5b90ceca1d24 Wed Sep 2 11:20:44 2026 -0700
    hgTracks, hgc, hgPhyloPlace: small fixes from the v503 Preview II review, refs #38172 hgTracks.c: skipBeyondDelimit returns NULL when the delimiter is absent, and the caller decremented and printed it without checking. The noYearDbs list keeps every assembly that reaches it carrying a '(' today, so nothing shows, but printf used to print "(null)" where htmlEncode now walks off the end. Fall back to the whole freeze name. gtexTracks.c: the guard before the in place truncation was two bytes stricter than the buffer needs, so a description just over the budget printed in full. Say what the buffer requirement actually is. vcfTrack.c has the same shape a byte the other way and is already right. hgPhyloPlace.c: initialize size, as the two sibling call sites do. cgiMemBlobFind always sets it when it returns a block, so this is for consistency. bigBedClick.c: hubEncode was called twice on the same string in one statement.
• src/hg/hgTracks/hgTracks.c
  • lines changed 6, context: html, text, full: html, text
    7007b23dcc178ac0d6a64642ecd5a1060c6f6a40 Thu Aug 27 13:47:54 2026 -0700
    hgTracks: measure how long the track image takes to reach the reader, refs #38109 We cannot choose a png compression level without knowing the reader's connection speed. Our own logs will not tell us: apache stops timing once the kernel has the bytes, so its duration field stays near two milliseconds whether the image is 20 KB or 500 KB. The reader's browser does know. It keeps a timing record for every image it loads, holding the bytes taken off the wire and the time waited. This reads that record for the track image and reports the numbers on the query string of DOT.gif, a 43 byte image that already sits in htdocs/images. No process starts, nothing touches the cart, and the client address, the time and the numbers all land in the same access log we already archive. New hg.conf setting pngTimingSampleRate. It is the N in one page load in N. Zero, or the setting left out, turns the whole thing off, which is the default. The report carries three numbers. ts is the bytes off the wire, d is the whole fetch, and x is only the time the bytes were arriving. x is the denominator for throughput, d is what the reader actually waited.
  • lines changed 7, context: html, text, full: html, text
    1ba03723b75a8d501529931b7e80b27ece5c34b1 Tue Sep 1 12:59:41 2026 -0700
    remove the obsolete gcOnTheFlyCoExist flag and hg.conf variable refs #38208
  • lines changed 4, context: html, text, full: html, text
    95807c7178fd38d52564f06df9fd5b90ceca1d24 Wed Sep 2 11:20:44 2026 -0700
    hgTracks, hgc, hgPhyloPlace: small fixes from the v503 Preview II review, refs #38172 hgTracks.c: skipBeyondDelimit returns NULL when the delimiter is absent, and the caller decremented and printed it without checking. The noYearDbs list keeps every assembly that reaches it carrying a '(' today, so nothing shows, but printf used to print "(null)" where htmlEncode now walks off the end. Fall back to the whole freeze name. gtexTracks.c: the guard before the in place truncation was two bytes stricter than the buffer needs, so a description just over the budget printed in full. Say what the buffer requirement actually is. vcfTrack.c has the same shape a byte the other way and is already right. hgPhyloPlace.c: initialize size, as the two sibling call sites do. cgiMemBlobFind always sets it when it returns a block, so this is for consistency. bigBedClick.c: hubEncode was called twice on the same string in one statement.
  • lines changed 13, context: html, text, full: html, text
    4c20412187794d30583e3b68a102b7a8612d29a1 Fri Sep 4 13:47:28 2026 -0700
    hgTracks: settle a composite subtrack's visibility before the parallel loaders start The same request could return two different images. A composite child's visibility is written lazily, the first time anything asks for it, by limitedVisFromComposite(). The load phase starts the parallel loader threads and only waits for them later, and in between the main thread runs compositeLoad() over every subtrack of the composite, including the ones a worker is loading right now. The loop condition there is what performs the write. So the main thread wrote the field while a worker read it, and whichever got there first decided what the track looked like. findLeavesForParallelLoad() already computes the same value, with the same test, for exactly the subtracks a worker will load, and then discards it. Keep it: write the visibility there, before pthread_create, and skip nothing else. The value and the guards are the ones limitedVisFromComposite() uses, so the write only moves earlier. Caught on clinvarSubLolly, whose loader divides its 128 pixel default height by 1.5 for pack mode, so the band came out 128 or 85 pixels. The effect is wider than one band: on BRCA1_BRCA2_ENIGMA_hg19 at chr17:41196312-41277500 in Helvetica, the race hid clinvarMain and clinvarSubLolly and collapsed the two gnomAD variant rows from about 15000 pixels each to 31, an image of 1259 rows instead of 31967. barChartTrack.c, chainTrack.c, gtexTracks.c and halSnakeTrack.c read the same field in their loaders, and three of them use it to decide what to load rather than how tall to draw. Verified with the 80 cell pixel sweep from #38094. Before: 33 of 80 cells render differently at one thread than at twenty, and the sweep reports differences when master is compared against itself. After: 0 of 80, and the control passes. refs #38254
• src/hg/hgTracks/hgTracks.h
  • lines changed 1, context: html, text, full: html, text
    9f5a3871b95f595b5a218f0d05395a8c72d58844 Tue Aug 25 12:57:10 2026 -0700
    remove the bigBedOnePath fallback, leaving a single bigBed load path, refs #36940 bigBedOnePath has defaulted to on since v492 and the setting has been removed from the hg.conf of the RR, Euro, Asia and hgwbeta. Drop the check and the code it guarded at all four sites: the field-count default in bigBedAddLinkedFeaturesFromExt, the method setup in commonBigBedMethods, the bedSize default in bigBedClick, and the bigBed case in hgc. The two branches in the hgc case differed only in a minimum the surviving path does not want. complexBedMethods lost its only isBigBed=TRUE caller with that branch, so drop the parameter. Retire the setting from the hg.conf catalog and its gate backlog, and repair the two hgc.c citations whose line numbers this commit shifted, refs #37925. Verified pixel-identical: 14 scenarios rendered before and after, covering the four QA sessions on the ticket plus mm10 knownGene, dbSnp155, rmsk, encRegTfbsClustered, clinvar and tandemDups. All AE=0.
• src/hg/hgTracks/mainMain.c
  • lines changed 4, context: html, text, full: html, text
    d2a2afcd6c5c19257f8ed2f746c3f94474f4ab78 Tue Sep 1 13:37:34 2026 -0700
    hgTracks: let hg.conf set how big a step malloc takes, refs #38225 hgTracks loads tracks in parallel threads, and each thread grows its own memory pool in the 128 kB steps glibc uses by default. A heavy render asks the kernel to enlarge a pool about 133,000 times, work that does nothing for the reader. The new hg.conf setting mallocTopPad is a number of bytes. When it is set, cfgSetMallocTopPad() passes it to mallopt(M_TOP_PAD) before anything else in main() has a chance to allocate, so the heap grows in that size step instead. Absent or zero, glibc is left alone and nothing changes. Measured with the #38094 harness over eight sessions and two positions, at 32 MB: renders take 0.926 of the time they did, against a noise floor of 1.006. With the setting absent the same build measures 1.004, inside that noise floor. Through apache on a sandbox it measures 0.906. The cost is about 65 MB more resident memory on a heavy session and 8 to 20 MB on an ordinary view, where it also saves no measurable time; the saving and the cost are the same effect and appear together. The rendered image is byte identical either way. Off by default, so a server opts in.
• src/hg/hgTracks/pubsTracks.c
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/hgTracks/renderMain.c
  • lines changed 5, context: html, text, full: html, text
    d2a2afcd6c5c19257f8ed2f746c3f94474f4ab78 Tue Sep 1 13:37:34 2026 -0700
    hgTracks: let hg.conf set how big a step malloc takes, refs #38225 hgTracks loads tracks in parallel threads, and each thread grows its own memory pool in the 128 kB steps glibc uses by default. A heavy render asks the kernel to enlarge a pool about 133,000 times, work that does nothing for the reader. The new hg.conf setting mallocTopPad is a number of bytes. When it is set, cfgSetMallocTopPad() passes it to mallopt(M_TOP_PAD) before anything else in main() has a chance to allocate, so the heap grows in that size step instead. Absent or zero, glibc is left alone and nothing changes. Measured with the #38094 harness over eight sessions and two positions, at 32 MB: renders take 0.926 of the time they did, against a noise floor of 1.006. With the setting absent the same build measures 1.004, inside that noise floor. Through apache on a sandbox it measures 0.906. The cost is about 65 MB more resident memory on a heavy session and 8 to 20 MB on an ordinary view, where it also saves no measurable time; the saving and the cost are the same effect and appear together. The rendered image is byte identical either way. Off by default, so a server opts in.
• src/hg/hgTracks/searchTracks.c
  • lines changed 6, context: html, text, full: html, text
    6d28e748f8aee8c252386600fc93d7692aa6929d Wed Sep 2 10:39:13 2026 -0700
    hgTracks: don't escape the Track Search title twice, refs #38172 The Track Search page built its blue bar title with htmlEncode() on the organism and the freeze name, then handed it to webStartWrapperDetailedNoArgs. That call prints the title through htmlTextOut, which already escapes & < > and the double quote, so the ampersand starting each entity htmlEncode() wrote was escaped a second time and the page printed the entity instead of the character. Every assembly whose dbDb description carries a '/' showed it: hg38 read "Search for Tracks in the Human Dec. 2013 (GRCh38&#x2F;hg38) Assembly". The title lands in element text between two divs rather than in an attribute, so htmlTextOut covers it on its own and the encode here is not needed. Checked against the assembly hub case that prompted the encode in the first place: a hub whose genomes.txt organism and description hold markup still comes back as text in the blue bar. Found by Max in the v503 Preview II code review.
• src/hg/hgTracks/simpleTracks.c
  • lines changed 4, context: html, text, full: html, text
    9f5a3871b95f595b5a218f0d05395a8c72d58844 Tue Aug 25 12:57:10 2026 -0700
    remove the bigBedOnePath fallback, leaving a single bigBed load path, refs #36940 bigBedOnePath has defaulted to on since v492 and the setting has been removed from the hg.conf of the RR, Euro, Asia and hgwbeta. Drop the check and the code it guarded at all four sites: the field-count default in bigBedAddLinkedFeaturesFromExt, the method setup in commonBigBedMethods, the bedSize default in bigBedClick, and the bigBed case in hgc. The two branches in the hgc case differed only in a minimum the surviving path does not want. complexBedMethods lost its only isBigBed=TRUE caller with that branch, so drop the parameter. Retire the setting from the hg.conf catalog and its gate backlog, and repair the two hgc.c citations whose line numbers this commit shifted, refs #37925. Verified pixel-identical: 14 scenarios rendered before and after, covering the four QA sessions on the ticket plus mm10 knownGene, dbSnp155, rmsk, encRegTfbsClustered, clinvar and tandemDups. All AE=0.
  • lines changed 93, context: html, text, full: html, text
    aeb00b82ee56facd622b8f5347ebbf85309c9403 Wed Sep 2 14:02:01 2026 -0700
    Ask the database once per track for RefSeq status, not once per gene, refs #38233 refGeneColor is the per-item color callback for the RefSeq gene tracks. Each call opened a database connection, checked that the status table existed, asked for one gene's status, and closed the connection again. A view with two RefSeq tracks in pack mode draws a few thousand genes, so the browser made a few thousand round trips to the database while it was drawing the image. refSeqStatusHashLoad now builds the whole name-to-status map with one batched query when the track loads, and leaves it on tg->customPt. refGeneColorByStatus reads that map, so drawing asks the database nothing. Both are static; nothing outside this file used either. The map is built after limitVisibility() and only for a track that will draw. A track limitVisibility hides needs no colors, and the tracks it hides are the ones with the most items, so they are exactly the ones whose query would be largest. The existence test goes through a connection rather than hTableExists(), because refSeqStatusTable carries its database (normally hgFixed.refSeqStatus) and hTableExists() looks a name up in one database's own list of tables. It answers FALSE for any name with a database prefix, which silently drops the shading for plain refGene and xenoRefGene. This also removes a latent errAbort: the old gate tested refSeqStatus OR ncbiRefSeqLink and then queried whichever table the track type wanted, so a database with only ncbiRefSeqLink sent a refGene track to query a table that was not there. Rendering is unchanged. Eight scenarios pixel-identical to genome-test, including whole chr1 and the plain refGene and xenoRefGene tracks. Across the seven clinical Recommended Track Sets on hg38, 14% faster on a 2.5 Mb view and 20% on a 25 Mb view, with peak memory unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 224, context: html, text, full: html, text
    b46fbfda9130992e290b222840f1d3b7bba15c6c Fri Sep 4 11:15:50 2026 -0700
    hgTracks: build item coverage from feature runs, not one counter per base A track drawn as a coverage graph asked for one unsigned counter per base of the window, added one to every base of every feature, then read the whole array back to produce about 1,200 pixels. On a 25 Mb view that is a 100 MB allocation per track and two passes over it, and the cost grows with the width of the view rather than with the amount of data. It was 38% of the render on one Recommended Track Set page and 29% on another, inside the drawing step where the reader waits for it. Coverage only changes where a feature starts or ends, so it is now held as a step function: two events per feature, sorted, swept into runs, and sampled with a forward cursor. The arithmetic at the two fractional bases of each pixel is deliberately unchanged, including its habit of squaring the weighted value rather than weighting the square, so the drawn image is identical. Whole bases fold a run at a time as cov*n, which is the same value the per-base loop reached, because those quantities are integers held in doubles. Verified with 96 pixel comparisons and no difference: the 80 case sweep is identical on every cell that renders deterministically, plus 18 comparisons on views narrower than the image, which is the only path the sweep does not cover. Geometric mean 0.81 over the eight page benchmark set, 0.51 on the worst cell, and about 100 MB off peak memory on a wide view. refs #38253
• src/hg/hgTracks/tests/README.md
  • lines changed 60, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/hgTracks/tests/defaultView.docent.yaml
  • lines changed 23, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/hgTracks/tests/makefile
  • lines changed 2, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/hgTracks/tests/pages/hgTracks.js
  • lines changed 190, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/hgTracks/tests/t01-render.js
  • lines changed 182, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/hgVai/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgc/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hgc/bigBedClick.c
  • lines changed 2, context: html, text, full: html, text
    9f5a3871b95f595b5a218f0d05395a8c72d58844 Tue Aug 25 12:57:10 2026 -0700
    remove the bigBedOnePath fallback, leaving a single bigBed load path, refs #36940 bigBedOnePath has defaulted to on since v492 and the setting has been removed from the hg.conf of the RR, Euro, Asia and hgwbeta. Drop the check and the code it guarded at all four sites: the field-count default in bigBedAddLinkedFeaturesFromExt, the method setup in commonBigBedMethods, the bedSize default in bigBedClick, and the bigBed case in hgc. The two branches in the hgc case differed only in a minimum the surviving path does not want. complexBedMethods lost its only isBigBed=TRUE caller with that branch, so drop the parameter. Retire the setting from the hg.conf catalog and its gate backlog, and repair the two hgc.c citations whose line numbers this commit shifted, refs #37925. Verified pixel-identical: 14 scenarios rendered before and after, covering the four QA sessions on the ticket plus mm10 knownGene, dbSnp155, rmsk, encRegTfbsClustered, clinvar and tandemDups. All AE=0.
  • lines changed 7, context: html, text, full: html, text
    0ca1bd9ec1196e15b2ea7cef508bd7f382bb5add Mon Aug 31 13:30:10 2026 -0700
    hgc: reject a bigBed type line that declares fewer than three fields, refs #36940 A bigBed always has chrom, chromStart and chromEnd. bigBedClick trusted the field count from the type line, so a track hub could declare "type bigBed 1", "type bigBed 2" or a negative count and reach the item detail page with that value. Two things then went wrong. The variable length array at the top of the interval loop was sized from it, which is undefined behaviour for a negative count. Then restBedFields went negative, so extraFields pointed before the start of restFields and getExtraFields read off the front of that stack array. The existing field count check sits twelve lines further down and never got the chance to stop either one. Check the count once, right after the zero sentinel has been resolved from the file, and abort with a message that names the track and the count it declared. The old path aborted too, just later and with wording that blamed a disagreement rather than the bad type line. The check has to come after the zero case is resolved. Zero means "take the count from the file", which is why the similar minimum in hgc.c was wrong and was removed earlier in this ticket. Tested with hubs declaring -5, 1, 2, 3, 6 and no count. The first three now stop with the new message; the last three are unchanged. Output for the valid counts is byte identical to a control build from the same tree, and a real "bigBed 3" track still renders.
  • lines changed 2, context: html, text, full: html, text
    95807c7178fd38d52564f06df9fd5b90ceca1d24 Wed Sep 2 11:20:44 2026 -0700
    hgTracks, hgc, hgPhyloPlace: small fixes from the v503 Preview II review, refs #38172 hgTracks.c: skipBeyondDelimit returns NULL when the delimiter is absent, and the caller decremented and printed it without checking. The noYearDbs list keeps every assembly that reaches it carrying a '(' today, so nothing shows, but printf used to print "(null)" where htmlEncode now walks off the end. Fall back to the whole freeze name. gtexTracks.c: the guard before the in place truncation was two bytes stricter than the buffer needs, so a description just over the budget printed in full. Say what the buffer requirement actually is. vcfTrack.c has the same shape a byte the other way and is already right. hgPhyloPlace.c: initialize size, as the two sibling call sites do. cgiMemBlobFind always sets it when it returns a block, so this is for consistency. bigBedClick.c: hubEncode was called twice on the same string in one statement.
• src/hg/hgc/hgc.c
  • lines changed 13, context: html, text, full: html, text
    9f5a3871b95f595b5a218f0d05395a8c72d58844 Tue Aug 25 12:57:10 2026 -0700
    remove the bigBedOnePath fallback, leaving a single bigBed load path, refs #36940 bigBedOnePath has defaulted to on since v492 and the setting has been removed from the hg.conf of the RR, Euro, Asia and hgwbeta. Drop the check and the code it guarded at all four sites: the field-count default in bigBedAddLinkedFeaturesFromExt, the method setup in commonBigBedMethods, the bedSize default in bigBedClick, and the bigBed case in hgc. The two branches in the hgc case differed only in a minimum the surviving path does not want. complexBedMethods lost its only isBigBed=TRUE caller with that branch, so drop the parameter. Retire the setting from the hg.conf catalog and its gate backlog, and repair the two hgc.c citations whose line numbers this commit shifted, refs #37925. Verified pixel-identical: 14 scenarios rendered before and after, covering the four QA sessions on the ticket plus mm10 knownGene, dbSnp155, rmsk, encRegTfbsClustered, clinvar and tandemDups. All AE=0.
  • lines changed 3, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
  • lines changed 23, context: html, text, full: html, text
    994528edb7d89068f371fef6d8a5fc72566f7334 Fri Sep 4 13:35:16 2026 -0700
    Adding an opt-in "Keep only last search" checkbox to the new BLAT search form. refs #36292 Behind a new hg.conf setting, blatOnlyLatestCheckbox (default off, documented in ex.hg.conf). When checked, each new BLAT search removes the user's earlier BLAT result tracks in hgc's buildBigPsl (cart variable blatOnlyLatest, remembered per browser in localStorage); unchecked keeps the current accumulate behavior, so site defaults are unchanged. Deletion requires both the hg.conf setting and the cart variable, so a stale cart cannot delete tracks where the feature is off. Independent of, and coexists with, the blatOldTracks hg.conf setting.
  • lines changed 12, context: html, text, full: html, text
    5915dcd2939889c51c286d540a60abef615fab77 Fri Sep 4 13:43:35 2026 -0700
    Fix new BLAT result tracks starting hidden due to stale cart visibility variables. refs #38086 Custom track ids are deterministic from the track name, so a re-created same-named BLAT result track inherited the visibility of a previously hidden and since-deleted one (the variable outlives the track). Seen with I'm feeling lucky: the user lands on hgTracks and the new result track is invisible. buildBigPsl now clears any stale visibility variable for the tracks it creates, after every block that can set hide.
  • lines changed 8, context: html, text, full: html, text
    cc23b76a1f2c7dc6a29c227661bfb0ce4d20eb54 Fri Sep 4 13:43:48 2026 -0700
    BLAT page polish from QA: Load example button, corrected input limits, wording fixes. refs #37996 Load example is a chip-style button with a mouseover instead of a hyperlink. Removed the Show input limits modal; corrected the limits in hgTracksHelp.html, stale since 2013 (DNA is 75,000 bases per sequence and 187,500 combined, protein and translated 10,000 and 25,000), added a #blatLimits anchor and pointed the two over-limit BLAT warnings and the form at it. Genome selector: section title Target assembly, type-to-search hint in the dropdown, Shortcuts: label, input and Query type select sized to match. Similar tools sidebar uses house-style hyphens and corrects the findMotif name and description. Share and rename text now states link lifetimes accurately (snapshot links last years, only sessions are permanent). Alignment page buttons match .gbPill, summary strip spacing fixed, New BLAT search button demoted from primary. Documents blatResultsGroup in ex.hg.conf.
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 1, context: html, text, full: html, text
    4be6477ebcb6be9290d5d439cdb48cb714e744b5 Sat Sep 5 11:07:29 2026 -0700
    hgc: let a quickLifted track fall back to the source hub's description again getHtmlFromSelfOrParent() reaches for the source assembly's description page only when the lifted track has none of its own, and it tested that with tdb->html == NULL. The lifted stanza in a quickLift hub carries the source hub's own relative html path, which cannot resolve from the hub's home in trash, so that fetch fails and the test used to hold. 1258d7f65e7 made trackHubAddOneDescription return early instead of assigning a failed fetch, so tdb->html keeps whatever it held rather than becoming NULL. The test then never holds, getTrackHtml is never called, and a track lifted from a GenArk assembly loses the description that #37389 added. Ask isEmpty() instead, which is the question this line meant: an empty description is the same as none here, and 1258d7f65e7 keeps what it was for. Bisected against genome-test: the description is drawn with trackHub.c at v503_base, at f759bf663de and at f001e39d156, and gone at 1258d7f65e7. Not shipped - v502 and v503 both draw it - so this is a fix before v504 rather than a patch. refs #38275
  • lines changed 27, context: html, text, full: html, text
    2eecd47992bf6fcfee294854359117d81167459a Sat Sep 5 15:51:03 2026 -0700
    hgc: show the trackDb iframeUrl frame with the item details, not above them The iframe from the iframeUrl setting was written out before anything else on the details page, which is unlike every other part of a details page. It now appears just below the "View DNA for this feature" line, with the rest of the details. Rather than move each of the printIframe() calls, which all happen before the position is known, printIframe() now only builds the html and a new printPendingIframe() writes it out from the position-printing routines. That covers bigBed, bigPsl and the generic click handler alike. The lorax details page prints it immediately, since there the iframe is the page itself. Also, a track with an iframeUrl but no url setting never showed its iframe at all, refs #37595
• src/hg/hgc/hgc.h
  • lines changed 8, context: html, text, full: html, text
    2eecd47992bf6fcfee294854359117d81167459a Sat Sep 5 15:51:03 2026 -0700
    hgc: show the trackDb iframeUrl frame with the item details, not above them The iframe from the iframeUrl setting was written out before anything else on the details page, which is unlike every other part of a details page. It now appears just below the "View DNA for this feature" line, with the rest of the details. Rather than move each of the printIframe() calls, which all happen before the position is known, printIframe() now only builds the html and a new printPendingIframe() writes it out from the position-printing routines. That covers bigBed, bigPsl and the generic click handler alike. The lorax details page prints it immediately, since there the iframe is the page itself. Also, a track with an iframeUrl but no url setting never showed its iframe at all, refs #37595
• src/hg/hgc/loraxClick.c
  • lines changed 3, context: html, text, full: html, text
    2eecd47992bf6fcfee294854359117d81167459a Sat Sep 5 15:51:03 2026 -0700
    hgc: show the trackDb iframeUrl frame with the item details, not above them The iframe from the iframeUrl setting was written out before anything else on the details page, which is unlike every other part of a details page. It now appears just below the "View DNA for this feature" line, with the rest of the details. Rather than move each of the printIframe() calls, which all happen before the position is known, printIframe() now only builds the html and a new printPendingIframe() writes it out from the position-printing routines. That covers bigBed, bigPsl and the generic click handler alike. The lorax details page prints it immediately, since there the iframe is the page itself. Also, a track with an iframeUrl but no url setting never showed its iframe at all, refs #37595
• src/hg/hgc/lowelab.c
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/hgc/mgcClick.c
  • lines changed 4, context: html, text, full: html, text
    752e20a9a1dc08793c5b529bbeac4ce5836a6be4 Fri Sep 4 17:13:55 2026 -0700
    hgc: stop five details-page handlers from aborting over a missing piece. chromSeqFileExists() opened its connection with sqlConnect() and only then asked whether the database existed, so it aborted before it could answer. The otherDb of a chain or net track is often not a local database at all -- a GenArk hub assembly, or one long retired -- so use sqlMayConnect() and return FALSE. This is 112 of the failures the TrackCheck robot reports, most of them hg38 net tracks against HPRC assemblies. Also moved the disconnect out of the if, where it leaked a connection whenever a database had no chromInfo table, and dropped the now-redundant sqlDatabaseExists() call, which was itself a second connection. mgcCloneInfoLoad() aborted when a clone carried no MGC: id in hgFixed.mrnaClone. Nothing on the page reads that field, and a clone can legitimately have only an IMAGE: id, so the check went away rather than the page. hDbOrganism() aborted for an assembly that has left dbDb but is still named by a maf component, which hg16.evofold does via mm3; it now falls back on the database name. The pgSnp SIFT and Polyphen prediction tables are loaded separately from the tracks that name them, so a machine can have the track and not the table, as hgwbeta and the RR do for hg18. Check with hTableExists first and say the predictions are unavailable instead of letting the query take the page down. Same treatment for the RNA fold diagram: a non-zero ghostscript exit now drops only the diagram and keeps the rest of the page, including the PseudoViewer link. Two of these report an unavailable piece through warn(), which still marks the page for the robot. That is deliberate -- the missing hg18 tables and the RNA fold diagram are real defects, and the log should keep naming them until they are fixed. On the RNA fold diagram in particular: RNAplot truncates the sequence id it is given to 42 characters, and the trash path we build is already 41, so it has never written the file ghostscript is asked to convert. That is worth its own fix. refs #37424
• src/hg/hgc/rnaFoldClick.c
  • lines changed 20, context: html, text, full: html, text
    752e20a9a1dc08793c5b529bbeac4ce5836a6be4 Fri Sep 4 17:13:55 2026 -0700
    hgc: stop five details-page handlers from aborting over a missing piece. chromSeqFileExists() opened its connection with sqlConnect() and only then asked whether the database existed, so it aborted before it could answer. The otherDb of a chain or net track is often not a local database at all -- a GenArk hub assembly, or one long retired -- so use sqlMayConnect() and return FALSE. This is 112 of the failures the TrackCheck robot reports, most of them hg38 net tracks against HPRC assemblies. Also moved the disconnect out of the if, where it leaked a connection whenever a database had no chromInfo table, and dropped the now-redundant sqlDatabaseExists() call, which was itself a second connection. mgcCloneInfoLoad() aborted when a clone carried no MGC: id in hgFixed.mrnaClone. Nothing on the page reads that field, and a clone can legitimately have only an IMAGE: id, so the check went away rather than the page. hDbOrganism() aborted for an assembly that has left dbDb but is still named by a maf component, which hg16.evofold does via mm3; it now falls back on the database name. The pgSnp SIFT and Polyphen prediction tables are loaded separately from the tracks that name them, so a machine can have the track and not the table, as hgwbeta and the RR do for hg18. Check with hTableExists first and say the predictions are unavailable instead of letting the query take the page down. Same treatment for the RNA fold diagram: a non-zero ghostscript exit now drops only the diagram and keeps the rest of the page, including the PseudoViewer link. Two of these report an unavailable piece through warn(), which still marks the page for the robot. That is deliberate -- the missing hg18 tables and the RNA fold diagram are real defects, and the log should keep naming them until they are fixed. On the RNA fold diagram in particular: RNAplot truncates the sequence id it is given to 42 characters, and the trash path we build is already 41, so it has never written the file ghostscript is asked to convert. That is worth its own fix. refs #37424
  • lines changed 37, context: html, text, full: html, text
    9a4d8c8cdd36457a18219cfb445abf9ce4e3f843 Fri Sep 4 17:28:09 2026 -0700
    hgc: make the RNA fold diagram actually appear. RNAplot takes the name of its output file from the sequence id on the fasta header it is given: it keeps the first 42 characters of that id and appends "_ss.ps". We were handing it "../trash/<table>/<table>_<name>.ps" and then asking ghostscript to read back exactly that path, which is wrong twice over. The "_ss.ps" means the name never matched even for a short item, and the path prefix alone is 41 characters, so RNAplot kept a single letter of the item name and every item in a track landed on the same file. Ghostscript was then pointed at something that had never been written, returned 1, and until the previous commit that killed the whole details page. There is no output-file option in RNAplot, and no way to hand our pipeline a working directory, so the id has to carry the path and still fit. The item name cannot be part of it: names in wuhCor1.rnaStructRangan are themselves 42 characters. So the id is now a short trash directory plus a 20 character hash of the track and item name, which is always 37 characters, and ghostscript reads the "_ss.ps" file RNAplot really wrote. All 112 items of wuhCor1.rnaStructRangan, the only rnaStruct track we have, now draw their own diagram: 112 distinct images where before there were none. Note for whoever looks at this next: RNAplot is not part of this tree and is not installed by it. It sits in the shared cgi-bin and in cgi-bin-beta, and it is absent from cgi-bin-$USER, so this feature cannot be exercised from a developer sandbox without setting rnaPlotPath in hg.conf. Worth confirming the binary is present in the RR's cgi-bin. Where it is missing the page now says the diagram could not be made rather than failing outright. refs #37424
• src/hg/hgc/vcfClick.c
  • lines changed 14, context: html, text, full: html, text
    44d8050c280e78571f46ea34b1ebffedca9758ce Wed Sep 2 11:20:27 2026 -0700
    vcfClick: print only the columns a tabular INFO value actually has, refs #38172 printTabularData() dropped the return value of chopByChar and then looped to the column count taken from the header row. chopByChar fills only the slots it uses, so a value carrying fewer pipe separated fields than its Description declares took the loop past the end of what was filled. looksTabular() requires just one of an element's values to match the header, so the rest of a multi-value element can be shorter. Keep the count and print an empty cell past it. Both functions also sized a stack array off VCF text with char copy[len+1], one from a record's INFO value and one from the header Description. Use the heap for those. Raised in the v503 Preview II code review.
• src/hg/htdocs/FAQ/FAQformat.html
  • lines changed 9, context: html, text, full: html, text
    059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
    Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245 This convention was previously only documented indirectly, as a SQL filtering tip on the Gene tracks FAQ page. Add it next to the cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as, sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and sangerGene.h, and mention it in FAQformat.html and bigGenePred.html (via the equivalent thickStart == thickEnd check).
• src/hg/htdocs/goldenPath/help/bigGenePred.html
  • lines changed 4, context: html, text, full: html, text
    059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
    Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245 This convention was previously only documented indirectly, as a SQL filtering tip on the Gene tracks FAQ page. Add it next to the cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as, sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and sangerGene.h, and mention it in FAQformat.html and bigGenePred.html (via the equivalent thickStart == thickEnd check).
• src/hg/htdocs/goldenPath/help/blatExample.fa
  • lines changed 279, context: html, text, full: html, text
    42b33234bee213325c2b6eede35edbb4fb2e566c Thu Sep 3 06:40:30 2026 -0700
    hgBlat: spinner while a search runs, shorter example, left-justified result columns Four items from Lou's note-37 on #38086. The search is a plain form POST, so nothing on screen changed between Submit and the arrival of the results - on an assembly served by a dynamic BLAT server that is a ten second wait with no sign that anything is happening. A busy overlay now covers the form: a generic .gbBusyBg/.gbSpinner in gbModern.css (CSS-drawn, since the page is already navigating away when it appears), put up by blatFormBusyWire() 350 ms after submit so a sub-second hg38 search does not flash a scrim, and swapping its text after 8 s to say that an index is probably being loaded on demand. Skipped for Clear and for an empty query; a pageshow handler takes it down again after a Back navigation. The example sequence goes from 13.8 kb of the SOD1 locus to 2.5 kb of PTP4A3, hg38 chr8:141,424,501-141,427,000. That window is carried by both chr8_KI270819v1_alt and chr8_MU273363v1_fix, so the example now demonstrates the alt and fix rows and their FAQ icons instead of returning one plain hit, and it covers two PTP4A3 exons so the result track is named after the gene. Every column of the results table is left-justified; the numeric ones keep tabular figures, and the score value gets a fixed width so its bars stay in a straight line. The alt/fix explanation icon is red rather than faint grey, and is drawn as the browser's own info-icon SVG rather than the U+24D8 glyph, which some system fonts do not have. refs #38086
• src/hg/htdocs/goldenPath/help/docker.html
  • lines changed 32, context: html, text, full: html, text
    3334b0dc02b7d88cd21a987b3765b30d581339cd Wed Sep 2 14:09:21 2026 -0700
    store, docker help: document the prebuilt Docker Hub image and correct the image size Braney pointed out on the ticket that neither page mentions genomebrowser/server, the image the weekly build pushes to Docker Hub (buildReleaseDocker.sh). It is rebuilt every release, latest currently points at v502, and one tag covers amd64 and arm64, so pulling it is the better path for most people than building the Dockerfile. Both pages now say so, and docker.html gains a section with the pull and run commands ahead of the build instructions. The store page said the built image is around 3GB, which was true a few releases ago. Building the current Dockerfile gives about 5.6GB on disk, and the published image is a 2.6GB download that unpacks to about 7.4GB, so the page now tells people to plan on 8GB for the image alone, separate from track data. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 5, context: html, text, full: html, text
    5b54eba6619b55ebe18c183dbc65c26e7da82e18 Thu Sep 3 09:39:15 2026 -0700
    docker help: promote the prebuilt image section to h2 and split out the build path Code review pointed out that the section 3334b0d added was an h3 nested under "Using Docker Desktop for UCSC Genome Browser" while every other entry in the page's table of contents is an h2. It is now an h2. That alone left the four build-your-own h3s that follow it (Obtaining a Dockerfile, Creating a Image, Creating a Container, Using Docker Desktop to Create a Container) nested under the prebuilt image section, which is the wrong parent since they describe building rather than pulling. They now sit under a new "Building the Image Yourself" h2, which also gets a table of contents entry. Every h2 on the page is now listed in the contents and every h3 is under the h2 it belongs to. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/htdocs/goldenPath/help/hgTracksHelp.html
  • lines changed 1, context: html, text, full: html, text
    f92f6c555664c22ac633538af5ebafa083264210 Fri Sep 4 09:40:23 2026 -0700
    Adding a Data file formats link to the Genome Browser User Guide table of contents. No RM. Reported by Chris.
  • lines changed 11, context: html, text, full: html, text
    cc23b76a1f2c7dc6a29c227661bfb0ce4d20eb54 Fri Sep 4 13:43:48 2026 -0700
    BLAT page polish from QA: Load example button, corrected input limits, wording fixes. refs #37996 Load example is a chip-style button with a mouseover instead of a hyperlink. Removed the Show input limits modal; corrected the limits in hgTracksHelp.html, stale since 2013 (DNA is 75,000 bases per sequence and 187,500 combined, protein and translated 10,000 and 25,000), added a #blatLimits anchor and pointed the two over-limit BLAT warnings and the form at it. Genome selector: section title Target assembly, type-to-search hint in the dropdown, Shortcuts: label, input and Query type select sized to match. Similar tools sidebar uses house-style hyphens and corrects the findMotif name and description. Share and rename text now states link lifetimes accurately (snapshot links last years, only sessions are permanent). Alignment page buttons match .gbPill, summary strip spacing fixed, New BLAT search button demoted from primary. Documents blatResultsGroup in ex.hg.conf.
• src/hg/htdocs/goldenPath/help/mirror.html
  • lines changed 11, context: html, text, full: html, text
    70ac948e82b41ef316523635c04e5e2c4a89c417 Fri Sep 4 16:57:54 2026 -0700
    trackLists: give the variant frequency projects their own table, move the page to goldenPath/help/mirrorTracks.html, add the otto cron line, refs #37781 Half the restricted list was national sequencing cohorts sitting under varFreqs and phasedVars, and in one alphabetical table they buried the tracks people actually write in about, OMIM, HGMD and DECIPHER. Those cohorts now get a table of their own below the rest. The split is read off the trackDb parent chain, so the next cohort added under varFreqs lands in the right table with no edit here. The page moves off the htdocs root to goldenPath/help/mirrorTracks.html, beside mirror.html, which now links to it. That link goes in src/product/README.txt, the pandoc source mirror.html is generated from. The licensing page link follows the move, and the page title changes with the file name. Also adds the weekly otto line, placed above the HGDB_CONF that would otherwise apply to it, and keeps the job from listing itself as a self-updating track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/htdocs/goldenPath/help/trackDb/changes.html
  • lines changed 99, context: html, text, full: html, text
    0e4e0c0af65eea70f64edbc68348ce0972c4bbf4 Fri Aug 28 16:45:00 2026 -0700
    Correct the track types listed for trackDb settings, and the hub settings list The "For Types" list in the trackDb docs was wrong for about sixty settings, so the docs named the wrong track types for settings that have always worked. Most named only the older type and left out its big* counterpart. The clearest case is the multiple-alignment family: a bigMaf track is drawn and configured by the same code as a wigMaf track, but only speciesOrder said so, while irows, itemFirstCharCase, speciesGroups, speciesCodonDefault, speciesDefaultOff, treeImage, pairwiseHeight and speciesUseFile all claimed wigMaf alone. The hapCluster settings said vcf and not vcfTabix. noScoreFilter said bed while its own example uses type bigBed 6 +. Six settings said "all" for something that only works on item tracks. Two documented settings do not exist. pslSequence describes a variable that was replaced by the baseColor family long before the setting was listed, and nothing has read either spelling since; it is removed. noStems is renamed to lollyNoStems, which is what the Browser actually reads. That one mattered: hubCheck builds its list of valid settings from trackDbHub.html, so it accepted the spelling that does nothing and rejected the one that works. Fourteen settings that work in hubs had no entry in the hub spec, so hubCheck reported them as unrecognized. They are listed now: chainColor, chainNormScoreAvailable, pairwiseHeight, barChartMatrixUrl, mouseOverFunction, intronGap, filterBy, baseColorTickColor, speciesGroups, speciesDefaultOff, speciesCodonDefault, itemFirstCharCase, irows, and canPack with configureByPopup and origAssembly. The last three, along with filterBy and baseColorTickColor, were marked "NOT FOR HUBS", which was wrong: the Browser reads them from a hub's trackDb the same way it reads them from ours. The type setting on the hub page listed every type the Browser knows, including ones that only work for tracks loaded into our own databases. It now shows only the types a hub can use. A hub-specific blurb for this already existed and had never been referenced. Three settings had no blurb at all, so the generated trackDbSettings.json never saw them: metadata, noInherit and useScore. Written, and the five hand-written copies in trackDbDoc.html that had drifted from the library are brought back into line. The library's header told the reader to always check their work in trackDbTestBlurbs.html, which was deleted in November 2025. It now points at "make settings" instead. That target regenerates trackDbSettings.yaml and .json, which are updated here, and its name map gains an entry so the hub-specific type blurb is still keyed as "type". refs #37908
  • lines changed 35, context: html, text, full: html, text
    92af4b0363b70062f69a5730648b20bd8e52a317 Mon Aug 31 13:34:03 2026 -0700
    Drop bed from searchIndex and searchTrix, and deprecate pslSequence instead of deleting it This is the reverse of the earlier passes. Instead of asking what a type list is missing, it asks which declared type nothing supports. searchIndex and searchTrix both declared bed. Neither works on a SQL bed table. The trackDb read for searchIndex is at bigBedFind.c:270 and it then needs a bbi file, from bigDataUrl or from the table's fileName, which a plain bed table does not have. searchTrix has two readers: bigBedFind.c:312 reads it from trackDb on the bigBed path, while hgFind.c:2393 reads it from hgFindSpec, which is how a SQL track gets trix search. Native usage agrees, with zero bed tracks for either setting. searchIndex also gains bigPsl, bigGenePred and bigBarChart, and searchTrix gains bigGenePred, all of which have real usage. pslSequence was deleted outright in 0e4e0c0af65. It is obsolete, but deleting the row was the wrong way to say so. 118 psl and 6 bigPsl native tracks set it, and so do three public-hub bigPsl tracks. hubCheck takes its vocabulary from this page, so with no row those hub authors would be told the setting "is not recognized. Check for typos", with a spelling suggestion, which is the wrong advice for a setting they took from our own documentation. The row is back at level-deprecated, which makes hubCheck say "is deprecated" instead, the way it already does for canPack, useScore, metadata and noInherit. Verified by running hubCheck against both spellings of the page. The library blurb is restored too, rewritten to say the setting is obsolete and to map its three values onto baseColorDefault, which replaced it in 8d32ba75938 in 2006. chainMinScore is deliberately left alone. It was proposed for deletion on the grounds that nothing reads it. Nothing does, but our own automation writes it: asmHubChainNetTrackDb.pl:101 and chainNetCompositeTrackDb.pl:186 emit "chainMinScore 5000" into the chainNet composite stanza, and findScores.pl converts -minScore into it. 40,920 native bed stanzas and 16 chain tracks carry it today. It records the minScore the chains were built with. Removing its documentation would leave those stanzas with an undocumented setting. changes.html gains a row for the type lists corrected across all three of these commits, and its existing pslSequence row is rewritten: it said the setting was removed, which is no longer what happened. Since the earlier commit has not shipped to the RR, no reader ever saw the "Removed" wording. trackDbSettings.yaml and .json are regenerated, and now hold 263 settings. refs #37908
• src/hg/htdocs/goldenPath/help/trackDb/trackDbDoc.html
  • lines changed 30, context: html, text, full: html, text
    0e4e0c0af65eea70f64edbc68348ce0972c4bbf4 Fri Aug 28 16:45:00 2026 -0700
    Correct the track types listed for trackDb settings, and the hub settings list The "For Types" list in the trackDb docs was wrong for about sixty settings, so the docs named the wrong track types for settings that have always worked. Most named only the older type and left out its big* counterpart. The clearest case is the multiple-alignment family: a bigMaf track is drawn and configured by the same code as a wigMaf track, but only speciesOrder said so, while irows, itemFirstCharCase, speciesGroups, speciesCodonDefault, speciesDefaultOff, treeImage, pairwiseHeight and speciesUseFile all claimed wigMaf alone. The hapCluster settings said vcf and not vcfTabix. noScoreFilter said bed while its own example uses type bigBed 6 +. Six settings said "all" for something that only works on item tracks. Two documented settings do not exist. pslSequence describes a variable that was replaced by the baseColor family long before the setting was listed, and nothing has read either spelling since; it is removed. noStems is renamed to lollyNoStems, which is what the Browser actually reads. That one mattered: hubCheck builds its list of valid settings from trackDbHub.html, so it accepted the spelling that does nothing and rejected the one that works. Fourteen settings that work in hubs had no entry in the hub spec, so hubCheck reported them as unrecognized. They are listed now: chainColor, chainNormScoreAvailable, pairwiseHeight, barChartMatrixUrl, mouseOverFunction, intronGap, filterBy, baseColorTickColor, speciesGroups, speciesDefaultOff, speciesCodonDefault, itemFirstCharCase, irows, and canPack with configureByPopup and origAssembly. The last three, along with filterBy and baseColorTickColor, were marked "NOT FOR HUBS", which was wrong: the Browser reads them from a hub's trackDb the same way it reads them from ours. The type setting on the hub page listed every type the Browser knows, including ones that only work for tracks loaded into our own databases. It now shows only the types a hub can use. A hub-specific blurb for this already existed and had never been referenced. Three settings had no blurb at all, so the generated trackDbSettings.json never saw them: metadata, noInherit and useScore. Written, and the five hand-written copies in trackDbDoc.html that had drifted from the library are brought back into line. The library's header told the reader to always check their work in trackDbTestBlurbs.html, which was deleted in November 2025. It now points at "make settings" instead. That target regenerates trackDbSettings.yaml and .json, which are updated here, and its name map gains an entry so the hub-specific type blurb is still keyed as "type". refs #37908
• src/hg/htdocs/goldenPath/help/trackDb/trackDbHub.v3.html
  • lines changed 60, context: html, text, full: html, text
    0e4e0c0af65eea70f64edbc68348ce0972c4bbf4 Fri Aug 28 16:45:00 2026 -0700
    Correct the track types listed for trackDb settings, and the hub settings list The "For Types" list in the trackDb docs was wrong for about sixty settings, so the docs named the wrong track types for settings that have always worked. Most named only the older type and left out its big* counterpart. The clearest case is the multiple-alignment family: a bigMaf track is drawn and configured by the same code as a wigMaf track, but only speciesOrder said so, while irows, itemFirstCharCase, speciesGroups, speciesCodonDefault, speciesDefaultOff, treeImage, pairwiseHeight and speciesUseFile all claimed wigMaf alone. The hapCluster settings said vcf and not vcfTabix. noScoreFilter said bed while its own example uses type bigBed 6 +. Six settings said "all" for something that only works on item tracks. Two documented settings do not exist. pslSequence describes a variable that was replaced by the baseColor family long before the setting was listed, and nothing has read either spelling since; it is removed. noStems is renamed to lollyNoStems, which is what the Browser actually reads. That one mattered: hubCheck builds its list of valid settings from trackDbHub.html, so it accepted the spelling that does nothing and rejected the one that works. Fourteen settings that work in hubs had no entry in the hub spec, so hubCheck reported them as unrecognized. They are listed now: chainColor, chainNormScoreAvailable, pairwiseHeight, barChartMatrixUrl, mouseOverFunction, intronGap, filterBy, baseColorTickColor, speciesGroups, speciesDefaultOff, speciesCodonDefault, itemFirstCharCase, irows, and canPack with configureByPopup and origAssembly. The last three, along with filterBy and baseColorTickColor, were marked "NOT FOR HUBS", which was wrong: the Browser reads them from a hub's trackDb the same way it reads them from ours. The type setting on the hub page listed every type the Browser knows, including ones that only work for tracks loaded into our own databases. It now shows only the types a hub can use. A hub-specific blurb for this already existed and had never been referenced. Three settings had no blurb at all, so the generated trackDbSettings.json never saw them: metadata, noInherit and useScore. Written, and the five hand-written copies in trackDbDoc.html that had drifted from the library are brought back into line. The library's header told the reader to always check their work in trackDbTestBlurbs.html, which was deleted in November 2025. It now points at "make settings" instead. That target regenerates trackDbSettings.yaml and .json, which are updated here, and its name map gains an entry so the hub-specific type blurb is still keyed as "type". refs #37908
  • lines changed 3, context: html, text, full: html, text
    92af4b0363b70062f69a5730648b20bd8e52a317 Mon Aug 31 13:34:03 2026 -0700
    Drop bed from searchIndex and searchTrix, and deprecate pslSequence instead of deleting it This is the reverse of the earlier passes. Instead of asking what a type list is missing, it asks which declared type nothing supports. searchIndex and searchTrix both declared bed. Neither works on a SQL bed table. The trackDb read for searchIndex is at bigBedFind.c:270 and it then needs a bbi file, from bigDataUrl or from the table's fileName, which a plain bed table does not have. searchTrix has two readers: bigBedFind.c:312 reads it from trackDb on the bigBed path, while hgFind.c:2393 reads it from hgFindSpec, which is how a SQL track gets trix search. Native usage agrees, with zero bed tracks for either setting. searchIndex also gains bigPsl, bigGenePred and bigBarChart, and searchTrix gains bigGenePred, all of which have real usage. pslSequence was deleted outright in 0e4e0c0af65. It is obsolete, but deleting the row was the wrong way to say so. 118 psl and 6 bigPsl native tracks set it, and so do three public-hub bigPsl tracks. hubCheck takes its vocabulary from this page, so with no row those hub authors would be told the setting "is not recognized. Check for typos", with a spelling suggestion, which is the wrong advice for a setting they took from our own documentation. The row is back at level-deprecated, which makes hubCheck say "is deprecated" instead, the way it already does for canPack, useScore, metadata and noInherit. Verified by running hubCheck against both spellings of the page. The library blurb is restored too, rewritten to say the setting is obsolete and to map its three values onto baseColorDefault, which replaced it in 8d32ba75938 in 2006. chainMinScore is deliberately left alone. It was proposed for deletion on the grounds that nothing reads it. Nothing does, but our own automation writes it: asmHubChainNetTrackDb.pl:101 and chainNetCompositeTrackDb.pl:186 emit "chainMinScore 5000" into the chainNet composite stanza, and findScores.pl converts -minScore into it. 40,920 native bed stanzas and 16 chain tracks carry it today. It records the minScore the chains were built with. Removing its documentation would leave those stanzas with an undocumented setting. changes.html gains a row for the type lists corrected across all three of these commits, and its existing pslSequence row is rewritten: it said the setting was removed, which is no longer what happened. Since the earlier commit has not shipped to the RR, no reader ever saw the "Removed" wording. trackDbSettings.yaml and .json are regenerated, and now hold 263 settings. refs #37908
• src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml
  • lines changed 169, context: html, text, full: html, text
    0e4e0c0af65eea70f64edbc68348ce0972c4bbf4 Fri Aug 28 16:45:00 2026 -0700
    Correct the track types listed for trackDb settings, and the hub settings list The "For Types" list in the trackDb docs was wrong for about sixty settings, so the docs named the wrong track types for settings that have always worked. Most named only the older type and left out its big* counterpart. The clearest case is the multiple-alignment family: a bigMaf track is drawn and configured by the same code as a wigMaf track, but only speciesOrder said so, while irows, itemFirstCharCase, speciesGroups, speciesCodonDefault, speciesDefaultOff, treeImage, pairwiseHeight and speciesUseFile all claimed wigMaf alone. The hapCluster settings said vcf and not vcfTabix. noScoreFilter said bed while its own example uses type bigBed 6 +. Six settings said "all" for something that only works on item tracks. Two documented settings do not exist. pslSequence describes a variable that was replaced by the baseColor family long before the setting was listed, and nothing has read either spelling since; it is removed. noStems is renamed to lollyNoStems, which is what the Browser actually reads. That one mattered: hubCheck builds its list of valid settings from trackDbHub.html, so it accepted the spelling that does nothing and rejected the one that works. Fourteen settings that work in hubs had no entry in the hub spec, so hubCheck reported them as unrecognized. They are listed now: chainColor, chainNormScoreAvailable, pairwiseHeight, barChartMatrixUrl, mouseOverFunction, intronGap, filterBy, baseColorTickColor, speciesGroups, speciesDefaultOff, speciesCodonDefault, itemFirstCharCase, irows, and canPack with configureByPopup and origAssembly. The last three, along with filterBy and baseColorTickColor, were marked "NOT FOR HUBS", which was wrong: the Browser reads them from a hub's trackDb the same way it reads them from ours. The type setting on the hub page listed every type the Browser knows, including ones that only work for tracks loaded into our own databases. It now shows only the types a hub can use. A hub-specific blurb for this already existed and had never been referenced. Three settings had no blurb at all, so the generated trackDbSettings.json never saw them: metadata, noInherit and useScore. Written, and the five hand-written copies in trackDbDoc.html that had drifted from the library are brought back into line. The library's header told the reader to always check their work in trackDbTestBlurbs.html, which was deleted in November 2025. It now points at "make settings" instead. That target regenerates trackDbSettings.yaml and .json, which are updated here, and its name map gains an entry so the hub-specific type blurb is still keyed as "type". refs #37908
  • lines changed 9, context: html, text, full: html, text
    2c66cf38bdd66fd2c5cb9a5a23de69ca2c24bec5 Mon Aug 31 13:14:15 2026 -0700
    Add the track types that a code read confirms for nine trackDb settings This is the first part of the Tier B pass on the "For Types" lists. Tier B holds the settings that real tracks use on a type the docs never mention. Usage alone is not proof, because a setting can be set on a track and do nothing there, so each row here was confirmed by finding the code that reads the setting and showing that it serves the added type. The barChart family, barChartBars, barChartLabel, barChartMetric and barChartUnit, listed bigBarChart alone. cfgTypeFromTdb sends both barChart and bigBarChart to cfgBarChart, and barChartUi.c serves the pair, so barChart is added to all four. indelDoubleInsert listed bam. One blurb covers it and indelQueryInsert and indelPolyA. indelEnabled (hui.c:1586) takes a trackDb and never looks at the type, and linkedFeaturesDrawAt calls it at simpleTracks.c:4391, so the whole linked-features family reaches it and not only bamTrack.c. psl and bigPsl are added: they are the two types with real usage, 3494 and 2338 tracks, and the hub spec has listed all three settings under bigPsl for years, so the hub page and the library have disagreed about this. mouseOver and mouseOverField gain bigLolly, which lollyTrack.c reads at lines 384 and 377. motifPwmTable gains bigBed, read on the bigBed details path at hgc/bigBedClick.c:584. logoMaf gains wig, read off tg->tdb with no type gate at wigTrack.c:2029, in a file that serves plain wig as well as bigWig. trackDbSettings.yaml and .json are regenerated. refs #37908
  • lines changed 13, context: html, text, full: html, text
    744c3f21361148e959397572e346516a81ea52ed Mon Aug 31 13:21:01 2026 -0700
    Replace the dead snp track type with bed in thirteen trackDb settings Thirteen settings in the library declared the type snp and nothing else. There is no such track any more. hg38 and hg19 both have zero tracks of that type, and nothing in the track engine dispatches on it: the only startsWith("snp", ...) left, at cgilib/snp125.c:318, tests the track name rather than the type. The dbSNP tracks are bigDbSnp and "bed 6 +". So snp is not a missing type here, it is a type that went away and took thirteen type lists with it. Every one of them is used on bed today, between 2 and 4047 tracks each, and each has a reader that a bed track reaches: snp125Ui.c for the ortho tables, hapmapTrack.c for hapmapPhase, variation.c for defaultMaxWeight, and hgc.c for the rest. defaultGeneTracks gets bed and bigDbSnp. Its 16 bigDbSnp tracks are real, not convention: snp153OfferGeneTracksForFunction reads the setting at hui.c:4923 and bigDbSnpCfgUi calls that function at hui.c:4999. None of the other twelve is read on a bigDbSnp path. Two of the thirteen were missing from the earlier survey of this family. chimpMacaqueOrthoTable is a separate setting from chimpOrangMacOrthoTable and is read at snp125Ui.c:16. codingAnnoLabel_<table> looked unused because hgc.c builds the setting name at run time, at hgc.c:20153 inside printSnpAlleleAndOrthos. The two remaining snp spans are left alone on purpose. Neither is a setting: one documents the snp track type itself and already says "type bed 6 +", and the other is the example track stanza beneath it. None of these thirteen has a row in the hub spec, which is correct, since they all name a SQL table or a SQL-backed gene track. So the generated trackDbSettings.yaml and .json do not change. refs #37908
  • lines changed 19, context: html, text, full: html, text
    92af4b0363b70062f69a5730648b20bd8e52a317 Mon Aug 31 13:34:03 2026 -0700
    Drop bed from searchIndex and searchTrix, and deprecate pslSequence instead of deleting it This is the reverse of the earlier passes. Instead of asking what a type list is missing, it asks which declared type nothing supports. searchIndex and searchTrix both declared bed. Neither works on a SQL bed table. The trackDb read for searchIndex is at bigBedFind.c:270 and it then needs a bbi file, from bigDataUrl or from the table's fileName, which a plain bed table does not have. searchTrix has two readers: bigBedFind.c:312 reads it from trackDb on the bigBed path, while hgFind.c:2393 reads it from hgFindSpec, which is how a SQL track gets trix search. Native usage agrees, with zero bed tracks for either setting. searchIndex also gains bigPsl, bigGenePred and bigBarChart, and searchTrix gains bigGenePred, all of which have real usage. pslSequence was deleted outright in 0e4e0c0af65. It is obsolete, but deleting the row was the wrong way to say so. 118 psl and 6 bigPsl native tracks set it, and so do three public-hub bigPsl tracks. hubCheck takes its vocabulary from this page, so with no row those hub authors would be told the setting "is not recognized. Check for typos", with a spelling suggestion, which is the wrong advice for a setting they took from our own documentation. The row is back at level-deprecated, which makes hubCheck say "is deprecated" instead, the way it already does for canPack, useScore, metadata and noInherit. Verified by running hubCheck against both spellings of the page. The library blurb is restored too, rewritten to say the setting is obsolete and to map its three values onto baseColorDefault, which replaced it in 8d32ba75938 in 2006. chainMinScore is deliberately left alone. It was proposed for deletion on the grounds that nothing reads it. Nothing does, but our own automation writes it: asmHubChainNetTrackDb.pl:101 and chainNetCompositeTrackDb.pl:186 emit "chainMinScore 5000" into the chainNet composite stanza, and findScores.pl converts -minScore into it. 40,920 native bed stanzas and 16 chain tracks carry it today. It records the minScore the chains were built with. Removing its documentation would leave those stanzas with an undocumented setting. changes.html gains a row for the type lists corrected across all three of these commits, and its existing pslSequence row is rewritten: it said the setting was removed, which is no longer what happened. Since the earlier commit has not shipped to the RR, no reader ever saw the "Removed" wording. trackDbSettings.yaml and .json are regenerated, and now hold 263 settings. refs #37908
• src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.json
  • lines changed 448, context: html, text, full: html, text
    0e4e0c0af65eea70f64edbc68348ce0972c4bbf4 Fri Aug 28 16:45:00 2026 -0700
    Correct the track types listed for trackDb settings, and the hub settings list The "For Types" list in the trackDb docs was wrong for about sixty settings, so the docs named the wrong track types for settings that have always worked. Most named only the older type and left out its big* counterpart. The clearest case is the multiple-alignment family: a bigMaf track is drawn and configured by the same code as a wigMaf track, but only speciesOrder said so, while irows, itemFirstCharCase, speciesGroups, speciesCodonDefault, speciesDefaultOff, treeImage, pairwiseHeight and speciesUseFile all claimed wigMaf alone. The hapCluster settings said vcf and not vcfTabix. noScoreFilter said bed while its own example uses type bigBed 6 +. Six settings said "all" for something that only works on item tracks. Two documented settings do not exist. pslSequence describes a variable that was replaced by the baseColor family long before the setting was listed, and nothing has read either spelling since; it is removed. noStems is renamed to lollyNoStems, which is what the Browser actually reads. That one mattered: hubCheck builds its list of valid settings from trackDbHub.html, so it accepted the spelling that does nothing and rejected the one that works. Fourteen settings that work in hubs had no entry in the hub spec, so hubCheck reported them as unrecognized. They are listed now: chainColor, chainNormScoreAvailable, pairwiseHeight, barChartMatrixUrl, mouseOverFunction, intronGap, filterBy, baseColorTickColor, speciesGroups, speciesDefaultOff, speciesCodonDefault, itemFirstCharCase, irows, and canPack with configureByPopup and origAssembly. The last three, along with filterBy and baseColorTickColor, were marked "NOT FOR HUBS", which was wrong: the Browser reads them from a hub's trackDb the same way it reads them from ours. The type setting on the hub page listed every type the Browser knows, including ones that only work for tracks loaded into our own databases. It now shows only the types a hub can use. A hub-specific blurb for this already existed and had never been referenced. Three settings had no blurb at all, so the generated trackDbSettings.json never saw them: metadata, noInherit and useScore. Written, and the five hand-written copies in trackDbDoc.html that had drifted from the library are brought back into line. The library's header told the reader to always check their work in trackDbTestBlurbs.html, which was deleted in November 2025. It now points at "make settings" instead. That target regenerates trackDbSettings.yaml and .json, which are updated here, and its name map gains an entry so the hub-specific type blurb is still keyed as "type". refs #37908
  • lines changed 12, context: html, text, full: html, text
    2c66cf38bdd66fd2c5cb9a5a23de69ca2c24bec5 Mon Aug 31 13:14:15 2026 -0700
    Add the track types that a code read confirms for nine trackDb settings This is the first part of the Tier B pass on the "For Types" lists. Tier B holds the settings that real tracks use on a type the docs never mention. Usage alone is not proof, because a setting can be set on a track and do nothing there, so each row here was confirmed by finding the code that reads the setting and showing that it serves the added type. The barChart family, barChartBars, barChartLabel, barChartMetric and barChartUnit, listed bigBarChart alone. cfgTypeFromTdb sends both barChart and bigBarChart to cfgBarChart, and barChartUi.c serves the pair, so barChart is added to all four. indelDoubleInsert listed bam. One blurb covers it and indelQueryInsert and indelPolyA. indelEnabled (hui.c:1586) takes a trackDb and never looks at the type, and linkedFeaturesDrawAt calls it at simpleTracks.c:4391, so the whole linked-features family reaches it and not only bamTrack.c. psl and bigPsl are added: they are the two types with real usage, 3494 and 2338 tracks, and the hub spec has listed all three settings under bigPsl for years, so the hub page and the library have disagreed about this. mouseOver and mouseOverField gain bigLolly, which lollyTrack.c reads at lines 384 and 377. motifPwmTable gains bigBed, read on the bigBed details path at hgc/bigBedClick.c:584. logoMaf gains wig, read off tg->tdb with no type gate at wigTrack.c:2029, in a file that serves plain wig as well as bigWig. trackDbSettings.yaml and .json are regenerated. refs #37908
  • lines changed 21, context: html, text, full: html, text
    92af4b0363b70062f69a5730648b20bd8e52a317 Mon Aug 31 13:34:03 2026 -0700
    Drop bed from searchIndex and searchTrix, and deprecate pslSequence instead of deleting it This is the reverse of the earlier passes. Instead of asking what a type list is missing, it asks which declared type nothing supports. searchIndex and searchTrix both declared bed. Neither works on a SQL bed table. The trackDb read for searchIndex is at bigBedFind.c:270 and it then needs a bbi file, from bigDataUrl or from the table's fileName, which a plain bed table does not have. searchTrix has two readers: bigBedFind.c:312 reads it from trackDb on the bigBed path, while hgFind.c:2393 reads it from hgFindSpec, which is how a SQL track gets trix search. Native usage agrees, with zero bed tracks for either setting. searchIndex also gains bigPsl, bigGenePred and bigBarChart, and searchTrix gains bigGenePred, all of which have real usage. pslSequence was deleted outright in 0e4e0c0af65. It is obsolete, but deleting the row was the wrong way to say so. 118 psl and 6 bigPsl native tracks set it, and so do three public-hub bigPsl tracks. hubCheck takes its vocabulary from this page, so with no row those hub authors would be told the setting "is not recognized. Check for typos", with a spelling suggestion, which is the wrong advice for a setting they took from our own documentation. The row is back at level-deprecated, which makes hubCheck say "is deprecated" instead, the way it already does for canPack, useScore, metadata and noInherit. Verified by running hubCheck against both spellings of the page. The library blurb is restored too, rewritten to say the setting is obsolete and to map its three values onto baseColorDefault, which replaced it in 8d32ba75938 in 2006. chainMinScore is deliberately left alone. It was proposed for deletion on the grounds that nothing reads it. Nothing does, but our own automation writes it: asmHubChainNetTrackDb.pl:101 and chainNetCompositeTrackDb.pl:186 emit "chainMinScore 5000" into the chainNet composite stanza, and findScores.pl converts -minScore into it. 40,920 native bed stanzas and 16 chain tracks carry it today. It records the minScore the chains were built with. Removing its documentation would leave those stanzas with an undocumented setting. changes.html gains a row for the type lists corrected across all three of these commits, and its existing pslSequence row is rewritten: it said the setting was removed, which is no longer what happened. Since the earlier commit has not shipped to the RR, no reader ever saw the "Removed" wording. trackDbSettings.yaml and .json are regenerated, and now hold 263 settings. refs #37908
• src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.yaml
  • lines changed 530, context: html, text, full: html, text
    0e4e0c0af65eea70f64edbc68348ce0972c4bbf4 Fri Aug 28 16:45:00 2026 -0700
    Correct the track types listed for trackDb settings, and the hub settings list The "For Types" list in the trackDb docs was wrong for about sixty settings, so the docs named the wrong track types for settings that have always worked. Most named only the older type and left out its big* counterpart. The clearest case is the multiple-alignment family: a bigMaf track is drawn and configured by the same code as a wigMaf track, but only speciesOrder said so, while irows, itemFirstCharCase, speciesGroups, speciesCodonDefault, speciesDefaultOff, treeImage, pairwiseHeight and speciesUseFile all claimed wigMaf alone. The hapCluster settings said vcf and not vcfTabix. noScoreFilter said bed while its own example uses type bigBed 6 +. Six settings said "all" for something that only works on item tracks. Two documented settings do not exist. pslSequence describes a variable that was replaced by the baseColor family long before the setting was listed, and nothing has read either spelling since; it is removed. noStems is renamed to lollyNoStems, which is what the Browser actually reads. That one mattered: hubCheck builds its list of valid settings from trackDbHub.html, so it accepted the spelling that does nothing and rejected the one that works. Fourteen settings that work in hubs had no entry in the hub spec, so hubCheck reported them as unrecognized. They are listed now: chainColor, chainNormScoreAvailable, pairwiseHeight, barChartMatrixUrl, mouseOverFunction, intronGap, filterBy, baseColorTickColor, speciesGroups, speciesDefaultOff, speciesCodonDefault, itemFirstCharCase, irows, and canPack with configureByPopup and origAssembly. The last three, along with filterBy and baseColorTickColor, were marked "NOT FOR HUBS", which was wrong: the Browser reads them from a hub's trackDb the same way it reads them from ours. The type setting on the hub page listed every type the Browser knows, including ones that only work for tracks loaded into our own databases. It now shows only the types a hub can use. A hub-specific blurb for this already existed and had never been referenced. Three settings had no blurb at all, so the generated trackDbSettings.json never saw them: metadata, noInherit and useScore. Written, and the five hand-written copies in trackDbDoc.html that had drifted from the library are brought back into line. The library's header told the reader to always check their work in trackDbTestBlurbs.html, which was deleted in November 2025. It now points at "make settings" instead. That target regenerates trackDbSettings.yaml and .json, which are updated here, and its name map gains an entry so the hub-specific type blurb is still keyed as "type". refs #37908
  • lines changed 9, context: html, text, full: html, text
    2c66cf38bdd66fd2c5cb9a5a23de69ca2c24bec5 Mon Aug 31 13:14:15 2026 -0700
    Add the track types that a code read confirms for nine trackDb settings This is the first part of the Tier B pass on the "For Types" lists. Tier B holds the settings that real tracks use on a type the docs never mention. Usage alone is not proof, because a setting can be set on a track and do nothing there, so each row here was confirmed by finding the code that reads the setting and showing that it serves the added type. The barChart family, barChartBars, barChartLabel, barChartMetric and barChartUnit, listed bigBarChart alone. cfgTypeFromTdb sends both barChart and bigBarChart to cfgBarChart, and barChartUi.c serves the pair, so barChart is added to all four. indelDoubleInsert listed bam. One blurb covers it and indelQueryInsert and indelPolyA. indelEnabled (hui.c:1586) takes a trackDb and never looks at the type, and linkedFeaturesDrawAt calls it at simpleTracks.c:4391, so the whole linked-features family reaches it and not only bamTrack.c. psl and bigPsl are added: they are the two types with real usage, 3494 and 2338 tracks, and the hub spec has listed all three settings under bigPsl for years, so the hub page and the library have disagreed about this. mouseOver and mouseOverField gain bigLolly, which lollyTrack.c reads at lines 384 and 377. motifPwmTable gains bigBed, read on the bigBed details path at hgc/bigBedClick.c:584. logoMaf gains wig, read off tg->tdb with no type gate at wigTrack.c:2029, in a file that serves plain wig as well as bigWig. trackDbSettings.yaml and .json are regenerated. refs #37908
  • lines changed 25, context: html, text, full: html, text
    92af4b0363b70062f69a5730648b20bd8e52a317 Mon Aug 31 13:34:03 2026 -0700
    Drop bed from searchIndex and searchTrix, and deprecate pslSequence instead of deleting it This is the reverse of the earlier passes. Instead of asking what a type list is missing, it asks which declared type nothing supports. searchIndex and searchTrix both declared bed. Neither works on a SQL bed table. The trackDb read for searchIndex is at bigBedFind.c:270 and it then needs a bbi file, from bigDataUrl or from the table's fileName, which a plain bed table does not have. searchTrix has two readers: bigBedFind.c:312 reads it from trackDb on the bigBed path, while hgFind.c:2393 reads it from hgFindSpec, which is how a SQL track gets trix search. Native usage agrees, with zero bed tracks for either setting. searchIndex also gains bigPsl, bigGenePred and bigBarChart, and searchTrix gains bigGenePred, all of which have real usage. pslSequence was deleted outright in 0e4e0c0af65. It is obsolete, but deleting the row was the wrong way to say so. 118 psl and 6 bigPsl native tracks set it, and so do three public-hub bigPsl tracks. hubCheck takes its vocabulary from this page, so with no row those hub authors would be told the setting "is not recognized. Check for typos", with a spelling suggestion, which is the wrong advice for a setting they took from our own documentation. The row is back at level-deprecated, which makes hubCheck say "is deprecated" instead, the way it already does for canPack, useScore, metadata and noInherit. Verified by running hubCheck against both spellings of the page. The library blurb is restored too, rewritten to say the setting is obsolete and to map its three values onto baseColorDefault, which replaced it in 8d32ba75938 in 2006. chainMinScore is deliberately left alone. It was proposed for deletion on the grounds that nothing reads it. Nothing does, but our own automation writes it: asmHubChainNetTrackDb.pl:101 and chainNetCompositeTrackDb.pl:186 emit "chainMinScore 5000" into the chainNet composite stanza, and findScores.pl converts -minScore into it. 40,920 native bed stanzas and 16 chain tracks carry it today. It records the minScore the chains were built with. Removing its documentation would leave those stanzas with an undocumented setting. changes.html gains a row for the type lists corrected across all three of these commits, and its existing pslSequence row is rewritten: it said the setting was removed, which is no longer what happened. Since the earlier commit has not shipped to the RR, no reader ever saw the "Removed" wording. trackDbSettings.yaml and .json are regenerated, and now hold 263 settings. refs #37908
• src/hg/htdocs/goldenPath/help/trackDb/trackDbSettingsGen.py
  • lines changed 1, context: html, text, full: html, text
    0e4e0c0af65eea70f64edbc68348ce0972c4bbf4 Fri Aug 28 16:45:00 2026 -0700
    Correct the track types listed for trackDb settings, and the hub settings list The "For Types" list in the trackDb docs was wrong for about sixty settings, so the docs named the wrong track types for settings that have always worked. Most named only the older type and left out its big* counterpart. The clearest case is the multiple-alignment family: a bigMaf track is drawn and configured by the same code as a wigMaf track, but only speciesOrder said so, while irows, itemFirstCharCase, speciesGroups, speciesCodonDefault, speciesDefaultOff, treeImage, pairwiseHeight and speciesUseFile all claimed wigMaf alone. The hapCluster settings said vcf and not vcfTabix. noScoreFilter said bed while its own example uses type bigBed 6 +. Six settings said "all" for something that only works on item tracks. Two documented settings do not exist. pslSequence describes a variable that was replaced by the baseColor family long before the setting was listed, and nothing has read either spelling since; it is removed. noStems is renamed to lollyNoStems, which is what the Browser actually reads. That one mattered: hubCheck builds its list of valid settings from trackDbHub.html, so it accepted the spelling that does nothing and rejected the one that works. Fourteen settings that work in hubs had no entry in the hub spec, so hubCheck reported them as unrecognized. They are listed now: chainColor, chainNormScoreAvailable, pairwiseHeight, barChartMatrixUrl, mouseOverFunction, intronGap, filterBy, baseColorTickColor, speciesGroups, speciesDefaultOff, speciesCodonDefault, itemFirstCharCase, irows, and canPack with configureByPopup and origAssembly. The last three, along with filterBy and baseColorTickColor, were marked "NOT FOR HUBS", which was wrong: the Browser reads them from a hub's trackDb the same way it reads them from ours. The type setting on the hub page listed every type the Browser knows, including ones that only work for tracks loaded into our own databases. It now shows only the types a hub can use. A hub-specific blurb for this already existed and had never been referenced. Three settings had no blurb at all, so the generated trackDbSettings.json never saw them: metadata, noInherit and useScore. Written, and the five hand-written copies in trackDbDoc.html that had drifted from the library are brought back into line. The library's header told the reader to always check their work in trackDbTestBlurbs.html, which was deleted in November 2025. It now points at "make settings" instead. That target regenerates trackDbSettings.yaml and .json, which are updated here, and its name map gains an entry so the hub-specific type blurb is still keyed as "type". refs #37908
• src/hg/htdocs/goldenPath/newsarch.html
  • lines changed 84, context: html, text, full: html, text
    0e28eac3fe0eaa0cdd5afec9c683e883e8836f95 Tue Sep 1 13:53:03 2026 -0700
    Add Sep. 1 2026 news announcement for the new faceted composite interface, with screenshot and indexNews entry, refs #36320
  • lines changed 51, context: html, text, full: html, text
    30985a03cc358c13aa88bf6cdb45d91b4ec2b581 Tue Sep 1 15:24:43 2026 -0700
    Making changes to the EVE and popEVE announcment, No Redmine
  • lines changed 13, context: html, text, full: html, text
    7e9195767501a6492dce8ecddc1a6d28d1980baf Tue Sep 1 16:04:49 2026 -0700
    Making slight edits to the bigCompositeUI announcement, refs #36320
  • lines changed 2, context: html, text, full: html, text
    7555d3078ece791540738922db687996c4c82b1b Wed Sep 2 11:43:14 2026 -0700
    Adding Eliza's session URL now that it is ready, refs #37763
  • lines changed 21, context: html, text, full: html, text
    ee5cab2ba250102cd4cd57bffead0d0ddcb7b082 Wed Sep 2 17:44:01 2026 -0700
    Restoring "both" so the sentence correctly refers to both track sets, and rewrapping lines over the 100-character limit, in the gnomAD v4.1.1/MPC news announcement per code review feedback on #38213, refs #38166
  • lines changed 80, context: html, text, full: html, text
    afdd5daeb07ccef3df67165f6d10e7abd4db21ff Thu Sep 3 14:12:39 2026 -0700
    Release the ENCODE Expression tracks for mm10 and mm39. refs #37001 Drop the alpha,beta release tags from all three include lines, so the container and the bigWig signal composite now ship to beta and public. Verified with tdbQuery -strict at all three release levels: no errors, and the tracks are visible at each one (320 on mm10, 3 on mm39). Point the pennantIcon at the news archive post and give it the release date: newsarch.html#090426 "Released Sep. 4, 2026", matching the convention used by the other released tracks. The liftOver pennants on the mm39 subtracks are unchanged. Add the Sep. 4 news announcement to newsarch.html and a matching entry to indexNews.html, dropping the oldest front-page item to stay at the six-item limit. The announcement links to the session at genome.ucsc.edu/s/Lou/ENCODEexpression and includes a screenshot of Gap43, whose expression rises through development and is highest in the four neural tissues. Add the screenshot image, which was previously untracked.
• src/hg/htdocs/images/mouseDevTimecourse.png
  • lines changed 0, context: html, text, full: html, text
    afdd5daeb07ccef3df67165f6d10e7abd4db21ff Thu Sep 3 14:12:39 2026 -0700
    Release the ENCODE Expression tracks for mm10 and mm39. refs #37001 Drop the alpha,beta release tags from all three include lines, so the container and the bigWig signal composite now ship to beta and public. Verified with tdbQuery -strict at all three release levels: no errors, and the tracks are visible at each one (320 on mm10, 3 on mm39). Point the pennantIcon at the news archive post and give it the release date: newsarch.html#090426 "Released Sep. 4, 2026", matching the convention used by the other released tracks. The liftOver pennants on the mm39 subtracks are unchanged. Add the Sep. 4 news announcement to newsarch.html and a matching entry to indexNews.html, dropping the oldest front-page item to stay at the six-item limit. The announcement links to the session at genome.ucsc.edu/s/Lou/ENCODEexpression and includes a screenshot of Gap43, whose expression rises through development and is highest in the four neural tissues. Add the screenshot image, which was previously untracked.
• src/hg/htdocs/images/newsArchImages/facetedComposite.png
  • lines changed 0, context: html, text, full: html, text
    0e28eac3fe0eaa0cdd5afec9c683e883e8836f95 Tue Sep 1 13:53:03 2026 -0700
    Add Sep. 1 2026 news announcement for the new faceted composite interface, with screenshot and indexNews entry, refs #36320
• src/hg/htdocs/images/newsArchImages/popEve.png
  • lines changed 0, context: html, text, full: html, text
    30985a03cc358c13aa88bf6cdb45d91b4ec2b581 Tue Sep 1 15:24:43 2026 -0700
    Making changes to the EVE and popEVE announcment, No Redmine
• src/hg/htdocs/inc/globalNavBar.inc
  • lines changed 3, context: html, text, full: html, text
    6e89bb12a35c65c0607719f2aa2c7377803d09f9 Fri Sep 4 14:03:07 2026 -0700
    Keep the top-right Login/Share a link menu icon inside the blue bar on narrow windows. On screens under 1100px the icon was positioned with position:fixed, which pins it to the top of the window. That is the top of the blue bar only on hgTracks; on hgGateway and the home page the bar sits below the logo banner, so the icon showed up in the white area above it. Use position:sticky instead, which keeps the icon in the bar on every page and still holds it at the right edge of the visible viewport while the 1000px-wide bar is scrolled sideways. The hardcoded background colour is replaced by the bar's own, which is not the same on all pages. The links now hang in their own span so that opening the menu no longer pushes the page content down, and the icon is left out entirely on mirrors that have no top-right links to show. refs #38251
• src/hg/htdocs/indexNews.html
  • lines changed 11, context: html, text, full: html, text
    0e28eac3fe0eaa0cdd5afec9c683e883e8836f95 Tue Sep 1 13:53:03 2026 -0700
    Add Sep. 1 2026 news announcement for the new faceted composite interface, with screenshot and indexNews entry, refs #36320
  • lines changed 1, context: html, text, full: html, text
    30985a03cc358c13aa88bf6cdb45d91b4ec2b581 Tue Sep 1 15:24:43 2026 -0700
    Making changes to the EVE and popEVE announcment, No Redmine
  • lines changed 11, context: html, text, full: html, text
    afdd5daeb07ccef3df67165f6d10e7abd4db21ff Thu Sep 3 14:12:39 2026 -0700
    Release the ENCODE Expression tracks for mm10 and mm39. refs #37001 Drop the alpha,beta release tags from all three include lines, so the container and the bigWig signal composite now ship to beta and public. Verified with tdbQuery -strict at all three release levels: no errors, and the tracks are visible at each one (320 on mm10, 3 on mm39). Point the pennantIcon at the news archive post and give it the release date: newsarch.html#090426 "Released Sep. 4, 2026", matching the convention used by the other released tracks. The liftOver pennants on the mm39 subtracks are unchanged. Add the Sep. 4 news announcement to newsarch.html and a matching entry to indexNews.html, dropping the oldest front-page item to stay at the six-item limit. The announcement links to the session at genome.ucsc.edu/s/Lou/ENCODEexpression and includes a screenshot of Gap43, whose expression rises through development and is highest in the four neural tissues. Add the screenshot image, which was previously untracked.
• src/hg/htdocs/license/index.html
  • lines changed 13, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
  • lines changed 1, context: html, text, full: html, text
    70ac948e82b41ef316523635c04e5e2c4a89c417 Fri Sep 4 16:57:54 2026 -0700
    trackLists: give the variant frequency projects their own table, move the page to goldenPath/help/mirrorTracks.html, add the otto cron line, refs #37781 Half the restricted list was national sequencing cohorts sitting under varFreqs and phasedVars, and in one alphabetical table they buried the tracks people actually write in about, OMIM, HGMD and DECIPHER. Those cohorts now get a table of their own below the rest. The split is read off the trackDb parent chain, so the next cohort added under varFreqs lands in the right table with no edit here. The page moves off the htdocs root to goldenPath/help/mirrorTracks.html, beside mirror.html, which now links to it. That link goes in src/product/README.txt, the pandoc source mirror.html is generated from. The licensing page link follows the move, and the page title changes with the file name. Also adds the weekly otto line, placed above the HGDB_CONF that would otherwise apply to it, and keeps the job from listing itself as a self-updating track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/htdocs/store.html
  • lines changed 151, context: html, text, full: html, text
    4a635b51567e14c426be09c8f027c2ff43179b57 Wed Sep 2 14:05:32 2026 -0700
    store: add chain file and Docker image sections, update the EULA to the Nov 2022 revision Adds a UCSC chain files section covering the per-user commercial license, and a Genome Browser Docker image section. The two LiftOver entries now point at the chain file section, since the over.chain files are licensed separately from the program. Replaces store/UCSC_GB_EULA.pdf with the November 22, 2022 revision. That is the revision whose Appendix B lists UCSC Chain Files and CRISPOR, so every product on the page can link to a single agreement. It also broadens "Commercial Purposes" to any use by a for-profit organization, which now applies to the source code, LiftOver, GBiC and Docker entries as well. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 4, context: html, text, full: html, text
    3334b0dc02b7d88cd21a987b3765b30d581339cd Wed Sep 2 14:09:21 2026 -0700
    store, docker help: document the prebuilt Docker Hub image and correct the image size Braney pointed out on the ticket that neither page mentions genomebrowser/server, the image the weekly build pushes to Docker Hub (buildReleaseDocker.sh). It is rebuilt every release, latest currently points at v502, and one tag covers amd64 and arm64, so pulling it is the better path for most people than building the Dockerfile. Both pages now say so, and docker.html gains a section with the pull and run commands ahead of the build instructions. The store page said the built image is around 3GB, which was true a few releases ago. Building the current Dockerfile gives about 5.6GB on disk, and the published image is a 2.6GB download that unpacks to about 7.4GB, so the page now tells people to plan on 8GB for the image alone, separate from track data. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 8, context: html, text, full: html, text
    d8a3c66525910314ea3cdcc00b694138e769e699 Wed Sep 2 14:22:52 2026 -0700
    store: make the internal links relative Every internal link on the page was written against https://genome.ucsc.edu, so on a sandbox or a mirror the page sent people to production instead of to the copy they were looking at. That bit us reviewing this ticket: the "Terms and conditions" links fetched the production EULA rather than the new one sitting next to the page. Eight links: the five EULA links, hgLiftOver, gbic.html and docker.html. The gbic.html link also had goldenpath in lowercase, which production tolerates but the tree does not, so it is now goldenPath like its neighbor. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 2, context: html, text, full: html, text
    ecb6ac9da06193c59e5a59dee24d6a02872eb407 Thu Sep 3 09:39:08 2026 -0700
    store: fix the last two absolute links on the page Code review caught two links d8a3c66 missed. The gbic.html one in the GBiC firewall list is the same problem that commit fixed and gets the same fix: it is now relative, so it resolves against whichever copy of the page you are reading. The other is not the same problem. It was a bare URL in parentheses rather than a link, and genome.ucsc.edu/downloads.html is not a page in htdocs at all, it is a redirect to hgdownload. Making it relative would have given a 404 on every host including production. It now links to hgdownload.gi.ucsc.edu/downloads.html, matching the two other download links already on this page, and the sentence loses the stray period it had before the parenthetical. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/htdocs/store/UCSC_GB_EULA.pdf
  • lines changed 0, context: html, text, full: html, text
    4a635b51567e14c426be09c8f027c2ff43179b57 Wed Sep 2 14:05:32 2026 -0700
    store: add chain file and Docker image sections, update the EULA to the Nov 2022 revision Adds a UCSC chain files section covering the per-user commercial license, and a Genome Browser Docker image section. The two LiftOver entries now point at the chain file section, since the over.chain files are licensed separately from the program. Replaces store/UCSC_GB_EULA.pdf with the November 22, 2022 revision. That is the revision whose Appendix B lists UCSC Chain Files and CRISPOR, so every product on the page can link to a single agreement. It also broadens "Commercial Purposes" to any use by a for-profit organization, which now applies to the source code, LiftOver, GBiC and Docker entries as well. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/htdocs/style/gbModern.css
  • lines changed 19, context: html, text, full: html, text
    42b33234bee213325c2b6eede35edbb4fb2e566c Thu Sep 3 06:40:30 2026 -0700
    hgBlat: spinner while a search runs, shorter example, left-justified result columns Four items from Lou's note-37 on #38086. The search is a plain form POST, so nothing on screen changed between Submit and the arrival of the results - on an assembly served by a dynamic BLAT server that is a ten second wait with no sign that anything is happening. A busy overlay now covers the form: a generic .gbBusyBg/.gbSpinner in gbModern.css (CSS-drawn, since the page is already navigating away when it appears), put up by blatFormBusyWire() 350 ms after submit so a sub-second hg38 search does not flash a scrim, and swapping its text after 8 s to say that an index is probably being loaded on demand. Skipped for Clear and for an empty query; a pageshow handler takes it down again after a Back navigation. The example sequence goes from 13.8 kb of the SOD1 locus to 2.5 kb of PTP4A3, hg38 chr8:141,424,501-141,427,000. That window is carried by both chr8_KI270819v1_alt and chr8_MU273363v1_fix, so the example now demonstrates the alt and fix rows and their FAQ icons instead of returning one plain hit, and it covers two PTP4A3 exons so the result track is named after the gene. Every column of the results table is left-justified; the numeric ones keep tabular figures, and the score value gets a fixed width so its bars stay in a straight line. The alt/fix explanation icon is red rather than faint grey, and is drawn as the browser's own info-icon SVG rather than the U+24D8 glyph, which some system fonts do not have. refs #38086
• src/hg/htdocs/style/hgBlat.css
  • lines changed 17, context: html, text, full: html, text
    42b33234bee213325c2b6eede35edbb4fb2e566c Thu Sep 3 06:40:30 2026 -0700
    hgBlat: spinner while a search runs, shorter example, left-justified result columns Four items from Lou's note-37 on #38086. The search is a plain form POST, so nothing on screen changed between Submit and the arrival of the results - on an assembly served by a dynamic BLAT server that is a ten second wait with no sign that anything is happening. A busy overlay now covers the form: a generic .gbBusyBg/.gbSpinner in gbModern.css (CSS-drawn, since the page is already navigating away when it appears), put up by blatFormBusyWire() 350 ms after submit so a sub-second hg38 search does not flash a scrim, and swapping its text after 8 s to say that an index is probably being loaded on demand. Skipped for Clear and for an empty query; a pageshow handler takes it down again after a Back navigation. The example sequence goes from 13.8 kb of the SOD1 locus to 2.5 kb of PTP4A3, hg38 chr8:141,424,501-141,427,000. That window is carried by both chr8_KI270819v1_alt and chr8_MU273363v1_fix, so the example now demonstrates the alt and fix rows and their FAQ icons instead of returning one plain hit, and it covers two PTP4A3 exons so the result track is named after the gene. Every column of the results table is left-justified; the numeric ones keep tabular figures, and the score value gets a fixed width so its bars stay in a straight line. The alt/fix explanation icon is red rather than faint grey, and is drawn as the browser's own info-icon SVG rather than the U+24D8 glyph, which some system fonts do not have. refs #38086
  • lines changed 16, context: html, text, full: html, text
    cc23b76a1f2c7dc6a29c227661bfb0ce4d20eb54 Fri Sep 4 13:43:48 2026 -0700
    BLAT page polish from QA: Load example button, corrected input limits, wording fixes. refs #37996 Load example is a chip-style button with a mouseover instead of a hyperlink. Removed the Show input limits modal; corrected the limits in hgTracksHelp.html, stale since 2013 (DNA is 75,000 bases per sequence and 187,500 combined, protein and translated 10,000 and 25,000), added a #blatLimits anchor and pointed the two over-limit BLAT warnings and the form at it. Genome selector: section title Target assembly, type-to-search hint in the dropdown, Shortcuts: label, input and Query type select sized to match. Similar tools sidebar uses house-style hyphens and corrects the findMotif name and description. Share and rename text now states link lifetimes accurately (snapshot links last years, only sessions are permanent). Alignment page buttons match .gbPill, summary strip spacing fixed, New BLAT search button demoted from primary. Documents blatResultsGroup in ex.hg.conf.
• src/hg/htdocs/style/nice_menu.css
  • lines changed 22, context: html, text, full: html, text
    6e89bb12a35c65c0607719f2aa2c7377803d09f9 Fri Sep 4 14:03:07 2026 -0700
    Keep the top-right Login/Share a link menu icon inside the blue bar on narrow windows. On screens under 1100px the icon was positioned with position:fixed, which pins it to the top of the window. That is the top of the blue bar only on hgTracks; on hgGateway and the home page the bar sits below the logo banner, so the icon showed up in the white area above it. Use position:sticky instead, which keeps the icon in the bar on every page and still holds it at the right edge of the visible viewport while the 1000px-wide bar is scrolled sideways. The hardcoded background colour is replaced by the bar's own, which is not the same on all pages. The links now hang in their own span so that opening the menu no longer pushes the page content down, and the icon is left out entirely on mirrors that have no top-right links to show. refs #38251
• src/hg/hubApi/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/hubApi/apiUtils.c
  • lines changed 11, context: html, text, full: html, text
    7a185fecc55ea44b360f04d250e64bcbaaf4d53e Wed Sep 2 13:45:33 2026 -0700
    remove references and functions related to the "debug" argument, becomes a harmless no-operation and eliminated potential security loop-holes no redmine
• src/hg/hubApi/findGenome.c
  • lines changed 69, context: html, text, full: html, text
    cd363560ba2735155c189ac52f636b66d5a3c469 Wed Sep 2 12:16:21 2026 -0700
    correctly use the new indexes on the assemblyList table - this was improved in June but the code here did not match the new indexes - will now be a few ms search on accession names instead of 10s of seconds - no redmine
• src/hg/hubApi/getData.c
  • lines changed 26, context: html, text, full: html, text
    7a185fecc55ea44b360f04d250e64bcbaaf4d53e Wed Sep 2 13:45:33 2026 -0700
    remove references and functions related to the "debug" argument, becomes a harmless no-operation and eliminated potential security loop-holes no redmine
• src/hg/hubApi/hubApi.c
  • lines changed 89, context: html, text, full: html, text
    7a185fecc55ea44b360f04d250e64bcbaaf4d53e Wed Sep 2 13:45:33 2026 -0700
    remove references and functions related to the "debug" argument, becomes a harmless no-operation and eliminated potential security loop-holes no redmine
• src/hg/hubApi/list.c
  • lines changed 1, context: html, text, full: html, text
    7a185fecc55ea44b360f04d250e64bcbaaf4d53e Wed Sep 2 13:45:33 2026 -0700
    remove references and functions related to the "debug" argument, becomes a harmless no-operation and eliminated potential security loop-holes no redmine
• src/hg/inc/chromInfo.h
  • lines changed 1, context: html, text, full: html, text
    752e20a9a1dc08793c5b529bbeac4ce5836a6be4 Fri Sep 4 17:13:55 2026 -0700
    hgc: stop five details-page handlers from aborting over a missing piece. chromSeqFileExists() opened its connection with sqlConnect() and only then asked whether the database existed, so it aborted before it could answer. The otherDb of a chain or net track is often not a local database at all -- a GenArk hub assembly, or one long retired -- so use sqlMayConnect() and return FALSE. This is 112 of the failures the TrackCheck robot reports, most of them hg38 net tracks against HPRC assemblies. Also moved the disconnect out of the if, where it leaked a connection whenever a database had no chromInfo table, and dropped the now-redundant sqlDatabaseExists() call, which was itself a second connection. mgcCloneInfoLoad() aborted when a clone carried no MGC: id in hgFixed.mrnaClone. Nothing on the page reads that field, and a clone can legitimately have only an IMAGE: id, so the check went away rather than the page. hDbOrganism() aborted for an assembly that has left dbDb but is still named by a maf component, which hg16.evofold does via mm3; it now falls back on the database name. The pgSnp SIFT and Polyphen prediction tables are loaded separately from the tracks that name them, so a machine can have the track and not the table, as hgwbeta and the RR do for hg18. Check with hTableExists first and say the predictions are unavailable instead of letting the query take the page down. Same treatment for the RNA fold diagram: a non-zero ghostscript exit now drops only the diagram and keeps the rest of the page, including the PseudoViewer link. Two of these report an unavailable piece through warn(), which still marks the page for the robot. That is deliberate -- the missing hg18 tables and the RNA fold diagram are real defects, and the log should keep naming them until they are fixed. On the RNA fold diagram in particular: RNAplot truncates the sequence id it is given to 42 characters, and the trash path we build is already 41, so it has never written the file ghostscript is asked to convert. That is worth its own fix. refs #37424
• src/hg/inc/genePred.h
  • lines changed 4, context: html, text, full: html, text
    059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
    Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245 This convention was previously only documented indirectly, as a SQL filtering tip on the Gene tracks FAQ page. Add it next to the cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as, sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and sangerGene.h, and mention it in FAQformat.html and bigGenePred.html (via the equivalent thickStart == thickEnd check).
• src/hg/inc/hdb.h
  • lines changed 4, context: html, text, full: html, text
    571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
    hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260 hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands each track to addTrackIfDataAccessible(), which frees any track whose data cannot be reached. That is right for the browser, which should degrade gracefully rather than draw a broken track, and wrong for hubCheck, whose main job is to say that the file is unreachable. The track was gone before the file check ran, so hubCheck printed nothing and exited 0: cd hubCheck/tests/input sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned Split the non-pruning part of trackDbPolishAfterLinkup() into trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour is unchanged. Added a tests/ case whose bigDataUrl does not exist. Before this change it produced no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb". Of the existing tests, only badType changes, and it gains a real error that the prune had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was already failing before this commit and still is, for an unrelated reason: seven tests carry expected output with per-track "missing description page" warnings that no longer appear, because those hubs point at remote URLs that no longer resolve and hubCheckTrack aborts at the file check before reaching the description check. That is stale test data, not addressed here.
• src/hg/inc/hgConfig.h
  • lines changed 5, context: html, text, full: html, text
    d2a2afcd6c5c19257f8ed2f746c3f94474f4ab78 Tue Sep 1 13:37:34 2026 -0700
    hgTracks: let hg.conf set how big a step malloc takes, refs #38225 hgTracks loads tracks in parallel threads, and each thread grows its own memory pool in the 128 kB steps glibc uses by default. A heavy render asks the kernel to enlarge a pool about 133,000 times, work that does nothing for the reader. The new hg.conf setting mallocTopPad is a number of bytes. When it is set, cfgSetMallocTopPad() passes it to mallopt(M_TOP_PAD) before anything else in main() has a chance to allocate, so the heap grows in that size step instead. Absent or zero, glibc is left alone and nothing changes. Measured with the #38094 harness over eight sessions and two positions, at 32 MB: renders take 0.926 of the time they did, against a noise floor of 1.006. With the setting absent the same build measures 1.004, inside that noise floor. Through apache on a sandbox it measures 0.906. The cost is about 65 MB more resident memory on a heavy session and 8 to 20 MB on an ordinary view, where it also saves no measurable time; the saving and the cost are the same effect and appear together. The rendered image is byte identical either way. Off by default, so a server opts in.
• src/hg/inc/sangerGene.h
  • lines changed 2, context: html, text, full: html, text
    059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
    Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245 This convention was previously only documented indirectly, as a SQL filtering tip on the Gene tracks FAQ page. Add it next to the cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as, sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and sangerGene.h, and mention it in FAQformat.html and bigGenePred.html (via the equivalent thickStart == thickEnd check).
• src/hg/inc/sessionData.h
  • lines changed 4, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
• src/hg/inc/snapshotSession.h
  • lines changed 73, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
  • lines changed 7, context: html, text, full: html, text
    89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
    snapshotSession: rename the snapshot "reaper" to "cleaner" Renames the snapshotReaper utility to snapshotCleaner and drops the word "reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and the doc/usage text now say clean/cleaned/cleaner). No behavior change. refs #38197
  • lines changed 6, context: html, text, full: html, text
    ef8d1004e3d984933a79686a7369d74825886c48 Fri Sep 4 11:35:31 2026 -0700
    snapshotSession: reject a snapshot missing its required var A "blat" snapshot is a dead link without blatLastBigBed (the pinned bigPsl). That variable is set by an async hgc buildBigPsl call, so a share clicked before the build finished - or after it failed - would have minted a link that reopens to nothing. A snapshotType may now name a requiredVar, and doSaveSessionJson refuses the save (asking the caller to retry) instead of handing out a broken link. Found in code review of the BLAT share wiring. refs #38197
• src/hg/inc/trackHub.h
  • lines changed 4, context: html, text, full: html, text
    f759bf663debc22e6015c6c38c22f922bfbdf45b Fri Aug 21 10:01:02 2026 -0700
    lib, cgilib: sanitize description HTML where it comes in, refs #38126 A track hub's track description, a custom track's documentation, and an assembly hub's genome description are all written by somebody else and were stored and printed as they arrived. Run each through htmlSanitize() at the point it is read, so that everything downstream sees text we are willing to print. Sanitizing on the way in is the only practical place: printTrackHtml() alone has more than a hundred callers, and hgTrackUi, hui.c, hgGene, hgGtexTrackSettings and hgTables all print tdb->html directly. The five places are trackHubAddOneDescription(), customTrack.c where it used to call jsStripJavascript(), the htmlUrl fetch in customFactory.c, hgPositionsHelpHtmlCart() and hAssemblyDescription(). In the last two, only the branch that fetches over the network is touched. The branch that reads a local file is left as it is, because those are our own description.html files. trackHubAddOneDescription() keeps its old shape. The fetch moves into a helper so that trackHubDescriptionRemovals() can hand hubCheck the list of things the filter takes out.
• src/hg/inc/wikiLink.h
  • lines changed 7, context: html, text, full: html, text
    d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
    hgLogin: let users set or change their recovery email address, refs #38197 Adds a page where a signed-in user can set or change the recovery address on their account, offered in the account menu next to "Change email" -- both in the top right blue bar popup and on the session page. It is off by default: set login.recovEmailChange=on in hg.conf to offer it. The page also needs login.cookieSalt, working outbound mail and the recovEmailVerified column, and stays hidden where any of those is missing. The new address is confirmed by mail before it takes effect, so whatever is on the account keeps working until the link is opened and a typo costs the user nothing. An account that has a password must supply it, since a confirmed recovery address can sign in. Once the address does change, the account's main address is told, the same notice that a change of the main address already sends. One signature now covers both the address given at signup and a later change, so there is a single confirmation path rather than two.
• src/hg/js/README
  • lines changed 8, context: html, text, full: html, text
    e46c62197a14b397bb01352c8309238a581ff935 Thu Aug 27 14:40:35 2026 -0700
    Remove the dead 2015 JsTestDriver setup under hg/js/test, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/js/hgBlat.js
  • lines changed 62, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
  • lines changed 103, context: html, text, full: html, text
    42b33234bee213325c2b6eede35edbb4fb2e566c Thu Sep 3 06:40:30 2026 -0700
    hgBlat: spinner while a search runs, shorter example, left-justified result columns Four items from Lou's note-37 on #38086. The search is a plain form POST, so nothing on screen changed between Submit and the arrival of the results - on an assembly served by a dynamic BLAT server that is a ten second wait with no sign that anything is happening. A busy overlay now covers the form: a generic .gbBusyBg/.gbSpinner in gbModern.css (CSS-drawn, since the page is already navigating away when it appears), put up by blatFormBusyWire() 350 ms after submit so a sub-second hg38 search does not flash a scrim, and swapping its text after 8 s to say that an index is probably being loaded on demand. Skipped for Clear and for an empty query; a pageshow handler takes it down again after a Back navigation. The example sequence goes from 13.8 kb of the SOD1 locus to 2.5 kb of PTP4A3, hg38 chr8:141,424,501-141,427,000. That window is carried by both chr8_KI270819v1_alt and chr8_MU273363v1_fix, so the example now demonstrates the alt and fix rows and their FAQ icons instead of returning one plain hit, and it covers two PTP4A3 exons so the result track is named after the gene. Every column of the results table is left-justified; the numeric ones keep tabular figures, and the score value gets a fixed width so its bars stay in a straight line. The alt/fix explanation icon is red rather than faint grey, and is drawn as the browser's own info-icon SVG rather than the U+24D8 glyph, which some system fonts do not have. refs #38086
  • lines changed 1, context: html, text, full: html, text
    89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
    snapshotSession: rename the snapshot "reaper" to "cleaner" Renames the snapshotReaper utility to snapshotCleaner and drops the word "reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and the doc/usage text now say clean/cleaned/cleaner). No behavior change. refs #38197
  • lines changed 35, context: html, text, full: html, text
    994528edb7d89068f371fef6d8a5fc72566f7334 Fri Sep 4 13:35:16 2026 -0700
    Adding an opt-in "Keep only last search" checkbox to the new BLAT search form. refs #36292 Behind a new hg.conf setting, blatOnlyLatestCheckbox (default off, documented in ex.hg.conf). When checked, each new BLAT search removes the user's earlier BLAT result tracks in hgc's buildBigPsl (cart variable blatOnlyLatest, remembered per browser in localStorage); unchecked keeps the current accumulate behavior, so site defaults are unchanged. Deletion requires both the hg.conf setting and the cart variable, so a stale cart cannot delete tracks where the feature is off. Independent of, and coexists with, the blatOldTracks hg.conf setting.
  • lines changed 56, context: html, text, full: html, text
    cc23b76a1f2c7dc6a29c227661bfb0ce4d20eb54 Fri Sep 4 13:43:48 2026 -0700
    BLAT page polish from QA: Load example button, corrected input limits, wording fixes. refs #37996 Load example is a chip-style button with a mouseover instead of a hyperlink. Removed the Show input limits modal; corrected the limits in hgTracksHelp.html, stale since 2013 (DNA is 75,000 bases per sequence and 187,500 combined, protein and translated 10,000 and 25,000), added a #blatLimits anchor and pointed the two over-limit BLAT warnings and the form at it. Genome selector: section title Target assembly, type-to-search hint in the dropdown, Shortcuts: label, input and Query type select sized to match. Similar tools sidebar uses house-style hyphens and corrects the findMotif name and description. Share and rename text now states link lifetimes accurately (snapshot links last years, only sessions are permanent). Alignment page buttons match .gbPill, summary strip spacing fixed, New BLAT search button demoted from primary. Documents blatResultsGroup in ex.hg.conf.
  • lines changed 5, context: html, text, full: html, text
    8b20ca320b01a4d283273fc4c0358a29319b2548 Fri Sep 4 15:02:46 2026 -0700
    New BLAT form banner: welcome wording with a link to the news announcement. refs #37996 Replaces "We are testing a new BLAT search page" with "Welcome to our new BLAT page", linking to the announcement's agreed anchor (goldenPath/newsarch.html#newBLAT); keeps the persistent opt-out link to the classic form and the feedback mailto.
  • lines changed 8, context: html, text, full: html, text
    3a3d2f00ba0408f55f02a8ef265dd4fc153a163b Fri Sep 4 15:06:35 2026 -0700
    New BLAT results banner: same welcome wording as the search form's banner. refs #37996
• src/hg/js/hgSession.js
  • lines changed 6, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
• src/hg/js/hgTracks.js
  • lines changed 49, context: html, text, full: html, text
    7007b23dcc178ac0d6a64642ecd5a1060c6f6a40 Thu Aug 27 13:47:54 2026 -0700
    hgTracks: measure how long the track image takes to reach the reader, refs #38109 We cannot choose a png compression level without knowing the reader's connection speed. Our own logs will not tell us: apache stops timing once the kernel has the bytes, so its duration field stays near two milliseconds whether the image is 20 KB or 500 KB. The reader's browser does know. It keeps a timing record for every image it loads, holding the bytes taken off the wire and the time waited. This reads that record for the track image and reports the numbers on the query string of DOT.gif, a 43 byte image that already sits in htdocs/images. No process starts, nothing touches the cart, and the client address, the time and the numbers all land in the same access log we already archive. New hg.conf setting pngTimingSampleRate. It is the N in one page load in N. Zero, or the setting left out, turns the whole thing off, which is the default. The report carries three numbers. ts is the bytes off the wire, d is the whole fetch, and x is only the time the bytes were arriving. x is the denominator for throughput, d is what the reader actually waited.
  • lines changed 14, context: html, text, full: html, text
    f01c7dc7ea48d00510033d7c9448b20bd9c68e19 Mon Aug 31 13:44:24 2026 -0700
    hgTracks: keep the fast png deliveries in the timing sample, refs #38109 Two fixes from a review of the timing beacon. The first threw away a real measurement. A download whose responseEnd and responseStart fall in the same clock tick was treated the same as an image that never crossed the wire. Firefox rounds resource timing to a millisecond, so a small track image on a fast link lands there. Dropping those samples would leave only the slow connections in the record, which is the wrong population to pick a compression level from. The beacon now sends them and reports x=0. Take d as the denominator when x is zero. The second was the beacon image itself. An Image with no reference to it can be collected before the request goes out. It now lives in a variable that outlives the function.
• src/hg/js/test/JsTestDriver-1.3.2.jar
  • lines changed 0, context: html, text, full: html, text
    e46c62197a14b397bb01352c8309238a581ff935 Thu Aug 27 14:40:35 2026 -0700
    Remove the dead 2015 JsTestDriver setup under hg/js/test, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/js/test/README
  • lines changed 130, context: html, text, full: html, text
    e46c62197a14b397bb01352c8309238a581ff935 Thu Aug 27 14:40:35 2026 -0700
    Remove the dead 2015 JsTestDriver setup under hg/js/test, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/js/test/coverage-1.3.2.jar
  • lines changed 0, context: html, text, full: html, text
    e46c62197a14b397bb01352c8309238a581ff935 Thu Aug 27 14:40:35 2026 -0700
    Remove the dead 2015 JsTestDriver setup under hg/js/test, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/js/test/jsTestCov
  • lines changed 3, context: html, text, full: html, text
    e46c62197a14b397bb01352c8309238a581ff935 Thu Aug 27 14:40:35 2026 -0700
    Remove the dead 2015 JsTestDriver setup under hg/js/test, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/js/test/jsTestDriver.conf
  • lines changed 12, context: html, text, full: html, text
    e46c62197a14b397bb01352c8309238a581ff935 Thu Aug 27 14:40:35 2026 -0700
    Remove the dead 2015 JsTestDriver setup under hg/js/test, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/js/test/jsTestRun
  • lines changed 3, context: html, text, full: html, text
    e46c62197a14b397bb01352c8309238a581ff935 Thu Aug 27 14:40:35 2026 -0700
    Remove the dead 2015 JsTestDriver setup under hg/js/test, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/js/test/jsTestServer
  • lines changed 3, context: html, text, full: html, text
    e46c62197a14b397bb01352c8309238a581ff935 Thu Aug 27 14:40:35 2026 -0700
    Remove the dead 2015 JsTestDriver setup under hg/js/test, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/js/test/testEncodeProject.js
  • lines changed 49, context: html, text, full: html, text
    e46c62197a14b397bb01352c8309238a581ff935 Thu Aug 27 14:40:35 2026 -0700
    Remove the dead 2015 JsTestDriver setup under hg/js/test, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/js/topLinks.js
  • lines changed 181, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
  • lines changed 3, context: html, text, full: html, text
    d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
    hgLogin: let users set or change their recovery email address, refs #38197 Adds a page where a signed-in user can set or change the recovery address on their account, offered in the account menu next to "Change email" -- both in the top right blue bar popup and on the session page. It is off by default: set login.recovEmailChange=on in hg.conf to offer it. The page also needs login.cookieSalt, working outbound mail and the recovEmailVerified column, and stays hidden where any of those is missing. The new address is confirmed by mail before it takes effect, so whatever is on the account keeps working until the link is opened and a typo costs the user nothing. An account that has a password must supply it, since a confirmed recovery address can sign in. Once the address does change, the account's main address is told, the same notice that a change of the main address already sends. One signature now covers both the address given at signup and a later change, so there is a single confirmation path rather than two.
  • lines changed 13, context: html, text, full: html, text
    7d0ceafa7c0753b9cd7e59b5e29641508e59ec87 Fri Sep 4 12:18:57 2026 -0700
    Close the Account dialog when one of its links is clicked, refs #38257 The dialog relied on navigation to remove it: every link in it leads to another page, and loading that page took the dialog away with it. "My Track Hubs" is the exception. On hgHubConnect it points at the page you are already on, and hgHubConnect.js cancels the click and switches to the Connected Hubs tab instead, so nothing ever removed the dialog. The dialog also swallowed the click on its way up, so hgHubConnect.js never saw it and the tab did not change either. Let clicks on links through, and close the dialog on the next tick, after the click has been fully dispatched.
  • lines changed 7, context: html, text, full: html, text
    cc23b76a1f2c7dc6a29c227661bfb0ce4d20eb54 Fri Sep 4 13:43:48 2026 -0700
    BLAT page polish from QA: Load example button, corrected input limits, wording fixes. refs #37996 Load example is a chip-style button with a mouseover instead of a hyperlink. Removed the Show input limits modal; corrected the limits in hgTracksHelp.html, stale since 2013 (DNA is 75,000 bases per sequence and 187,500 combined, protein and translated 10,000 and 25,000), added a #blatLimits anchor and pointed the two over-limit BLAT warnings and the form at it. Genome selector: section title Target assembly, type-to-search hint in the dropdown, Shortcuts: label, input and Query type select sized to match. Similar tools sidebar uses house-style hyphens and corrects the findMotif name and description. Share and rename text now states link lifetimes accurately (snapshot links last years, only sessions are permanent). Alignment page buttons match .gbPill, summary strip spacing fixed, New BLAT search button demoted from primary. Documents blatResultsGroup in ex.hg.conf.
  • lines changed 4, context: html, text, full: html, text
    6e89bb12a35c65c0607719f2aa2c7377803d09f9 Fri Sep 4 14:03:07 2026 -0700
    Keep the top-right Login/Share a link menu icon inside the blue bar on narrow windows. On screens under 1100px the icon was positioned with position:fixed, which pins it to the top of the window. That is the top of the blue bar only on hgTracks; on hgGateway and the home page the bar sits below the logo banner, so the icon showed up in the white area above it. Use position:sticky instead, which keeps the icon in the bar on every page and still holds it at the right edge of the visible viewport while the 1000px-wide bar is scrolled sideways. The hardcoded background colour is replaced by the bar's own, which is not the same on all pages. The links now hang in their own span so that opening the menu no longer pushes the page content down, and the icon is left out entirely on mirrors that have no top-right links to show. refs #38251
• src/hg/js/utils.js
  • lines changed 6, context: html, text, full: html, text
    f7bfbc23bb2aaccb6113e055608ca8374beca5ad Thu Sep 3 15:24:03 2026 -0700
    Make dragReorder.init() idempotent so that when users hide multiple visible tracks one after the other we don't spuriously attach event handlers, refs #38212
  • lines changed 8, context: html, text, full: html, text
    cc23b76a1f2c7dc6a29c227661bfb0ce4d20eb54 Fri Sep 4 13:43:48 2026 -0700
    BLAT page polish from QA: Load example button, corrected input limits, wording fixes. refs #37996 Load example is a chip-style button with a mouseover instead of a hyperlink. Removed the Show input limits modal; corrected the limits in hgTracksHelp.html, stale since 2013 (DNA is 75,000 bases per sequence and 187,500 combined, protein and translated 10,000 and 25,000), added a #blatLimits anchor and pointed the two over-limit BLAT warnings and the form at it. Genome selector: section title Target assembly, type-to-search hint in the dropdown, Shortcuts: label, input and Query type select sized to match. Similar tools sidebar uses house-style hyphens and corrects the findMotif name and description. Share and rename text now states link lifetimes accurately (snapshot links last years, only sessions are permanent). Alignment page buttons match .gbPill, summary strip spacing fixed, New BLAT search button demoted from primary. Documents blatResultsGroup in ex.hg.conf.
• src/hg/lib/cart.c
  • lines changed 29, context: html, text, full: html, text
    a6a7e658e255ff09b7128f43b84de3a5ad1d74d2 Tue Aug 25 09:28:08 2026 -0700
    cart.c: resolve a GenArk db= again after a session load, refs #38184 A URL with both db=GCF_... and a session load failed with an unknown database error. fixUpDb turns db=GCF_... into genome= plus hubUrl=, but the session load empties the cart and then restores the raw CGI variables, so the bare accession reached hDbConnect. Split the GenArk part of fixUpDb into resolveGenarkDb and call that once more after the session-load block. Only the rewrite runs there, not the bad-database errAbort, so a session that carries a retired db fails the way it does now. Covers hgS_doLoadUrl and hgS_doOtherUser.
  • lines changed 8, context: html, text, full: html, text
    b0f8840863d5673f41d45932502249141a9e3241 Mon Aug 31 10:00:49 2026 -0700
    cart.c: keep the session position when db= resolves to the same assembly, refs #38184 resolveGenarkDb takes db out of the cart, so the block that runs after hubConnectLoadHubs saw an empty db and called every GenArk db= a database change. It then set the old db to "none", which getDbAndGenome reads as a switch of assembly: the position was replaced with the assembly default and virtMode, virtModeType and nonVirtPosition were dropped. A session loaded with db=GCF_... came up at the default position instead of the one it saved. Only call it a database change when the value the cart had before the CGI variables were applied names a different database. A session that really is on another assembly still gets the default position.
  • lines changed 1, context: html, text, full: html, text
    5abe782bada3c851f1844c94b747cf7c769c71f9 Fri Sep 4 09:12:46 2026 -0700
    cart.c: gate the captcha session-id debug print behind hg.conf captchaDebug, refs #38247 Register captchaDebug in the hg.conf catalog as a permanent debugging knob rather than a release gate.
• src/hg/lib/chromInfo.c
  • lines changed 12, context: html, text, full: html, text
    752e20a9a1dc08793c5b529bbeac4ce5836a6be4 Fri Sep 4 17:13:55 2026 -0700
    hgc: stop five details-page handlers from aborting over a missing piece. chromSeqFileExists() opened its connection with sqlConnect() and only then asked whether the database existed, so it aborted before it could answer. The otherDb of a chain or net track is often not a local database at all -- a GenArk hub assembly, or one long retired -- so use sqlMayConnect() and return FALSE. This is 112 of the failures the TrackCheck robot reports, most of them hg38 net tracks against HPRC assemblies. Also moved the disconnect out of the if, where it leaked a connection whenever a database had no chromInfo table, and dropped the now-redundant sqlDatabaseExists() call, which was itself a second connection. mgcCloneInfoLoad() aborted when a clone carried no MGC: id in hgFixed.mrnaClone. Nothing on the page reads that field, and a clone can legitimately have only an IMAGE: id, so the check went away rather than the page. hDbOrganism() aborted for an assembly that has left dbDb but is still named by a maf component, which hg16.evofold does via mm3; it now falls back on the database name. The pgSnp SIFT and Polyphen prediction tables are loaded separately from the tracks that name them, so a machine can have the track and not the table, as hgwbeta and the RR do for hg18. Check with hTableExists first and say the predictions are unavailable instead of letting the query take the page down. Same treatment for the RNA fold diagram: a non-zero ghostscript exit now drops only the diagram and keeps the rest of the page, including the PseudoViewer link. Two of these report an unavailable piece through warn(), which still marks the page for the robot. That is deliberate -- the missing hg18 tables and the RNA fold diagram are real defects, and the log should keep naming them until they are fixed. On the RNA fold diagram in particular: RNAplot truncates the sequence id it is given to 42 characters, and the trash path we build is already 41, so it has never written the file ghostscript is asked to convert. That is worth its own fix. refs #37424
• src/hg/lib/customFactory.c
  • lines changed 5, context: html, text, full: html, text
    f759bf663debc22e6015c6c38c22f922bfbdf45b Fri Aug 21 10:01:02 2026 -0700
    lib, cgilib: sanitize description HTML where it comes in, refs #38126 A track hub's track description, a custom track's documentation, and an assembly hub's genome description are all written by somebody else and were stored and printed as they arrived. Run each through htmlSanitize() at the point it is read, so that everything downstream sees text we are willing to print. Sanitizing on the way in is the only practical place: printTrackHtml() alone has more than a hundred callers, and hgTrackUi, hui.c, hgGene, hgGtexTrackSettings and hgTables all print tdb->html directly. The five places are trackHubAddOneDescription(), customTrack.c where it used to call jsStripJavascript(), the htmlUrl fetch in customFactory.c, hgPositionsHelpHtmlCart() and hAssemblyDescription(). In the last two, only the branch that fetches over the network is touched. The branch that reads a local file is left as it is, because those are our own description.html files. trackHubAddOneDescription() keeps its old shape. The fetch moves into a helper so that trackHubDescriptionRemovals() can hand hubCheck the list of things the filter takes out.
• src/hg/lib/customTrack.c
  • lines changed 2, context: html, text, full: html, text
    f759bf663debc22e6015c6c38c22f922bfbdf45b Fri Aug 21 10:01:02 2026 -0700
    lib, cgilib: sanitize description HTML where it comes in, refs #38126 A track hub's track description, a custom track's documentation, and an assembly hub's genome description are all written by somebody else and were stored and printed as they arrived. Run each through htmlSanitize() at the point it is read, so that everything downstream sees text we are willing to print. Sanitizing on the way in is the only practical place: printTrackHtml() alone has more than a hundred callers, and hgTrackUi, hui.c, hgGene, hgGtexTrackSettings and hgTables all print tdb->html directly. The five places are trackHubAddOneDescription(), customTrack.c where it used to call jsStripJavascript(), the htmlUrl fetch in customFactory.c, hgPositionsHelpHtmlCart() and hAssemblyDescription(). In the last two, only the branch that fetches over the network is touched. The branch that reads a local file is left as it is, because those are our own description.html files. trackHubAddOneDescription() keeps its old shape. The fetch moves into a helper so that trackHubDescriptionRemovals() can hand hubCheck the list of things the filter takes out.
• src/hg/lib/ensGene.as
  • lines changed 2, context: html, text, full: html, text
    059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
    Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245 This convention was previously only documented indirectly, as a SQL filtering tip on the Gene tracks FAQ page. Add it next to the cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as, sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and sangerGene.h, and mention it in FAQformat.html and bigGenePred.html (via the equivalent thickStart == thickEnd check).
• src/hg/lib/genePred.as
  • lines changed 2, context: html, text, full: html, text
    059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
    Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245 This convention was previously only documented indirectly, as a SQL filtering tip on the Gene tracks FAQ page. Add it next to the cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as, sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and sangerGene.h, and mention it in FAQformat.html and bigGenePred.html (via the equivalent thickStart == thickEnd check).
• src/hg/lib/genePredExt.as
  • lines changed 2, context: html, text, full: html, text
    059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
    Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245 This convention was previously only documented indirectly, as a SQL filtering tip on the Gene tracks FAQ page. Add it next to the cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as, sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and sangerGene.h, and mention it in FAQformat.html and bigGenePred.html (via the equivalent thickStart == thickEnd check).
• src/hg/lib/hdb.c
  • lines changed 13, context: html, text, full: html, text
    571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
    hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260 hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands each track to addTrackIfDataAccessible(), which frees any track whose data cannot be reached. That is right for the browser, which should degrade gracefully rather than draw a broken track, and wrong for hubCheck, whose main job is to say that the file is unreachable. The track was gone before the file check ran, so hubCheck printed nothing and exited 0: cd hubCheck/tests/input sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned Split the non-pruning part of trackDbPolishAfterLinkup() into trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour is unchanged. Added a tests/ case whose bigDataUrl does not exist. Before this change it produced no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb". Of the existing tests, only badType changes, and it gains a real error that the prune had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was already failing before this commit and still is, for an unrelated reason: seven tests carry expected output with per-track "missing description page" warnings that no longer appear, because those hubs point at remote URLs that no longer resolve and hubCheckTrack aborts at the file check before reaching the description check. That is stale test data, not addressed here.
• src/hg/lib/hgConfig.c
  • lines changed 23, context: html, text, full: html, text
    d2a2afcd6c5c19257f8ed2f746c3f94474f4ab78 Tue Sep 1 13:37:34 2026 -0700
    hgTracks: let hg.conf set how big a step malloc takes, refs #38225 hgTracks loads tracks in parallel threads, and each thread grows its own memory pool in the 128 kB steps glibc uses by default. A heavy render asks the kernel to enlarge a pool about 133,000 times, work that does nothing for the reader. The new hg.conf setting mallocTopPad is a number of bytes. When it is set, cfgSetMallocTopPad() passes it to mallopt(M_TOP_PAD) before anything else in main() has a chance to allocate, so the heap grows in that size step instead. Absent or zero, glibc is left alone and nothing changes. Measured with the #38094 harness over eight sessions and two positions, at 32 MB: renders take 0.926 of the time they did, against a noise floor of 1.006. With the setting absent the same build measures 1.004, inside that noise floor. Through apache on a sandbox it measures 0.906. The cost is about 65 MB more resident memory on a heavy session and 8 to 20 MB on an ordinary view, where it also saves no measurable time; the saving and the cost are the same effect and appear together. The rendered image is byte identical either way. Off by default, so a server opts in.
• src/hg/lib/hgFind.c
  • lines changed 7, context: html, text, full: html, text
    f759bf663debc22e6015c6c38c22f922bfbdf45b Fri Aug 21 10:01:02 2026 -0700
    lib, cgilib: sanitize description HTML where it comes in, refs #38126 A track hub's track description, a custom track's documentation, and an assembly hub's genome description are all written by somebody else and were stored and printed as they arrived. Run each through htmlSanitize() at the point it is read, so that everything downstream sees text we are willing to print. Sanitizing on the way in is the only practical place: printTrackHtml() alone has more than a hundred callers, and hgTrackUi, hui.c, hgGene, hgGtexTrackSettings and hgTables all print tdb->html directly. The five places are trackHubAddOneDescription(), customTrack.c where it used to call jsStripJavascript(), the htmlUrl fetch in customFactory.c, hgPositionsHelpHtmlCart() and hAssemblyDescription(). In the last two, only the branch that fetches over the network is touched. The branch that reads a local file is left as it is, because those are our own description.html files. trackHubAddOneDescription() keeps its old shape. The fetch moves into a helper so that trackHubDescriptionRemovals() can hand hubCheck the list of things the filter takes out.
• src/hg/lib/hgHgvs.c
  • lines changed 9, context: html, text, full: html, text
    71bb7711b1374deea53e3dd4fa91fd4da8f21042 Fri Sep 4 17:52:53 2026 -0700
    Fall back to the RefSeq Historical protein table when an NP_ accession is a deprecated version. getProteinSeq read only ncbiRefSeqPepTable, so an HGVS protein term naming a superseded NP_ failed validation and the search fell through to a whole-gene match, or to nothing. getCdnaSeq has had the equivalent historical fallback for years; this gives the protein path the same one, guarded by hTableExists so it degrades cleanly where the new table is not loaded yet. refs #38248
  • lines changed 4, context: html, text, full: html, text
    4c1caa46a5cd5bb553e81373872a2b0fcbd618ae Fri Sep 4 17:53:17 2026 -0700
    Pick the newest deprecated transcript when mapping an HGVS protein term. Several deprecated transcript versions can share one protein version, and the lookup took whichever row came back first. Sort by length before value so that .10 beats .9 rather than losing a string comparison. refs #38248
• src/hg/lib/hubConnect.c
  • lines changed 9, context: html, text, full: html, text
    9ce14583f270ff891278d2cf4f0c6432bf79fc61 Wed Sep 2 11:20:27 2026 -0700
    hubConnect: hubEncode covers custom tracks as well as hub tracks, refs #38172 hubEncode() keyed off isHubTrack() alone, so the thirty-odd call sites that use it treated a custom track's data fields as our own. A custom track's data comes from outside the same way a hub's does, and customFactory.c only looks at the track and label lines, not the data fields, which are what hubEncode() is called on. Add isCustomTrack() to the test. Raised in the v503 Preview II code review.
• src/hg/lib/hubSearchText.c
  • lines changed 2, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/lib/hui.c
  • lines changed 5, context: html, text, full: html, text
    cdb348c551227ac24dc0fab9f18516459b47b962 Sat Sep 5 11:33:32 2026 -0700
    hgTrackUi: don't try MySQL for a GenArk quickLift source assembly, refs #38272 The track settings page swaps in the track's quickLiftDb for the database it passes down to the cfg routines. For a GenArk source that is a bare accession with no MySQL database behind it, and asForDb() connected anyway, so the page stopped after the display-mode row. trackHubDatabase() does not catch this because the source hub is not loaded in that request; hgc got the matching isGenArk() guard in #37388. Two filter blocks in the same file guarded with isHubTrack(), a plain hub_ prefix test that a bare accession also slips past. Same guard added there.
  • lines changed 8, context: html, text, full: html, text
    be590a4f6b10055e567a9ac717919cd99481272a Fri Sep 4 17:01:05 2026 -0700
    Let a dataVersion file under /gbdb be read for hub tracks A track may point dataVersion at a local file, and otto jobs use that to show a live version string. Hub tracks were excluded, because a hub is supplied by the user and could otherwise name any file on the server. That also excludes the curated-hub assemblies. hs1 is served to the browser as a hub, so its tracks are hub tracks, and hgTrackUi printed the raw path where the version should be. Paths under /gbdb are safe to make an exception for: that tree is public data, mirrored on hgdownload, so reading one discloses nothing that is not already downloadable. Also resolves $D against the plain database name, so it does not expand to hub_NNNNN_hs1 on a curated hub. refs #38268
  • lines changed 28, context: html, text, full: html, text
    1a85cdaa59f02695ccecc1aac8a9d40bac188064 Sun Sep 6 07:10:47 2026 -0700
    hgTrackUi: tighten the dataVersion path check, refs #38268 checkDataVersion() decides whether a dataVersion setting that looks like a local path may be opened. Do that test on the path after $D substitution rather than on the raw setting, and accept only a plain path in the /gbdb tree. A quickLifted track no longer needs a case of its own, its file is under /gbdb as well.
• src/hg/lib/knownGene.as
  • lines changed 2, context: html, text, full: html, text
    059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
    Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245 This convention was previously only documented indirectly, as a SQL filtering tip on the Gene tracks FAQ page. Add it next to the cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as, sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and sangerGene.h, and mention it in FAQformat.html and bigGenePred.html (via the equivalent thickStart == thickEnd check).
• src/hg/lib/makefile
  • lines changed 1, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
• src/hg/lib/refFlat.as
  • lines changed 2, context: html, text, full: html, text
    059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
    Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245 This convention was previously only documented indirectly, as a SQL filtering tip on the Gene tracks FAQ page. Add it next to the cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as, sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and sangerGene.h, and mention it in FAQformat.html and bigGenePred.html (via the equivalent thickStart == thickEnd check).
• src/hg/lib/sangerGene.as
  • lines changed 2, context: html, text, full: html, text
    059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
    Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245 This convention was previously only documented indirectly, as a SQL filtering tip on the Gene tracks FAQ page. Add it next to the cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as, sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and sangerGene.h, and mention it in FAQformat.html and bigGenePred.html (via the equivalent thickStart == thickEnd check).
• src/hg/lib/snapshotSession.c
  • lines changed 279, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
  • lines changed 6, context: html, text, full: html, text
    89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
    snapshotSession: rename the snapshot "reaper" to "cleaner" Renames the snapshotReaper utility to snapshotCleaner and drops the word "reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and the doc/usage text now say clean/cleaned/cleaner). No behavior change. refs #38197
  • lines changed 9, context: html, text, full: html, text
    ef8d1004e3d984933a79686a7369d74825886c48 Fri Sep 4 11:35:31 2026 -0700
    snapshotSession: reject a snapshot missing its required var A "blat" snapshot is a dead link without blatLastBigBed (the pinned bigPsl). That variable is set by an async hgc buildBigPsl call, so a share clicked before the build finished - or after it failed - would have minted a link that reopens to nothing. A snapshotType may now name a requiredVar, and doSaveSessionJson refuses the save (asking the caller to retry) instead of handing out a broken link. Found in code review of the BLAT share wiring. refs #38197
• src/hg/lib/trackHub.c
  • lines changed 26, context: html, text, full: html, text
    f759bf663debc22e6015c6c38c22f922bfbdf45b Fri Aug 21 10:01:02 2026 -0700
    lib, cgilib: sanitize description HTML where it comes in, refs #38126 A track hub's track description, a custom track's documentation, and an assembly hub's genome description are all written by somebody else and were stored and printed as they arrived. Run each through htmlSanitize() at the point it is read, so that everything downstream sees text we are willing to print. Sanitizing on the way in is the only practical place: printTrackHtml() alone has more than a hundred callers, and hgTrackUi, hui.c, hgGene, hgGtexTrackSettings and hgTables all print tdb->html directly. The five places are trackHubAddOneDescription(), customTrack.c where it used to call jsStripJavascript(), the htmlUrl fetch in customFactory.c, hgPositionsHelpHtmlCart() and hAssemblyDescription(). In the last two, only the branch that fetches over the network is touched. The branch that reads a local file is left as it is, because those are our own description.html files. trackHubAddOneDescription() keeps its old shape. The fetch moves into a helper so that trackHubDescriptionRemovals() can hand hubCheck the list of things the filter takes out.
  • lines changed 169, context: html, text, full: html, text
    f001e39d1561c86eb6832c556888e73dc200f8d0 Thu Aug 27 14:10:09 2026 -0700
    quickLift: let a second lift update a track already in the hub, refs #38198 The hub writer skipped any track whose name was already in the hub file. The first lift writes a container together with all of its children, including the ones switched off, so those names were taken from then on. A later lift skipped the container and everything inside it: a subtrack switched on after the first lift never reached the target, and a visibility change on a track that had already been lifted never reached it either. walkTree no longer takes the set of names already in the file and no longer skips a track for being there. It writes every visible track into a dyString, and the new writeMergedHubFile merges that against the file: a generated stanza replaces the old stanza of the same track, in the slot the old one held, so parents stay ahead of their children; a track in the file that was not generated this time is kept as it was, so tracks still accumulate across lifts in one session. The skip could not simply be dropped because a duplicate track name makes hub loading abort, and replacing in place keeps names unique. The stanza parser inside quickLiftHubRemoveTrack is now readStanzas, shared by both paths. A container is many stanzas there, one per track line, which is what makes the per-track replacement work. The file is written to a temporary name and renamed into place. Every lift now rewrites the whole file and the target assembly reads that same file, so an in-place rewrite could hand a reader a truncated hub. outTrack also frees the dyString it gets from trackDbString.
  • lines changed 2, context: html, text, full: html, text
    1258d7f65e7058408390246f0b592ae00e1b4d26 Mon Aug 31 13:48:21 2026 -0700
    trackHub: keep a description a track inherited from its parent, refs #38126 trackHubAddOneDescription used to return early when a track had no html setting of its own, which left tdb->html alone. Sanitizing where the text comes in made it assign every time, so it cleared a description that trackHubAddDescription had copied down from an ancestor. No caller loses a page over this today, because each one re-fetches afterward, but the function should not be destructive.
  • lines changed 14, context: html, text, full: html, text
    bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
    hubCheck: stop resolving bigDataUrl a second time, refs #38256 trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against genome->trackDbFile, which prepended the hub's directory a second time. For a local hub reached by a relative path that turned a file which is present into a file which is reported missing: hubCheck out/hub.txt Couldn't open out/out/hg19/x.bb Remote hubs and local hubs given by an absolute path were unaffected, because the second resolution is a no-op once the path is a URL or already absolute, which is why this went unnoticed. The tests/makefile has been working around it: the fieldCountMismatch case has to cd into input/ before it can find its bigBed. Added a tests/ case that does not cd, with the data file in a subdirectory below hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report that mismatch if it really opened the file. Before this change the test reported "Couldn't open input/relPath/input/relPath/data/test6field.bb". Verified against a binary built from the unmodified tree: this changes the output of no other test in hubCheck/tests. Seven of those tests were already failing before this commit and still are.
• src/hg/lib/web.c
  • lines changed 6, context: html, text, full: html, text
    d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
    hgLogin: let users set or change their recovery email address, refs #38197 Adds a page where a signed-in user can set or change the recovery address on their account, offered in the account menu next to "Change email" -- both in the top right blue bar popup and on the session page. It is off by default: set login.recovEmailChange=on in hg.conf to offer it. The page also needs login.cookieSalt, working outbound mail and the recovEmailVerified column, and stays hidden where any of those is missing. The new address is confirmed by mail before it takes effect, so whatever is on the account keeps working until the link is opened and a typo costs the user nothing. An account that has a password must supply it, since a confirmed recovery address can sign in. Once the address does change, the account's main address is told, the same notice that a change of the main address already sends. One signature now covers both the address given at signup and a later change, so there is a single confirmation path rather than two.
• src/hg/lib/wikiLink.c
  • lines changed 26, context: html, text, full: html, text
    d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
    hgLogin: let users set or change their recovery email address, refs #38197 Adds a page where a signed-in user can set or change the recovery address on their account, offered in the account menu next to "Change email" -- both in the top right blue bar popup and on the session page. It is off by default: set login.recovEmailChange=on in hg.conf to offer it. The page also needs login.cookieSalt, working outbound mail and the recovEmailVerified column, and stays hidden where any of those is missing. The new address is confirmed by mail before it takes effect, so whatever is on the account keeps working until the link is opened and a typo costs the user nothing. An account that has a password must supply it, since a confirmed recovery address can sign in. Once the address does change, the account's main address is told, the same notice that a change of the main address already sends. One signature now covers both the address given at signup and a later change, so there is a single confirmation path rather than two.
  • lines changed 6, context: html, text, full: html, text
    a1b16efb1479af46ea0ed5d6464f084479ce8490 Wed Sep 2 11:20:43 2026 -0700
    wikiLink: returnUrlSchemeIsSafe should refuse a scheme-relative URL, refs #38172 The function allows an http URL, an https URL or a relative one. It found the scheme by looking for a colon, so "//host/path" fell through the first test and was accepted as relative, though it names another host. Refuse it, and say so in the comment. Raised in the v503 Preview II code review.
• src/hg/liftOver/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/logCrawl/dbTrackAndSearchUsage/generateUsageStats.py
  • lines changed 36, context: html, text, full: html, text
    4395aff08f55db6aea215e9717ceed746054840b Wed Sep 2 15:34:32 2026 -0700
    Fix and extend the monthly usage stats cron. refs #38232 Adds the GenArk and hubSpace reporting asked for in the ticket, and fixes several things that were quietly producing wrong numbers. New in the report: a summed GenArk row in the database usage table, a section ranking the top GenArk assemblies, and a closing section listing hubSpace usage with one row per user. The non-public hub list now drops GenArk and curated hubs, keeps only each hubSpace user's busiest hub, and reports track collections and ENCODE search hubs as one summed line each - previously a single person uploading a couple of hundred hubs took over most of the list. Fixes in generateUsageStats.py: a missing space meant db= was never passed to hgTracks, so every assembly was reported with hg38's default tracks. Only the first ~800 byte block of hgTracks' track list was read, so hg38 was analysed with 40 of its 91 defaults. Hub backed databases are now skipped, since hub ids differ between hgcentrals and their track names could never match the logs. Fixes in assemblyStatsCron.py: hgTracks was run from the wrong directory and died before logging anything, so the default track list was parsed out of timestamps and CGI_TIME lines - the "non-default track usage" section was really listing default tracks. The startup cleanup was deleting the hubStatus file that genome-asia copies over each month, which dropped asia-only hubs from the counts from February 2025 onward. The report label was computed as today minus 30 days, so the March run labelled itself January and overwrote it, and no February report has ever been published. Hub lookups no longer stop at the first 20 hubs, and are resolved by streaming each mirror's hubStatus once instead of grepping a 341MB file per hub. Counts for hubs are now keyed on the resolved hub URL rather than the track name, so unrelated hubs sharing a common track name are no longer added together. This moves some numbers: CADD reads 280 rather than 293, because three separate CADD download URLs were previously summed into one figure. Table columns, headers included, are now aligned in the emailed output. Warnings about an unreachable mirror or a missing asia file appear in the report body instead of a stderr stream that the cron discards.
  • lines changed 5, context: html, text, full: html, text
    32515a6797b6e68daa3aa94c663760216b24dd67 Thu Sep 3 13:30:26 2026 -0700
    Usage stats cron fixes from code review. refs #38232 Apply the same db mismatch guard to assemblyStatsCron.py that generateUsageStats.py already had. Both files ask hgTracks for one assembly's default tracks and parse the answer off stderr, but only one of them checked that hgTracks answered about the assembly it was asked about. Without it, a repeat of the db= bug this ticket fixed would quietly fill the default track filter with another assembly's tracks and say nothing. Drop any database name that does not look like one before interpolating it into a shell command. The trimmed logs only ever carry real assembly names, so this is defence in depth rather than a live hole. Fix a comment in resolveHub that still described the lastOkTime fallback, which was replaced by mirror order earlier in this ticket. No change to the report. The row counts for the database usage and non-public hub tables were cut from 15 to 10 in the previous commit, which its message did not mention; that was intentional, to make room for the new GenArk and hubSpace sections.
• src/hg/makeDb/doc/archaeaAsmHub/archaea.orderList.tsv
  • lines changed 7, context: html, text, full: html, text
    6e9cae86505025606c69d9f06ca08a1426f6eed9 Mon Aug 31 16:52:26 2026 -0700
    VGP update refs @29545
• src/hg/makeDb/doc/birdsAsmHub/birds.orderList.tsv
  • lines changed 80, context: html, text, full: html, text
    6e9cae86505025606c69d9f06ca08a1426f6eed9 Mon Aug 31 16:52:26 2026 -0700
    VGP update refs @29545
• src/hg/makeDb/doc/contrib/hprc2annot.txt
  • lines changed 228, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/doc/danRer11/danioCode.txt
  • lines changed 140, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
• src/hg/makeDb/doc/fishAsmHub/fish.orderList.tsv
  • lines changed 215, context: html, text, full: html, text
    6e9cae86505025606c69d9f06ca08a1426f6eed9 Mon Aug 31 16:52:26 2026 -0700
    VGP update refs @29545
• src/hg/makeDb/doc/hg38/alphaGenome.txt
  • lines changed 57, context: html, text, full: html, text
    8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
    AlphaGenome Variant Impact (AVI) score track for hg38 New track under the Deleteriousness Predictions container showing Google DeepMind's AlphaGenome Variant Impact score for every possible single-base substitution, as four bigWigs, one per alternate allele. alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per alternate allele. It copies the PHRED column through as text so no precision is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a long contiguous run. The input has no row where alt == ref, so each per-allele file is missing roughly every fourth position; without the fill the wig fragments into blocks of about 3.5 values and the bigWig doubles in size. All 8,812,917,339 input lines were used, none skipped. The data cannot be redistributed, so the stanzas set tableBrowser off and the description page sends users to the AlphaGenome Atlas. Alpha only for now. No reference yet, the score is unpublished. refs #38261
• src/hg/makeDb/doc/hg38/imprinting.txt
  • lines changed 53, context: html, text, full: html, text
    684ad73a17eebb717fba52f0aa25f3d9d7adb2de Thu Sep 3 12:03:37 2026 -0700
    New Imprinting track collection on hg38, first subtrack is MethBase2 ASM Adds the "Imprinting" superTrack to the Regulation group and its first subtrack, the allele-specific methylation summary from the MethBase2 hub (http://smithlab.usc.edu/trackdata/methylation/hub.txt). The bigWig gives, for each of 29,401,795 CpG sites, the fraction of MethBase2 human methylomes in which the site fell inside an allelically methylated region called by dnmtools amrfinder. Taken from upstream unchanged. Alpha only for now, refs #37599
  • lines changed 200, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/doc/hg38/lrSv.txt
  • lines changed 72, context: html, text, full: html, text
    b687dd9018670941ce30f8a4582d6597c5a974d8 Tue Sep 1 14:57:43 2026 -0700
    lrSv1kLin: fix 2bp insertion span, drop dead numConsolidated field, refresh lrSvAll merge. refs #38099 The Lin 1218 VCFs set INFO/END = POS+1 on insertions, and the converter took chromEnd from END, so every insertion was drawn 2bp wide with svLen 2 instead of the 1bp anchor base. That contradicted the track's own description page and the coordinate convention in the makeDoc, and it kept 107,980 Lin insertions from merging in lrSvAll. Insertions now clamp chromEnd to the anchor; deletions are unchanged and still verify span == |SVLEN| against the source VCFs. Dropped numConsolidated from the converter and the .as: the NumConsolidated INFO key is declared in the VCF header but never appears on a data line, so the column was 0 on all 1.2M rows and added a meaningless line to every detail page. Rebuilt lin1218 on hg38 and hs1 (item counts and variant names unchanged) and re-ran the merge: lrSvAll 2,963,093 -> 2,855,267 rows as the duplicate insertion rows collapse. Bumped seven filter.svLen/insLen maxima in lrSv.ra that were short of the data after the August deletion narrowing, three of them only visible on hs1. lrSvAll.html said the 1000 Genomes linear set was not in the merge, which is no longer true, and gave no warning that sourceCount double-counts because Lin1218 already absorbs HPRC, HGSVC3 and both 1KG ONT callsets. Corrected the merge key description and refreshed ten stale cells in the lrSv.html summary table.
• src/hg/makeDb/doc/hg38/ncbiRefSeq.txt
  • lines changed 29, context: html, text, full: html, text
    c8c1b1a9adac6dfa7a869326ca3f774a724a50db Fri Sep 4 17:51:48 2026 -0700
    Document the RefSeq Historical protein table build and add a reminder to the notifier. The historical rna.gbff we already download carries a /translation for every coding record, and gbProcess has a -pepFa flag that writes them out, so this is one extra pass over a file we have rather than a new download. Adds the build steps to the RS_2024_08 makedoc section, and two lines to the weekly notifier so the next refresh does not leave the protein table behind. refs #38248
• src/hg/makeDb/doc/hg38/singleCellSignalsPeaks.txt
  • lines changed 12, context: html, text, full: html, text
    3522a9acc8256a35230b07f2e7ab3fc08c827f33 Sat Sep 5 17:19:20 2026 -0700
    singleCellSignalsPeaks: correct the hg38 description page counts and cite BrainVar, refs #38219 From the v503 code review. Commit 3aac3982aa2 dropped 5 hg38 subtracks and updated the makeDoc, but the dataset list on the description page was missed, so the page disagreed with both the .ra and the facet menu: - Risk Loci in Alzheimer's and Parkinson's: 1 peak subtrack, not 2 (neuro-degen-atac/peaks.bb was the one dropped) - BrainVar: 9 signal subtracks, not 13, and the text still described tracks "for all nuclei together" -- the 4 combined-stage tracks are exactly the ones dropped, so every remaining BrainVar track carries a life stage Counts re-derived from the .ra by type (bigWig = signal, else peak): the other 7 hg38 datasets and all 9 mm10 datasets were already right, 929 and 587 total. BrainVar was also the only hg38 dataset with no citation, while its methods text had grown quite specific. It now cites Werling et al. 2020, which described the cohort, with the caveat that the single-nucleus data shown here were not part of that paper -- the same distinction the Cell Browser desc.conf makes. Citing it bare would credit a bulk RNA-seq and WGS study for 10x Multiome data. References are alphabetical, so hg38 is now 10 and mm10 still 7. copySingleCellSignalsPeaksFiles.py: colors_json() reads the palette in a with block, and a malformed R,G,B row now raises the SystemExit the rest of the file uses, naming the file, line number and offending field, instead of a bare ValueError or TypeError from int() or the %02X format. Output is unchanged. makeDoc: the mm10 cell-class note now records that the bare "Progenitor" row in celltype-class.tsv is live rather than leftover -- build_stanzas' class_key() collapses plurals, so BrainVar's "Progenitors" looks up under the singular key and takes its class and color from that one row. Retiring or qualifying the row would grey out that track. A non-neural label needs a specific cell type added instead, which is how "Nephron progenitors" comes out Stromal. Also fixed a contradiction there: the file claimed celltype-class.tsv is built by build_celltype_crosswalks.py a hundred lines above the note saying, correctly, that it is hand-curated and not generated. The submitters confirmed BrainVar is 100 bp tiles, so the 1 kb in their methods text is an error and the page is right; recorded in the hg38 makeDoc. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/makeDb/doc/hg38/strchive.txt
  • lines changed 32, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
• src/hg/makeDb/doc/hs1/lrSv.txt
  • lines changed 7, context: html, text, full: html, text
    b687dd9018670941ce30f8a4582d6597c5a974d8 Tue Sep 1 14:57:43 2026 -0700
    lrSv1kLin: fix 2bp insertion span, drop dead numConsolidated field, refresh lrSvAll merge. refs #38099 The Lin 1218 VCFs set INFO/END = POS+1 on insertions, and the converter took chromEnd from END, so every insertion was drawn 2bp wide with svLen 2 instead of the 1bp anchor base. That contradicted the track's own description page and the coordinate convention in the makeDoc, and it kept 107,980 Lin insertions from merging in lrSvAll. Insertions now clamp chromEnd to the anchor; deletions are unchanged and still verify span == |SVLEN| against the source VCFs. Dropped numConsolidated from the converter and the .as: the NumConsolidated INFO key is declared in the VCF header but never appears on a data line, so the column was 0 on all 1.2M rows and added a meaningless line to every detail page. Rebuilt lin1218 on hg38 and hs1 (item counts and variant names unchanged) and re-ran the merge: lrSvAll 2,963,093 -> 2,855,267 rows as the duplicate insertion rows collapse. Bumped seven filter.svLen/insLen maxima in lrSv.ra that were short of the data after the August deletion narrowing, three of them only visible on hs1. lrSvAll.html said the 1000 Genomes linear set was not in the merge, which is no longer true, and gave no warning that sourceCount double-counts because Lin1218 already absorbs HPRC, HGSVC3 and both 1KG ONT callsets. Corrected the merge key description and refreshed ten stale cells in the lrSv.html summary table.
• src/hg/makeDb/doc/invertebrateAsmHub/invertebrate.orderList.tsv
  • lines changed 4, context: html, text, full: html, text
    6e9cae86505025606c69d9f06ca08a1426f6eed9 Mon Aug 31 16:52:26 2026 -0700
    VGP update refs @29545
• src/hg/makeDb/doc/mammalsAsmHub/mammals.orderList.tsv
  • lines changed 54, context: html, text, full: html, text
    6e9cae86505025606c69d9f06ca08a1426f6eed9 Mon Aug 31 16:52:26 2026 -0700
    VGP update refs @29545
• src/hg/makeDb/doc/mm10/singleCellSignalsPeaks.txt
  • lines changed 14, context: html, text, full: html, text
    3522a9acc8256a35230b07f2e7ab3fc08c827f33 Sat Sep 5 17:19:20 2026 -0700
    singleCellSignalsPeaks: correct the hg38 description page counts and cite BrainVar, refs #38219 From the v503 code review. Commit 3aac3982aa2 dropped 5 hg38 subtracks and updated the makeDoc, but the dataset list on the description page was missed, so the page disagreed with both the .ra and the facet menu: - Risk Loci in Alzheimer's and Parkinson's: 1 peak subtrack, not 2 (neuro-degen-atac/peaks.bb was the one dropped) - BrainVar: 9 signal subtracks, not 13, and the text still described tracks "for all nuclei together" -- the 4 combined-stage tracks are exactly the ones dropped, so every remaining BrainVar track carries a life stage Counts re-derived from the .ra by type (bigWig = signal, else peak): the other 7 hg38 datasets and all 9 mm10 datasets were already right, 929 and 587 total. BrainVar was also the only hg38 dataset with no citation, while its methods text had grown quite specific. It now cites Werling et al. 2020, which described the cohort, with the caveat that the single-nucleus data shown here were not part of that paper -- the same distinction the Cell Browser desc.conf makes. Citing it bare would credit a bulk RNA-seq and WGS study for 10x Multiome data. References are alphabetical, so hg38 is now 10 and mm10 still 7. copySingleCellSignalsPeaksFiles.py: colors_json() reads the palette in a with block, and a malformed R,G,B row now raises the SystemExit the rest of the file uses, naming the file, line number and offending field, instead of a bare ValueError or TypeError from int() or the %02X format. Output is unchanged. makeDoc: the mm10 cell-class note now records that the bare "Progenitor" row in celltype-class.tsv is live rather than leftover -- build_stanzas' class_key() collapses plurals, so BrainVar's "Progenitors" looks up under the singular key and takes its class and color from that one row. Retiring or qualifying the row would grey out that track. A non-neural label needs a specific cell type added instead, which is how "Nephron progenitors" comes out Stromal. Also fixed a contradiction there: the file claimed celltype-class.tsv is built by build_celltype_crosswalks.py a hundred lines above the note saying, correctly, that it is hand-curated and not generated. The submitters confirmed BrainVar is 100 bp tiles, so the 1 kb in their methods text is an error and the page is right; recorded in the hg38 makeDoc. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/makeDb/doc/plantsAsmHub/plants.orderList.tsv
  • lines changed 5, context: html, text, full: html, text
    6e9cae86505025606c69d9f06ca08a1426f6eed9 Mon Aug 31 16:52:26 2026 -0700
    VGP update refs @29545
• src/hg/makeDb/doc/primatesAsmHub/primates.orderList.tsv
  • lines changed 4, context: html, text, full: html, text
    6e9cae86505025606c69d9f06ca08a1426f6eed9 Mon Aug 31 16:52:26 2026 -0700
    VGP update refs @29545
• src/hg/makeDb/doc/vertebrateAsmHub/vertebrate.orderList.tsv
  • lines changed 79, context: html, text, full: html, text
    6e9cae86505025606c69d9f06ca08a1426f6eed9 Mon Aug 31 16:52:26 2026 -0700
    VGP update refs @29545
• src/hg/makeDb/outside/cgapSage/cgapSageFind/uninteresting.c
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/makeDb/outside/hgGtex/hgGtex.c
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/makeDb/scripts/danioCode/danioCodeDownload.sh
  • lines changed 38, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
• src/hg/makeDb/scripts/danioCode/danioCodeHubToRa.py
  • lines changed 425, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
• src/hg/makeDb/scripts/hprc2annot/bigGenePred.as
  • lines changed 25, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/scripts/hprc2annot/hprc2annotBuild.sh
  • lines changed 108, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/scripts/hprc2annot/hprc2annotBuildOne.sh
  • lines changed 247, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/scripts/hprc2annot/hprc2annotFillCdsPhase.py
  • lines changed 62, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/scripts/hprc2annot/hprc2annotFixBed.sh
  • lines changed 47, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/scripts/hprc2annot/hprc2annotMakeTrackDb.py
  • lines changed 114, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/scripts/hprc2annot/pclai.as
  • lines changed 16, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/scripts/hprc2annot/segdups.as
  • lines changed 19, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/scripts/imprinting/akbariIdmr.as
  • lines changed 24, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/scripts/imprinting/akbariIdmrToBed.py
  • lines changed 143, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/scripts/imprinting/geneimprint.as
  • lines changed 22, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/scripts/imprinting/geneimprintToBed.py
  • lines changed 329, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/scripts/imprinting/omimImprint.as
  • lines changed 19, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/scripts/imprinting/omimImprintToBed.py
  • lines changed 201, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/scripts/lrSv/lrSv1kLin1218.as
  • lines changed 1, context: html, text, full: html, text
    b687dd9018670941ce30f8a4582d6597c5a974d8 Tue Sep 1 14:57:43 2026 -0700
    lrSv1kLin: fix 2bp insertion span, drop dead numConsolidated field, refresh lrSvAll merge. refs #38099 The Lin 1218 VCFs set INFO/END = POS+1 on insertions, and the converter took chromEnd from END, so every insertion was drawn 2bp wide with svLen 2 instead of the 1bp anchor base. That contradicted the track's own description page and the coordinate convention in the makeDoc, and it kept 107,980 Lin insertions from merging in lrSvAll. Insertions now clamp chromEnd to the anchor; deletions are unchanged and still verify span == |SVLEN| against the source VCFs. Dropped numConsolidated from the converter and the .as: the NumConsolidated INFO key is declared in the VCF header but never appears on a data line, so the column was 0 on all 1.2M rows and added a meaningless line to every detail page. Rebuilt lin1218 on hg38 and hs1 (item counts and variant names unchanged) and re-ran the merge: lrSvAll 2,963,093 -> 2,855,267 rows as the duplicate insertion rows collapse. Bumped seven filter.svLen/insLen maxima in lrSv.ra that were short of the data after the August deletion narrowing, three of them only visible on hs1. lrSvAll.html said the 1000 Genomes linear set was not in the merge, which is no longer true, and gave no warning that sourceCount double-counts because Lin1218 already absorbs HPRC, HGSVC3 and both 1KG ONT callsets. Corrected the merge key description and refreshed ten stale cells in the lrSv.html summary table.
• src/hg/makeDb/scripts/lrSv/lrSv1kLin1218VcfToBed.py
  • lines changed 15, context: html, text, full: html, text
    b687dd9018670941ce30f8a4582d6597c5a974d8 Tue Sep 1 14:57:43 2026 -0700
    lrSv1kLin: fix 2bp insertion span, drop dead numConsolidated field, refresh lrSvAll merge. refs #38099 The Lin 1218 VCFs set INFO/END = POS+1 on insertions, and the converter took chromEnd from END, so every insertion was drawn 2bp wide with svLen 2 instead of the 1bp anchor base. That contradicted the track's own description page and the coordinate convention in the makeDoc, and it kept 107,980 Lin insertions from merging in lrSvAll. Insertions now clamp chromEnd to the anchor; deletions are unchanged and still verify span == |SVLEN| against the source VCFs. Dropped numConsolidated from the converter and the .as: the NumConsolidated INFO key is declared in the VCF header but never appears on a data line, so the column was 0 on all 1.2M rows and added a meaningless line to every detail page. Rebuilt lin1218 on hg38 and hs1 (item counts and variant names unchanged) and re-ran the merge: lrSvAll 2,963,093 -> 2,855,267 rows as the duplicate insertion rows collapse. Bumped seven filter.svLen/insLen maxima in lrSv.ra that were short of the data after the August deletion narrowing, three of them only visible on hs1. lrSvAll.html said the 1000 Genomes linear set was not in the merge, which is no longer true, and gave no warning that sourceCount double-counts because Lin1218 already absorbs HPRC, HGSVC3 and both 1KG ONT callsets. Corrected the merge key description and refreshed ten stale cells in the lrSv.html summary table.
• src/hg/makeDb/scripts/singleCellSignalsPeaks/copySingleCellSignalsPeaksFiles.py
  • lines changed 7, context: html, text, full: html, text
    3522a9acc8256a35230b07f2e7ab3fc08c827f33 Sat Sep 5 17:19:20 2026 -0700
    singleCellSignalsPeaks: correct the hg38 description page counts and cite BrainVar, refs #38219 From the v503 code review. Commit 3aac3982aa2 dropped 5 hg38 subtracks and updated the makeDoc, but the dataset list on the description page was missed, so the page disagreed with both the .ra and the facet menu: - Risk Loci in Alzheimer's and Parkinson's: 1 peak subtrack, not 2 (neuro-degen-atac/peaks.bb was the one dropped) - BrainVar: 9 signal subtracks, not 13, and the text still described tracks "for all nuclei together" -- the 4 combined-stage tracks are exactly the ones dropped, so every remaining BrainVar track carries a life stage Counts re-derived from the .ra by type (bigWig = signal, else peak): the other 7 hg38 datasets and all 9 mm10 datasets were already right, 929 and 587 total. BrainVar was also the only hg38 dataset with no citation, while its methods text had grown quite specific. It now cites Werling et al. 2020, which described the cohort, with the caveat that the single-nucleus data shown here were not part of that paper -- the same distinction the Cell Browser desc.conf makes. Citing it bare would credit a bulk RNA-seq and WGS study for 10x Multiome data. References are alphabetical, so hg38 is now 10 and mm10 still 7. copySingleCellSignalsPeaksFiles.py: colors_json() reads the palette in a with block, and a malformed R,G,B row now raises the SystemExit the rest of the file uses, naming the file, line number and offending field, instead of a bare ValueError or TypeError from int() or the %02X format. Output is unchanged. makeDoc: the mm10 cell-class note now records that the bare "Progenitor" row in celltype-class.tsv is live rather than leftover -- build_stanzas' class_key() collapses plurals, so BrainVar's "Progenitors" looks up under the singular key and takes its class and color from that one row. Retiring or qualifying the row would grey out that track. A non-neural label needs a specific cell type added instead, which is how "Nephron progenitors" comes out Stromal. Also fixed a contradiction there: the file claimed celltype-class.tsv is built by build_celltype_crosswalks.py a hundred lines above the note saying, correctly, that it is hand-curated and not generated. The submitters confirmed BrainVar is 100 bp tiles, so the 1 kb in their methods text is an error and the page is right; recorded in the hg38 makeDoc. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/makeDb/trackDb/betaGenArk.txt
  • lines changed 1, context: html, text, full: html, text
    bd5952e27a963cccacf526649b5be02b9049e8e0 Tue Sep 1 12:43:00 2026 -0700
    testing the RCPediaVGP_v1 contrib tracks refs #38223
• src/hg/makeDb/trackDb/buildTrix
  • lines changed 1, context: html, text, full: html, text
    cec595b267a0dae66dab498417df29b3bb8b60d0 Wed Sep 2 10:21:29 2026 -0700
    trackDb: rsync to the destination string as it stands, refs #35489 #38211 Code review noted that "${dest%%:*}:${dest#*:}/" is hard to read. It is also a no-op. It takes the machine:path string apart at the first colon and puts it back together with a colon, which is the same string as "$dest/".
• src/hg/makeDb/trackDb/checkMetaDb
  • lines changed 3, context: html, text, full: html, text
    65bfad661897f4f834dd07b89c507bae0a19bdde Tue Sep 1 00:41:08 2026 -0700
    trackDb: clear MAKEFLAGS for the metaDb sub-make, refs #35489 checkMetaDb runs a plain make in the per-database metaDb directory. The recipe that calls it is not a recursive make, so under -j the parent has already closed the jobserver pipe and the sub-make prints make[2]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule. makeStrictBeta.csh greps the log for "error|warn" and exits 1 on any hit, so those warnings would stop the build even though the trackDb work succeeded. The beta path has three of these sub-makes (hg18, hg19, mm9), so three hits. Clearing MAKEFLAGS for that one command silences it. The sub-make only needs DB and TABLE, and both are passed on the command line. Tested with the real script in the same three-level shape the build uses: before, serial gives zero hits on that grep and -O -j 8 gives three; after, both give zero, and the metaDb recipe still runs in all four cases.
• src/hg/makeDb/trackDb/contrib/RCPediaVGP_v1/RCPediaVGP_v1.html
  • lines changed 67, context: html, text, full: html, text
    9f4f893d208b9f082b1e64dc4932e8df8afbcb5a Wed Sep 2 08:53:55 2026 -0700
    html and trackDb for RCPediaVGP_v1 contrib tracks refs #38224
• src/hg/makeDb/trackDb/contrib/RCPediaVGP_v1/RCPediaVGP_v1.trackDb.txt
  • lines changed 8, context: html, text, full: html, text
    9f4f893d208b9f082b1e64dc4932e8df8afbcb5a Wed Sep 2 08:53:55 2026 -0700
    html and trackDb for RCPediaVGP_v1 contrib tracks refs #38224
• src/hg/makeDb/trackDb/contrib/hprc2annot/catGenes.html
  • lines changed 78, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/trackDb/contrib/hprc2annot/censat.html
  • lines changed 71, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/trackDb/contrib/hprc2annot/censatCentromeres.html
  • lines changed 63, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt
  • lines changed 112, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/trackDb/contrib/hprc2annot/liftoffGenes.html
  • lines changed 83, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/trackDb/contrib/hprc2annot/methylation.html
  • lines changed 62, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/trackDb/contrib/hprc2annot/pclai.html
  • lines changed 85, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/trackDb/contrib/hprc2annot/segdups.html
  • lines changed 105, context: html, text, full: html, text
    e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
    hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415
• src/hg/makeDb/trackDb/human/alphaGenome.html
  • lines changed 77, context: html, text, full: html, text
    8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
    AlphaGenome Variant Impact (AVI) score track for hg38 New track under the Deleteriousness Predictions container showing Google DeepMind's AlphaGenome Variant Impact score for every possible single-base substitution, as four bigWigs, one per alternate allele. alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per alternate allele. It copies the PHRED column through as text so no precision is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a long contiguous run. The input has no row where alt == ref, so each per-allele file is missing roughly every fourth position; without the fill the wig fragments into blocks of about 3.5 values and the bigWig doubles in size. All 8,812,917,339 input lines were used, none skipped. The data cannot be redistributed, so the stanzas set tableBrowser off and the description page sends users to the AlphaGenome Atlas. Alpha only for now. No reference yet, the score is unpublished. refs #38261
• src/hg/makeDb/trackDb/human/alphaGenome.ra
  • lines changed 72, context: html, text, full: html, text
    8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
    AlphaGenome Variant Impact (AVI) score track for hg38 New track under the Deleteriousness Predictions container showing Google DeepMind's AlphaGenome Variant Impact score for every possible single-base substitution, as four bigWigs, one per alternate allele. alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per alternate allele. It copies the PHRED column through as text so no precision is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a long contiguous run. The input has no row where alt == ref, so each per-allele file is missing roughly every fourth position; without the fill the wig fragments into blocks of about 3.5 values and the bigWig doubles in size. All 8,812,917,339 input lines were used, none skipped. The data cannot be redistributed, so the stanzas set tableBrowser off and the description page sends users to the AlphaGenome Atlas. Alpha only for now. No reference yet, the score is unpublished. refs #38261
• src/hg/makeDb/trackDb/human/hg38/akbariIdmr.html
  • lines changed 228, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/trackDb/human/hg38/encode4LongRnaTranscripts.html
  • lines changed 6, context: html, text, full: html, text
    3f640d70bab0bc124024a977d8836756318c1852 Fri Sep 4 14:39:51 2026 -0700
    Fixing malformed HTML in three track description pages caught by hgTablesTest: encode4LongRnaTranscripts.html had a stray html/body wrapper, and knownGeneVM33.html/knownGeneVM36.html each had an extra closing </ul>. No RM.
• src/hg/makeDb/trackDb/human/hg38/geneimprint.html
  • lines changed 148, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/trackDb/human/hg38/imprinting.html
  • lines changed 36, context: html, text, full: html, text
    684ad73a17eebb717fba52f0aa25f3d9d7adb2de Thu Sep 3 12:03:37 2026 -0700
    New Imprinting track collection on hg38, first subtrack is MethBase2 ASM Adds the "Imprinting" superTrack to the Regulation group and its first subtrack, the allele-specific methylation summary from the MethBase2 hub (http://smithlab.usc.edu/trackdata/methylation/hub.txt). The bigWig gives, for each of 29,401,795 CpG sites, the fraction of MethBase2 human methylomes in which the site fell inside an allelically methylated region called by dnmtools amrfinder. Taken from upstream unchanged. Alpha only for now, refs #37599
  • lines changed 48, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/trackDb/human/hg38/imprinting.ra
  • lines changed 25, context: html, text, full: html, text
    684ad73a17eebb717fba52f0aa25f3d9d7adb2de Thu Sep 3 12:03:37 2026 -0700
    New Imprinting track collection on hg38, first subtrack is MethBase2 ASM Adds the "Imprinting" superTrack to the Regulation group and its first subtrack, the allele-specific methylation summary from the MethBase2 hub (http://smithlab.usc.edu/trackdata/methylation/hub.txt). The bigWig gives, for each of 29,401,795 CpG sites, the fraction of MethBase2 human methylomes in which the site fell inside an allelically methylated region called by dnmtools amrfinder. Taken from upstream unchanged. Alpha only for now, refs #37599
  • lines changed 67, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/trackDb/human/hg38/methBaseAsm.html
  • lines changed 150, context: html, text, full: html, text
    684ad73a17eebb717fba52f0aa25f3d9d7adb2de Thu Sep 3 12:03:37 2026 -0700
    New Imprinting track collection on hg38, first subtrack is MethBase2 ASM Adds the "Imprinting" superTrack to the Regulation group and its first subtrack, the allele-specific methylation summary from the MethBase2 hub (http://smithlab.usc.edu/trackdata/methylation/hub.txt). The bigWig gives, for each of 29,401,795 CpG sites, the fraction of MethBase2 human methylomes in which the site fell inside an allelically methylated region called by dnmtools amrfinder. Taken from upstream unchanged. Alpha only for now, refs #37599
  • lines changed 15, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/trackDb/human/hg38/omimImprint.html
  • lines changed 148, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
• src/hg/makeDb/trackDb/human/hg38/singleCellSignalsPeaks.html
  • lines changed 18, context: html, text, full: html, text
    3522a9acc8256a35230b07f2e7ab3fc08c827f33 Sat Sep 5 17:19:20 2026 -0700
    singleCellSignalsPeaks: correct the hg38 description page counts and cite BrainVar, refs #38219 From the v503 code review. Commit 3aac3982aa2 dropped 5 hg38 subtracks and updated the makeDoc, but the dataset list on the description page was missed, so the page disagreed with both the .ra and the facet menu: - Risk Loci in Alzheimer's and Parkinson's: 1 peak subtrack, not 2 (neuro-degen-atac/peaks.bb was the one dropped) - BrainVar: 9 signal subtracks, not 13, and the text still described tracks "for all nuclei together" -- the 4 combined-stage tracks are exactly the ones dropped, so every remaining BrainVar track carries a life stage Counts re-derived from the .ra by type (bigWig = signal, else peak): the other 7 hg38 datasets and all 9 mm10 datasets were already right, 929 and 587 total. BrainVar was also the only hg38 dataset with no citation, while its methods text had grown quite specific. It now cites Werling et al. 2020, which described the cohort, with the caveat that the single-nucleus data shown here were not part of that paper -- the same distinction the Cell Browser desc.conf makes. Citing it bare would credit a bulk RNA-seq and WGS study for 10x Multiome data. References are alphabetical, so hg38 is now 10 and mm10 still 7. copySingleCellSignalsPeaksFiles.py: colors_json() reads the palette in a with block, and a malformed R,G,B row now raises the SystemExit the rest of the file uses, naming the file, line number and offending field, instead of a bare ValueError or TypeError from int() or the %02X format. Output is unchanged. makeDoc: the mm10 cell-class note now records that the bare "Progenitor" row in celltype-class.tsv is live rather than leftover -- build_stanzas' class_key() collapses plurals, so BrainVar's "Progenitors" looks up under the singular key and takes its class and color from that one row. Retiring or qualifying the row would grey out that track. A non-neural label needs a specific cell type added instead, which is how "Nephron progenitors" comes out Stromal. Also fixed a contradiction there: the file claimed celltype-class.tsv is built by build_celltype_crosswalks.py a hundred lines above the note saying, correctly, that it is hand-curated and not generated. The submitters confirmed BrainVar is 100 bp tiles, so the 1 kb in their methods text is an error and the page is right; recorded in the hg38 makeDoc. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/makeDb/trackDb/human/hg38/trackDb.ra
  • lines changed 2, context: html, text, full: html, text
    684ad73a17eebb717fba52f0aa25f3d9d7adb2de Thu Sep 3 12:03:37 2026 -0700
    New Imprinting track collection on hg38, first subtrack is MethBase2 ASM Adds the "Imprinting" superTrack to the Regulation group and its first subtrack, the allele-specific methylation summary from the MethBase2 hub (http://smithlab.usc.edu/trackdata/methylation/hub.txt). The bigWig gives, for each of 29,401,795 CpG sites, the fraction of MethBase2 human methylomes in which the site fell inside an allelically methylated region called by dnmtools amrfinder. Taken from upstream unchanged. Alpha only for now, refs #37599
  • lines changed 4, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
• src/hg/makeDb/trackDb/human/lrSv.html
  • lines changed 10, context: html, text, full: html, text
    b687dd9018670941ce30f8a4582d6597c5a974d8 Tue Sep 1 14:57:43 2026 -0700
    lrSv1kLin: fix 2bp insertion span, drop dead numConsolidated field, refresh lrSvAll merge. refs #38099 The Lin 1218 VCFs set INFO/END = POS+1 on insertions, and the converter took chromEnd from END, so every insertion was drawn 2bp wide with svLen 2 instead of the 1bp anchor base. That contradicted the track's own description page and the coordinate convention in the makeDoc, and it kept 107,980 Lin insertions from merging in lrSvAll. Insertions now clamp chromEnd to the anchor; deletions are unchanged and still verify span == |SVLEN| against the source VCFs. Dropped numConsolidated from the converter and the .as: the NumConsolidated INFO key is declared in the VCF header but never appears on a data line, so the column was 0 on all 1.2M rows and added a meaningless line to every detail page. Rebuilt lin1218 on hg38 and hs1 (item counts and variant names unchanged) and re-ran the merge: lrSvAll 2,963,093 -> 2,855,267 rows as the duplicate insertion rows collapse. Bumped seven filter.svLen/insLen maxima in lrSv.ra that were short of the data after the August deletion narrowing, three of them only visible on hs1. lrSvAll.html said the 1000 Genomes linear set was not in the merge, which is no longer true, and gave no warning that sourceCount double-counts because Lin1218 already absorbs HPRC, HGSVC3 and both 1KG ONT callsets. Corrected the merge key description and refreshed ten stale cells in the lrSv.html summary table.
• src/hg/makeDb/trackDb/human/lrSv.ra
  • lines changed 7, context: html, text, full: html, text
    b687dd9018670941ce30f8a4582d6597c5a974d8 Tue Sep 1 14:57:43 2026 -0700
    lrSv1kLin: fix 2bp insertion span, drop dead numConsolidated field, refresh lrSvAll merge. refs #38099 The Lin 1218 VCFs set INFO/END = POS+1 on insertions, and the converter took chromEnd from END, so every insertion was drawn 2bp wide with svLen 2 instead of the 1bp anchor base. That contradicted the track's own description page and the coordinate convention in the makeDoc, and it kept 107,980 Lin insertions from merging in lrSvAll. Insertions now clamp chromEnd to the anchor; deletions are unchanged and still verify span == |SVLEN| against the source VCFs. Dropped numConsolidated from the converter and the .as: the NumConsolidated INFO key is declared in the VCF header but never appears on a data line, so the column was 0 on all 1.2M rows and added a meaningless line to every detail page. Rebuilt lin1218 on hg38 and hs1 (item counts and variant names unchanged) and re-ran the merge: lrSvAll 2,963,093 -> 2,855,267 rows as the duplicate insertion rows collapse. Bumped seven filter.svLen/insLen maxima in lrSv.ra that were short of the data after the August deletion narrowing, three of them only visible on hs1. lrSvAll.html said the 1000 Genomes linear set was not in the merge, which is no longer true, and gave no warning that sourceCount double-counts because Lin1218 already absorbs HPRC, HGSVC3 and both 1KG ONT callsets. Corrected the merge key description and refreshed ten stale cells in the lrSv.html summary table.
• src/hg/makeDb/trackDb/human/lrSvAll.html
  • lines changed 19, context: html, text, full: html, text
    b687dd9018670941ce30f8a4582d6597c5a974d8 Tue Sep 1 14:57:43 2026 -0700
    lrSv1kLin: fix 2bp insertion span, drop dead numConsolidated field, refresh lrSvAll merge. refs #38099 The Lin 1218 VCFs set INFO/END = POS+1 on insertions, and the converter took chromEnd from END, so every insertion was drawn 2bp wide with svLen 2 instead of the 1bp anchor base. That contradicted the track's own description page and the coordinate convention in the makeDoc, and it kept 107,980 Lin insertions from merging in lrSvAll. Insertions now clamp chromEnd to the anchor; deletions are unchanged and still verify span == |SVLEN| against the source VCFs. Dropped numConsolidated from the converter and the .as: the NumConsolidated INFO key is declared in the VCF header but never appears on a data line, so the column was 0 on all 1.2M rows and added a meaningless line to every detail page. Rebuilt lin1218 on hg38 and hs1 (item counts and variant names unchanged) and re-ran the merge: lrSvAll 2,963,093 -> 2,855,267 rows as the duplicate insertion rows collapse. Bumped seven filter.svLen/insLen maxima in lrSv.ra that were short of the data after the August deletion narrowing, three of them only visible on hs1. lrSvAll.html said the 1000 Genomes linear set was not in the merge, which is no longer true, and gave no warning that sourceCount double-counts because Lin1218 already absorbs HPRC, HGSVC3 and both 1KG ONT callsets. Corrected the merge key description and refreshed ten stale cells in the lrSv.html summary table.
• src/hg/makeDb/trackDb/human/predictionScoresSuper.html
  • lines changed 25, context: html, text, full: html, text
    8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
    AlphaGenome Variant Impact (AVI) score track for hg38 New track under the Deleteriousness Predictions container showing Google DeepMind's AlphaGenome Variant Impact score for every possible single-base substitution, as four bigWigs, one per alternate allele. alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per alternate allele. It copies the PHRED column through as text so no precision is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a long contiguous run. The input has no row where alt == ref, so each per-allele file is missing roughly every fourth position; without the fill the wig fragments into blocks of about 3.5 values and the bigWig doubles in size. All 8,812,917,339 input lines were used, none skipped. The data cannot be redistributed, so the stanzas set tableBrowser off and the description page sends users to the AlphaGenome Atlas. Alpha only for now. No reference yet, the score is unpublished. refs #38261
• src/hg/makeDb/trackDb/human/predictionScoresSuper.ra
  • lines changed 1, context: html, text, full: html, text
    8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
    AlphaGenome Variant Impact (AVI) score track for hg38 New track under the Deleteriousness Predictions container showing Google DeepMind's AlphaGenome Variant Impact score for every possible single-base substitution, as four bigWigs, one per alternate allele. alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per alternate allele. It copies the PHRED column through as text so no precision is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a long contiguous run. The input has no row where alt == ref, so each per-allele file is missing roughly every fourth position; without the fill the wig fragments into blocks of about 3.5 values and the bigWig doubles in size. All 8,812,917,339 input lines were used, none skipped. The data cannot be redistributed, so the stanzas set tableBrowser off and the description page sends users to the AlphaGenome Atlas. Alpha only for now. No reference yet, the score is unpublished. refs #38261
• src/hg/makeDb/trackDb/human/strVarNew.html
  • lines changed 56, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
• src/hg/makeDb/trackDb/human/strVarNew.ra
  • lines changed 125, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
• src/hg/makeDb/trackDb/human/strchiveNew.html
  • lines changed 128, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
• src/hg/makeDb/trackDb/human/trackDb.ra
  • lines changed 10, context: html, text, full: html, text
    115c6d24a46472bd44400f1fe9b38541364138ad Fri Sep 4 17:53:17 2026 -0700
    Add a search rule so a deprecated versioned NP_ accession resolves to RefSeq Historical. A superseded protein accession was answered by hgFixed.refLink, which stores accessions without versions, so NP_000005.2 and NP_000005.3 returned the same gene. The new rule looks the accession up in ncbiRefSeqLinkHistorical and jumps to the deprecated transcript instead. It excludes accessions that are still current so their searches are untouched, and is semiShortCircuit so it replaces the version-blind answer rather than adding a second result. refs #38248
  • lines changed 7, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
• src/hg/makeDb/trackDb/makefile
  • lines changed 28, context: html, text, full: html, text
    6d3ac828268e706ba9dbc4baa01d444839c9aec2 Wed Sep 2 13:05:49 2026 -0700
    trackDb: write the gbdb file list to $TMPDIR, not the scanned directory The parallel beta make (0b36c1276f7) aborted the v503 final build and both attempts at the #38231 build patch: /cluster/bin/x86_64/tdbQuery -check -release=beta -strict \ 'select count(*) from mm10' -root=.../trackDb No such file or directory stat failed in listDirX: .../trackDb/musFur1.gbdbList.txt make[1]: *** [makefile:281: mm10_beta] Error 255 Note the two different databases -- that is the signature. The %_beta recipe writes $*.gbdbList.txt into the trackDb directory, uses it, then removes it. loadTracks runs tdbQuery -check with -root set to that same directory, and listDirX stats every entry it finds there, so at -j 8 one database unlinks its list file between another database's readdir and stat. Naming each file after its database, which is what the original commit relied on, prevents two recipes from writing the same name but not one recipe from seeing another's file. Put the list in $TMPDIR instead, where nothing scans it, in all three recipes that build one (beta, publicTest, public). Fixing listDirX to tolerate ENOENT would not have helped: the recipe runs the installed /cluster/bin/x86_64/tdbQuery, not the binary the build compiles. This also keeps stray list files out of the working directory, where they have occasionally been picked up by a commit. The header comment claimed per-database naming was sufficient for -j; correct it and say where scratch files belong. Verified by expansion with make -n; a real parallel run has to wait for the next beta make. refs #35489 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/makeDb/trackDb/mouse/mm10/mouseDevTimecourse.ra
  • lines changed 1, context: html, text, full: html, text
    afdd5daeb07ccef3df67165f6d10e7abd4db21ff Thu Sep 3 14:12:39 2026 -0700
    Release the ENCODE Expression tracks for mm10 and mm39. refs #37001 Drop the alpha,beta release tags from all three include lines, so the container and the bigWig signal composite now ship to beta and public. Verified with tdbQuery -strict at all three release levels: no errors, and the tracks are visible at each one (320 on mm10, 3 on mm39). Point the pennantIcon at the news archive post and give it the release date: newsarch.html#090426 "Released Sep. 4, 2026", matching the convention used by the other released tracks. The liftOver pennants on the mm39 subtracks are unchanged. Add the Sep. 4 news announcement to newsarch.html and a matching entry to indexNews.html, dropping the oldest front-page item to stay at the six-item limit. The announcement links to the session at genome.ucsc.edu/s/Lou/ENCODEexpression and includes a screenshot of Gap43, whose expression rises through development and is highest in the four neural tissues. Add the screenshot image, which was previously untracked.
• src/hg/makeDb/trackDb/mouse/mm10/trackDb.ra
  • lines changed 2, context: html, text, full: html, text
    afdd5daeb07ccef3df67165f6d10e7abd4db21ff Thu Sep 3 14:12:39 2026 -0700
    Release the ENCODE Expression tracks for mm10 and mm39. refs #37001 Drop the alpha,beta release tags from all three include lines, so the container and the bigWig signal composite now ship to beta and public. Verified with tdbQuery -strict at all three release levels: no errors, and the tracks are visible at each one (320 on mm10, 3 on mm39). Point the pennantIcon at the news archive post and give it the release date: newsarch.html#090426 "Released Sep. 4, 2026", matching the convention used by the other released tracks. The liftOver pennants on the mm39 subtracks are unchanged. Add the Sep. 4 news announcement to newsarch.html and a matching entry to indexNews.html, dropping the oldest front-page item to stay at the six-item limit. The announcement links to the session at genome.ucsc.edu/s/Lou/ENCODEexpression and includes a screenshot of Gap43, whose expression rises through development and is highest in the four neural tissues. Add the screenshot image, which was previously untracked.
• src/hg/makeDb/trackDb/mouse/mm39/knownGeneVM33.html
  • lines changed 1, context: html, text, full: html, text
    3f640d70bab0bc124024a977d8836756318c1852 Fri Sep 4 14:39:51 2026 -0700
    Fixing malformed HTML in three track description pages caught by hgTablesTest: encode4LongRnaTranscripts.html had a stray html/body wrapper, and knownGeneVM33.html/knownGeneVM36.html each had an extra closing </ul>. No RM.
• src/hg/makeDb/trackDb/mouse/mm39/knownGeneVM36.html
  • lines changed 1, context: html, text, full: html, text
    3f640d70bab0bc124024a977d8836756318c1852 Fri Sep 4 14:39:51 2026 -0700
    Fixing malformed HTML in three track description pages caught by hgTablesTest: encode4LongRnaTranscripts.html had a stray html/body wrapper, and knownGeneVM33.html/knownGeneVM36.html each had an extra closing </ul>. No RM.
• src/hg/makeDb/trackDb/mouse/mm39/mouseDevTimecourse.ra
  • lines changed 1, context: html, text, full: html, text
    afdd5daeb07ccef3df67165f6d10e7abd4db21ff Thu Sep 3 14:12:39 2026 -0700
    Release the ENCODE Expression tracks for mm10 and mm39. refs #37001 Drop the alpha,beta release tags from all three include lines, so the container and the bigWig signal composite now ship to beta and public. Verified with tdbQuery -strict at all three release levels: no errors, and the tracks are visible at each one (320 on mm10, 3 on mm39). Point the pennantIcon at the news archive post and give it the release date: newsarch.html#090426 "Released Sep. 4, 2026", matching the convention used by the other released tracks. The liftOver pennants on the mm39 subtracks are unchanged. Add the Sep. 4 news announcement to newsarch.html and a matching entry to indexNews.html, dropping the oldest front-page item to stay at the six-item limit. The announcement links to the session at genome.ucsc.edu/s/Lou/ENCODEexpression and includes a screenshot of Gap43, whose expression rises through development and is highest in the four neural tissues. Add the screenshot image, which was previously untracked.
• src/hg/makeDb/trackDb/mouse/mm39/trackDb.ra
  • lines changed 1, context: html, text, full: html, text
    afdd5daeb07ccef3df67165f6d10e7abd4db21ff Thu Sep 3 14:12:39 2026 -0700
    Release the ENCODE Expression tracks for mm10 and mm39. refs #37001 Drop the alpha,beta release tags from all three include lines, so the container and the bigWig signal composite now ship to beta and public. Verified with tdbQuery -strict at all three release levels: no errors, and the tracks are visible at each one (320 on mm10, 3 on mm39). Point the pennantIcon at the news archive post and give it the release date: newsarch.html#090426 "Released Sep. 4, 2026", matching the convention used by the other released tracks. The liftOver pennants on the mm39 subtracks are unchanged. Add the Sep. 4 news announcement to newsarch.html and a matching entry to indexNews.html, dropping the oldest front-page item to stay at the six-item limit. The announcement links to the session at genome.ucsc.edu/s/Lou/ENCODEexpression and includes a screenshot of Gap43, whose expression rises through development and is highest in the four neural tissues. Add the screenshot image, which was previously untracked.
• src/hg/makeDb/trackDb/relatedTracks.ra
  • lines changed 3, context: html, text, full: html, text
    22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
    Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM Geneimprint: the human catalog of imprinted and candidate imprinted genes from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band, so symbols are resolved against the HGNC track, with fallbacks for small RNA clusters and for symbols HGNC has retired. Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their published coordinates are 1-based inclusive, not bed, as their own PatMat reader shows, so the starts are shifted. OMIM: the 459 loci that OMIM curates as imprinted. That call is published only through GeneScout and is in no OMIM download file, so the input is a GeneScout export made by hand from a browser. OMIM phenotype entries, which are mapped disease regions up to 90 Mb rather than gene positions, are left out. Also unifies the color scheme across the collection, so vermillion always means the maternal copy, blue the paternal copy and gray no parent of origin, and relates the collection to the Human Methylation Atlas. refs #37599
  • lines changed 338, context: html, text, full: html, text
    d92252cc474cc9fc079cbad1b020446d97d50428 Fri Sep 4 10:04:48 2026 -0700
    relatedTracks.ra: 275 new track relationships from the #38016 audit Audits all six assemblies that appear in relatedTracks.ra (hg38, hg19, mm39, mm10, hs1, danRer11) for track pairs that a reader would want linked and that the file did not already connect. Adds 172 relationships, which come to 275 lines, taking the file from 139 relationship lines to 414. Version successions run one way, from the older track to the newer one, on the grounds that someone reading the current track does not need a pointer back to the retired one, while someone who lands on the retired track does need sending forward. Archives are the exception and stay reciprocal, since browsing an archive to reach a specific release is the point of it. Direction was checked against each track's longLabel rather than inferred from the table name, which mattered for the COVID GWAS pair: covidHgiGwas is release 3 and covidHgiGwasR4Pval is release 4, so the table name reads older than the track is. The variant impact predictors are capped at eight related tracks rather than five, because comparing one score against another is the main reason anyone opens them. That cap raise is what let spliceImpactSuper and its four children, the three constraintSuper children, mutScore and caddSuper1_7 get any related tracks at all; every one of them had none. Also relates three pairs one way where the older half was frozen years ago: Platinum Genomes to Genome In A Bottle, the Coriell cell line CNVs to ClinGen, and CCDS to MANE. refs #38016 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 12, context: html, text, full: html, text
    a98dfd4707e69c1617982e26ddf3310a8b7a3277 Fri Sep 4 10:11:06 2026 -0700
    relatedTracks.ra: relate OMIM to GenCC, Orphanet and G2P The gene-disease cluster added in the previous commit was built around GenCC as a hub, with every spoke sharing the reason "one of the sources GenCC aggregates". OMIM is not a GenCC submitter, so it never matched that phrasing and ended up connected only to GeneReviews, despite being the reference the other resources cite. The related-track cap was not the constraint; GenCC was at four of five and OMIM at one of five. Adds three reciprocal pairs on hg38 and hg19: OMIM with GenCC, with Orphanet and with G2P. GenCC now sits at its cap of five and OMIM at four. PanelApp is left out on purpose. Its panels do cite OMIM, but including it would put five gene-disease resources on OMIM's page mostly describing one another, and PanelApp is the most specialized of them. refs #38016 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 70, context: html, text, full: html, text
    da5860e01a546d10d1276880758dcfaa8e573467 Fri Sep 4 10:24:18 2026 -0700
    relatedTracks.ra: connect RepeatMasker, GWAS Catalog, HGNC and CRISPR Follow-up to the #38016 audit, which organized its work by writing one reason template per cluster of tracks. That approach only linked a track if it matched some cluster's phrasing, so tracks that are conceptually adjacent but described differently were skipped. Four such cases, all of which had no related tracks at all before this: RepeatMasker. The audit's repeat cluster was written around tandem repeats, so rmsk, which annotates interspersed repeats, fell outside the wording. It is one of the most widely used tracks in the browser and had nothing. Now relates to Interrupted Repeats and RepeatMasker Visualization, which are derived from it, to Simple Repeats as the tandem counterpart, and to WindowMasker + SDust as the library-free alternative. GWAS Catalog, SNPedia and Variants in Papers. The audit built a cluster for variant impact prediction and another for gene-disease curation, but none for variants reported in the literature, so all three were left isolated. GWAS Catalog also now relates to the GTEx cis-eQTLs, since looking for the gene behind an association signal is a common reason to open either one. HGNC. The gene symbol authority, unlinked from the gene tracks that use its nomenclature. Now relates to GENCODE, NCBI RefSeq and MANE. CRISPR. The 200 bp window track is superseded by the 10 kb one, computed by the same pipeline over a wider region, so it links forward to it and not back, the same rule the audit applies to other version successions. Counts: 32 relationships, 70 lines, taking the file to 487 relationship lines. refs #38016 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/makeDb/trackDb/zebrafish/danRer11/danioCode.html
  • lines changed 157, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 1, context: html, text, full: html, text
    650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
    DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/danioCode.ra
  • lines changed 10647, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dc3PseqComposite.html
  • lines changed 117, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 1, context: html, text, full: html, text
    650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
    DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
  • lines changed 5, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcCAGEseqComposite.html
  • lines changed 123, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 1, context: html, text, full: html, text
    650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
    DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
  • lines changed 5, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcChIPseqComposite.html
  • lines changed 130, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 1, context: html, text, full: html, text
    650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
    DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
  • lines changed 5, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcComp.html
  • lines changed 169, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 6, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcComp_cell_type.html
  • lines changed 124, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 1, context: html, text, full: html, text
    650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
    DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
  • lines changed 5, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcComparativeGenomics.html
  • lines changed 97, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 5, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcConsensus_promoters.html
  • lines changed 115, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 1, context: html, text, full: html, text
    650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
    DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
  • lines changed 5, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcCopes_and_dopes.html
  • lines changed 121, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 1, context: html, text, full: html, text
    650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
    DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
  • lines changed 5, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcEvalidation.html
  • lines changed 114, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 1, context: html, text, full: html, text
    650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
    DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
  • lines changed 5, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcHiC_Composite.html
  • lines changed 126, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 1, context: html, text, full: html, text
    650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
    DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
  • lines changed 5, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcRNAseqComposite.html
  • lines changed 120, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
  • lines changed 1, context: html, text, full: html, text
    650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
    DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
  • lines changed 7, context: html, text, full: html, text
    76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
    DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/trackDb.ra
  • lines changed 2, context: html, text, full: html, text
    f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
    DANIO-CODE as native danRer11 tracks, alpha only. Converts the DANIO-CODE consortium's public track hub for danRer11 into a native trackDb: 897 stanzas under one superTrack, with 11 containers for RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types, COPEs/DOPEs, validated enhancers, conservation and consensus promoters. The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are byte-identical to the consortium's copies. Four cell-type subtracks are left out because their files 404 on the consortium's server. refs #38265
• src/hg/makeDb/trackDbRaFormat/trackDbRaFormat.c
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/near/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/hg/oneShot/alphaGenomeToWig/alphaGenomeToWig.c
  • lines changed 227, context: html, text, full: html, text
    8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
    AlphaGenome Variant Impact (AVI) score track for hg38 New track under the Deleteriousness Predictions container showing Google DeepMind's AlphaGenome Variant Impact score for every possible single-base substitution, as four bigWigs, one per alternate allele. alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per alternate allele. It copies the PHRED column through as text so no precision is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a long contiguous run. The input has no row where alt == ref, so each per-allele file is missing roughly every fourth position; without the fill the wig fragments into blocks of about 3.5 values and the bigWig doubles in size. All 8,812,917,339 input lines were used, none skipped. The data cannot be redistributed, so the stanzas set tableBrowser off and the description page sends users to the AlphaGenome Atlas. Alpha only for now. No reference yet, the score is unpublished. refs #38261
• src/hg/oneShot/alphaGenomeToWig/makefile
  • lines changed 19, context: html, text, full: html, text
    8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
    AlphaGenome Variant Impact (AVI) score track for hg38 New track under the Deleteriousness Predictions container showing Google DeepMind's AlphaGenome Variant Impact score for every possible single-base substitution, as four bigWigs, one per alternate allele. alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per alternate allele. It copies the PHRED column through as text so no precision is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a long contiguous run. The input has no row where alt == ref, so each per-allele file is missing roughly every fourth position; without the fill the wig fragments into blocks of about 3.5 values and the bigWig doubles in size. All 8,812,917,339 input lines were used, none skipped. The data cannot be redistributed, so the stanzas set tableBrowser off and the description page sends users to the AlphaGenome Atlas. Alpha only for now. No reference yet, the score is unpublished. refs #38261
• src/hg/pslPseudo/checkExp/checkExp.c
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/automation/asmHubGc5Percent.pl
  • lines changed 20, context: html, text, full: html, text
    67731d2ba246fec0ba68843228e1fe7d18afd64c Tue Sep 1 13:35:40 2026 -0700
    enable the gcOnFly track for the GenArk hubs refs #35958
• src/hg/utils/automation/asmHubTrackDb.sh
  • lines changed 37, context: html, text, full: html, text
    67731d2ba246fec0ba68843228e1fe7d18afd64c Tue Sep 1 13:35:40 2026 -0700
    enable the gcOnFly track for the GenArk hubs refs #35958
  • lines changed 1, context: html, text, full: html, text
    583e765f8a3035ac100f0e841f29ec8d79fb3633 Wed Sep 2 08:37:53 2026 -0700
    correct bigDataUrl for the gcOnFly.bw file error found by claude review refs #35958
• src/hg/utils/cartTrackVarCatalog/cartTrackVarCatalog.py
  • lines changed 63, context: html, text, full: html, text
    754f5637634694624fd359811c60513966f3c1a9 Sat Sep 5 07:58:46 2026 -0700
    cartTrackVarCatalog: read a filename as a filename, not as a cart variable. The harvester looks for a track-scoped name built as safef(buf, size, "%s.%s", track, SUFFIX). Code that builds "%s.tmp" from a filename has exactly that shape, so every such site arrived as a name a person had to write down in cartVarsNotCataloged.txt as not-a-cart-variable. There were 15 of them, and they were arriving at a rate of one every few weeks: .tmp came in on 2026-08-27 with writeMergedHubFile, _ss.ps on 2026-09-04 with the RNA fold fix. The docstring predicted the class from the start and still asked for it to be thrown away by hand. harvestCartVars.py now answers the question with fileNameLike(), which asks whether the name's trailing dot-separated component is a file extension. FILE_SUFFIXES holds only the extensions the tree builds today plus tbi beside bai: each entry is a name nobody classifies again, so a guessed one adds a way to lose a real cart variable and buys nothing. Two cleverer tests were tried and rejected, and the reasons are in the docstring: the destination buffer's declaration does not decide it, since the .bai and .link.bb sites format into a plain buf and buffer while psName and tmpName are char[PATH_LEN]; and neither does the argument being formatted, which is a filename at some sites, a url at others and a table name at a third set. The records still carry these names, because a harvest that hides what it saw cannot be checked. What changed is that --reconcile no longer asks a person about them, and --update-baseline no longer writes them back. They are listed under --reconcile --verbose, and harvestCartVars.py --filenames prints the rule's claims with their call sites. Two things guard against the rule going wrong in the direction that would matter. --reconcile tests cataloged() before the filename rule, so a name the catalog describes can never be suppressed by it. And --check now fails if any cataloged name would be read as a filename, which is the case where the two halves of this file disagree about what a name is; verified with a planted row that it reports rather than passing. Baseline 70 names to 55. refs #37838
  • lines changed 42, context: html, text, full: html, text
    2f60825bdf72a8d0c99b81552266c9537e9ee2dc Sat Sep 5 08:03:52 2026 -0700
    cartTrackVarCatalog: tell an hg.conf name from a cart name. An hg.conf setting and a track-scoped cart variable are built the same way. jksql.c:1325 does safef(cfgName, sizeof cfgName, "%s.excludeDbs", failoverProf->name) and reads the result with cfgOption on the next line, which the scan cannot distinguish from safef(buf, size, "%s.heightPer", track). So excludeDbs sat in cartVarsNotCataloged.txt: correctly, in that it is not a cart variable, but that left the setting described in no registry at all and suppressed in the wrong one. The accessor that reads the buffer back is what tells them apart, so hgConfRead() looks for a cfg* call taking the same identifier within six lines of the safef. Six because the read is normally the next line and a buffer reused later in the function for something else must not excuse an unrelated name; the destination has to be a plain identifier, or we do not know what was filled in. It claims exactly one name in the tree today, and harvestCartVars.py --hgconf prints it with its call site. --reconcile no longer asks about such a name and --update-baseline no longer writes it back, so the baseline is 55 names to 54. The reverse case is now an error rather than a silence: if this catalog ever describes a name the tree reads with a cfg* accessor, --reconcile says so and exits 1, because that is one of the two registries being wrong about what the name is rather than a matter of taste. Verified by planting excludeDbs in the catalog and confirming the report. hgConfCatalog gained the row in the companion commit. refs #37838 #37925
  • lines changed 145, context: html, text, full: html, text
    287583e8d72b4abc90ac33694147f604e1238e93 Sat Sep 5 08:17:39 2026 -0700
    cartTrackVarCatalog: walk the 31 names left in the baseline, catalog 23. The baseline's own header said that a name in it was not evidence anybody had looked at it, because the first version was accepted wholesale. Every remaining name has now been read at its call site. Twenty-three turned out to be real cart variables and get rows; the other 31 are not cart variables at all, and the header now says which five things they are instead. The 23: My Variants edit form, 18 rows, a new group in OTHER_CGIS. The hgc details page prints a form named <track>_<field>, the form posts so the values land in the cart like any request variable, and hgTracks reads them out of the cart on the next render, writes the row to SQL and removes them. Cart variables by mechanism, one-shot commands by intent, which is why _id is the trigger and nothing has a default. <field>FilterLabel, <field>FilterValuesDefault, <field>HighlightType. Each of these settings is read through cartOrTdbString in three spellings: filterLabel.<field>, <field>.FilterLabel, <field>FilterLabel. The catalog had the first and, for four sibling settings, the third; for these three it had neither, which is the only reason they showed up as unknown while <field>FilterType did not. <container>.defaults, the track UI reset button, read with cartUsualInt for a composite or a superTrack and then used to clear that track's cart variables and its children's. <track>.minAc, the VCF minimum allele count, so a reader can hide singletons with 2 instead of a frequency cutoff. What the other 31 are: gvfItemName's item labels (_unk, _dnovo and six more appended to an item's name for display, which is not a variable at all), HTML element ids, table and db.table names, one submit button read with cgiOptionalString rather than from the cart, and one printf artifact. projectSelect is an id, not a name: the <select> that writes it carries no name attribute, so nothing is ever submitted. One cost, measured rather than guessed, and written into the new group's note. The My Variants suffixes are BED field names, so they collide with hgTables' per-field cart variables, whose last component is also a column name. In the 6,620 saved sessions that moves 24 names out of sessionCartAudit's "matched only by a catch-all" bucket and into the known one. The real cause is in that audit: peel() only tries suffixes beginning after a separator, so a left-anchored row such as hgta_fs.check.<db>.<table>.<field> can never match the name it describes, which is why 4,299 hgta_ names are in that bucket already. Fixing it would make the longer match win and take the 24 back. Not fixed here; it is the audit's own bug and its numbers are published. Catalog 341 rows to 364, baseline 54 names to 31. refs #37838 #37979
• src/hg/utils/cartTrackVarCatalog/cartVarsNotCataloged.txt
  • lines changed 2, context: html, text, full: html, text
    ff4d234bc327996bad7dfe2d7a26a4a579c20444 Sat Sep 5 07:29:21 2026 -0700
    cartTrackVarCatalog: baseline two filenames the scan reads as cart vars. Both are the false-positive class the harvester's own docstring names beside .bai and .tbi: a filename built with safef, where the literal in the format string looks like a track-scoped suffix. ss.ps hgc/rnaFoldClick.c:426, "%s_ss.ps", the PostScript file RNAplot writes for the RNA fold diagram. tmp lib/trackHub.c:2197, "%s.tmp", the temporary file writeMergedHubFile renames into place so a reader on the target assembly never sees a half-written quickLift hub. Neither is a cart variable, so they go in cartVarsNotCataloged.txt and the nightly reconcile goes quiet again. refs #37838
  • lines changed 23, context: html, text, full: html, text
    754f5637634694624fd359811c60513966f3c1a9 Sat Sep 5 07:58:46 2026 -0700
    cartTrackVarCatalog: read a filename as a filename, not as a cart variable. The harvester looks for a track-scoped name built as safef(buf, size, "%s.%s", track, SUFFIX). Code that builds "%s.tmp" from a filename has exactly that shape, so every such site arrived as a name a person had to write down in cartVarsNotCataloged.txt as not-a-cart-variable. There were 15 of them, and they were arriving at a rate of one every few weeks: .tmp came in on 2026-08-27 with writeMergedHubFile, _ss.ps on 2026-09-04 with the RNA fold fix. The docstring predicted the class from the start and still asked for it to be thrown away by hand. harvestCartVars.py now answers the question with fileNameLike(), which asks whether the name's trailing dot-separated component is a file extension. FILE_SUFFIXES holds only the extensions the tree builds today plus tbi beside bai: each entry is a name nobody classifies again, so a guessed one adds a way to lose a real cart variable and buys nothing. Two cleverer tests were tried and rejected, and the reasons are in the docstring: the destination buffer's declaration does not decide it, since the .bai and .link.bb sites format into a plain buf and buffer while psName and tmpName are char[PATH_LEN]; and neither does the argument being formatted, which is a filename at some sites, a url at others and a table name at a third set. The records still carry these names, because a harvest that hides what it saw cannot be checked. What changed is that --reconcile no longer asks a person about them, and --update-baseline no longer writes them back. They are listed under --reconcile --verbose, and harvestCartVars.py --filenames prints the rule's claims with their call sites. Two things guard against the rule going wrong in the direction that would matter. --reconcile tests cataloged() before the filename rule, so a name the catalog describes can never be suppressed by it. And --check now fails if any cataloged name would be read as a filename, which is the case where the two halves of this file disagree about what a name is; verified with a planted row that it reports rather than passing. Baseline 70 names to 55. refs #37838
  • lines changed 1, context: html, text, full: html, text
    2f60825bdf72a8d0c99b81552266c9537e9ee2dc Sat Sep 5 08:03:52 2026 -0700
    cartTrackVarCatalog: tell an hg.conf name from a cart name. An hg.conf setting and a track-scoped cart variable are built the same way. jksql.c:1325 does safef(cfgName, sizeof cfgName, "%s.excludeDbs", failoverProf->name) and reads the result with cfgOption on the next line, which the scan cannot distinguish from safef(buf, size, "%s.heightPer", track). So excludeDbs sat in cartVarsNotCataloged.txt: correctly, in that it is not a cart variable, but that left the setting described in no registry at all and suppressed in the wrong one. The accessor that reads the buffer back is what tells them apart, so hgConfRead() looks for a cfg* call taking the same identifier within six lines of the safef. Six because the read is normally the next line and a buffer reused later in the function for something else must not excuse an unrelated name; the destination has to be a plain identifier, or we do not know what was filled in. It claims exactly one name in the tree today, and harvestCartVars.py --hgconf prints it with its call site. --reconcile no longer asks about such a name and --update-baseline no longer writes it back, so the baseline is 55 names to 54. The reverse case is now an error rather than a silence: if this catalog ever describes a name the tree reads with a cfg* accessor, --reconcile says so and exits 1, because that is one of the two registries being wrong about what the name is rather than a matter of taste. Verified by planting excludeDbs in the catalog and confirming the report. hgConfCatalog gained the row in the companion commit. refs #37838 #37925
  • lines changed 40, context: html, text, full: html, text
    287583e8d72b4abc90ac33694147f604e1238e93 Sat Sep 5 08:17:39 2026 -0700
    cartTrackVarCatalog: walk the 31 names left in the baseline, catalog 23. The baseline's own header said that a name in it was not evidence anybody had looked at it, because the first version was accepted wholesale. Every remaining name has now been read at its call site. Twenty-three turned out to be real cart variables and get rows; the other 31 are not cart variables at all, and the header now says which five things they are instead. The 23: My Variants edit form, 18 rows, a new group in OTHER_CGIS. The hgc details page prints a form named <track>_<field>, the form posts so the values land in the cart like any request variable, and hgTracks reads them out of the cart on the next render, writes the row to SQL and removes them. Cart variables by mechanism, one-shot commands by intent, which is why _id is the trigger and nothing has a default. <field>FilterLabel, <field>FilterValuesDefault, <field>HighlightType. Each of these settings is read through cartOrTdbString in three spellings: filterLabel.<field>, <field>.FilterLabel, <field>FilterLabel. The catalog had the first and, for four sibling settings, the third; for these three it had neither, which is the only reason they showed up as unknown while <field>FilterType did not. <container>.defaults, the track UI reset button, read with cartUsualInt for a composite or a superTrack and then used to clear that track's cart variables and its children's. <track>.minAc, the VCF minimum allele count, so a reader can hide singletons with 2 instead of a frequency cutoff. What the other 31 are: gvfItemName's item labels (_unk, _dnovo and six more appended to an item's name for display, which is not a variable at all), HTML element ids, table and db.table names, one submit button read with cgiOptionalString rather than from the cart, and one printf artifact. projectSelect is an id, not a name: the <select> that writes it carries no name attribute, so nothing is ever submitted. One cost, measured rather than guessed, and written into the new group's note. The My Variants suffixes are BED field names, so they collide with hgTables' per-field cart variables, whose last component is also a column name. In the 6,620 saved sessions that moves 24 names out of sessionCartAudit's "matched only by a catch-all" bucket and into the known one. The real cause is in that audit: peel() only tries suffixes beginning after a separator, so a left-anchored row such as hgta_fs.check.<db>.<table>.<field> can never match the name it describes, which is why 4,299 hgta_ names are in that bucket already. Fixing it would make the longer match win and take the 24 back. Not fixed here; it is the audit's own bug and its numbers are published. Catalog 341 rows to 364, baseline 54 names to 31. refs #37838 #37979
• src/hg/utils/cartTrackVarCatalog/harvestCartVars.py
  • lines changed 58, context: html, text, full: html, text
    754f5637634694624fd359811c60513966f3c1a9 Sat Sep 5 07:58:46 2026 -0700
    cartTrackVarCatalog: read a filename as a filename, not as a cart variable. The harvester looks for a track-scoped name built as safef(buf, size, "%s.%s", track, SUFFIX). Code that builds "%s.tmp" from a filename has exactly that shape, so every such site arrived as a name a person had to write down in cartVarsNotCataloged.txt as not-a-cart-variable. There were 15 of them, and they were arriving at a rate of one every few weeks: .tmp came in on 2026-08-27 with writeMergedHubFile, _ss.ps on 2026-09-04 with the RNA fold fix. The docstring predicted the class from the start and still asked for it to be thrown away by hand. harvestCartVars.py now answers the question with fileNameLike(), which asks whether the name's trailing dot-separated component is a file extension. FILE_SUFFIXES holds only the extensions the tree builds today plus tbi beside bai: each entry is a name nobody classifies again, so a guessed one adds a way to lose a real cart variable and buys nothing. Two cleverer tests were tried and rejected, and the reasons are in the docstring: the destination buffer's declaration does not decide it, since the .bai and .link.bb sites format into a plain buf and buffer while psName and tmpName are char[PATH_LEN]; and neither does the argument being formatted, which is a filename at some sites, a url at others and a table name at a third set. The records still carry these names, because a harvest that hides what it saw cannot be checked. What changed is that --reconcile no longer asks a person about them, and --update-baseline no longer writes them back. They are listed under --reconcile --verbose, and harvestCartVars.py --filenames prints the rule's claims with their call sites. Two things guard against the rule going wrong in the direction that would matter. --reconcile tests cataloged() before the filename rule, so a name the catalog describes can never be suppressed by it. And --check now fails if any cataloged name would be read as a filename, which is the case where the two halves of this file disagree about what a name is; verified with a planted row that it reports rather than passing. Baseline 70 names to 55. refs #37838
  • lines changed 78, context: html, text, full: html, text
    2f60825bdf72a8d0c99b81552266c9537e9ee2dc Sat Sep 5 08:03:52 2026 -0700
    cartTrackVarCatalog: tell an hg.conf name from a cart name. An hg.conf setting and a track-scoped cart variable are built the same way. jksql.c:1325 does safef(cfgName, sizeof cfgName, "%s.excludeDbs", failoverProf->name) and reads the result with cfgOption on the next line, which the scan cannot distinguish from safef(buf, size, "%s.heightPer", track). So excludeDbs sat in cartVarsNotCataloged.txt: correctly, in that it is not a cart variable, but that left the setting described in no registry at all and suppressed in the wrong one. The accessor that reads the buffer back is what tells them apart, so hgConfRead() looks for a cfg* call taking the same identifier within six lines of the safef. Six because the read is normally the next line and a buffer reused later in the function for something else must not excuse an unrelated name; the destination has to be a plain identifier, or we do not know what was filled in. It claims exactly one name in the tree today, and harvestCartVars.py --hgconf prints it with its call site. --reconcile no longer asks about such a name and --update-baseline no longer writes it back, so the baseline is 55 names to 54. The reverse case is now an error rather than a silence: if this catalog ever describes a name the tree reads with a cfg* accessor, --reconcile says so and exits 1, because that is one of the two registries being wrong about what the name is rather than a matter of taste. Verified by planting excludeDbs in the catalog and confirming the report. hgConfCatalog gained the row in the companion commit. refs #37838 #37925
• src/hg/utils/docent/README.md
  • lines changed 8, context: html, text, full: html, text
    e8fd1a37d1959a9e73943e946088d1b86d2b1bb4 Fri Sep 4 11:17:35 2026 -0700
    docent: correct the claim that expect: is the only verb that can fail a run An audit of tests/README.txt against the code found the claim wrong in both READMEs. docent.js turns any step's throw into "step N (verb) failed" and exit 1, so mouseover:, loadSession:, drag:, convert: and go: all fail a run when they cannot do what they were told. Verified by running a script with no expect: step in it at all: it exited 1 from loadSession:. The conclusion drawn from the claim still holds and is kept: expect: is the only verb that CHECKS anything, and track: does not even check its own input, since a name no assembly has is sent as name=mode and the run exits 0 (also verified). Also documents why make derive strips one line of trackDb provenance: the listing is cached for a day, a cold fetch prints a line a warm run does not, and the first run of any day otherwise fails against a baseline captured warm. refs #37892
  • lines changed 1, context: html, text, full: html, text
    6af92621b7dba437be69c05f3ff49079ba637383 Fri Sep 4 11:43:39 2026 -0700
    docent: let convert: take a bare string, like every other verb convert: hs1 left the argument as a String, so o.to was undefined and o.search picked up String.prototype.search. The run then reported convert: "function search() { [native code] }" matched nothing which says nothing about what is wrong with the script. A bare string is now the target assembly, matching hub:, addCustomTrack:, loadSession: and the rest. quicklift: is still never implied, so convert: hs1 is a plain coordinate convert; the README row says so now. Found while writing the first of the regression tests for #38252, fourteen of which open with convert:. refs #37892
  • lines changed 2, context: html, text, full: html, text
    ce8cbfc3d2ef6853f94bf80b0f8b5f389c60025b Sat Sep 5 09:21:32 2026 -0700
    docent: two more expect: checks, a positional click:, and three fixes found writing tests Everything here was needed by a regression script that could not otherwise be written, or by one that failed for a reason that was not a bug. expect: {url:, noUrl:} is a substring check on the current address. Some things are visible nowhere else: which CGI a click reached, and what a form put in a query string. #36387's fix strips zero-width characters out of a search term before the position box submits it, and the character is invisible in the rendered page, so whether %E2%80%8B survives into the URL is the only evidence either way. expect: {has:, noHas:} takes a CSS selector, for a bug whose whole signature is WHERE something sits. #37785 attached a squishyPack track's center label to the wrong row: same rows drawn, same image height, same pixels, and only the row the label's image map hangs off changed, so rows:, height: and text: are all blind to it. Documented as a last resort, since an assertion on hgTracks' own ids breaks easily for reasons that are not bugs. click: now takes the positional forms mouseover: already had (at:/frac:/x:) and follows the item box nearest that point. Some tracks have no item that can be named at all: every GIAB Problematic Regions subtrack is type bigBed 3, so hgTracks writes an EMPTY i= into the hgc href and gives every box the title "Start of Exon (1/1)". Neither item: nor title: can pick one, and a raw mouse click on the data area is swallowed by the drag-select handler. The "item not found" message now picks the row's map boxes by MAP NAME instead of by a y-band, reports how many boxes are in the row, and falls back to a box's title when it has no name. On a quickLift target the band test dropped every item box, so the message said the GIAB row held three things when it held twenty-six, and the three it named were a density control and two exon arrows. goShow now scrolls to the top and takes the Login/Share links out of the way before clicking Search. Those links sit in an absolutely-positioned container at the top right of the header bar and on a wide page land on top of #goButton; Playwright then retries for the full 30s and fails with "<a id=loginLink ...> subtree intercepts pointer events", which reads like a broken Search button rather than a covered one. Pressing Enter in the position box is not a substitute: by the time a 30s click timeout has been caught, the navigation wait armed before it has already expired. tests/pagechecks and tests/pagechecks.xfail cover all of it, per the rule in tests/README.txt that a verb we touch and find untested belongs on its list. The xfail aims all four new checks the wrong way at once, because a check that cannot fail is not a check. refs #38252
  • lines changed 1, context: html, text, full: html, text
    2b8d9275548bcc9bd161b68f24848b545e1069ef Sun Sep 6 14:00:28 2026 -0700
    docent: click raw:, and the first two scripts that do anything twice Every one of the 29 scripts in the suite was a straight line: fresh cart, a few steps, assert once. A bug that only exists on the repeat was invisible to that shape, and the closed-bug pool holds several. These are the first two that repeat a gesture. rm36805 click a TOGA item, dismiss the pop-up, click the same item again. One click passes on the broken build as happily as on the fixed one, so the second click is the test. rm27113 three clicks on the centre of the base-position ruler. A single click zooms 3x about the base under the cursor, and the bug drifted that centre one base left each time. Both needed a gesture the language could not express. click: {track, item} follows the item's own map-box href, which is right when the assertion is about the hgc PAGE, but hgTracks answers a real click with an ajax DIALOG (popUpHgcOrHgGene.hgc) and following the href never opens one. raw: true presses the mouse where a user presses it and lets the page answer -- a navigation, a dialog, or a new image in place, whichever arrives, waited for rather than slept through. With no item name it is a bare point on the row, which is the only way to click the ruler at all: the ruler carries no hgc map boxes for areaXY to snap to. Two things measured while writing these, both in the script comments. jQuery UI HIDES a dialog on close rather than removing it, so the assertion has to be has: "#hgcDialog:visible" -- without :visible the second half of rm36805 passes whether or not the pop-up ever comes back. And one base is invisible at the 13kb windows the rest of the suite uses, where a pixel is fourteen bases, so rm27113 starts at 100 bases and its three expected windows are arithmetic rather than three observed strings: every one of them is centred on base 155,806,200. #37014 is the other repeat-click bug in the pool and is deliberately NOT written. Its reproducer no longer reaches the code it was about: 9e9ee32a4c8 (#37878) later excluded crossTissue* tracks from the pop-up path altogether, and the session's track is crossTissueMapsTissueCellType, so the click now navigates and no dialog is involved. Writing it against a different bar chart track would be pinning the ticket to something it was never about. refs #38252
• src/hg/utils/docent/docent.js
  • lines changed 6, context: html, text, full: html, text
    6af92621b7dba437be69c05f3ff49079ba637383 Fri Sep 4 11:43:39 2026 -0700
    docent: let convert: take a bare string, like every other verb convert: hs1 left the argument as a String, so o.to was undefined and o.search picked up String.prototype.search. The run then reported convert: "function search() { [native code] }" matched nothing which says nothing about what is wrong with the script. A bare string is now the target assembly, matching hub:, addCustomTrack:, loadSession: and the rest. quicklift: is still never implied, so convert: hs1 is a plain coordinate convert; the README row says so now. Found while writing the first of the regression tests for #38252, fourteen of which open with convert:. refs #37892
  • lines changed 156, context: html, text, full: html, text
    ce8cbfc3d2ef6853f94bf80b0f8b5f389c60025b Sat Sep 5 09:21:32 2026 -0700
    docent: two more expect: checks, a positional click:, and three fixes found writing tests Everything here was needed by a regression script that could not otherwise be written, or by one that failed for a reason that was not a bug. expect: {url:, noUrl:} is a substring check on the current address. Some things are visible nowhere else: which CGI a click reached, and what a form put in a query string. #36387's fix strips zero-width characters out of a search term before the position box submits it, and the character is invisible in the rendered page, so whether %E2%80%8B survives into the URL is the only evidence either way. expect: {has:, noHas:} takes a CSS selector, for a bug whose whole signature is WHERE something sits. #37785 attached a squishyPack track's center label to the wrong row: same rows drawn, same image height, same pixels, and only the row the label's image map hangs off changed, so rows:, height: and text: are all blind to it. Documented as a last resort, since an assertion on hgTracks' own ids breaks easily for reasons that are not bugs. click: now takes the positional forms mouseover: already had (at:/frac:/x:) and follows the item box nearest that point. Some tracks have no item that can be named at all: every GIAB Problematic Regions subtrack is type bigBed 3, so hgTracks writes an EMPTY i= into the hgc href and gives every box the title "Start of Exon (1/1)". Neither item: nor title: can pick one, and a raw mouse click on the data area is swallowed by the drag-select handler. The "item not found" message now picks the row's map boxes by MAP NAME instead of by a y-band, reports how many boxes are in the row, and falls back to a box's title when it has no name. On a quickLift target the band test dropped every item box, so the message said the GIAB row held three things when it held twenty-six, and the three it named were a density control and two exon arrows. goShow now scrolls to the top and takes the Login/Share links out of the way before clicking Search. Those links sit in an absolutely-positioned container at the top right of the header bar and on a wide page land on top of #goButton; Playwright then retries for the full 30s and fails with "<a id=loginLink ...> subtree intercepts pointer events", which reads like a broken Search button rather than a covered one. Pressing Enter in the position box is not a substitute: by the time a 30s click timeout has been caught, the navigation wait armed before it has already expired. tests/pagechecks and tests/pagechecks.xfail cover all of it, per the rule in tests/README.txt that a verb we touch and find untested belongs on its list. The xfail aims all four new checks the wrong way at once, because a check that cannot fail is not a check. refs #38252
  • lines changed 26, context: html, text, full: html, text
    2b8d9275548bcc9bd161b68f24848b545e1069ef Sun Sep 6 14:00:28 2026 -0700
    docent: click raw:, and the first two scripts that do anything twice Every one of the 29 scripts in the suite was a straight line: fresh cart, a few steps, assert once. A bug that only exists on the repeat was invisible to that shape, and the closed-bug pool holds several. These are the first two that repeat a gesture. rm36805 click a TOGA item, dismiss the pop-up, click the same item again. One click passes on the broken build as happily as on the fixed one, so the second click is the test. rm27113 three clicks on the centre of the base-position ruler. A single click zooms 3x about the base under the cursor, and the bug drifted that centre one base left each time. Both needed a gesture the language could not express. click: {track, item} follows the item's own map-box href, which is right when the assertion is about the hgc PAGE, but hgTracks answers a real click with an ajax DIALOG (popUpHgcOrHgGene.hgc) and following the href never opens one. raw: true presses the mouse where a user presses it and lets the page answer -- a navigation, a dialog, or a new image in place, whichever arrives, waited for rather than slept through. With no item name it is a bare point on the row, which is the only way to click the ruler at all: the ruler carries no hgc map boxes for areaXY to snap to. Two things measured while writing these, both in the script comments. jQuery UI HIDES a dialog on close rather than removing it, so the assertion has to be has: "#hgcDialog:visible" -- without :visible the second half of rm36805 passes whether or not the pop-up ever comes back. And one base is invisible at the 13kb windows the rest of the suite uses, where a pixel is fourteen bases, so rm27113 starts at 100 bases and its three expected windows are arithmetic rather than three observed strings: every one of them is centred on base 155,806,200. #37014 is the other repeat-click bug in the pool and is deliberately NOT written. Its reproducer no longer reaches the code it was about: 9e9ee32a4c8 (#37878) later excluded crossTissue* tracks from the pop-up path altogether, and the session's track is crossTissueMapsTissueCellType, so the click now navigates and no dialog is involved. Writing it against a different bar chart track would be pinning the ticket to something it was never about. refs #38252
• src/hg/utils/docent/tests/README.txt
  • lines changed 15, context: html, text, full: html, text
    e8fd1a37d1959a9e73943e946088d1b86d2b1bb4 Fri Sep 4 11:17:35 2026 -0700
    docent: correct the claim that expect: is the only verb that can fail a run An audit of tests/README.txt against the code found the claim wrong in both READMEs. docent.js turns any step's throw into "step N (verb) failed" and exit 1, so mouseover:, loadSession:, drag:, convert: and go: all fail a run when they cannot do what they were told. Verified by running a script with no expect: step in it at all: it exited 1 from loadSession:. The conclusion drawn from the claim still holds and is kept: expect: is the only verb that CHECKS anything, and track: does not even check its own input, since a name no assembly has is sent as name=mode and the run exits 0 (also verified). Also documents why make derive strips one line of trackDb provenance: the listing is cached for a day, a cold fetch prints a line a warm run does not, and the first run of any day otherwise fails against a baseline captured warm. refs #37892
  • lines changed 9, context: html, text, full: html, text
    ce8cbfc3d2ef6853f94bf80b0f8b5f389c60025b Sat Sep 5 09:21:32 2026 -0700
    docent: two more expect: checks, a positional click:, and three fixes found writing tests Everything here was needed by a regression script that could not otherwise be written, or by one that failed for a reason that was not a bug. expect: {url:, noUrl:} is a substring check on the current address. Some things are visible nowhere else: which CGI a click reached, and what a form put in a query string. #36387's fix strips zero-width characters out of a search term before the position box submits it, and the character is invisible in the rendered page, so whether %E2%80%8B survives into the URL is the only evidence either way. expect: {has:, noHas:} takes a CSS selector, for a bug whose whole signature is WHERE something sits. #37785 attached a squishyPack track's center label to the wrong row: same rows drawn, same image height, same pixels, and only the row the label's image map hangs off changed, so rows:, height: and text: are all blind to it. Documented as a last resort, since an assertion on hgTracks' own ids breaks easily for reasons that are not bugs. click: now takes the positional forms mouseover: already had (at:/frac:/x:) and follows the item box nearest that point. Some tracks have no item that can be named at all: every GIAB Problematic Regions subtrack is type bigBed 3, so hgTracks writes an EMPTY i= into the hgc href and gives every box the title "Start of Exon (1/1)". Neither item: nor title: can pick one, and a raw mouse click on the data area is swallowed by the drag-select handler. The "item not found" message now picks the row's map boxes by MAP NAME instead of by a y-band, reports how many boxes are in the row, and falls back to a box's title when it has no name. On a quickLift target the band test dropped every item box, so the message said the GIAB row held three things when it held twenty-six, and the three it named were a density control and two exon arrows. goShow now scrolls to the top and takes the Login/Share links out of the way before clicking Search. Those links sit in an absolutely-positioned container at the top right of the header bar and on a wide page land on top of #goButton; Playwright then retries for the full 30s and fails with "<a id=loginLink ...> subtree intercepts pointer events", which reads like a broken Search button rather than a covered one. Pressing Enter in the position box is not a substitute: by the time a 30s click timeout has been caught, the navigation wait armed before it has already expired. tests/pagechecks and tests/pagechecks.xfail cover all of it, per the rule in tests/README.txt that a verb we touch and find untested belongs on its list. The xfail aims all four new checks the wrong way at once, because a check that cannot fail is not a check. refs #38252
• src/hg/utils/docent/tests/docentTest.mk
  • lines changed 116, context: html, text, full: html, text
    67efa330d2830c30741f9524abfd72b59f8791f8 Fri Sep 4 11:17:46 2026 -0700
    docent: share the test rules, add tests/regress, and stop an empty suite passing Moves the test rules out of tests/makefile into tests/docentTest.mk so a second directory of Docent tests runs the same code rather than a copy. tests/makefile and the new tests/regress/makefile each set DOCENT and include it. tests/regress/ is for one script per already-fixed bug, asserting the behavior its ticket says is correct, to be run nightly against genome-test (#38252). It is kept out of tests/ so that suite stays short enough to run before a commit: measured, its eleven scripts take 56 seconds. Two fixes to the rules while they moved: - make derive strips "trackDb: N tracks for DB from .../hubApi" from both the run and the baseline. docent.js caches the trackDb listing for a day and prints that line only on a cold fetch, so the first make derive of any day reported CHANGED against a baseline captured warm. Verified cold and warm, and make derive-accept reproduces the three committed baselines byte for byte. - make test with no *.docent.yaml, and make derive with no baseline, now fail instead of printing "docent tests passed". An empty tests/regress reported a pass before this. refs #37892 #38252
  • lines changed 12, context: html, text, full: html, text
    e128682fd48d974c0eaa74366087d2be47080bf5 Fri Sep 4 11:43:52 2026 -0700
    docent: add make preflight, which checks the fixtures a test suite does not own A Docent test that loads a saved session or attaches a hub depends on something outside the tree, and the failure when that thing goes away is silent rather than loud. A session that has been renamed or deleted is not an error: hgTracks answers HTTP 200 with a page titled "Very Early Error" whose body reads "Could not find session NAME for user USER", the page carries no track image, and every noText: assertion on it passes. The run goes green having tested nothing. Verified both ways: a bogus session name passes a test whose only check is noText:, and fails once the script also asserts noText: "Could not find session". preflight.js reads the fixtures out of the scripts themselves -- loadSession:, hub:, addHub:, addCustomTrack: url:, and hubUrl= inside a goto: -- so the list cannot drift from what the scripts actually use. It needs no browser, runs in a couple of seconds, and exits non-zero if anything is unreachable, which is what lets a nightly run tell "the fixtures are gone" from "a bug came back". Checking it against the sessions cited by the tickets in #38252 found four that do not exist on genome-test because they were saved on the RR or on beta, and corrected one I had wrongly called missing: session names store a dash as %2D, so a MySQL LIKE with a literal dash misses them. The HTTP check has no such problem, which is a reason to prefer it over a query against namedSessionDb. refs #37892 #38252
• src/hg/utils/docent/tests/makefile
  • lines changed 85, context: html, text, full: html, text
    67efa330d2830c30741f9524abfd72b59f8791f8 Fri Sep 4 11:17:46 2026 -0700
    docent: share the test rules, add tests/regress, and stop an empty suite passing Moves the test rules out of tests/makefile into tests/docentTest.mk so a second directory of Docent tests runs the same code rather than a copy. tests/makefile and the new tests/regress/makefile each set DOCENT and include it. tests/regress/ is for one script per already-fixed bug, asserting the behavior its ticket says is correct, to be run nightly against genome-test (#38252). It is kept out of tests/ so that suite stays short enough to run before a commit: measured, its eleven scripts take 56 seconds. Two fixes to the rules while they moved: - make derive strips "trackDb: N tracks for DB from .../hubApi" from both the run and the baseline. docent.js caches the trackDb listing for a day and prints that line only on a cold fetch, so the first make derive of any day reported CHANGED against a baseline captured warm. Verified cold and warm, and make derive-accept reproduces the three committed baselines byte for byte. - make test with no *.docent.yaml, and make derive with no baseline, now fail instead of printing "docent tests passed". An empty tests/regress reported a pass before this. refs #37892 #38252
  • lines changed 2, context: html, text, full: html, text
    e128682fd48d974c0eaa74366087d2be47080bf5 Fri Sep 4 11:43:52 2026 -0700
    docent: add make preflight, which checks the fixtures a test suite does not own A Docent test that loads a saved session or attaches a hub depends on something outside the tree, and the failure when that thing goes away is silent rather than loud. A session that has been renamed or deleted is not an error: hgTracks answers HTTP 200 with a page titled "Very Early Error" whose body reads "Could not find session NAME for user USER", the page carries no track image, and every noText: assertion on it passes. The run goes green having tested nothing. Verified both ways: a bogus session name passes a test whose only check is noText:, and fails once the script also asserts noText: "Could not find session". preflight.js reads the fixtures out of the scripts themselves -- loadSession:, hub:, addHub:, addCustomTrack: url:, and hubUrl= inside a goto: -- so the list cannot drift from what the scripts actually use. It needs no browser, runs in a couple of seconds, and exits non-zero if anything is unreachable, which is what lets a nightly run tell "the fixtures are gone" from "a bug came back". Checking it against the sessions cited by the tickets in #38252 found four that do not exist on genome-test because they were saved on the RR or on beta, and corrected one I had wrongly called missing: session names store a dash as %2D, so a MySQL LIKE with a literal dash misses them. The HTTP check has no such problem, which is a reason to prefer it over a query against namedSessionDb. refs #37892 #38252
• src/hg/utils/docent/tests/pagechecks.docent.yaml
  • lines changed 56, context: html, text, full: html, text
    ce8cbfc3d2ef6853f94bf80b0f8b5f389c60025b Sat Sep 5 09:21:32 2026 -0700
    docent: two more expect: checks, a positional click:, and three fixes found writing tests Everything here was needed by a regression script that could not otherwise be written, or by one that failed for a reason that was not a bug. expect: {url:, noUrl:} is a substring check on the current address. Some things are visible nowhere else: which CGI a click reached, and what a form put in a query string. #36387's fix strips zero-width characters out of a search term before the position box submits it, and the character is invisible in the rendered page, so whether %E2%80%8B survives into the URL is the only evidence either way. expect: {has:, noHas:} takes a CSS selector, for a bug whose whole signature is WHERE something sits. #37785 attached a squishyPack track's center label to the wrong row: same rows drawn, same image height, same pixels, and only the row the label's image map hangs off changed, so rows:, height: and text: are all blind to it. Documented as a last resort, since an assertion on hgTracks' own ids breaks easily for reasons that are not bugs. click: now takes the positional forms mouseover: already had (at:/frac:/x:) and follows the item box nearest that point. Some tracks have no item that can be named at all: every GIAB Problematic Regions subtrack is type bigBed 3, so hgTracks writes an EMPTY i= into the hgc href and gives every box the title "Start of Exon (1/1)". Neither item: nor title: can pick one, and a raw mouse click on the data area is swallowed by the drag-select handler. The "item not found" message now picks the row's map boxes by MAP NAME instead of by a y-band, reports how many boxes are in the row, and falls back to a box's title when it has no name. On a quickLift target the band test dropped every item box, so the message said the GIAB row held three things when it held twenty-six, and the three it named were a density control and two exon arrows. goShow now scrolls to the top and takes the Login/Share links out of the way before clicking Search. Those links sit in an absolutely-positioned container at the top right of the header bar and on a wide page land on top of #goButton; Playwright then retries for the full 30s and fails with "<a id=loginLink ...> subtree intercepts pointer events", which reads like a broken Search button rather than a covered one. Pressing Enter in the position box is not a substitute: by the time a 30s click timeout has been caught, the navigation wait armed before it has already expired. tests/pagechecks and tests/pagechecks.xfail cover all of it, per the rule in tests/README.txt that a verb we touch and find untested belongs on its list. The xfail aims all four new checks the wrong way at once, because a check that cannot fail is not a check. refs #38252
• src/hg/utils/docent/tests/pagechecks.xfail.docent.yaml
  • lines changed 30, context: html, text, full: html, text
    ce8cbfc3d2ef6853f94bf80b0f8b5f389c60025b Sat Sep 5 09:21:32 2026 -0700
    docent: two more expect: checks, a positional click:, and three fixes found writing tests Everything here was needed by a regression script that could not otherwise be written, or by one that failed for a reason that was not a bug. expect: {url:, noUrl:} is a substring check on the current address. Some things are visible nowhere else: which CGI a click reached, and what a form put in a query string. #36387's fix strips zero-width characters out of a search term before the position box submits it, and the character is invisible in the rendered page, so whether %E2%80%8B survives into the URL is the only evidence either way. expect: {has:, noHas:} takes a CSS selector, for a bug whose whole signature is WHERE something sits. #37785 attached a squishyPack track's center label to the wrong row: same rows drawn, same image height, same pixels, and only the row the label's image map hangs off changed, so rows:, height: and text: are all blind to it. Documented as a last resort, since an assertion on hgTracks' own ids breaks easily for reasons that are not bugs. click: now takes the positional forms mouseover: already had (at:/frac:/x:) and follows the item box nearest that point. Some tracks have no item that can be named at all: every GIAB Problematic Regions subtrack is type bigBed 3, so hgTracks writes an EMPTY i= into the hgc href and gives every box the title "Start of Exon (1/1)". Neither item: nor title: can pick one, and a raw mouse click on the data area is swallowed by the drag-select handler. The "item not found" message now picks the row's map boxes by MAP NAME instead of by a y-band, reports how many boxes are in the row, and falls back to a box's title when it has no name. On a quickLift target the band test dropped every item box, so the message said the GIAB row held three things when it held twenty-six, and the three it named were a density control and two exon arrows. goShow now scrolls to the top and takes the Login/Share links out of the way before clicking Search. Those links sit in an absolutely-positioned container at the top right of the header bar and on a wide page land on top of #goButton; Playwright then retries for the full 30s and fails with "<a id=loginLink ...> subtree intercepts pointer events", which reads like a broken Search button rather than a covered one. Pressing Enter in the position box is not a substitute: by the time a 30s click timeout has been caught, the navigation wait armed before it has already expired. tests/pagechecks and tests/pagechecks.xfail cover all of it, per the rule in tests/README.txt that a verb we touch and find untested belongs on its list. The xfail aims all four new checks the wrong way at once, because a check that cannot fail is not a check. refs #38252
• src/hg/utils/docent/tests/preflight.js
  • lines changed 175, context: html, text, full: html, text
    e128682fd48d974c0eaa74366087d2be47080bf5 Fri Sep 4 11:43:52 2026 -0700
    docent: add make preflight, which checks the fixtures a test suite does not own A Docent test that loads a saved session or attaches a hub depends on something outside the tree, and the failure when that thing goes away is silent rather than loud. A session that has been renamed or deleted is not an error: hgTracks answers HTTP 200 with a page titled "Very Early Error" whose body reads "Could not find session NAME for user USER", the page carries no track image, and every noText: assertion on it passes. The run goes green having tested nothing. Verified both ways: a bogus session name passes a test whose only check is noText:, and fails once the script also asserts noText: "Could not find session". preflight.js reads the fixtures out of the scripts themselves -- loadSession:, hub:, addHub:, addCustomTrack: url:, and hubUrl= inside a goto: -- so the list cannot drift from what the scripts actually use. It needs no browser, runs in a couple of seconds, and exits non-zero if anything is unreachable, which is what lets a nightly run tell "the fixtures are gone" from "a bug came back". Checking it against the sessions cited by the tickets in #38252 found four that do not exist on genome-test because they were saved on the RR or on beta, and corrected one I had wrongly called missing: session names store a dash as %2D, so a MySQL LIKE with a literal dash misses them. The HTTP check has no such problem, which is a reason to prefer it over a query against namedSessionDb. refs #37892 #38252
  • lines changed 10, context: html, text, full: html, text
    2201c5c416938253128eebcac00c0a240574a889 Fri Sep 4 12:47:16 2026 -0700
    docent: nightly cron wrapper for the regression suite, refs #38252 Mails a report every night whether anything failed or not, matching the catalogNightly job: no mail means the cron has stopped, not that the browser is fine. The script mails on its own and always exits 0, so cron adds nothing. Two things it does deliberately: - It runs the COMMITTED tests, listed with git ls-files rather than by globbing the directory. The same directory holds scripts written against a ticket whose recipe is not right yet, and those must not mail a failure every night. A newly committed test is picked up with no edit. - It reports preflight separately from the tests, and preflight is now given an explicit script list (preflight.js takes names after the directory) so a dead fixture belonging to a work-in-progress script is not reported against a run that never included it. A missing session and a returning bug are different news and should not arrive as the same red. Verified: the pass path, the fail path (broke one committed assertion, restored it from git), and a run under `env -i` with only HOME and SHELL set, which is what cron will give it. Installed as `10 4 * * *` in braney's crontab, reading the tests out of the worktree because they are on this branch and not yet on master. It should move to a checkout of its own when the branch lands, so that a day's editing cannot change what the cron measures.
• src/hg/utils/docent/tests/regress/README.txt
  • lines changed 36, context: html, text, full: html, text
    67efa330d2830c30741f9524abfd72b59f8791f8 Fri Sep 4 11:17:46 2026 -0700
    docent: share the test rules, add tests/regress, and stop an empty suite passing Moves the test rules out of tests/makefile into tests/docentTest.mk so a second directory of Docent tests runs the same code rather than a copy. tests/makefile and the new tests/regress/makefile each set DOCENT and include it. tests/regress/ is for one script per already-fixed bug, asserting the behavior its ticket says is correct, to be run nightly against genome-test (#38252). It is kept out of tests/ so that suite stays short enough to run before a commit: measured, its eleven scripts take 56 seconds. Two fixes to the rules while they moved: - make derive strips "trackDb: N tracks for DB from .../hubApi" from both the run and the baseline. docent.js caches the trackDb listing for a day and prints that line only on a cold fetch, so the first make derive of any day reported CHANGED against a baseline captured warm. Verified cold and warm, and make derive-accept reproduces the three committed baselines byte for byte. - make test with no *.docent.yaml, and make derive with no baseline, now fail instead of printing "docent tests passed". An empty tests/regress reported a pass before this. refs #37892 #38252
• src/hg/utils/docent/tests/regress/makefile
  • lines changed 27, context: html, text, full: html, text
    67efa330d2830c30741f9524abfd72b59f8791f8 Fri Sep 4 11:17:46 2026 -0700
    docent: share the test rules, add tests/regress, and stop an empty suite passing Moves the test rules out of tests/makefile into tests/docentTest.mk so a second directory of Docent tests runs the same code rather than a copy. tests/makefile and the new tests/regress/makefile each set DOCENT and include it. tests/regress/ is for one script per already-fixed bug, asserting the behavior its ticket says is correct, to be run nightly against genome-test (#38252). It is kept out of tests/ so that suite stays short enough to run before a commit: measured, its eleven scripts take 56 seconds. Two fixes to the rules while they moved: - make derive strips "trackDb: N tracks for DB from .../hubApi" from both the run and the baseline. docent.js caches the trackDb listing for a day and prints that line only on a cold fetch, so the first make derive of any day reported CHANGED against a baseline captured warm. Verified cold and warm, and make derive-accept reproduces the three committed baselines byte for byte. - make test with no *.docent.yaml, and make derive with no baseline, now fail instead of printing "docent tests passed". An empty tests/regress reported a pass before this. refs #37892 #38252
  • lines changed 2, context: html, text, full: html, text
    e128682fd48d974c0eaa74366087d2be47080bf5 Fri Sep 4 11:43:52 2026 -0700
    docent: add make preflight, which checks the fixtures a test suite does not own A Docent test that loads a saved session or attaches a hub depends on something outside the tree, and the failure when that thing goes away is silent rather than loud. A session that has been renamed or deleted is not an error: hgTracks answers HTTP 200 with a page titled "Very Early Error" whose body reads "Could not find session NAME for user USER", the page carries no track image, and every noText: assertion on it passes. The run goes green having tested nothing. Verified both ways: a bogus session name passes a test whose only check is noText:, and fails once the script also asserts noText: "Could not find session". preflight.js reads the fixtures out of the scripts themselves -- loadSession:, hub:, addHub:, addCustomTrack: url:, and hubUrl= inside a goto: -- so the list cannot drift from what the scripts actually use. It needs no browser, runs in a couple of seconds, and exits non-zero if anything is unreachable, which is what lets a nightly run tell "the fixtures are gone" from "a bug came back". Checking it against the sessions cited by the tickets in #38252 found four that do not exist on genome-test because they were saved on the RR or on beta, and corrected one I had wrongly called missing: session names store a dash as %2D, so a MySQL LIKE with a literal dash misses them. The HTTP check has no such problem, which is a reason to prefer it over a query against namedSessionDb. refs #37892 #38252
• src/hg/utils/docent/tests/regress/nightly.sh
  • lines changed 95, context: html, text, full: html, text
    2201c5c416938253128eebcac00c0a240574a889 Fri Sep 4 12:47:16 2026 -0700
    docent: nightly cron wrapper for the regression suite, refs #38252 Mails a report every night whether anything failed or not, matching the catalogNightly job: no mail means the cron has stopped, not that the browser is fine. The script mails on its own and always exits 0, so cron adds nothing. Two things it does deliberately: - It runs the COMMITTED tests, listed with git ls-files rather than by globbing the directory. The same directory holds scripts written against a ticket whose recipe is not right yet, and those must not mail a failure every night. A newly committed test is picked up with no edit. - It reports preflight separately from the tests, and preflight is now given an explicit script list (preflight.js takes names after the directory) so a dead fixture belonging to a work-in-progress script is not reported against a run that never included it. A missing session and a returning bug are different news and should not arrive as the same red. Verified: the pass path, the fail path (broke one committed assertion, restored it from git), and a run under `env -i` with only HOME and SHELL set, which is what cron will give it. Installed as `10 4 * * *` in braney's crontab, reading the tests out of the worktree because they are on this branch and not yet on master. It should move to a checkout of its own when the branch lands, so that a day's editing cannot change what the cron measures.
  • lines changed 61, context: html, text, full: html, text
    7f5ea857be45e43cece2cfb37093c6c51b23f79d Sun Sep 6 08:23:15 2026 -0700
    docent: run the nightly regression out of a checkout of its own The header of this script has said since it was written that once the tests were on master the cron should read them out of a clone of its own, the way catalogNightly has one under /hive/users/braney, so that an ordinary day's editing in ~/kent cannot change what the cron measures. The suite landed on master yesterday, so this does it. The clone is /hive/users/braney/docentNightly/kent, beside the log directory the job already writes to. Nothing here is built, so it needs no submodules and no make. --update brings that clone to origin/master and then re-runs this script from the result. The re-exec is why this is a flag here rather than a separate driver script beside the crontab: the whole job stays in the tree where it can be reviewed and committed, and a change to this file takes effect the same night as a change to a test instead of a night later. --update will only ever touch a checkout that is a pristine mirror of origin/master. A working tree with uncommitted edits, or one holding a commit that has not been pushed, is left alone and reported rather than reset, because `reset --hard` in ~/kent would throw away work. That guard is what makes it safe for the flag to live in a script that also sits in a working tree. Both refusals mail with the same subject shape as a finished run, since a night when nothing ran must not read as a quiet night. refs #38252
• src/hg/utils/docent/tests/regress/rm27113.docent.yaml
  • lines changed 56, context: html, text, full: html, text
    2b8d9275548bcc9bd161b68f24848b545e1069ef Sun Sep 6 14:00:28 2026 -0700
    docent: click raw:, and the first two scripts that do anything twice Every one of the 29 scripts in the suite was a straight line: fresh cart, a few steps, assert once. A bug that only exists on the repeat was invisible to that shape, and the closed-bug pool holds several. These are the first two that repeat a gesture. rm36805 click a TOGA item, dismiss the pop-up, click the same item again. One click passes on the broken build as happily as on the fixed one, so the second click is the test. rm27113 three clicks on the centre of the base-position ruler. A single click zooms 3x about the base under the cursor, and the bug drifted that centre one base left each time. Both needed a gesture the language could not express. click: {track, item} follows the item's own map-box href, which is right when the assertion is about the hgc PAGE, but hgTracks answers a real click with an ajax DIALOG (popUpHgcOrHgGene.hgc) and following the href never opens one. raw: true presses the mouse where a user presses it and lets the page answer -- a navigation, a dialog, or a new image in place, whichever arrives, waited for rather than slept through. With no item name it is a bare point on the row, which is the only way to click the ruler at all: the ruler carries no hgc map boxes for areaXY to snap to. Two things measured while writing these, both in the script comments. jQuery UI HIDES a dialog on close rather than removing it, so the assertion has to be has: "#hgcDialog:visible" -- without :visible the second half of rm36805 passes whether or not the pop-up ever comes back. And one base is invisible at the 13kb windows the rest of the suite uses, where a pixel is fourteen bases, so rm27113 starts at 100 bases and its three expected windows are arithmetic rather than three observed strings: every one of them is centred on base 155,806,200. #37014 is the other repeat-click bug in the pool and is deliberately NOT written. Its reproducer no longer reaches the code it was about: 9e9ee32a4c8 (#37878) later excluded crossTissue* tracks from the pop-up path altogether, and the session's track is crossTissueMapsTissueCellType, so the click now navigates and no dialog is involved. Writing it against a different bar chart track would be pinning the ticket to something it was never about. refs #38252
• src/hg/utils/docent/tests/regress/rm35333.docent.yaml
  • lines changed 33, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm35580.docent.yaml
  • lines changed 46, context: html, text, full: html, text
    e497eccfb431be6af38d22c5bc2bb0e745c89b2d Sat Sep 5 09:22:17 2026 -0700
    docent: eight more regression scripts, one per fixed bug rm35580 "Show <alt> placed on its chromosome" errored on the window rm36061 DECIPHER SNVs details missing after a quickLift rm36335 a quickLifted GIAB item click printed a garbled "can't port" rm36340 a search-result click carried the source coordinate rm37491 hgConvert named subtracks the user never showed as not liftable rm37562 a stale lastPosition banner on the lifted search-results page rm37615 a quickLifted bigBed tooltip reported pre-lift coordinates rm37785 a squishyPack track stole a neighbouring track's center label Each asserts the behavior its ticket says is correct, on genome-test, and names in noText:/noUrl: the exact string the buggy build produced rather than a generic "Error". Where the last page has no track image the noText: is paired with a positive text: or url: check, because a bare noText: passes just as happily on a blank page or an early error. Four things found writing these, all of them recorded in the scripts: A ticket's stated position and its stated item can disagree. #36061 says to go to chr7:156,982,676-156,996,015 and click item 517898; that item is at chr7:155,803,191, and decipherSnvs draws nothing at all in the ticket's window. hideKids cannot express "the user deselected this subtrack". It writes a <track>=hide cart variable, and isSubtrackVisible() (hg/lib/trackHub.c) begins with overrideComposite = (NULL != cartOptionalString(cart, tdb->track)) and then forces enabled = TRUE, so the sibling counts as visible to hgConvert and lands back in the "failed to lift" warning. A first rm37491 failed on exactly that and looked like the bug being back. A deselected subtrack is <track>_sel=0 with no visibility variable, which is what hgTrackUi's checkbox writes and what the goto: in that script sets. #37615's fix is not the one the plan for this suite described. The first commit appended a note saying the coordinates came from the source assembly; the second replaced it with the real fix, which puts the TARGET coordinates in the tooltip. A test for the note would fail. decipherSnvs (SNVs) and decipher (CNVs) are two different members of the superTrack decipherContainer, and the row that turns up when the SNVs track draws nothing is the CNVs one, which reads like the same track under a shorter id. refs #38252
• src/hg/utils/docent/tests/regress/rm35865.docent.yaml
  • lines changed 50, context: html, text, full: html, text
    79aa96ab96eaf59b3d4779e60b9e5a4f012884f3 Sun Sep 6 15:04:38 2026 -0700
    docent: three scripts that assert a value rather than the shape of a page 34 scripts, all green. I told Brian this class had no coverage. That was wrong, and the correction matters more than the three scripts: rm36061 already asserts five fields off a Decipher hgc page, rm37615 asserts an exact lifted coordinate and names the source coordinate in noText:, and rm37326 asserts a CRISPR score triple. What is thin is not value assertions, it is value assertions anywhere other than an hgc page reached by clicking an item. These three are each somewhere else. rm37489 a lifted track printed the TEMPLATE of its dataVersion setting, "/gbdb/$D/bbi/clinvar/version.txt", where a release date belongs. The positive check is the prefix "ClinVar Release:" only -- the date after it changes every month and asserting it would fail on the next ClinVar update, which is not a regression. rm36810 trackDb's `urls` statement stopped turning data fields into links. The assertion is the HREF, a value the browser BUILT from the data rather than text it copied, and it asks for /clinvar/RCV specifically: every ClinVar hgc page carries NCBI links in its boilerplate, so a bare ncbi.nlm.nih.gov would have passed on the broken page. Measured: exactly one RCV link on the page, and it is the substituted one. rm35865 hgGene's Microarray Expression Data section was empty. The heading is on the page either way, so the two data set names under it are the check. First script here to open hgGene at all. Three things learned, all in the script comments. A lifted session cannot be reused: Gerardo's RM_37489 comes back as plain hg38 with no hub_ prefix on anything, because a lift dies with the trash it points at, so a script that needs one has to make it. A lifted composite has no hgTrackUi link for its subtrack -- every settings link points at the container, and there are four of them. And hgGene writes its sections with style='display:none', so expect: text cannot see the contents until the + is clicked; a has: written to dodge that would have passed on the empty page the ticket is about. Nothing in these three names a transcript accession or a variant accession. Those are data, and the next knownGene or ClinVar build can retire them. Items are taken by gene symbol or by position instead. refs #38252
• src/hg/utils/docent/tests/regress/rm35920.docent.yaml
  • lines changed 39, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm36029.docent.yaml
  • lines changed 41, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm36061.docent.yaml
  • lines changed 46, context: html, text, full: html, text
    e497eccfb431be6af38d22c5bc2bb0e745c89b2d Sat Sep 5 09:22:17 2026 -0700
    docent: eight more regression scripts, one per fixed bug rm35580 "Show <alt> placed on its chromosome" errored on the window rm36061 DECIPHER SNVs details missing after a quickLift rm36335 a quickLifted GIAB item click printed a garbled "can't port" rm36340 a search-result click carried the source coordinate rm37491 hgConvert named subtracks the user never showed as not liftable rm37562 a stale lastPosition banner on the lifted search-results page rm37615 a quickLifted bigBed tooltip reported pre-lift coordinates rm37785 a squishyPack track stole a neighbouring track's center label Each asserts the behavior its ticket says is correct, on genome-test, and names in noText:/noUrl: the exact string the buggy build produced rather than a generic "Error". Where the last page has no track image the noText: is paired with a positive text: or url: check, because a bare noText: passes just as happily on a blank page or an early error. Four things found writing these, all of them recorded in the scripts: A ticket's stated position and its stated item can disagree. #36061 says to go to chr7:156,982,676-156,996,015 and click item 517898; that item is at chr7:155,803,191, and decipherSnvs draws nothing at all in the ticket's window. hideKids cannot express "the user deselected this subtrack". It writes a <track>=hide cart variable, and isSubtrackVisible() (hg/lib/trackHub.c) begins with overrideComposite = (NULL != cartOptionalString(cart, tdb->track)) and then forces enabled = TRUE, so the sibling counts as visible to hgConvert and lands back in the "failed to lift" warning. A first rm37491 failed on exactly that and looked like the bug being back. A deselected subtrack is <track>_sel=0 with no visibility variable, which is what hgTrackUi's checkbox writes and what the goto: in that script sets. #37615's fix is not the one the plan for this suite described. The first commit appended a note saying the coordinates came from the source assembly; the second replaced it with the real fix, which puts the TARGET coordinates in the tooltip. A test for the note would fail. decipherSnvs (SNVs) and decipher (CNVs) are two different members of the superTrack decipherContainer, and the row that turns up when the SNVs track draws nothing is the CNVs one, which reads like the same track under a shorter id. refs #38252
• src/hg/utils/docent/tests/regress/rm36331.docent.yaml
  • lines changed 37, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm36335.docent.yaml
  • lines changed 55, context: html, text, full: html, text
    e497eccfb431be6af38d22c5bc2bb0e745c89b2d Sat Sep 5 09:22:17 2026 -0700
    docent: eight more regression scripts, one per fixed bug rm35580 "Show <alt> placed on its chromosome" errored on the window rm36061 DECIPHER SNVs details missing after a quickLift rm36335 a quickLifted GIAB item click printed a garbled "can't port" rm36340 a search-result click carried the source coordinate rm37491 hgConvert named subtracks the user never showed as not liftable rm37562 a stale lastPosition banner on the lifted search-results page rm37615 a quickLifted bigBed tooltip reported pre-lift coordinates rm37785 a squishyPack track stole a neighbouring track's center label Each asserts the behavior its ticket says is correct, on genome-test, and names in noText:/noUrl: the exact string the buggy build produced rather than a generic "Error". Where the last page has no track image the noText: is paired with a positive text: or url: check, because a bare noText: passes just as happily on a blank page or an early error. Four things found writing these, all of them recorded in the scripts: A ticket's stated position and its stated item can disagree. #36061 says to go to chr7:156,982,676-156,996,015 and click item 517898; that item is at chr7:155,803,191, and decipherSnvs draws nothing at all in the ticket's window. hideKids cannot express "the user deselected this subtrack". It writes a <track>=hide cart variable, and isSubtrackVisible() (hg/lib/trackHub.c) begins with overrideComposite = (NULL != cartOptionalString(cart, tdb->track)) and then forces enabled = TRUE, so the sibling counts as visible to hgConvert and lands back in the "failed to lift" warning. A first rm37491 failed on exactly that and looked like the bug being back. A deselected subtrack is <track>_sel=0 with no visibility variable, which is what hgTrackUi's checkbox writes and what the goto: in that script sets. #37615's fix is not the one the plan for this suite described. The first commit appended a note saying the coordinates came from the source assembly; the second replaced it with the real fix, which puts the TARGET coordinates in the tooltip. A test for the note would fail. decipherSnvs (SNVs) and decipher (CNVs) are two different members of the superTrack decipherContainer, and the row that turns up when the SNVs track draws nothing is the CNVs one, which reads like the same track under a shorter id. refs #38252
• src/hg/utils/docent/tests/regress/rm36340.docent.yaml
  • lines changed 50, context: html, text, full: html, text
    e497eccfb431be6af38d22c5bc2bb0e745c89b2d Sat Sep 5 09:22:17 2026 -0700
    docent: eight more regression scripts, one per fixed bug rm35580 "Show <alt> placed on its chromosome" errored on the window rm36061 DECIPHER SNVs details missing after a quickLift rm36335 a quickLifted GIAB item click printed a garbled "can't port" rm36340 a search-result click carried the source coordinate rm37491 hgConvert named subtracks the user never showed as not liftable rm37562 a stale lastPosition banner on the lifted search-results page rm37615 a quickLifted bigBed tooltip reported pre-lift coordinates rm37785 a squishyPack track stole a neighbouring track's center label Each asserts the behavior its ticket says is correct, on genome-test, and names in noText:/noUrl: the exact string the buggy build produced rather than a generic "Error". Where the last page has no track image the noText: is paired with a positive text: or url: check, because a bare noText: passes just as happily on a blank page or an early error. Four things found writing these, all of them recorded in the scripts: A ticket's stated position and its stated item can disagree. #36061 says to go to chr7:156,982,676-156,996,015 and click item 517898; that item is at chr7:155,803,191, and decipherSnvs draws nothing at all in the ticket's window. hideKids cannot express "the user deselected this subtrack". It writes a <track>=hide cart variable, and isSubtrackVisible() (hg/lib/trackHub.c) begins with overrideComposite = (NULL != cartOptionalString(cart, tdb->track)) and then forces enabled = TRUE, so the sibling counts as visible to hgConvert and lands back in the "failed to lift" warning. A first rm37491 failed on exactly that and looked like the bug being back. A deselected subtrack is <track>_sel=0 with no visibility variable, which is what hgTrackUi's checkbox writes and what the goto: in that script sets. #37615's fix is not the one the plan for this suite described. The first commit appended a note saying the coordinates came from the source assembly; the second replaced it with the real fix, which puts the TARGET coordinates in the tooltip. A test for the note would fail. decipherSnvs (SNVs) and decipher (CNVs) are two different members of the superTrack decipherContainer, and the row that turns up when the SNVs track draws nothing is the CNVs one, which reads like the same track under a shorter id. refs #38252
• src/hg/utils/docent/tests/regress/rm36387.docent.yaml
  • lines changed 47, context: html, text, full: html, text
    e040e51d90b1b7af95a56c14199e4238e6912b48 Sat Sep 5 09:21:54 2026 -0700
    docent: the four regression scripts that were failing, now green These were written yesterday, failed, and were left out of the last commit so the suite stayed green. None of the four was a browser bug. Each was the test being wrong, and each was wrong in a way worth writing into the script. rm36387 had lost the bug. The term the ticket quotes carries a ZERO-WIDTH SPACE (U+200B) between "):" and "n.46G>A", which is how it arrived from a clinical report; hgTracks passed it to encodeURIComponent unstripped, hgSuggest put it in a LIKE, and MySQL died on a collation mismatch. Without that character there is no bug at all and the test passes on any build ever shipped, so the term is now written as a ​ escape. The fixed behavior is also not what the test assumed: the position box submits to hgSearch, which reports no results for an HGVS term. The old script waited for #imgTbl and timed out. rm37805 sat at a position where the track cannot draw. gnomadGenomesVariantsV3_1_1 carries filterValuesDefault.annot pLoF,missense,synonymous, and all 258 v3.1.1 variants in chr17:43044295-43045295 are annotated "other", so the row does not exist. Moved to chr17:43091000-43092000, inside BRCA1 exon 11, which holds 80 missense, 13 pLoF and 29 synonymous. gnomadVariants is also a superTrack, so hideKids is needed to keep its coverage siblings off. rm37326 hovered into a gap. at: sets x from the coordinate but forces y to the MIDDLE of the row, and in pack mode that track is 324px of stacked guides. It now hovers by title:, which lands on the item's own row and waits for that item's tooltip. rm37553 asserted label text that does not exist. "GFF example" is not a label: the file says `track name=GFF example description=` with the name unquoted, so hgCustom takes the name as "GFF". Nor is the description on the page, since hgTracks puts a longLabel in the control's title attribute, which innerText does not see. All 46 tracks were loading the whole time. refs #38252
• src/hg/utils/docent/tests/regress/rm36514.docent.yaml
  • lines changed 31, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm36540.xfail.docent.yaml
  • lines changed 32, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm36702.docent.yaml
  • lines changed 32, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm36798.docent.yaml
  • lines changed 27, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm36805.docent.yaml
  • lines changed 48, context: html, text, full: html, text
    2b8d9275548bcc9bd161b68f24848b545e1069ef Sun Sep 6 14:00:28 2026 -0700
    docent: click raw:, and the first two scripts that do anything twice Every one of the 29 scripts in the suite was a straight line: fresh cart, a few steps, assert once. A bug that only exists on the repeat was invisible to that shape, and the closed-bug pool holds several. These are the first two that repeat a gesture. rm36805 click a TOGA item, dismiss the pop-up, click the same item again. One click passes on the broken build as happily as on the fixed one, so the second click is the test. rm27113 three clicks on the centre of the base-position ruler. A single click zooms 3x about the base under the cursor, and the bug drifted that centre one base left each time. Both needed a gesture the language could not express. click: {track, item} follows the item's own map-box href, which is right when the assertion is about the hgc PAGE, but hgTracks answers a real click with an ajax DIALOG (popUpHgcOrHgGene.hgc) and following the href never opens one. raw: true presses the mouse where a user presses it and lets the page answer -- a navigation, a dialog, or a new image in place, whichever arrives, waited for rather than slept through. With no item name it is a bare point on the row, which is the only way to click the ruler at all: the ruler carries no hgc map boxes for areaXY to snap to. Two things measured while writing these, both in the script comments. jQuery UI HIDES a dialog on close rather than removing it, so the assertion has to be has: "#hgcDialog:visible" -- without :visible the second half of rm36805 passes whether or not the pop-up ever comes back. And one base is invisible at the 13kb windows the rest of the suite uses, where a pixel is fourteen bases, so rm27113 starts at 100 bases and its three expected windows are arithmetic rather than three observed strings: every one of them is centred on base 155,806,200. #37014 is the other repeat-click bug in the pool and is deliberately NOT written. Its reproducer no longer reaches the code it was about: 9e9ee32a4c8 (#37878) later excluded crossTissue* tracks from the pop-up path altogether, and the session's track is crossTissueMapsTissueCellType, so the click now navigates and no dialog is involved. Writing it against a different bar chart track would be pinning the ticket to something it was never about. refs #38252
• src/hg/utils/docent/tests/regress/rm36810.docent.yaml
  • lines changed 44, context: html, text, full: html, text
    79aa96ab96eaf59b3d4779e60b9e5a4f012884f3 Sun Sep 6 15:04:38 2026 -0700
    docent: three scripts that assert a value rather than the shape of a page 34 scripts, all green. I told Brian this class had no coverage. That was wrong, and the correction matters more than the three scripts: rm36061 already asserts five fields off a Decipher hgc page, rm37615 asserts an exact lifted coordinate and names the source coordinate in noText:, and rm37326 asserts a CRISPR score triple. What is thin is not value assertions, it is value assertions anywhere other than an hgc page reached by clicking an item. These three are each somewhere else. rm37489 a lifted track printed the TEMPLATE of its dataVersion setting, "/gbdb/$D/bbi/clinvar/version.txt", where a release date belongs. The positive check is the prefix "ClinVar Release:" only -- the date after it changes every month and asserting it would fail on the next ClinVar update, which is not a regression. rm36810 trackDb's `urls` statement stopped turning data fields into links. The assertion is the HREF, a value the browser BUILT from the data rather than text it copied, and it asks for /clinvar/RCV specifically: every ClinVar hgc page carries NCBI links in its boilerplate, so a bare ncbi.nlm.nih.gov would have passed on the broken page. Measured: exactly one RCV link on the page, and it is the substituted one. rm35865 hgGene's Microarray Expression Data section was empty. The heading is on the page either way, so the two data set names under it are the check. First script here to open hgGene at all. Three things learned, all in the script comments. A lifted session cannot be reused: Gerardo's RM_37489 comes back as plain hg38 with no hub_ prefix on anything, because a lift dies with the trash it points at, so a script that needs one has to make it. A lifted composite has no hgTrackUi link for its subtrack -- every settings link points at the container, and there are four of them. And hgGene writes its sections with style='display:none', so expect: text cannot see the contents until the + is clicked; a has: written to dodge that would have passed on the empty page the ticket is about. Nothing in these three names a transcript accession or a variant accession. Those are data, and the next knownGene or ClinVar build can retire them. Items are taken by gene symbol or by position instead. refs #38252
• src/hg/utils/docent/tests/regress/rm36836.docent.yaml
  • lines changed 40, context: html, text, full: html, text
    62132f503d586f5172a397772c6ca6af3e8eccfb Sun Sep 6 13:16:05 2026 -0700
    docent: four more regression scripts, three of them watched to fail 29 scripts now, 28 of the 36 tickets that can have one. All green against genome-test. The suite's standing weakness is that almost nothing in it was ever seen to fail on a build that still had its bug, so the whole weight sits on assertion tightness. Three of these four do not have that problem. hgwbeta is running v503 and two of these fixes ship in v504, so the broken behavior was still there to be read this morning and each assertion was checked against it: rm38185 empty CGI pair. hgwbeta lands on chr7:155,799,529-155,812,871, hg38's default, because the position= after the && is silently lost; the same pair at the end returns "Mangled CGI input string &". The assertion is the position itself, since losing a variable produces no error to look for. rm38126 hub description filtering. The fixture hub is ours and deliberately malformed. Every structural check flips: on hgwbeta the chosen id, the script element and the form input are all still in the DOM, and on genome-test the id carries the descPage- prefix and the other two are gone. has:/noHas: rather than text:, because a script element's contents are not in innerText on either build -- a noText: there would have passed everywhere and asserted nothing. The two text checks deliberately do not flip: they are what stops the run passing on a page where the description simply failed to load. rm36836 a GenArk assembly and an assembly hub opened by URL. This one was only ever broken on hgwdev, so genome-test is the only server it could have been caught on. It was fixed by reverting #36835, so what the script guards is a second attempt at that work. rm38108 the caller-supplied upload address. No server left to watch it fail on: the fix is in v503_branch so hgwbeta has it, and the RR is still on v502 and so still vulnerable -- deliberately not driven there, since the reproducer's whole effect is to kill the CGI serving it. hgSession shares the fix but answers the crafted GET with the sign-in page, so it is not in the script. New fixture ~braney/docentFixtures/descFilterHub, for rm38126 only. hubCheck reports its script and form on purpose: that reporting is the other half of #38126, and the hub is not to be tidied up. refs #38252
• src/hg/utils/docent/tests/regress/rm37326.docent.yaml
  • lines changed 34, context: html, text, full: html, text
    e040e51d90b1b7af95a56c14199e4238e6912b48 Sat Sep 5 09:21:54 2026 -0700
    docent: the four regression scripts that were failing, now green These were written yesterday, failed, and were left out of the last commit so the suite stayed green. None of the four was a browser bug. Each was the test being wrong, and each was wrong in a way worth writing into the script. rm36387 had lost the bug. The term the ticket quotes carries a ZERO-WIDTH SPACE (U+200B) between "):" and "n.46G>A", which is how it arrived from a clinical report; hgTracks passed it to encodeURIComponent unstripped, hgSuggest put it in a LIKE, and MySQL died on a collation mismatch. Without that character there is no bug at all and the test passes on any build ever shipped, so the term is now written as a ​ escape. The fixed behavior is also not what the test assumed: the position box submits to hgSearch, which reports no results for an HGVS term. The old script waited for #imgTbl and timed out. rm37805 sat at a position where the track cannot draw. gnomadGenomesVariantsV3_1_1 carries filterValuesDefault.annot pLoF,missense,synonymous, and all 258 v3.1.1 variants in chr17:43044295-43045295 are annotated "other", so the row does not exist. Moved to chr17:43091000-43092000, inside BRCA1 exon 11, which holds 80 missense, 13 pLoF and 29 synonymous. gnomadVariants is also a superTrack, so hideKids is needed to keep its coverage siblings off. rm37326 hovered into a gap. at: sets x from the coordinate but forces y to the MIDDLE of the row, and in pack mode that track is 324px of stacked guides. It now hovers by title:, which lands on the item's own row and waits for that item's tooltip. rm37553 asserted label text that does not exist. "GFF example" is not a label: the file says `track name=GFF example description=` with the name unquoted, so hgCustom takes the name as "GFF". Nor is the description on the page, since hgTracks puts a longLabel in the control's title attribute, which innerText does not see. All 46 tracks were loading the whole time. refs #38252
• src/hg/utils/docent/tests/regress/rm37388.docent.yaml
  • lines changed 30, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm37389.docent.yaml
  • lines changed 0, context: html, text, full: html, text
    12d4ad442f780318c6a9a7bd93ae64da5019b689 Sat Sep 5 11:22:24 2026 -0700
    docent: rm37389 is no longer an xfail, the bug it pins is fixed #38275 was the second cause of #37389's symptom: 1258d7f65e7 made trackHubAddOneDescription return early rather than assign a failed fetch, so tdb->html kept a non-NULL value and hgc's `tdb->html == NULL` test stopped reaching the quickLift fallback. Fixed on master in 4be6477ebcb, pushed, and verified on genome-test once it was rebuilt: the lifted details page is back to 3047 characters, the same as the RR. So the script started passing and the .xfail had to go, which is the signal the makefile was built to give. Renamed, and the comment now records both causes - the v498 fix that works and the commit that defeated it - because a reader who sees only one of them will not understand why the script is worded as it is. The assertion is tighter than it was. "Description" alone is the heading printTrackHtml writes; the script now also asks for a phrase out of the hub's own description file, which is the only thing that cannot be on the page unless the file was really fetched from the GenArk hub. This is the one script in the suite that has watched its behaviour break and come back, rather than only asserting a fix it never saw fail. refs #38252
• src/hg/utils/docent/tests/regress/rm37389.xfail.docent.yaml
  • lines changed 60, context: html, text, full: html, text
    c6b4a4bd0e3024efed0f59f21dc01b5aa3c351f1 Sat Sep 5 09:22:30 2026 -0700
    docent: pin two Closed tickets whose symptom is still live, as xfail Both were measured on genome-test (v503) on 2026-09-05, both are Closed and out on the RR, and both are pinned rather than deleted so the suite reports it when they start passing. Same treatment as rm36540. rm38272 is NOT a second copy of rm37388. That script covers hgc, where the fix landed and works. This one follows the same lifted track's own settings link, and hgTrackUi still does what #37388 described: Couldn't set connection database to GCA_018466835.2 mySQL error 1049: Unknown database 'GCA_018466835.2' #37388's fix was per-CGI. 921a40c472e guarded the one call in hgc (hgc.c:4984, !trackHubDatabase(liftDb) && !isGenArk(liftDb) before hAllocConnTrack) and the branch in getTrackHtml. hgTrackUi has a path of its own that was never guarded: specificUi() (hgTrackUi.c:3432) reassigns db to the quickLiftDb setting and hands it to cfgByCfgType(), labelCfgUi() and extraUiLinks(), which connect to it. Where the page stops fits -- specificUi() is called at :4202, after the "Remove from QuickLift" link at :4118, and that link is the last thing the 451-character page contains. It reproduces from scratch with no saved session and no hub of ours, and it is live on genome.ucsc.edu, hgwbeta and genome-test alike: open GenArk GCA_018466835.2 at CM089257.1:80,200,000-80,360,000, leave the default tracks alone, QuickLift to hg38, click the gear on RefSeq mRNAs. Filed as #38272. The script uses braney/crash1 instead because that is the cheap way to the same state and rm37388 already depends on that session. rm37389 asserts the track description block that is still absent from a quickLifted hgc page whose source is a GenArk. Four things were checked rather than assumed and none of them explains it away: GCA_018466835.2 is in the genark table, the hub declares its xenoRefGene html file, that file is served 200 from hgdownload, and the fix (5aed2d465f1) is still in hg/lib/hui.c. hgc prints no warning of its own, which points at getHtmlFromSelfOrParent (hgc.c:3715) never calling getTrackHtml, since it only does so `if (liftDb && ...)`. Not chased further. This is a different path from the hgTrackUi one above, not the same bug seen twice. The verification session on the ticket does not settle it either: the only lifted track with clickable items at its position is a bigWig, whose hgc page has no description section in any build. refs #38252
• src/hg/utils/docent/tests/regress/rm37489.docent.yaml
  • lines changed 50, context: html, text, full: html, text
    79aa96ab96eaf59b3d4779e60b9e5a4f012884f3 Sun Sep 6 15:04:38 2026 -0700
    docent: three scripts that assert a value rather than the shape of a page 34 scripts, all green. I told Brian this class had no coverage. That was wrong, and the correction matters more than the three scripts: rm36061 already asserts five fields off a Decipher hgc page, rm37615 asserts an exact lifted coordinate and names the source coordinate in noText:, and rm37326 asserts a CRISPR score triple. What is thin is not value assertions, it is value assertions anywhere other than an hgc page reached by clicking an item. These three are each somewhere else. rm37489 a lifted track printed the TEMPLATE of its dataVersion setting, "/gbdb/$D/bbi/clinvar/version.txt", where a release date belongs. The positive check is the prefix "ClinVar Release:" only -- the date after it changes every month and asserting it would fail on the next ClinVar update, which is not a regression. rm36810 trackDb's `urls` statement stopped turning data fields into links. The assertion is the HREF, a value the browser BUILT from the data rather than text it copied, and it asks for /clinvar/RCV specifically: every ClinVar hgc page carries NCBI links in its boilerplate, so a bare ncbi.nlm.nih.gov would have passed on the broken page. Measured: exactly one RCV link on the page, and it is the substituted one. rm35865 hgGene's Microarray Expression Data section was empty. The heading is on the page either way, so the two data set names under it are the check. First script here to open hgGene at all. Three things learned, all in the script comments. A lifted session cannot be reused: Gerardo's RM_37489 comes back as plain hg38 with no hub_ prefix on anything, because a lift dies with the trash it points at, so a script that needs one has to make it. A lifted composite has no hgTrackUi link for its subtrack -- every settings link points at the container, and there are four of them. And hgGene writes its sections with style='display:none', so expect: text cannot see the contents until the + is clicked; a has: written to dodge that would have passed on the empty page the ticket is about. Nothing in these three names a transcript accession or a variant accession. Those are data, and the next knownGene or ClinVar build can retire them. Items are taken by gene symbol or by position instead. refs #38252
• src/hg/utils/docent/tests/regress/rm37491.docent.yaml
  • lines changed 60, context: html, text, full: html, text
    e497eccfb431be6af38d22c5bc2bb0e745c89b2d Sat Sep 5 09:22:17 2026 -0700
    docent: eight more regression scripts, one per fixed bug rm35580 "Show <alt> placed on its chromosome" errored on the window rm36061 DECIPHER SNVs details missing after a quickLift rm36335 a quickLifted GIAB item click printed a garbled "can't port" rm36340 a search-result click carried the source coordinate rm37491 hgConvert named subtracks the user never showed as not liftable rm37562 a stale lastPosition banner on the lifted search-results page rm37615 a quickLifted bigBed tooltip reported pre-lift coordinates rm37785 a squishyPack track stole a neighbouring track's center label Each asserts the behavior its ticket says is correct, on genome-test, and names in noText:/noUrl: the exact string the buggy build produced rather than a generic "Error". Where the last page has no track image the noText: is paired with a positive text: or url: check, because a bare noText: passes just as happily on a blank page or an early error. Four things found writing these, all of them recorded in the scripts: A ticket's stated position and its stated item can disagree. #36061 says to go to chr7:156,982,676-156,996,015 and click item 517898; that item is at chr7:155,803,191, and decipherSnvs draws nothing at all in the ticket's window. hideKids cannot express "the user deselected this subtrack". It writes a <track>=hide cart variable, and isSubtrackVisible() (hg/lib/trackHub.c) begins with overrideComposite = (NULL != cartOptionalString(cart, tdb->track)) and then forces enabled = TRUE, so the sibling counts as visible to hgConvert and lands back in the "failed to lift" warning. A first rm37491 failed on exactly that and looked like the bug being back. A deselected subtrack is <track>_sel=0 with no visibility variable, which is what hgTrackUi's checkbox writes and what the goto: in that script sets. #37615's fix is not the one the plan for this suite described. The first commit appended a note saying the coordinates came from the source assembly; the second replaced it with the real fix, which puts the TARGET coordinates in the tooltip. A test for the note would fail. decipherSnvs (SNVs) and decipher (CNVs) are two different members of the superTrack decipherContainer, and the row that turns up when the SNVs track draws nothing is the CNVs one, which reads like the same track under a shorter id. refs #38252
• src/hg/utils/docent/tests/regress/rm37520.docent.yaml
  • lines changed 32, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm37553.docent.yaml
  • lines changed 47, context: html, text, full: html, text
    e040e51d90b1b7af95a56c14199e4238e6912b48 Sat Sep 5 09:21:54 2026 -0700
    docent: the four regression scripts that were failing, now green These were written yesterday, failed, and were left out of the last commit so the suite stayed green. None of the four was a browser bug. Each was the test being wrong, and each was wrong in a way worth writing into the script. rm36387 had lost the bug. The term the ticket quotes carries a ZERO-WIDTH SPACE (U+200B) between "):" and "n.46G>A", which is how it arrived from a clinical report; hgTracks passed it to encodeURIComponent unstripped, hgSuggest put it in a LIKE, and MySQL died on a collation mismatch. Without that character there is no bug at all and the test passes on any build ever shipped, so the term is now written as a ​ escape. The fixed behavior is also not what the test assumed: the position box submits to hgSearch, which reports no results for an HGVS term. The old script waited for #imgTbl and timed out. rm37805 sat at a position where the track cannot draw. gnomadGenomesVariantsV3_1_1 carries filterValuesDefault.annot pLoF,missense,synonymous, and all 258 v3.1.1 variants in chr17:43044295-43045295 are annotated "other", so the row does not exist. Moved to chr17:43091000-43092000, inside BRCA1 exon 11, which holds 80 missense, 13 pLoF and 29 synonymous. gnomadVariants is also a superTrack, so hideKids is needed to keep its coverage siblings off. rm37326 hovered into a gap. at: sets x from the coordinate but forces y to the MIDDLE of the row, and in pack mode that track is 324px of stacked guides. It now hovers by title:, which lands on the item's own row and waits for that item's tooltip. rm37553 asserted label text that does not exist. "GFF example" is not a label: the file says `track name=GFF example description=` with the name unquoted, so hgCustom takes the name as "GFF". Nor is the description on the page, since hgTracks puts a longLabel in the control's title attribute, which innerText does not see. All 46 tracks were loading the whole time. refs #38252
• src/hg/utils/docent/tests/regress/rm37562.docent.yaml
  • lines changed 46, context: html, text, full: html, text
    e497eccfb431be6af38d22c5bc2bb0e745c89b2d Sat Sep 5 09:22:17 2026 -0700
    docent: eight more regression scripts, one per fixed bug rm35580 "Show <alt> placed on its chromosome" errored on the window rm36061 DECIPHER SNVs details missing after a quickLift rm36335 a quickLifted GIAB item click printed a garbled "can't port" rm36340 a search-result click carried the source coordinate rm37491 hgConvert named subtracks the user never showed as not liftable rm37562 a stale lastPosition banner on the lifted search-results page rm37615 a quickLifted bigBed tooltip reported pre-lift coordinates rm37785 a squishyPack track stole a neighbouring track's center label Each asserts the behavior its ticket says is correct, on genome-test, and names in noText:/noUrl: the exact string the buggy build produced rather than a generic "Error". Where the last page has no track image the noText: is paired with a positive text: or url: check, because a bare noText: passes just as happily on a blank page or an early error. Four things found writing these, all of them recorded in the scripts: A ticket's stated position and its stated item can disagree. #36061 says to go to chr7:156,982,676-156,996,015 and click item 517898; that item is at chr7:155,803,191, and decipherSnvs draws nothing at all in the ticket's window. hideKids cannot express "the user deselected this subtrack". It writes a <track>=hide cart variable, and isSubtrackVisible() (hg/lib/trackHub.c) begins with overrideComposite = (NULL != cartOptionalString(cart, tdb->track)) and then forces enabled = TRUE, so the sibling counts as visible to hgConvert and lands back in the "failed to lift" warning. A first rm37491 failed on exactly that and looked like the bug being back. A deselected subtrack is <track>_sel=0 with no visibility variable, which is what hgTrackUi's checkbox writes and what the goto: in that script sets. #37615's fix is not the one the plan for this suite described. The first commit appended a note saying the coordinates came from the source assembly; the second replaced it with the real fix, which puts the TARGET coordinates in the tooltip. A test for the note would fail. decipherSnvs (SNVs) and decipher (CNVs) are two different members of the superTrack decipherContainer, and the row that turns up when the SNVs track draws nothing is the CNVs one, which reads like the same track under a shorter id. refs #38252
• src/hg/utils/docent/tests/regress/rm37615.docent.yaml
  • lines changed 45, context: html, text, full: html, text
    e497eccfb431be6af38d22c5bc2bb0e745c89b2d Sat Sep 5 09:22:17 2026 -0700
    docent: eight more regression scripts, one per fixed bug rm35580 "Show <alt> placed on its chromosome" errored on the window rm36061 DECIPHER SNVs details missing after a quickLift rm36335 a quickLifted GIAB item click printed a garbled "can't port" rm36340 a search-result click carried the source coordinate rm37491 hgConvert named subtracks the user never showed as not liftable rm37562 a stale lastPosition banner on the lifted search-results page rm37615 a quickLifted bigBed tooltip reported pre-lift coordinates rm37785 a squishyPack track stole a neighbouring track's center label Each asserts the behavior its ticket says is correct, on genome-test, and names in noText:/noUrl: the exact string the buggy build produced rather than a generic "Error". Where the last page has no track image the noText: is paired with a positive text: or url: check, because a bare noText: passes just as happily on a blank page or an early error. Four things found writing these, all of them recorded in the scripts: A ticket's stated position and its stated item can disagree. #36061 says to go to chr7:156,982,676-156,996,015 and click item 517898; that item is at chr7:155,803,191, and decipherSnvs draws nothing at all in the ticket's window. hideKids cannot express "the user deselected this subtrack". It writes a <track>=hide cart variable, and isSubtrackVisible() (hg/lib/trackHub.c) begins with overrideComposite = (NULL != cartOptionalString(cart, tdb->track)) and then forces enabled = TRUE, so the sibling counts as visible to hgConvert and lands back in the "failed to lift" warning. A first rm37491 failed on exactly that and looked like the bug being back. A deselected subtrack is <track>_sel=0 with no visibility variable, which is what hgTrackUi's checkbox writes and what the goto: in that script sets. #37615's fix is not the one the plan for this suite described. The first commit appended a note saying the coordinates came from the source assembly; the second replaced it with the real fix, which puts the TARGET coordinates in the tooltip. A test for the note would fail. decipherSnvs (SNVs) and decipher (CNVs) are two different members of the superTrack decipherContainer, and the row that turns up when the SNVs track draws nothing is the CNVs one, which reads like the same track under a shorter id. refs #38252
• src/hg/utils/docent/tests/regress/rm37785.docent.yaml
  • lines changed 47, context: html, text, full: html, text
    e497eccfb431be6af38d22c5bc2bb0e745c89b2d Sat Sep 5 09:22:17 2026 -0700
    docent: eight more regression scripts, one per fixed bug rm35580 "Show <alt> placed on its chromosome" errored on the window rm36061 DECIPHER SNVs details missing after a quickLift rm36335 a quickLifted GIAB item click printed a garbled "can't port" rm36340 a search-result click carried the source coordinate rm37491 hgConvert named subtracks the user never showed as not liftable rm37562 a stale lastPosition banner on the lifted search-results page rm37615 a quickLifted bigBed tooltip reported pre-lift coordinates rm37785 a squishyPack track stole a neighbouring track's center label Each asserts the behavior its ticket says is correct, on genome-test, and names in noText:/noUrl: the exact string the buggy build produced rather than a generic "Error". Where the last page has no track image the noText: is paired with a positive text: or url: check, because a bare noText: passes just as happily on a blank page or an early error. Four things found writing these, all of them recorded in the scripts: A ticket's stated position and its stated item can disagree. #36061 says to go to chr7:156,982,676-156,996,015 and click item 517898; that item is at chr7:155,803,191, and decipherSnvs draws nothing at all in the ticket's window. hideKids cannot express "the user deselected this subtrack". It writes a <track>=hide cart variable, and isSubtrackVisible() (hg/lib/trackHub.c) begins with overrideComposite = (NULL != cartOptionalString(cart, tdb->track)) and then forces enabled = TRUE, so the sibling counts as visible to hgConvert and lands back in the "failed to lift" warning. A first rm37491 failed on exactly that and looked like the bug being back. A deselected subtrack is <track>_sel=0 with no visibility variable, which is what hgTrackUi's checkbox writes and what the goto: in that script sets. #37615's fix is not the one the plan for this suite described. The first commit appended a note saying the coordinates came from the source assembly; the second replaced it with the real fix, which puts the TARGET coordinates in the tooltip. A test for the note would fail. decipherSnvs (SNVs) and decipher (CNVs) are two different members of the superTrack decipherContainer, and the row that turns up when the SNVs track draws nothing is the CNVs one, which reads like the same track under a shorter id. refs #38252
• src/hg/utils/docent/tests/regress/rm37805.docent.yaml
  • lines changed 42, context: html, text, full: html, text
    e040e51d90b1b7af95a56c14199e4238e6912b48 Sat Sep 5 09:21:54 2026 -0700
    docent: the four regression scripts that were failing, now green These were written yesterday, failed, and were left out of the last commit so the suite stayed green. None of the four was a browser bug. Each was the test being wrong, and each was wrong in a way worth writing into the script. rm36387 had lost the bug. The term the ticket quotes carries a ZERO-WIDTH SPACE (U+200B) between "):" and "n.46G>A", which is how it arrived from a clinical report; hgTracks passed it to encodeURIComponent unstripped, hgSuggest put it in a LIKE, and MySQL died on a collation mismatch. Without that character there is no bug at all and the test passes on any build ever shipped, so the term is now written as a ​ escape. The fixed behavior is also not what the test assumed: the position box submits to hgSearch, which reports no results for an HGVS term. The old script waited for #imgTbl and timed out. rm37805 sat at a position where the track cannot draw. gnomadGenomesVariantsV3_1_1 carries filterValuesDefault.annot pLoF,missense,synonymous, and all 258 v3.1.1 variants in chr17:43044295-43045295 are annotated "other", so the row does not exist. Moved to chr17:43091000-43092000, inside BRCA1 exon 11, which holds 80 missense, 13 pLoF and 29 synonymous. gnomadVariants is also a superTrack, so hideKids is needed to keep its coverage siblings off. rm37326 hovered into a gap. at: sets x from the coordinate but forces y to the MIDDLE of the row, and in pack mode that track is 324px of stacked guides. It now hovers by title:, which lands on the item's own row and waits for that item's tooltip. rm37553 asserted label text that does not exist. "GFF example" is not a label: the file says `track name=GFF example description=` with the name unquoted, so hgCustom takes the name as "GFF". Nor is the description on the page, since hgTracks puts a longLabel in the control's title attribute, which innerText does not see. All 46 tracks were loading the whole time. refs #38252
• src/hg/utils/docent/tests/regress/rm37906.docent.yaml
  • lines changed 37, context: html, text, full: html, text
    443d7aad57553f2606be41a934cb4dab46019e09 Fri Sep 4 12:10:00 2026 -0700
    docent: first eleven regression tests, one per fixed bug, refs #38252 Ten assert the behavior their ticket says is correct; the eleventh is an xfail. All eleven pass against genome-test, 40s for the set. rm35333 bigBed schema vs trackDb type mismatch, no SEGFAULT rm35920 wrong bigBed type in a hub, no crash and no garbage item label rm36029 a MAF displays after several phyloP tracks are on rm36331 quickLift of the GENCODE Archive container, no "Unknown database" rm36514 a chromosome search after a quickLift hop rm36702 quickLift hg19 to hs1, no "Unknown database" rm36798 the two OMIM tracks survive a configure submit rm37388 hgc on a quickLifted hub item, no connect to the source assembly rm37520 lifted tracks survive a hop to another genome and back rm37906 Neandertal tracks draw data in a narrow window on hg18 rm36540 is the xfail, and it is worth reading. The ticket is Closed, but the symptom is present on genome.ucsc.edu (v502), hgwbeta (v503) and genome-test (v503), measured with the reporter's own hub URL as well as with our copy of it: genome= instead of db= for a hub-backed assembly still reaches a query against a chromInfo table that does not exist. There is no fixed behavior to assert, so it is pinned as an xfail and the run fails if it ever starts passing. Six of these build their own state rather than loading the session their ticket names, which is deliberate: a session on someone's account can be renamed or deleted, and hgTracks answers a missing session with a 200 and an early-error page that every noText: assertion passes on. Where a session is genuinely the cheapest way to a state (rm37388), the script also asserts noText: "Could not find session" so that a deleted session fails loudly instead of quietly. Four traps cost a run each and are written into the scripts that hit them, since the next twenty-nine will hit them too: a container never gets an img_data_ row (the rows carry its children's names); a leaf track with no features at the ticket's position has no row either, so a lifted view is better checked by its own quickLiftChain; `track:` sends a plain name and cannot turn on a track in an ATTACHED hub, whose cart variable carries the hub prefix; and asking for one view of a composite turns its sibling views on as well.
• src/hg/utils/docent/tests/regress/rm38108.docent.yaml
  • lines changed 57, context: html, text, full: html, text
    62132f503d586f5172a397772c6ca6af3e8eccfb Sun Sep 6 13:16:05 2026 -0700
    docent: four more regression scripts, three of them watched to fail 29 scripts now, 28 of the 36 tickets that can have one. All green against genome-test. The suite's standing weakness is that almost nothing in it was ever seen to fail on a build that still had its bug, so the whole weight sits on assertion tightness. Three of these four do not have that problem. hgwbeta is running v503 and two of these fixes ship in v504, so the broken behavior was still there to be read this morning and each assertion was checked against it: rm38185 empty CGI pair. hgwbeta lands on chr7:155,799,529-155,812,871, hg38's default, because the position= after the && is silently lost; the same pair at the end returns "Mangled CGI input string &". The assertion is the position itself, since losing a variable produces no error to look for. rm38126 hub description filtering. The fixture hub is ours and deliberately malformed. Every structural check flips: on hgwbeta the chosen id, the script element and the form input are all still in the DOM, and on genome-test the id carries the descPage- prefix and the other two are gone. has:/noHas: rather than text:, because a script element's contents are not in innerText on either build -- a noText: there would have passed everywhere and asserted nothing. The two text checks deliberately do not flip: they are what stops the run passing on a page where the description simply failed to load. rm36836 a GenArk assembly and an assembly hub opened by URL. This one was only ever broken on hgwdev, so genome-test is the only server it could have been caught on. It was fixed by reverting #36835, so what the script guards is a second attempt at that work. rm38108 the caller-supplied upload address. No server left to watch it fail on: the fix is in v503_branch so hgwbeta has it, and the RR is still on v502 and so still vulnerable -- deliberately not driven there, since the reproducer's whole effect is to kill the CGI serving it. hgSession shares the fix but answers the crafted GET with the sign-in page, so it is not in the script. New fixture ~braney/docentFixtures/descFilterHub, for rm38126 only. hubCheck reports its script and form on purpose: that reporting is the other half of #38126, and the hub is not to be tidied up. refs #38252
• src/hg/utils/docent/tests/regress/rm38126.docent.yaml
  • lines changed 61, context: html, text, full: html, text
    62132f503d586f5172a397772c6ca6af3e8eccfb Sun Sep 6 13:16:05 2026 -0700
    docent: four more regression scripts, three of them watched to fail 29 scripts now, 28 of the 36 tickets that can have one. All green against genome-test. The suite's standing weakness is that almost nothing in it was ever seen to fail on a build that still had its bug, so the whole weight sits on assertion tightness. Three of these four do not have that problem. hgwbeta is running v503 and two of these fixes ship in v504, so the broken behavior was still there to be read this morning and each assertion was checked against it: rm38185 empty CGI pair. hgwbeta lands on chr7:155,799,529-155,812,871, hg38's default, because the position= after the && is silently lost; the same pair at the end returns "Mangled CGI input string &". The assertion is the position itself, since losing a variable produces no error to look for. rm38126 hub description filtering. The fixture hub is ours and deliberately malformed. Every structural check flips: on hgwbeta the chosen id, the script element and the form input are all still in the DOM, and on genome-test the id carries the descPage- prefix and the other two are gone. has:/noHas: rather than text:, because a script element's contents are not in innerText on either build -- a noText: there would have passed everywhere and asserted nothing. The two text checks deliberately do not flip: they are what stops the run passing on a page where the description simply failed to load. rm36836 a GenArk assembly and an assembly hub opened by URL. This one was only ever broken on hgwdev, so genome-test is the only server it could have been caught on. It was fixed by reverting #36835, so what the script guards is a second attempt at that work. rm38108 the caller-supplied upload address. No server left to watch it fail on: the fix is in v503_branch so hgwbeta has it, and the RR is still on v502 and so still vulnerable -- deliberately not driven there, since the reproducer's whole effect is to kill the CGI serving it. hgSession shares the fix but answers the crafted GET with the sign-in page, so it is not in the script. New fixture ~braney/docentFixtures/descFilterHub, for rm38126 only. hubCheck reports its script and form on purpose: that reporting is the other half of #38126, and the hub is not to be tidied up. refs #38252
• src/hg/utils/docent/tests/regress/rm38185.docent.yaml
  • lines changed 48, context: html, text, full: html, text
    62132f503d586f5172a397772c6ca6af3e8eccfb Sun Sep 6 13:16:05 2026 -0700
    docent: four more regression scripts, three of them watched to fail 29 scripts now, 28 of the 36 tickets that can have one. All green against genome-test. The suite's standing weakness is that almost nothing in it was ever seen to fail on a build that still had its bug, so the whole weight sits on assertion tightness. Three of these four do not have that problem. hgwbeta is running v503 and two of these fixes ship in v504, so the broken behavior was still there to be read this morning and each assertion was checked against it: rm38185 empty CGI pair. hgwbeta lands on chr7:155,799,529-155,812,871, hg38's default, because the position= after the && is silently lost; the same pair at the end returns "Mangled CGI input string &". The assertion is the position itself, since losing a variable produces no error to look for. rm38126 hub description filtering. The fixture hub is ours and deliberately malformed. Every structural check flips: on hgwbeta the chosen id, the script element and the form input are all still in the DOM, and on genome-test the id carries the descPage- prefix and the other two are gone. has:/noHas: rather than text:, because a script element's contents are not in innerText on either build -- a noText: there would have passed everywhere and asserted nothing. The two text checks deliberately do not flip: they are what stops the run passing on a page where the description simply failed to load. rm36836 a GenArk assembly and an assembly hub opened by URL. This one was only ever broken on hgwdev, so genome-test is the only server it could have been caught on. It was fixed by reverting #36835, so what the script guards is a second attempt at that work. rm38108 the caller-supplied upload address. No server left to watch it fail on: the fix is in v503_branch so hgwbeta has it, and the RR is still on v502 and so still vulnerable -- deliberately not driven there, since the reproducer's whole effect is to kill the CGI serving it. hgSession shares the fix but answers the crafted GET with the sign-in page, so it is not in the script. New fixture ~braney/docentFixtures/descFilterHub, for rm38126 only. hubCheck reports its script and form on purpose: that reporting is the other half of #38126, and the hub is not to be tidied up. refs #38252
• src/hg/utils/docent/tests/regress/rm38272.docent.yaml
  • lines changed 54, context: html, text, full: html, text
    7f68cb2a6a108e5b09074ce5a08c3b6a78509fe9 Sun Sep 6 07:31:02 2026 -0700
    docent: rm38272 is no longer an xfail, the bug it pins is fixed The nightly run mailed FAIL this morning with 24 ok and rm38272.xfail passing, which is the signal the makefile was built to give. cdb348c5512 added the isGenArk() guard to asForDb() and to two trackDbFilter blocks in hui.c, so hgTrackUi no longer tries a MySQL connect to a quickLifted GenArk source assembly. genome-test picked it up overnight. Renamed, and the comment now records the fix rather than the symptom. The assertions are tighter than they were, and for the same reason rm37389's were. "RefSeq mRNAs Track Settings" was the only positive check, and the 451-character broken page carries that heading too, so it never told the two apart -- the noText on the MySQL warning was doing all the work. The checks are now on "Color track by codons" and "Data schema/format description and download", which come from cfgByCfgType() and extraUiLinks(), two of the three routines that were handed the bad db, and neither is in the broken page. Both pages were measured on 2026-09-06, not inferred: hgwbeta still returned the 451-character page with the warning while genome-test returned 2800 characters with none. The fix ships in v504. refs #38252
• src/hg/utils/docent/tests/regress/rm38272.xfail.docent.yaml
  • lines changed 59, context: html, text, full: html, text
    c6b4a4bd0e3024efed0f59f21dc01b5aa3c351f1 Sat Sep 5 09:22:30 2026 -0700
    docent: pin two Closed tickets whose symptom is still live, as xfail Both were measured on genome-test (v503) on 2026-09-05, both are Closed and out on the RR, and both are pinned rather than deleted so the suite reports it when they start passing. Same treatment as rm36540. rm38272 is NOT a second copy of rm37388. That script covers hgc, where the fix landed and works. This one follows the same lifted track's own settings link, and hgTrackUi still does what #37388 described: Couldn't set connection database to GCA_018466835.2 mySQL error 1049: Unknown database 'GCA_018466835.2' #37388's fix was per-CGI. 921a40c472e guarded the one call in hgc (hgc.c:4984, !trackHubDatabase(liftDb) && !isGenArk(liftDb) before hAllocConnTrack) and the branch in getTrackHtml. hgTrackUi has a path of its own that was never guarded: specificUi() (hgTrackUi.c:3432) reassigns db to the quickLiftDb setting and hands it to cfgByCfgType(), labelCfgUi() and extraUiLinks(), which connect to it. Where the page stops fits -- specificUi() is called at :4202, after the "Remove from QuickLift" link at :4118, and that link is the last thing the 451-character page contains. It reproduces from scratch with no saved session and no hub of ours, and it is live on genome.ucsc.edu, hgwbeta and genome-test alike: open GenArk GCA_018466835.2 at CM089257.1:80,200,000-80,360,000, leave the default tracks alone, QuickLift to hg38, click the gear on RefSeq mRNAs. Filed as #38272. The script uses braney/crash1 instead because that is the cheap way to the same state and rm37388 already depends on that session. rm37389 asserts the track description block that is still absent from a quickLifted hgc page whose source is a GenArk. Four things were checked rather than assumed and none of them explains it away: GCA_018466835.2 is in the genark table, the hub declares its xenoRefGene html file, that file is served 200 from hgdownload, and the fix (5aed2d465f1) is still in hg/lib/hui.c. hgc prints no warning of its own, which points at getHtmlFromSelfOrParent (hgc.c:3715) never calling getTrackHtml, since it only does so `if (liftDb && ...)`. Not chased further. This is a different path from the hgTrackUi one above, not the same bug seen twice. The verification session on the ticket does not settle it either: the only lifted track with clickable items at its position is a bigWig, whose hgc page has no description section in any build. refs #38252
  • lines changed 59, context: html, text, full: html, text
    7f68cb2a6a108e5b09074ce5a08c3b6a78509fe9 Sun Sep 6 07:31:02 2026 -0700
    docent: rm38272 is no longer an xfail, the bug it pins is fixed The nightly run mailed FAIL this morning with 24 ok and rm38272.xfail passing, which is the signal the makefile was built to give. cdb348c5512 added the isGenArk() guard to asForDb() and to two trackDbFilter blocks in hui.c, so hgTrackUi no longer tries a MySQL connect to a quickLifted GenArk source assembly. genome-test picked it up overnight. Renamed, and the comment now records the fix rather than the symptom. The assertions are tighter than they were, and for the same reason rm37389's were. "RefSeq mRNAs Track Settings" was the only positive check, and the 451-character broken page carries that heading too, so it never told the two apart -- the noText on the MySQL warning was doing all the work. The checks are now on "Color track by codons" and "Data schema/format description and download", which come from cfgByCfgType() and extraUiLinks(), two of the three routines that were handed the bad db, and neither is in the broken page. Both pages were measured on 2026-09-06, not inferred: hgwbeta still returned the 451-character page with the warning while genome-test returned 2800 characters with none. The fix ships in v504. refs #38252
• src/hg/utils/hgConfCatalog/hgConfCatalog.py
  • lines changed 7, context: html, text, full: html, text
    9f5a3871b95f595b5a218f0d05395a8c72d58844 Tue Aug 25 12:57:10 2026 -0700
    remove the bigBedOnePath fallback, leaving a single bigBed load path, refs #36940 bigBedOnePath has defaulted to on since v492 and the setting has been removed from the hg.conf of the RR, Euro, Asia and hgwbeta. Drop the check and the code it guarded at all four sites: the field-count default in bigBedAddLinkedFeaturesFromExt, the method setup in commonBigBedMethods, the bedSize default in bigBedClick, and the bigBed case in hgc. The two branches in the hgc case differed only in a minimum the surviving path does not want. complexBedMethods lost its only isBigBed=TRUE caller with that branch, so drop the parameter. Retire the setting from the hg.conf catalog and its gate backlog, and repair the two hgc.c citations whose line numbers this commit shifted, refs #37925. Verified pixel-identical: 14 scenarios rendered before and after, covering the four QA sessions on the ticket plus mm10 knownGene, dbSnp155, rmsk, encRegTfbsClustered, clinvar and tandemDups. All AE=0.
  • lines changed 57, context: html, text, full: html, text
    d44a9daabcca7a322db6907add7ccac336394c85 Mon Aug 31 12:21:36 2026 -0700
    Merge branch 'onePath36940': remove the bigBedOnePath fallback, refs #36940 Brings in the single bigBed load path. bigBedOnePath has defaulted to on since v492, so this drops the check and the code it guarded at four sites: bigBedAddLinkedFeaturesFromExt, commonBigBedMethods, bigBedClick and the bigBed case in hgc. complexBedMethods lost its only isBigBed=TRUE caller, so the parameter is gone. Conflict in hgConfCatalog.py, resolved in favour of master. Master has since changed every citation to name the file without a line number, so the branch's two hgc.c line-number repairs are obsolete and are dropped. What survives from the branch is the part that still applies: the bigBedOnePath row leaves the catalog and the gate backlog, refs #37925. Verified after the merge: hgTracks and hgc both build clean under -Wall -Werror, and hgConfCatalog.py --check reports 0 problems.
  • lines changed 7, context: html, text, full: html, text
    520b1f069967ad1a0a0d39eb3a30acc0ab99fb42 Thu Sep 3 02:30:35 2026 -0700
    hgConfCatalog: register snapshot.ttlDays, refs #37925 Written by nightlyRegister.sh, which records the settings the tree reads that the catalog was missing. Only facts copied off the call site are filled in. No classification is guessed: a new boolean gets no role=, because calling a release gate a knob would hide it from the sunset report for good, and every row lands in the 'Awaiting review' section until somebody reads the call site. snapshot.ttlDays hg/utils/snapshotReaper/snapshotReaper.c:46 wrote 1 row to the 'Awaiting review' section of hgConfCatalog.py; classify it and move it out
  • lines changed 20, context: html, text, full: html, text
    4e816d84ac0d3079b74b96b0c827fa9d213eba2b Thu Sep 3 07:00:40 2026 -0700
    hgConfCatalog: register login.recovEmailChange as a knob, refs #37925 #38197
  • lines changed 6, context: html, text, full: html, text
    5abe782bada3c851f1844c94b747cf7c769c71f9 Fri Sep 4 09:12:46 2026 -0700
    cart.c: gate the captcha session-id debug print behind hg.conf captchaDebug, refs #38247 Register captchaDebug in the hg.conf catalog as a permanent debugging knob rather than a release gate.
  • lines changed 2, context: html, text, full: html, text
    ea12fc3b55a64c2f8f4abdbd6e78f7c26db9c03d Fri Sep 4 11:53:47 2026 -0700
    hgConfCatalog: fix stale snapshotReaper path in snapshot.ttlDays note, refs #38197 The auto-registered note for snapshot.ttlDays still pointed at hg/utils/snapshotReaper/snapshotReaper.c after the snapshotReaper -> snapshotCleaner rename. Update to hg/utils/snapshotCleaner/snapshotCleaner.c.
  • lines changed 23, context: html, text, full: html, text
    c430b31cb48da35050fd294fd29934e6e6fa033e Sat Sep 5 02:30:49 2026 -0700
    hgConfCatalog: register 3 settings the tree reads, refs #37925 Written by nightlyRegister.sh, which records the settings the tree reads that the catalog was missing. Only facts copied off the call site are filled in. No classification is guessed: a new boolean gets no role=, because calling a release gate a knob would hide it from the sunset report for good, and every row lands in the 'Awaiting review' section until somebody reads the call site. blatNewFormNewsUrl hg/hgBlat/hgBlat.c:848 (37996) blatNewFormSwitchDate hg/hgBlat/hgBlat.c:843 (37996) blatOnlyLatestCheckbox hg/hgBlat/hgBlat.c:2815 (36292) wrote 3 rows to the 'Awaiting review' section of hgConfCatalog.py; classify them and move them out
  • lines changed 124, context: html, text, full: html, text
    25458068678366409e5e4923a382da4ab1344844 Sat Sep 5 07:28:59 2026 -0700
    hgConfCatalog: classify the six settings waiting for review. --auto-register had recorded six reads that nobody had classified, and --reconcile counted every one of them as a problem, so the nightly cron mailed a finding every morning. All six call sites are now read and the rows have moved into the section each one belongs in with verified=True. mallocTopPad Limits, a limit. mallopt(M_TOP_PAD) at CGI startup, hgConfig.c:396. refs #38225 snapshot.ttlDays Limits, a limit. How long an anonymous Share-a-link snapshot lives before snapshotCleaner deletes it. pngTimingSampleRate Logging, a debug setting. One page load in this many reports how long the track image took to reach the reader. refs #38109 blatNewFormSwitchDate Branding, site text. The date the classic BLAT banner names. refs #37996 blatNewFormNewsUrl Branding, a url. Where that banner's news link points. refs #37996 blatOnlyLatestCheckbox Mirror knobs, internal. Whether the BLAT form offers the "Keep only last search" checkbox. refs #36292 None of the six is a boolean flag, so none of them needed a gate or knob call and none enters the sunset arithmetic. Two things came out of reading the call sites rather than the harvest. blatOnlyLatestCheckbox has a second call site the auto-registered row did not name, hgc.c:27616, which is where the tracks are actually deleted; the row names both, since the two have to agree. And it is a string compared with sameString against "on", so only that exact value turns the feature on: true, 1 and yes do not, which is not how a flag read through cfgOptionBooleanDefault behaves. The row says so. refs #37925
  • lines changed 16, context: html, text, full: html, text
    06e8572fc3c80c01b547e5616191347674d971cc Sat Sep 5 08:03:52 2026 -0700
    hgConfCatalog: register excludeDbs and slow-db, the failover profile. hg/lib/jksql.c:1325 builds "<profile>.excludeDbs" with safef from a failover profile's name and reads it with cfgOption. So it is an hg.conf setting, and this catalog did not have it: not in the profile suffix list, not anywhere else. The harvester cannot see it, because the name never appears as a literal, which is the same blind spot login.oauth.<provider>.<field> sits in. It is not obscure. product/ex.hg.conf:36 documents slow-db.excludeDbs and goldenPath/help/gbib.html explains it twice. It is a comma-separated list of databases that exist only on the local server, so the failover connection is never opened for them. It is the fifteenth profile suffix, and the only one that is not read through cfgOption2 and the only one that belongs to a failover profile rather than to any profile. The section text now says both, since the suffix list alone would claim central.excludeDbs works. slow-db joins knownProfiles while I am here. It is a real profile, named by failoverProfPrefix at jksql.c:106 as "slow-" plus the main profile's name, and ex.hg.conf documents slow-db.host, .user and .password. Where it came from: excludeDbs had been written down in the CART variable baseline, one registry over, as a name that is not a track-scoped cart variable. True, but it left the setting described nowhere and suppressed in the wrong place. The companion commit teaches harvestCartVars.py to tell the two apart, so it is no longer in that file either. refs #37925
• src/hg/utils/hgConfCatalog/hgConfGateBacklog.txt
  • lines changed 1, context: html, text, full: html, text
    9f5a3871b95f595b5a218f0d05395a8c72d58844 Tue Aug 25 12:57:10 2026 -0700
    remove the bigBedOnePath fallback, leaving a single bigBed load path, refs #36940 bigBedOnePath has defaulted to on since v492 and the setting has been removed from the hg.conf of the RR, Euro, Asia and hgwbeta. Drop the check and the code it guarded at all four sites: the field-count default in bigBedAddLinkedFeaturesFromExt, the method setup in commonBigBedMethods, the bedSize default in bigBedClick, and the bigBed case in hgc. The two branches in the hgc case differed only in a minimum the surviving path does not want. complexBedMethods lost its only isBigBed=TRUE caller with that branch, so drop the parameter. Retire the setting from the hg.conf catalog and its gate backlog, and repair the two hgc.c citations whose line numbers this commit shifted, refs #37925. Verified pixel-identical: 14 scenarios rendered before and after, covering the four QA sessions on the ticket plus mm10 knownGene, dbSnp155, rmsk, encRegTfbsClustered, clinvar and tandemDups. All AE=0.
• src/hg/utils/hubCheck/hubCheck.c
  • lines changed 17, context: html, text, full: html, text
    717a40d56723cebacd7cbb8c240f9c2e72a4160a Fri Aug 21 10:01:12 2026 -0700
    hubCheck: name the parts of a description page we do not print, refs #38126 Now that a hub description is filtered on the way in, a hub author could find out about it from a page that comes out looking wrong. Warn instead, one line per thing removed, up to ten, then a count of the rest. Only things a reader would notice are named: an element dropped with its contents, an event attribute, a link with a scheme we do not print, a style value holding url() or expression. A class attribute or an unlisted style property goes quietly. Over the 5390 pages of the public hubs, 118 draw at least one of these warnings.
  • lines changed 1, context: html, text, full: html, text
    8983998334c36e7dfefe608b9a4effe3a10ae9ab Fri Aug 21 10:21:02 2026 -0700
    hubCheck: give chopByChar the number of slots, not the size of the array, no redmine splitMessages holds sixteen pointers, and the call passed sizeof(splitMessages), which is the byte count. A track that gathers more than sixteen warning lines therefore wrote past the end of it. Twenty filterByRange settings with no matching filter setting is enough to do it, and hgHubConnect runs hubCheck with -htmlOut on any hub address somebody types in. Pass ArraySize instead. The mistake dates from 702afb4c465 in 2020.
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 4, context: html, text, full: html, text
    571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
    hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260 hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands each track to addTrackIfDataAccessible(), which frees any track whose data cannot be reached. That is right for the browser, which should degrade gracefully rather than draw a broken track, and wrong for hubCheck, whose main job is to say that the file is unreachable. The track was gone before the file check ran, so hubCheck printed nothing and exited 0: cd hubCheck/tests/input sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned Split the non-pruning part of trackDbPolishAfterLinkup() into trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour is unchanged. Added a tests/ case whose bigDataUrl does not exist. Before this change it produced no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb". Of the existing tests, only badType changes, and it gains a real error that the prune had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was already failing before this commit and still is, for an unrelated reason: seven tests carry expected output with per-track "missing description page" warnings that no longer appear, because those hubs point at remote URLs that no longer resolve and hubCheckTrack aborts at the file check before reaching the description check. That is stale test data, not addressed here.
• src/hg/utils/hubCheck/tests/expected/missingFile.output.txt
  • lines changed 2, context: html, text, full: html, text
    571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
    hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260 hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands each track to addTrackIfDataAccessible(), which frees any track whose data cannot be reached. That is right for the browser, which should degrade gracefully rather than draw a broken track, and wrong for hubCheck, whose main job is to say that the file is unreachable. The track was gone before the file check ran, so hubCheck printed nothing and exited 0: cd hubCheck/tests/input sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned Split the non-pruning part of trackDbPolishAfterLinkup() into trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour is unchanged. Added a tests/ case whose bigDataUrl does not exist. Before this change it produced no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb". Of the existing tests, only badType changes, and it gains a real error that the prune had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was already failing before this commit and still is, for an unrelated reason: seven tests carry expected output with per-track "missing description page" warnings that no longer appear, because those hubs point at remote URLs that no longer resolve and hubCheckTrack aborts at the file check before reaching the description check. That is stale test data, not addressed here.
• src/hg/utils/hubCheck/tests/expected/relPath.output.txt
  • lines changed 2, context: html, text, full: html, text
    bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
    hubCheck: stop resolving bigDataUrl a second time, refs #38256 trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against genome->trackDbFile, which prepended the hub's directory a second time. For a local hub reached by a relative path that turned a file which is present into a file which is reported missing: hubCheck out/hub.txt Couldn't open out/out/hg19/x.bb Remote hubs and local hubs given by an absolute path were unaffected, because the second resolution is a no-op once the path is a URL or already absolute, which is why this went unnoticed. The tests/makefile has been working around it: the fieldCountMismatch case has to cd into input/ before it can find its bigBed. Added a tests/ case that does not cd, with the data file in a subdirectory below hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report that mismatch if it really opened the file. Before this change the test reported "Couldn't open input/relPath/input/relPath/data/test6field.bb". Verified against a binary built from the unmodified tree: this changes the output of no other test in hubCheck/tests. Seven of those tests were already failing before this commit and still are.
• src/hg/utils/hubCheck/tests/input/relPath/data/test6field.bb
  • lines changed 0, context: html, text, full: html, text
    bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
    hubCheck: stop resolving bigDataUrl a second time, refs #38256 trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against genome->trackDbFile, which prepended the hub's directory a second time. For a local hub reached by a relative path that turned a file which is present into a file which is reported missing: hubCheck out/hub.txt Couldn't open out/out/hg19/x.bb Remote hubs and local hubs given by an absolute path were unaffected, because the second resolution is a no-op once the path is a URL or already absolute, which is why this went unnoticed. The tests/makefile has been working around it: the fieldCountMismatch case has to cd into input/ before it can find its bigBed. Added a tests/ case that does not cd, with the data file in a subdirectory below hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report that mismatch if it really opened the file. Before this change the test reported "Couldn't open input/relPath/input/relPath/data/test6field.bb". Verified against a binary built from the unmodified tree: this changes the output of no other test in hubCheck/tests. Seven of those tests were already failing before this commit and still are.
• src/hg/utils/hubCheck/tests/input/relPath/hub.txt
  • lines changed 20, context: html, text, full: html, text
    bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
    hubCheck: stop resolving bigDataUrl a second time, refs #38256 trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against genome->trackDbFile, which prepended the hub's directory a second time. For a local hub reached by a relative path that turned a file which is present into a file which is reported missing: hubCheck out/hub.txt Couldn't open out/out/hg19/x.bb Remote hubs and local hubs given by an absolute path were unaffected, because the second resolution is a no-op once the path is a URL or already absolute, which is why this went unnoticed. The tests/makefile has been working around it: the fieldCountMismatch case has to cd into input/ before it can find its bigBed. Added a tests/ case that does not cd, with the data file in a subdirectory below hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report that mismatch if it really opened the file. Before this change the test reported "Couldn't open input/relPath/input/relPath/data/test6field.bb". Verified against a binary built from the unmodified tree: this changes the output of no other test in hubCheck/tests. Seven of those tests were already failing before this commit and still are.
• src/hg/utils/hubCheck/tests/input/relPath/hubDescription.html
  • lines changed 2, context: html, text, full: html, text
    bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
    hubCheck: stop resolving bigDataUrl a second time, refs #38256 trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against genome->trackDbFile, which prepended the hub's directory a second time. For a local hub reached by a relative path that turned a file which is present into a file which is reported missing: hubCheck out/hub.txt Couldn't open out/out/hg19/x.bb Remote hubs and local hubs given by an absolute path were unaffected, because the second resolution is a no-op once the path is a URL or already absolute, which is why this went unnoticed. The tests/makefile has been working around it: the fieldCountMismatch case has to cd into input/ before it can find its bigBed. Added a tests/ case that does not cd, with the data file in a subdirectory below hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report that mismatch if it really opened the file. Before this change the test reported "Couldn't open input/relPath/input/relPath/data/test6field.bb". Verified against a binary built from the unmodified tree: this changes the output of no other test in hubCheck/tests. Seven of those tests were already failing before this commit and still are.
• src/hg/utils/hubCheck/tests/input/relPath/missingFile.txt
  • lines changed 18, context: html, text, full: html, text
    571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
    hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260 hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands each track to addTrackIfDataAccessible(), which frees any track whose data cannot be reached. That is right for the browser, which should degrade gracefully rather than draw a broken track, and wrong for hubCheck, whose main job is to say that the file is unreachable. The track was gone before the file check ran, so hubCheck printed nothing and exited 0: cd hubCheck/tests/input sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned Split the non-pruning part of trackDbPolishAfterLinkup() into trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour is unchanged. Added a tests/ case whose bigDataUrl does not exist. Before this change it produced no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb". Of the existing tests, only badType changes, and it gains a real error that the prune had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was already failing before this commit and still is, for an unrelated reason: seven tests carry expected output with per-track "missing description page" warnings that no longer appear, because those hubs point at remote URLs that no longer resolve and hubCheckTrack aborts at the file check before reaching the description check. That is stale test data, not addressed here.
• src/hg/utils/hubCheck/tests/makefile
  • lines changed 10, context: html, text, full: html, text
    bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
    hubCheck: stop resolving bigDataUrl a second time, refs #38256 trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against genome->trackDbFile, which prepended the hub's directory a second time. For a local hub reached by a relative path that turned a file which is present into a file which is reported missing: hubCheck out/hub.txt Couldn't open out/out/hg19/x.bb Remote hubs and local hubs given by an absolute path were unaffected, because the second resolution is a no-op once the path is a URL or already absolute, which is why this went unnoticed. The tests/makefile has been working around it: the fieldCountMismatch case has to cd into input/ before it can find its bigBed. Added a tests/ case that does not cd, with the data file in a subdirectory below hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report that mismatch if it really opened the file. Before this change the test reported "Couldn't open input/relPath/input/relPath/data/test6field.bb". Verified against a binary built from the unmodified tree: this changes the output of no other test in hubCheck/tests. Seven of those tests were already failing before this commit and still are.
  • lines changed 7, context: html, text, full: html, text
    571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
    hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260 hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands each track to addTrackIfDataAccessible(), which frees any track whose data cannot be reached. That is right for the browser, which should degrade gracefully rather than draw a broken track, and wrong for hubCheck, whose main job is to say that the file is unreachable. The track was gone before the file check ran, so hubCheck printed nothing and exited 0: cd hubCheck/tests/input sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned Split the non-pruning part of trackDbPolishAfterLinkup() into trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour is unchanged. Added a tests/ case whose bigDataUrl does not exist. Before this change it produced no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb". Of the existing tests, only badType changes, and it gains a real error that the prune had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was already failing before this commit and still is, for an unrelated reason: seven tests carry expected output with per-track "missing description page" warnings that no longer appear, because those hubs point at remote URLs that no longer resolve and hubCheckTrack aborts at the file check before reaching the description check. That is stale test data, not addressed here.
• src/hg/utils/makefile
  • lines changed 1, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
  • lines changed 1, context: html, text, full: html, text
    89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
    snapshotSession: rename the snapshot "reaper" to "cleaner" Renames the snapshotReaper utility to snapshotCleaner and drops the word "reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and the doc/usage text now say clean/cleaned/cleaner). No behavior change. refs #38197
• src/hg/utils/otto/otto.crontab
  • lines changed 6, context: html, text, full: html, text
    3181982d60858062c80fd6db95294172f41525be Fri Sep 4 16:21:50 2026 -0700
    Add the otto cron line for cbPingHidden, the monthly reminder to submitters of hidden Cell Browser datasets. It goes in the block whose MAILTO is cells@ucsc.edu, next to the other Cell Browser jobs. The run only queues one draft email per dataset that is due and mails the report; nothing reaches an author until someone reads the queue and runs "cbPingHidden send". Three departures from the line suggested in the script's README. The report address is spelled out, because a bare --mail-report falls back to $USER@ucsc.edu and $USER here is otto. The umask keeps the drafts otto writes into the queue group-writable, so they can still be edited by hand before they go out. And the schedule is the 1st of the month rather than the first Monday: "1-7 * 1" would have been read as the 1st through the 7th OR every Monday, which is roughly ten runs a month, not one. refs #38246 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 7, context: html, text, full: html, text
    70ac948e82b41ef316523635c04e5e2c4a89c417 Fri Sep 4 16:57:54 2026 -0700
    trackLists: give the variant frequency projects their own table, move the page to goldenPath/help/mirrorTracks.html, add the otto cron line, refs #37781 Half the restricted list was national sequencing cohorts sitting under varFreqs and phasedVars, and in one alphabetical table they buried the tracks people actually write in about, OMIM, HGMD and DECIPHER. Those cohorts now get a table of their own below the rest. The split is read off the trackDb parent chain, so the next cohort added under varFreqs lands in the right table with no edit here. The page moves off the htdocs root to goldenPath/help/mirrorTracks.html, beside mirror.html, which now links to it. That link goes in src/product/README.txt, the pandoc source mirror.html is generated from. The licensing page link follows the move, and the page title changes with the file name. Also adds the weekly otto line, placed above the HGDB_CONF that would otherwise apply to it, and keeps the job from listing itself as a self-updating track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 5, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
• src/hg/utils/otto/refSeqHistorical/checkRefSeqHistoricalUpdate.sh
  • lines changed 2, context: html, text, full: html, text
    c8c1b1a9adac6dfa7a869326ca3f774a724a50db Fri Sep 4 17:51:48 2026 -0700
    Document the RefSeq Historical protein table build and add a reminder to the notifier. The historical rna.gbff we already download carries a /translation for every coding record, and gbProcess has a -pepFa flag that writes them out, so this is one extra pass over a file we have rather than a new download. Adds the build steps to the RS_2024_08 makedoc section, and two lines to the weekly notifier so the next refresh does not leave the protein table behind. refs #38248
• src/hg/utils/otto/strchive/README.md
  • lines changed 92, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
• src/hg/utils/otto/strchive/makefile
  • lines changed 18, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
• src/hg/utils/otto/strchive/strchiveAutoPush
  • lines changed 53, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
  • lines changed 7, context: html, text, full: html, text
    4274e5376b9d13e83e67cfe1bce8a973643f0afa Sun Sep 6 07:44:02 2026 -0700
    strchiveAutoPush: report and return the real rsync exit code The failure branch read $? after two echo commands had already replaced it, so the log said "code 0" whatever rsync did and the script exited 0 on a failed push. Save the status right after rsync and use it in both messages and the exit. The script is still the uninstalled draft for the sysadmins. refs #38268
• src/hg/utils/otto/strchive/strchiveOtto.py
  • lines changed 234, context: html, text, full: html, text
    0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
    STRchive track now updates itself, and is on hg19 and hs1 as well as hg38 STRchive builds a Genome Browser bigBed with every release now and publishes it on their GitHub releases page, so there is nothing left for us to convert. A new otto job checks weekly for a new release, downloads the file into its own releases/<tag> directory, sanity-checks the locus count, and repoints the /gbdb symlinks at it. Silent when there is no new release. See STRchive issue 333 for the correspondence that set this up. Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all three, so the new strVar stanza lives in human/strVarNew.ra rather than under hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose file is missing, so each assembly gets only the tracks that were built for it: STRchive on all three, the other four on hg38 only. The description page is rewritten for the new evidence-level coloring, gains an evidence filter, and now says that the track updates itself and that older versions are on GitHub rather than here. All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet, so releasing this trackDb would have it promise data the RR does not have. The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public, until the sysadmins install the strchiveAutoPush cron drafted alongside the otto job. The header of strVarNew.ra lists what to delete and rename then. refs #38268
• src/hg/utils/otto/trackLists/README.txt
  • lines changed 64, context: html, text, full: html, text
    dcb1fa309fa0157eb7076263b1a0a36b0c7babec Thu Sep 3 18:28:29 2026 -0700
    Add trackLists otto generator: tracks we cannot redistribute, tracks that update themselves, and contributed tracks, refs #37781 Builds one page answering the three questions mirror sites keep asking. List 1 unions several tests rather than relying on one, because no single trackDb setting marks every restricted track: tableBrowser off is the usual marker but OMIM instead sets tableBrowser noGenome with a noGenomeReason naming its distribution terms, so a query for "off" alone misses it. Not all noGenome is about licensing either -- CRISPR and JASPAR set it because a genome-wide query times out -- so the reason text is what separates them. The convention of putting restricted files under an underscore directory is real but partial: decipher, mexbb, spliceAI, cosmicRegions and hgmd are restricted and are not under one. It also checks the download server both directions. A trackDb track whose MySQL table exists here but is missing from hgdownload is almost certainly restricted, and a file we call restricted that hgdownload still serves is a bug the script prints so cron mails it. mkPage.py omits that second cross-check unless --internal is given, since naming reachable restricted files on a world-readable page would defeat the point. The generated page is not committed, matching allTips.html and thumbNailLinks.html, which live only in htdocs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 6, context: html, text, full: html, text
    fbca0ee885a9c343e160f7ab7d584d44fa1cf8a5 Thu Sep 3 18:36:22 2026 -0700
    trackLists: emit the page in the standard static-page style, refs #37781 Drops the hand-rolled stylesheet and uses the house shape instead: the gbPageStart/gbPageEnd SSI includes for the menu bar and footer, h2 per section with an h6 contents list, and bare tables, which gbStatic.css already styles through the .gbsPage wrapper. Source is ASCII-only, lowercase tags, and no line over 100 characters, per the QA conventions; long cells wrap rather than run off. The deploy step now chmods the page to 775. Apache runs the SSI includes on a .html file only when its execute bit is set (XBitHack), so without it the page is served verbatim and the reader gets bare content with no menu bar and no stylesheets. allTips.html is in that state on the RR today, which is how the behaviour came to light. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 15, context: html, text, full: html, text
    70ac948e82b41ef316523635c04e5e2c4a89c417 Fri Sep 4 16:57:54 2026 -0700
    trackLists: give the variant frequency projects their own table, move the page to goldenPath/help/mirrorTracks.html, add the otto cron line, refs #37781 Half the restricted list was national sequencing cohorts sitting under varFreqs and phasedVars, and in one alphabetical table they buried the tracks people actually write in about, OMIM, HGMD and DECIPHER. Those cohorts now get a table of their own below the rest. The split is read off the trackDb parent chain, so the next cohort added under varFreqs lands in the right table with no edit here. The page moves off the htdocs root to goldenPath/help/mirrorTracks.html, beside mirror.html, which now links to it. That link goes in src/product/README.txt, the pandoc source mirror.html is generated from. The licensing page link follows the move, and the page title changes with the file name. Also adds the weekly otto line, placed above the HGDB_CONF that would otherwise apply to it, and keeps the job from listing itself as a self-updating track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/otto/trackLists/collect.py
  • lines changed 357, context: html, text, full: html, text
    dcb1fa309fa0157eb7076263b1a0a36b0c7babec Thu Sep 3 18:28:29 2026 -0700
    Add trackLists otto generator: tracks we cannot redistribute, tracks that update themselves, and contributed tracks, refs #37781 Builds one page answering the three questions mirror sites keep asking. List 1 unions several tests rather than relying on one, because no single trackDb setting marks every restricted track: tableBrowser off is the usual marker but OMIM instead sets tableBrowser noGenome with a noGenomeReason naming its distribution terms, so a query for "off" alone misses it. Not all noGenome is about licensing either -- CRISPR and JASPAR set it because a genome-wide query times out -- so the reason text is what separates them. The convention of putting restricted files under an underscore directory is real but partial: decipher, mexbb, spliceAI, cosmicRegions and hgmd are restricted and are not under one. It also checks the download server both directions. A trackDb track whose MySQL table exists here but is missing from hgdownload is almost certainly restricted, and a file we call restricted that hgdownload still serves is a bug the script prints so cron mails it. mkPage.py omits that second cross-check unless --internal is given, since naming reachable restricted files on a world-readable page would defeat the point. The generated page is not committed, matching allTips.html and thumbNailLinks.html, which live only in htdocs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 28, context: html, text, full: html, text
    497f2788d49fda531c0836618688f954486eb87c Fri Sep 4 09:44:13 2026 -0700
    trackLists: do not let the public page claim the download cross-check is clean, refs #37781 The static-page rewrite collapsed mkPage.py's three-way branch on the hgdownload cross-check into two, so the public run fell into the all-clear wording whether or not collect.py had found a restricted file that is still reachable. collect.py's current output has three such files, so the page was asserting the opposite of what the check found. Restored as three cases: the public page says only that the check runs and that anything found is reported privately, the --internal page keeps the table of paths, and the all-clear wording is used only when the list really is empty. The comment explaining why the public branch must not be merged back into the all-clear one is back too. collect.py counted any HTTP code that was not literally 404 as reachable, which made a curl timeout or a failed connection (empty output, or 000) look like an exposed file. That mails false alarms and, now that the branch above depends on it, would drop the all-clear line from the public page on a network blip. Only 2xx and 3xx count as served, 4xx as blocked, and anything else is reported separately as not checked, in collected.json and on stderr. Also: the remaining unquoted interpolations into shell=True commands go through the existing q() helper, rows_by_track() no longer builds the "why" set that the new table does not use, and US spelling throughout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 28, context: html, text, full: html, text
    70ac948e82b41ef316523635c04e5e2c4a89c417 Fri Sep 4 16:57:54 2026 -0700
    trackLists: give the variant frequency projects their own table, move the page to goldenPath/help/mirrorTracks.html, add the otto cron line, refs #37781 Half the restricted list was national sequencing cohorts sitting under varFreqs and phasedVars, and in one alphabetical table they buried the tracks people actually write in about, OMIM, HGMD and DECIPHER. Those cohorts now get a table of their own below the rest. The split is read off the trackDb parent chain, so the next cohort added under varFreqs lands in the right table with no edit here. The page moves off the htdocs root to goldenPath/help/mirrorTracks.html, beside mirror.html, which now links to it. That link goes in src/product/README.txt, the pandoc source mirror.html is generated from. The licensing page link follows the move, and the page title changes with the file name. Also adds the weekly otto line, placed above the HGDB_CONF that would otherwise apply to it, and keeps the job from listing itself as a self-updating track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 2, context: html, text, full: html, text
    d953601aa1ab7e04ad424f10ec36d7bbd2c8c34b Sat Sep 5 17:13:58 2026 -0700
    trackLists: say what the malacards table is actually for, refs #37781 The row read "which no track points at", which is true and misleading in the same breath. hgGene's malaCardsSection joins the table against kgXref and builds the MalaCards links on the gene details page, so a mirror that skips the table loses that section. Nothing about the schedule or the hg38-only load changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/otto/trackLists/mkPage.py
  • lines changed 178, context: html, text, full: html, text
    dcb1fa309fa0157eb7076263b1a0a36b0c7babec Thu Sep 3 18:28:29 2026 -0700
    Add trackLists otto generator: tracks we cannot redistribute, tracks that update themselves, and contributed tracks, refs #37781 Builds one page answering the three questions mirror sites keep asking. List 1 unions several tests rather than relying on one, because no single trackDb setting marks every restricted track: tableBrowser off is the usual marker but OMIM instead sets tableBrowser noGenome with a noGenomeReason naming its distribution terms, so a query for "off" alone misses it. Not all noGenome is about licensing either -- CRISPR and JASPAR set it because a genome-wide query times out -- so the reason text is what separates them. The convention of putting restricted files under an underscore directory is real but partial: decipher, mexbb, spliceAI, cosmicRegions and hgmd are restricted and are not under one. It also checks the download server both directions. A trackDb track whose MySQL table exists here but is missing from hgdownload is almost certainly restricted, and a file we call restricted that hgdownload still serves is a bug the script prints so cron mails it. mkPage.py omits that second cross-check unless --internal is given, since naming reachable restricted files on a world-readable page would defeat the point. The generated page is not committed, matching allTips.html and thumbNailLinks.html, which live only in htdocs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 281, context: html, text, full: html, text
    fbca0ee885a9c343e160f7ab7d584d44fa1cf8a5 Thu Sep 3 18:36:22 2026 -0700
    trackLists: emit the page in the standard static-page style, refs #37781 Drops the hand-rolled stylesheet and uses the house shape instead: the gbPageStart/gbPageEnd SSI includes for the menu bar and footer, h2 per section with an h6 contents list, and bare tables, which gbStatic.css already styles through the .gbsPage wrapper. Source is ASCII-only, lowercase tags, and no line over 100 characters, per the QA conventions; long cells wrap rather than run off. The deploy step now chmods the page to 775. Apache runs the SSI includes on a .html file only when its execute bit is set (XBitHack), so without it the page is served verbatim and the reader gets bare content with no menu bar and no stylesheets. allTips.html is in that state on the RR today, which is how the behaviour came to light. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 22, context: html, text, full: html, text
    497f2788d49fda531c0836618688f954486eb87c Fri Sep 4 09:44:13 2026 -0700
    trackLists: do not let the public page claim the download cross-check is clean, refs #37781 The static-page rewrite collapsed mkPage.py's three-way branch on the hgdownload cross-check into two, so the public run fell into the all-clear wording whether or not collect.py had found a restricted file that is still reachable. collect.py's current output has three such files, so the page was asserting the opposite of what the check found. Restored as three cases: the public page says only that the check runs and that anything found is reported privately, the --internal page keeps the table of paths, and the all-clear wording is used only when the list really is empty. The comment explaining why the public branch must not be merged back into the all-clear one is back too. collect.py counted any HTTP code that was not literally 404 as reachable, which made a curl timeout or a failed connection (empty output, or 000) look like an exposed file. That mails false alarms and, now that the branch above depends on it, would drop the all-clear line from the public page on a network blip. Only 2xx and 3xx count as served, 4xx as blocked, and anything else is reported separately as not checked, in collected.json and on stderr. Also: the remaining unquoted interpolations into shell=True commands go through the existing q() helper, rows_by_track() no longer builds the "why" set that the new table does not use, and US spelling throughout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 54, context: html, text, full: html, text
    70ac948e82b41ef316523635c04e5e2c4a89c417 Fri Sep 4 16:57:54 2026 -0700
    trackLists: give the variant frequency projects their own table, move the page to goldenPath/help/mirrorTracks.html, add the otto cron line, refs #37781 Half the restricted list was national sequencing cohorts sitting under varFreqs and phasedVars, and in one alphabetical table they buried the tracks people actually write in about, OMIM, HGMD and DECIPHER. Those cohorts now get a table of their own below the rest. The split is read off the trackDb parent chain, so the next cohort added under varFreqs lands in the right table with no edit here. The page moves off the htdocs root to goldenPath/help/mirrorTracks.html, beside mirror.html, which now links to it. That link goes in src/product/README.txt, the pandoc source mirror.html is generated from. The licensing page link follows the move, and the page title changes with the file name. Also adds the weekly otto line, placed above the HGDB_CONF that would otherwise apply to it, and keeps the job from listing itself as a self-updating track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/otto/trackLists/trackLists.sh
  • lines changed 39, context: html, text, full: html, text
    dcb1fa309fa0157eb7076263b1a0a36b0c7babec Thu Sep 3 18:28:29 2026 -0700
    Add trackLists otto generator: tracks we cannot redistribute, tracks that update themselves, and contributed tracks, refs #37781 Builds one page answering the three questions mirror sites keep asking. List 1 unions several tests rather than relying on one, because no single trackDb setting marks every restricted track: tableBrowser off is the usual marker but OMIM instead sets tableBrowser noGenome with a noGenomeReason naming its distribution terms, so a query for "off" alone misses it. Not all noGenome is about licensing either -- CRISPR and JASPAR set it because a genome-wide query times out -- so the reason text is what separates them. The convention of putting restricted files under an underscore directory is real but partial: decipher, mexbb, spliceAI, cosmicRegions and hgmd are restricted and are not under one. It also checks the download server both directions. A trackDb track whose MySQL table exists here but is missing from hgdownload is almost certainly restricted, and a file we call restricted that hgdownload still serves is a bug the script prints so cron mails it. mkPage.py omits that second cross-check unless --internal is given, since naming reachable restricted files on a world-readable page would defeat the point. The generated page is not committed, matching allTips.html and thumbNailLinks.html, which live only in htdocs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 6, context: html, text, full: html, text
    fbca0ee885a9c343e160f7ab7d584d44fa1cf8a5 Thu Sep 3 18:36:22 2026 -0700
    trackLists: emit the page in the standard static-page style, refs #37781 Drops the hand-rolled stylesheet and uses the house shape instead: the gbPageStart/gbPageEnd SSI includes for the menu bar and footer, h2 per section with an h6 contents list, and bare tables, which gbStatic.css already styles through the .gbsPage wrapper. Source is ASCII-only, lowercase tags, and no line over 100 characters, per the QA conventions; long cells wrap rather than run off. The deploy step now chmods the page to 775. Apache runs the SSI includes on a .html file only when its execute bit is set (XBitHack), so without it the page is served verbatim and the reader gets bare content with no menu bar and no stylesheets. allTips.html is in that state on the RR today, which is how the behaviour came to light. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 1, context: html, text, full: html, text
    497f2788d49fda531c0836618688f954486eb87c Fri Sep 4 09:44:13 2026 -0700
    trackLists: do not let the public page claim the download cross-check is clean, refs #37781 The static-page rewrite collapsed mkPage.py's three-way branch on the hgdownload cross-check into two, so the public run fell into the all-clear wording whether or not collect.py had found a restricted file that is still reachable. collect.py's current output has three such files, so the page was asserting the opposite of what the check found. Restored as three cases: the public page says only that the check runs and that anything found is reported privately, the --internal page keeps the table of paths, and the all-clear wording is used only when the list really is empty. The comment explaining why the public branch must not be merged back into the all-clear one is back too. collect.py counted any HTTP code that was not literally 404 as reachable, which made a curl timeout or a failed connection (empty output, or 000) look like an exposed file. That mails false alarms and, now that the branch above depends on it, would drop the all-clear line from the public page on a network blip. Only 2xx and 3xx count as served, 4xx as blocked, and anything else is reported separately as not checked, in collected.json and on stderr. Also: the remaining unquoted interpolations into shell=True commands go through the existing q() helper, rows_by_track() no longer builds the "why" set that the new table does not use, and US spelling throughout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 6, context: html, text, full: html, text
    70ac948e82b41ef316523635c04e5e2c4a89c417 Fri Sep 4 16:57:54 2026 -0700
    trackLists: give the variant frequency projects their own table, move the page to goldenPath/help/mirrorTracks.html, add the otto cron line, refs #37781 Half the restricted list was national sequencing cohorts sitting under varFreqs and phasedVars, and in one alphabetical table they buried the tracks people actually write in about, OMIM, HGMD and DECIPHER. Those cohorts now get a table of their own below the rest. The split is read off the trackDb parent chain, so the next cohort added under varFreqs lands in the right table with no edit here. The page moves off the htdocs root to goldenPath/help/mirrorTracks.html, beside mirror.html, which now links to it. That link goes in src/product/README.txt, the pandoc source mirror.html is generated from. The licensing page link follows the move, and the page title changes with the file name. Also adds the weekly otto line, placed above the HGDB_CONF that would otherwise apply to it, and keeps the job from listing itself as a self-updating track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/pngTiming/pngTimingReport.py
  • lines changed 502, context: html, text, full: html, text
    7b4952c3b7ec30772836c67d40db442a13b83476 Wed Sep 2 11:00:09 2026 -0700
    pngTimingReport: read the track image timing beacons out of an apache log, refs #38109 hgTracks reports how long its track image took to reach the reader on the query string of a 43 byte image, so the apache log line is the whole record. This turns those lines into a throughput distribution, split by continent, which is the input the png compression level decision needs. Reads plain logs and .gz archives, and defaults to the live hgwdev log. Country comes from geoIpCountry6, the table geoMirror.c uses, read once and bisected here. A delivery that finished inside one clock tick (x=0) is counted in its own column and kept out of the percentiles, because it cannot give a throughput. Our own headless test runs are dropped by default.
• src/hg/utils/registryPages/correlate.css
  • lines changed 91, context: html, text, full: html, text
    3c814b674f49f9a30d4b8d227e0fe7061a18766a Tue Sep 1 09:21:28 2026 -0700
    registryPages: correlate the trackDb settings docs with the cart, refs #37908 #37838 Two files in the tree say which track types a setting applies to, and they were written from different evidence. trackDbLibrary.shtml carries a hand-written types list per setting, which is what #37908 has been correcting. cartTrackVarCatalog files each cart variable under the config function that reads it and records the trackDb types that function serves, which came from reading hui.c and the per-type Ui functions. Where a trackDb setting and a cart variable are the same knob, the two are answering the same question, so they can be compared. 60 of the 261 documented settings have a runtime override. 46 of those pairs are comparable, 12 agree exactly, and 28 have a type the config code serves that the docs do not list. The output is a candidate list, not a verdict, and the page says so. Two known false-positive shapes are called out on it: a variable read by two config functions collects the types of both, and a pair joined by tdbDefault rather than by name is weaker evidence, so those are reported separately. Also factors the palette and the shared reset out of venn.css and index.css into tokens.css, since a third page now needs them.
• src/hg/utils/registryPages/index.css
  • lines changed 133, context: html, text, full: html, text
    cab003c85820ef181c537ad402aad0044e4e3878 Tue Sep 1 09:06:56 2026 -0700
    registryPages: draw two pages from all four configuration catalogs, refs #37838 #37923 #37925 #37623 Each of the four catalogs answers for its own surface and none of them knows the others exist. This reads all four and draws the two pages that need them at once: a Venn of the registries, and an index of every name they describe with its description on hover. The reason it is a program and not two files: every count on both pages is computed from the catalogs, so a page cannot drift from the tree. A hand-built version of the Venn had the track registry 12 names short and one region count wrong, and neither error was visible on the page. The one judgement is which names two registries describe as the same variable. That is matched on the cart variable a row names, not on the suffix the track catalog stores, because comparing bare names both misses the four track-scoped names and invents six overlaps that are not there. KNOWN_SHARED records the eleven confirmed by reading the call sites, and --check fails when the computed set no longer matches it, the same contract the sibling catalogs' --reconcile has. --check writes nothing and is the mode for a cron. --audit adds the saved session counts from sessionCartAudit and needs the database.
  • lines changed 27, context: html, text, full: html, text
    3c814b674f49f9a30d4b8d227e0fe7061a18766a Tue Sep 1 09:21:28 2026 -0700
    registryPages: correlate the trackDb settings docs with the cart, refs #37908 #37838 Two files in the tree say which track types a setting applies to, and they were written from different evidence. trackDbLibrary.shtml carries a hand-written types list per setting, which is what #37908 has been correcting. cartTrackVarCatalog files each cart variable under the config function that reads it and records the trackDb types that function serves, which came from reading hui.c and the per-type Ui functions. Where a trackDb setting and a cart variable are the same knob, the two are answering the same question, so they can be compared. 60 of the 261 documented settings have a runtime override. 46 of those pairs are comparable, 12 agree exactly, and 28 have a type the config code serves that the docs do not list. The output is a candidate list, not a verdict, and the page says so. Two known false-positive shapes are called out on it: a variable read by two config functions collects the types of both, and a pair joined by tdbDefault rather than by name is weaker evidence, so those are reported separately. Also factors the palette and the shared reset out of venn.css and index.css into tokens.css, since a third page now needs them.
• src/hg/utils/registryPages/index.js
  • lines changed 133, context: html, text, full: html, text
    cab003c85820ef181c537ad402aad0044e4e3878 Tue Sep 1 09:06:56 2026 -0700
    registryPages: draw two pages from all four configuration catalogs, refs #37838 #37923 #37925 #37623 Each of the four catalogs answers for its own surface and none of them knows the others exist. This reads all four and draws the two pages that need them at once: a Venn of the registries, and an index of every name they describe with its description on hover. The reason it is a program and not two files: every count on both pages is computed from the catalogs, so a page cannot drift from the tree. A hand-built version of the Venn had the track registry 12 names short and one region count wrong, and neither error was visible on the page. The one judgement is which names two registries describe as the same variable. That is matched on the cart variable a row names, not on the suffix the track catalog stores, because comparing bare names both misses the four track-scoped names and invents six overlaps that are not there. KNOWN_SHARED records the eleven confirmed by reading the call sites, and --check fails when the computed set no longer matches it, the same contract the sibling catalogs' --reconcile has. --check writes nothing and is the mode for a cron. --audit adds the saved session counts from sessionCartAudit and needs the database.
• src/hg/utils/registryPages/registryData.py
  • lines changed 473, context: html, text, full: html, text
    cab003c85820ef181c537ad402aad0044e4e3878 Tue Sep 1 09:06:56 2026 -0700
    registryPages: draw two pages from all four configuration catalogs, refs #37838 #37923 #37925 #37623 Each of the four catalogs answers for its own surface and none of them knows the others exist. This reads all four and draws the two pages that need them at once: a Venn of the registries, and an index of every name they describe with its description on hover. The reason it is a program and not two files: every count on both pages is computed from the catalogs, so a page cannot drift from the tree. A hand-built version of the Venn had the track registry 12 names short and one region count wrong, and neither error was visible on the page. The one judgement is which names two registries describe as the same variable. That is matched on the cart variable a row names, not on the suffix the track catalog stores, because comparing bare names both misses the four track-scoped names and invents six overlaps that are not there. KNOWN_SHARED records the eleven confirmed by reading the call sites, and --check fails when the computed set no longer matches it, the same contract the sibling catalogs' --reconcile has. --check writes nothing and is the mode for a cron. --audit adds the saved session counts from sessionCartAudit and needs the database.
• src/hg/utils/registryPages/registryPages.py
  • lines changed 716, context: html, text, full: html, text
    cab003c85820ef181c537ad402aad0044e4e3878 Tue Sep 1 09:06:56 2026 -0700
    registryPages: draw two pages from all four configuration catalogs, refs #37838 #37923 #37925 #37623 Each of the four catalogs answers for its own surface and none of them knows the others exist. This reads all four and draws the two pages that need them at once: a Venn of the registries, and an index of every name they describe with its description on hover. The reason it is a program and not two files: every count on both pages is computed from the catalogs, so a page cannot drift from the tree. A hand-built version of the Venn had the track registry 12 names short and one region count wrong, and neither error was visible on the page. The one judgement is which names two registries describe as the same variable. That is matched on the cart variable a row names, not on the suffix the track catalog stores, because comparing bare names both misses the four track-scoped names and invents six overlaps that are not there. KNOWN_SHARED records the eleven confirmed by reading the call sites, and --check fails when the computed set no longer matches it, the same contract the sibling catalogs' --reconcile has. --check writes nothing and is the mode for a cron. --audit adds the saved session counts from sessionCartAudit and needs the database.
  • lines changed 264, context: html, text, full: html, text
    3c814b674f49f9a30d4b8d227e0fe7061a18766a Tue Sep 1 09:21:28 2026 -0700
    registryPages: correlate the trackDb settings docs with the cart, refs #37908 #37838 Two files in the tree say which track types a setting applies to, and they were written from different evidence. trackDbLibrary.shtml carries a hand-written types list per setting, which is what #37908 has been correcting. cartTrackVarCatalog files each cart variable under the config function that reads it and records the trackDb types that function serves, which came from reading hui.c and the per-type Ui functions. Where a trackDb setting and a cart variable are the same knob, the two are answering the same question, so they can be compared. 60 of the 261 documented settings have a runtime override. 46 of those pairs are comparable, 12 agree exactly, and 28 have a type the config code serves that the docs do not list. The output is a candidate list, not a verdict, and the page says so. Two known false-positive shapes are called out on it: a variable read by two config functions collects the types of both, and a pair joined by tdbDefault rather than by name is weaker evidence, so those are reported separately. Also factors the palette and the shared reset out of venn.css and index.css into tokens.css, since a third page now needs them.
• src/hg/utils/registryPages/tokens.css
  • lines changed 52, context: html, text, full: html, text
    3c814b674f49f9a30d4b8d227e0fe7061a18766a Tue Sep 1 09:21:28 2026 -0700
    registryPages: correlate the trackDb settings docs with the cart, refs #37908 #37838 Two files in the tree say which track types a setting applies to, and they were written from different evidence. trackDbLibrary.shtml carries a hand-written types list per setting, which is what #37908 has been correcting. cartTrackVarCatalog files each cart variable under the config function that reads it and records the trackDb types that function serves, which came from reading hui.c and the per-type Ui functions. Where a trackDb setting and a cart variable are the same knob, the two are answering the same question, so they can be compared. 60 of the 261 documented settings have a runtime override. 46 of those pairs are comparable, 12 agree exactly, and 28 have a type the config code serves that the docs do not list. The output is a candidate list, not a verdict, and the page says so. Two known false-positive shapes are called out on it: a variable read by two config functions collects the types of both, and a pair joined by tdbDefault rather than by name is weaker evidence, so those are reported separately. Also factors the palette and the shared reset out of venn.css and index.css into tokens.css, since a third page now needs them.
• src/hg/utils/registryPages/trackDbData.py
  • lines changed 238, context: html, text, full: html, text
    3c814b674f49f9a30d4b8d227e0fe7061a18766a Tue Sep 1 09:21:28 2026 -0700
    registryPages: correlate the trackDb settings docs with the cart, refs #37908 #37838 Two files in the tree say which track types a setting applies to, and they were written from different evidence. trackDbLibrary.shtml carries a hand-written types list per setting, which is what #37908 has been correcting. cartTrackVarCatalog files each cart variable under the config function that reads it and records the trackDb types that function serves, which came from reading hui.c and the per-type Ui functions. Where a trackDb setting and a cart variable are the same knob, the two are answering the same question, so they can be compared. 60 of the 261 documented settings have a runtime override. 46 of those pairs are comparable, 12 agree exactly, and 28 have a type the config code serves that the docs do not list. The output is a candidate list, not a verdict, and the page says so. Two known false-positive shapes are called out on it: a variable read by two config functions collects the types of both, and a pair joined by tdbDefault rather than by name is weaker evidence, so those are reported separately. Also factors the palette and the shared reset out of venn.css and index.css into tokens.css, since a third page now needs them.
• src/hg/utils/registryPages/venn.css
  • lines changed 125, context: html, text, full: html, text
    cab003c85820ef181c537ad402aad0044e4e3878 Tue Sep 1 09:06:56 2026 -0700
    registryPages: draw two pages from all four configuration catalogs, refs #37838 #37923 #37925 #37623 Each of the four catalogs answers for its own surface and none of them knows the others exist. This reads all four and draws the two pages that need them at once: a Venn of the registries, and an index of every name they describe with its description on hover. The reason it is a program and not two files: every count on both pages is computed from the catalogs, so a page cannot drift from the tree. A hand-built version of the Venn had the track registry 12 names short and one region count wrong, and neither error was visible on the page. The one judgement is which names two registries describe as the same variable. That is matched on the cart variable a row names, not on the suffix the track catalog stores, because comparing bare names both misses the four track-scoped names and invents six overlaps that are not there. KNOWN_SHARED records the eleven confirmed by reading the call sites, and --check fails when the computed set no longer matches it, the same contract the sibling catalogs' --reconcile has. --check writes nothing and is the mode for a cron. --audit adds the saved session counts from sessionCartAudit and needs the database.
  • lines changed 31, context: html, text, full: html, text
    3c814b674f49f9a30d4b8d227e0fe7061a18766a Tue Sep 1 09:21:28 2026 -0700
    registryPages: correlate the trackDb settings docs with the cart, refs #37908 #37838 Two files in the tree say which track types a setting applies to, and they were written from different evidence. trackDbLibrary.shtml carries a hand-written types list per setting, which is what #37908 has been correcting. cartTrackVarCatalog files each cart variable under the config function that reads it and records the trackDb types that function serves, which came from reading hui.c and the per-type Ui functions. Where a trackDb setting and a cart variable are the same knob, the two are answering the same question, so they can be compared. 60 of the 261 documented settings have a runtime override. 46 of those pairs are comparable, 12 agree exactly, and 28 have a type the config code serves that the docs do not list. The output is a candidate list, not a verdict, and the page says so. Two known false-positive shapes are called out on it: a variable read by two config functions collects the types of both, and a pair joined by tdbDefault rather than by name is weaker evidence, so those are reported separately. Also factors the palette and the shared reset out of venn.css and index.css into tokens.css, since a third page now needs them.
• src/hg/utils/snapshotCleaner/makefile
  • lines changed 1, context: html, text, full: html, text
    89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
    snapshotSession: rename the snapshot "reaper" to "cleaner" Renames the snapshotReaper utility to snapshotCleaner and drops the word "reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and the doc/usage text now say clean/cleaned/cleaner). No behavior change. refs #38197
• src/hg/utils/snapshotCleaner/snapshotCleaner.c
  • lines changed 10, context: html, text, full: html, text
    89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
    snapshotSession: rename the snapshot "reaper" to "cleaner" Renames the snapshotReaper utility to snapshotCleaner and drops the word "reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and the doc/usage text now say clean/cleaned/cleaner). No behavior change. refs #38197
• src/hg/utils/snapshotReaper/makefile
  • lines changed 3, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
• src/hg/utils/snapshotReaper/snapshotReaper.c
  • lines changed 60, context: html, text, full: html, text
    32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
    Reusable "view snapshot" sessions for durable, minimal Share-a-link links "Share a link" could only share the whole cart: it saved a session holding every track, setting and position, which bloated the central db and leaked the sharer's unrelated tracks to whoever opened the link. Anonymous share sessions (under the reserved user "l") were also never reaped, so they accumulated forever, and their 8-char names were generated client-side with no uniqueness check, so two shares could collide and silently overwrite. Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a snapshot stores only the handful of cart variables a feature declares (a registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only those variables' trash files into durable sessionData storage, and is saved under a "__"-prefixed name. The "__" marks it machine-made: hidden from the My Sessions list by default and eligible for reaping. For the anonymous "l" owner the durable files fan out over two extra hash levels so one directory never fills with millions of entries. Every anonymous link now shares one server-side name generator (snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token, so tokens never collide. The hgSession doSaveSessionJson endpoint gained hgS_snapshotType (save a minimal snapshot rather than the whole cart) and hgS_doAnonName (reserve a unique anonymous name without saving, so the top-right dialog can preview the exact link before it is created). Wired three callers to the facility: - hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a full-cart anonymous session. - hgBlat results "Share a link": creates a "blat" snapshot on click and reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the existing doShareReopen), replacing the trash-only reveal. - top-right "Share a link": anonymous links use the reserved server name; logged-in named shares are unchanged. snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse is bumped on every open by the existing session load, so a link stays alive as long as it is used. Meant to run from the trash-cleaner cron. Also folds in the recent Share-dialog work in these files: the auto share name is a short "_" prefix instead of "share_", the dialog previews the link and creates it only when the button is clicked (no orphan session just from opening the dialog), an optional name field with an overwrite warning, and the anonymous save path forces the reap-eligible "__" name. refs #38197
• src/hg/utils/trackDbConditions/conditionBaseline.txt
  • lines changed 45, context: html, text, full: html, text
    0c504951ad80a920183bcb6406ab5f6f1264cc53 Tue Sep 1 10:24:09 2026 -0700
    trackDbConditions: work out what has to be true before a setting does anything, refs #37908 The types list in trackDbLibrary.shtml is flat, so it cannot say "only in coverage mode", "only when the track is in pack", "only when another setting is on". Those conditions are real and there are a lot of them. This finds them by reading the C. harvestConditions.py records, for every read of a track setting, the conditions that enclose it, using offset ranges rather than brace depth so that `if (x) return;` is handled as exactly as a braced block, and an early return contributes its negation to the rest of the function. That alone finds little, because the read is usually plain and the test sits at the caller, so it also builds a call graph and computes the conditions that hold on EVERY path into each function. Only those are reported, which keeps every claim a necessary condition rather than a guess. A function whose address is taken can be a track method and is reached from outside the scan, so it reports nothing rather than something false. Render and config are scanned as separate call graphs. What has to be true for a setting to change the picture is a different question from what has to be true for its control to appear, and mixing them empties every intersection. trackDbConditions.py sorts the conditions into kinds and reports them against the documented settings. --check is the cron mode and fails when a documented setting gains its first condition or loses its last, since either way the documentation and the code have parted company. Where it stands: 325 settings read across 779 sites. In the render scope 96 have a condition that holds at every read, 43 of them documented, and 27 of those turn on something other than the track type. Known limits, all in the module docstrings: a name built with safef is invisible, a condition carried in a variable to a later use is not followed, and the boundary between drawing and the configuration popup inside hgTracks.c is not clean, which is why configurable and filterBy come back with popup plumbing in their lists.
  • lines changed 14, context: html, text, full: html, text
    fd2771d51dfa6526d51778a1a3e2e553aa75290c Tue Sep 1 10:37:16 2026 -0700
    trackDbConditions: follow a setting to where its value is used, refs #37908 The first pass only saw a condition when it enclosed the read. That misses the commonest shape in the drawing code: the setting is read plainly at the top of a loader, carried into a struct, and used far below behind a test of a different setting. bamColorTag came back unconditional even though it does nothing unless bamColorMode is tag. So follow the value. A name that holds exactly one setting across a file is taken to carry it, including into a struct field of the same name, which is how the value usually travels. Then find where the value is used and intersect the conditions guarding those uses. Two distinctions do the work. A mention at paren depth zero is one side of an assignment or an element of an initializer list, which only moves the value somewhere else, so it is not a use; inside a call it is an argument and it is. Counting the struct initializer as a use put an unguarded site in the set and emptied every intersection. And matching drops the field prefix, so the test written sameString(colorMode, ...) in the loader is the same condition as sameString(btd->colorMode, ...) in the drawer. These are reported as "when used" and kept apart from "always". The every-path conditions are necessary by construction; a use-site condition is only as good as the set of uses found, so it is a strong hint rather than a claim, and it keeps the strict key rather than the loose one for that reason. 43 settings in the render scope are read plainly and used only under a condition, 21 of them documented. Among them: bamColorTag needs bamColorMode=tag, pairSearchRange needs pairEndsByName, speciesCodonDefault needs mafChain and frames, and speciesOrder, speciesGroups and speciesDefaultOff turn out to gate each other.
  • lines changed 14, context: html, text, full: html, text
    b712b918c9fdf2aac65b5f6ceb076e4af584231c Tue Sep 1 12:29:36 2026 -0700
    trackDbConditions: scan the whole library, and check the scanner against known cases, refs #37908 Four refinements, one of which fixes wrong output rather than noisy output. The scanned file list was hand-kept, and it had silently missed netCart.c, chainCart.c, pgSnp.c, hgMaf.c and a dozen more that read track settings on the drawing path. An unscanned read site is worse than an unclassified one: the every-path analysis was claiming a condition holds at every read of a setting while never having seen one of the reads. Three settings were carrying false claims because of it, barChartBars, barChartCategoryUrl and bigDataUrl. So scan hg/lib and hg/cgilib whole and let reachability decide which side of the browser each read belongs to. Coverage goes from 325 settings to 347. An early return is treated as a precondition only near the top of a function. The same shape four hundred lines down is sound but says nothing about the setting, and it was how the jsonp output check at the tail of doTrackForm came to look like a condition on filterBy. A negated disjunction is a conjunction, so NOT (A || B) now splits into NOT A and NOT B. The squishyPack guard was one unsplittable string that was neither a visibility condition nor a coverage one; it is now correctly both. --self-test checks the harvest against ten cases read out of the C by hand. Every one of them broke at least once while this was being built, usually silently, so they are checked rather than trusted, and --check runs them first and refuses to report anything if the scanner itself has moved. It earned its place immediately by catching two misclassifications in the same commit that added it. Also caches the harvest in a temp file keyed on the newest source mtime, since scanning takes forty seconds and reading the output takes several runs. Warm runs are now instant.
  • lines changed 9, context: html, text, full: html, text
    fd380f3ae9c678f8e92b4728d5f614a71764fecd Tue Sep 1 12:56:18 2026 -0700
    trackDbConditions: four fixes found by walking the results against the code, refs #37908 Walking the worklist row by row is what found these; none of them was visible from the summary counts. A read wrapped in another call was invisible to the value map, because it stopped at the first call name. cloneString(trackDbSetting(...)) and atoi(cartOrTdbString(...)) are both common. This is why hideEmptySubtracksSourcesUrl came back as merely composite-only when it in fact also needs hideEmptySubtracks and hideEmptySubtracksMultiBedUrl. A macro in a condition was read as an unknown word, so cartVarExistsAnyLevel(cart, tdb, FALSE, MAF_CHAIN_VAR) did not resolve. That hid the fact that irows is consulted only when mafChain is absent from the cart. Same indirection trap as the chained defines, in a different place. The variable-to-setting map kept only the last assignment. wigFetchMinMaxYWithCart assigns defaultViewLimits from defaultViewLimits and then, if that came back NULL, from viewLimits, so the test in between looked like viewLimits testing itself. It is now resolved at the position of the test, which turns an artifact into the real finding: viewLimits is read only when defaultViewLimits is absent. Two classes of noise removed. "Read the trackDb value when the cart has none" is how every setting with a default resolves, and it was a third of the worklist. A guard on the trackDb type line having words is a sanity check that is true of every track, and it was the whole of what the scan had to say about chainNormScoreAvailable, lollyMaxSize and lollyNoStems.
• src/hg/utils/trackDbConditions/harvestConditions.py
  • lines changed 656, context: html, text, full: html, text
    0c504951ad80a920183bcb6406ab5f6f1264cc53 Tue Sep 1 10:24:09 2026 -0700
    trackDbConditions: work out what has to be true before a setting does anything, refs #37908 The types list in trackDbLibrary.shtml is flat, so it cannot say "only in coverage mode", "only when the track is in pack", "only when another setting is on". Those conditions are real and there are a lot of them. This finds them by reading the C. harvestConditions.py records, for every read of a track setting, the conditions that enclose it, using offset ranges rather than brace depth so that `if (x) return;` is handled as exactly as a braced block, and an early return contributes its negation to the rest of the function. That alone finds little, because the read is usually plain and the test sits at the caller, so it also builds a call graph and computes the conditions that hold on EVERY path into each function. Only those are reported, which keeps every claim a necessary condition rather than a guess. A function whose address is taken can be a track method and is reached from outside the scan, so it reports nothing rather than something false. Render and config are scanned as separate call graphs. What has to be true for a setting to change the picture is a different question from what has to be true for its control to appear, and mixing them empties every intersection. trackDbConditions.py sorts the conditions into kinds and reports them against the documented settings. --check is the cron mode and fails when a documented setting gains its first condition or loses its last, since either way the documentation and the code have parted company. Where it stands: 325 settings read across 779 sites. In the render scope 96 have a condition that holds at every read, 43 of them documented, and 27 of those turn on something other than the track type. Known limits, all in the module docstrings: a name built with safef is invisible, a condition carried in a variable to a later use is not followed, and the boundary between drawing and the configuration popup inside hgTracks.c is not clean, which is why configurable and filterBy come back with popup plumbing in their lists.
  • lines changed 81, context: html, text, full: html, text
    fd2771d51dfa6526d51778a1a3e2e553aa75290c Tue Sep 1 10:37:16 2026 -0700
    trackDbConditions: follow a setting to where its value is used, refs #37908 The first pass only saw a condition when it enclosed the read. That misses the commonest shape in the drawing code: the setting is read plainly at the top of a loader, carried into a struct, and used far below behind a test of a different setting. bamColorTag came back unconditional even though it does nothing unless bamColorMode is tag. So follow the value. A name that holds exactly one setting across a file is taken to carry it, including into a struct field of the same name, which is how the value usually travels. Then find where the value is used and intersect the conditions guarding those uses. Two distinctions do the work. A mention at paren depth zero is one side of an assignment or an element of an initializer list, which only moves the value somewhere else, so it is not a use; inside a call it is an argument and it is. Counting the struct initializer as a use put an unguarded site in the set and emptied every intersection. And matching drops the field prefix, so the test written sameString(colorMode, ...) in the loader is the same condition as sameString(btd->colorMode, ...) in the drawer. These are reported as "when used" and kept apart from "always". The every-path conditions are necessary by construction; a use-site condition is only as good as the set of uses found, so it is a strong hint rather than a claim, and it keeps the strict key rather than the loose one for that reason. 43 settings in the render scope are read plainly and used only under a condition, 21 of them documented. Among them: bamColorTag needs bamColorMode=tag, pairSearchRange needs pairEndsByName, speciesCodonDefault needs mafChain and frames, and speciesOrder, speciesGroups and speciesDefaultOff turn out to gate each other.
  • lines changed 60, context: html, text, full: html, text
    b712b918c9fdf2aac65b5f6ceb076e4af584231c Tue Sep 1 12:29:36 2026 -0700
    trackDbConditions: scan the whole library, and check the scanner against known cases, refs #37908 Four refinements, one of which fixes wrong output rather than noisy output. The scanned file list was hand-kept, and it had silently missed netCart.c, chainCart.c, pgSnp.c, hgMaf.c and a dozen more that read track settings on the drawing path. An unscanned read site is worse than an unclassified one: the every-path analysis was claiming a condition holds at every read of a setting while never having seen one of the reads. Three settings were carrying false claims because of it, barChartBars, barChartCategoryUrl and bigDataUrl. So scan hg/lib and hg/cgilib whole and let reachability decide which side of the browser each read belongs to. Coverage goes from 325 settings to 347. An early return is treated as a precondition only near the top of a function. The same shape four hundred lines down is sound but says nothing about the setting, and it was how the jsonp output check at the tail of doTrackForm came to look like a condition on filterBy. A negated disjunction is a conjunction, so NOT (A || B) now splits into NOT A and NOT B. The squishyPack guard was one unsplittable string that was neither a visibility condition nor a coverage one; it is now correctly both. --self-test checks the harvest against ten cases read out of the C by hand. Every one of them broke at least once while this was being built, usually silently, so they are checked rather than trusted, and --check runs them first and refuses to report anything if the scanner itself has moved. It earned its place immediately by catching two misclassifications in the same commit that added it. Also caches the harvest in a temp file keyed on the newest source mtime, since scanning takes forty seconds and reading the output takes several runs. Warm runs are now instant.
  • lines changed 47, context: html, text, full: html, text
    fd380f3ae9c678f8e92b4728d5f614a71764fecd Tue Sep 1 12:56:18 2026 -0700
    trackDbConditions: four fixes found by walking the results against the code, refs #37908 Walking the worklist row by row is what found these; none of them was visible from the summary counts. A read wrapped in another call was invisible to the value map, because it stopped at the first call name. cloneString(trackDbSetting(...)) and atoi(cartOrTdbString(...)) are both common. This is why hideEmptySubtracksSourcesUrl came back as merely composite-only when it in fact also needs hideEmptySubtracks and hideEmptySubtracksMultiBedUrl. A macro in a condition was read as an unknown word, so cartVarExistsAnyLevel(cart, tdb, FALSE, MAF_CHAIN_VAR) did not resolve. That hid the fact that irows is consulted only when mafChain is absent from the cart. Same indirection trap as the chained defines, in a different place. The variable-to-setting map kept only the last assignment. wigFetchMinMaxYWithCart assigns defaultViewLimits from defaultViewLimits and then, if that came back NULL, from viewLimits, so the test in between looked like viewLimits testing itself. It is now resolved at the position of the test, which turns an artifact into the real finding: viewLimits is read only when defaultViewLimits is absent. Two classes of noise removed. "Read the trackDb value when the cart has none" is how every setting with a default resolves, and it was a third of the worklist. A guard on the trackDb type line having words is a sanity check that is true of every track, and it was the whole of what the scan had to say about chainNormScoreAvailable, lollyMaxSize and lollyNoStems.
• src/hg/utils/trackDbConditions/trackDbConditions.py
  • lines changed 328, context: html, text, full: html, text
    0c504951ad80a920183bcb6406ab5f6f1264cc53 Tue Sep 1 10:24:09 2026 -0700
    trackDbConditions: work out what has to be true before a setting does anything, refs #37908 The types list in trackDbLibrary.shtml is flat, so it cannot say "only in coverage mode", "only when the track is in pack", "only when another setting is on". Those conditions are real and there are a lot of them. This finds them by reading the C. harvestConditions.py records, for every read of a track setting, the conditions that enclose it, using offset ranges rather than brace depth so that `if (x) return;` is handled as exactly as a braced block, and an early return contributes its negation to the rest of the function. That alone finds little, because the read is usually plain and the test sits at the caller, so it also builds a call graph and computes the conditions that hold on EVERY path into each function. Only those are reported, which keeps every claim a necessary condition rather than a guess. A function whose address is taken can be a track method and is reached from outside the scan, so it reports nothing rather than something false. Render and config are scanned as separate call graphs. What has to be true for a setting to change the picture is a different question from what has to be true for its control to appear, and mixing them empties every intersection. trackDbConditions.py sorts the conditions into kinds and reports them against the documented settings. --check is the cron mode and fails when a documented setting gains its first condition or loses its last, since either way the documentation and the code have parted company. Where it stands: 325 settings read across 779 sites. In the render scope 96 have a condition that holds at every read, 43 of them documented, and 27 of those turn on something other than the track type. Known limits, all in the module docstrings: a name built with safef is invisible, a condition carried in a variable to a later use is not followed, and the boundary between drawing and the configuration popup inside hgTracks.c is not clean, which is why configurable and filterBy come back with popup plumbing in their lists.
  • lines changed 25, context: html, text, full: html, text
    fd2771d51dfa6526d51778a1a3e2e553aa75290c Tue Sep 1 10:37:16 2026 -0700
    trackDbConditions: follow a setting to where its value is used, refs #37908 The first pass only saw a condition when it enclosed the read. That misses the commonest shape in the drawing code: the setting is read plainly at the top of a loader, carried into a struct, and used far below behind a test of a different setting. bamColorTag came back unconditional even though it does nothing unless bamColorMode is tag. So follow the value. A name that holds exactly one setting across a file is taken to carry it, including into a struct field of the same name, which is how the value usually travels. Then find where the value is used and intersect the conditions guarding those uses. Two distinctions do the work. A mention at paren depth zero is one side of an assignment or an element of an initializer list, which only moves the value somewhere else, so it is not a use; inside a call it is an argument and it is. Counting the struct initializer as a use put an unguarded site in the set and emptied every intersection. And matching drops the field prefix, so the test written sameString(colorMode, ...) in the loader is the same condition as sameString(btd->colorMode, ...) in the drawer. These are reported as "when used" and kept apart from "always". The every-path conditions are necessary by construction; a use-site condition is only as good as the set of uses found, so it is a strong hint rather than a claim, and it keeps the strict key rather than the loose one for that reason. 43 settings in the render scope are read plainly and used only under a condition, 21 of them documented. Among them: bamColorTag needs bamColorMode=tag, pairSearchRange needs pairEndsByName, speciesCodonDefault needs mafChain and frames, and speciesOrder, speciesGroups and speciesDefaultOff turn out to gate each other.
  • lines changed 129, context: html, text, full: html, text
    b712b918c9fdf2aac65b5f6ceb076e4af584231c Tue Sep 1 12:29:36 2026 -0700
    trackDbConditions: scan the whole library, and check the scanner against known cases, refs #37908 Four refinements, one of which fixes wrong output rather than noisy output. The scanned file list was hand-kept, and it had silently missed netCart.c, chainCart.c, pgSnp.c, hgMaf.c and a dozen more that read track settings on the drawing path. An unscanned read site is worse than an unclassified one: the every-path analysis was claiming a condition holds at every read of a setting while never having seen one of the reads. Three settings were carrying false claims because of it, barChartBars, barChartCategoryUrl and bigDataUrl. So scan hg/lib and hg/cgilib whole and let reachability decide which side of the browser each read belongs to. Coverage goes from 325 settings to 347. An early return is treated as a precondition only near the top of a function. The same shape four hundred lines down is sound but says nothing about the setting, and it was how the jsonp output check at the tail of doTrackForm came to look like a condition on filterBy. A negated disjunction is a conjunction, so NOT (A || B) now splits into NOT A and NOT B. The squishyPack guard was one unsplittable string that was neither a visibility condition nor a coverage one; it is now correctly both. --self-test checks the harvest against ten cases read out of the C by hand. Every one of them broke at least once while this was being built, usually silently, so they are checked rather than trusted, and --check runs them first and refuses to report anything if the scanner itself has moved. It earned its place immediately by catching two misclassifications in the same commit that added it. Also caches the harvest in a temp file keyed on the newest source mtime, since scanning takes forty seconds and reading the output takes several runs. Warm runs are now instant.
  • lines changed 10, context: html, text, full: html, text
    fd380f3ae9c678f8e92b4728d5f614a71764fecd Tue Sep 1 12:56:18 2026 -0700
    trackDbConditions: four fixes found by walking the results against the code, refs #37908 Walking the worklist row by row is what found these; none of them was visible from the summary counts. A read wrapped in another call was invisible to the value map, because it stopped at the first call name. cloneString(trackDbSetting(...)) and atoi(cartOrTdbString(...)) are both common. This is why hideEmptySubtracksSourcesUrl came back as merely composite-only when it in fact also needs hideEmptySubtracks and hideEmptySubtracksMultiBedUrl. A macro in a condition was read as an unknown word, so cartVarExistsAnyLevel(cart, tdb, FALSE, MAF_CHAIN_VAR) did not resolve. That hid the fact that irows is consulted only when mafChain is absent from the cart. Same indirection trap as the chained defines, in a different place. The variable-to-setting map kept only the last assignment. wigFetchMinMaxYWithCart assigns defaultViewLimits from defaultViewLimits and then, if that came back NULL, from viewLimits, so the test in between looked like viewLimits testing itself. It is now resolved at the position of the test, which turns an artifact into the real finding: viewLimits is read only when defaultViewLimits is absent. Two classes of noise removed. "Read the trackDb value when the cart has none" is how every setting with a default resolves, and it was a third of the worklist. A guard on the trackDb type line having words is a sanity check that is true of every track, and it was the whole of what the scan had to say about chainNormScoreAvailable, lollyMaxSize and lollyNoStems.
• src/hg/utils/uiTest/README.md
  • lines changed 189, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/WRITING-TESTS.md
  • lines changed 183, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/lib/browser.js
  • lines changed 85, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/lib/env.js
  • lines changed 317, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/lib/pages.js
  • lines changed 29, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/lib/pw.js
  • lines changed 137, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/lib/run.js
  • lines changed 296, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/lib/site.js
  • lines changed 227, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/lib/wait.js
  • lines changed 113, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/sample.hg.uiTest.conf
  • lines changed 58, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/tests/assert.js
  • lines changed 39, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/tests/makefile
  • lines changed 22, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/tests/t01-env.js
  • lines changed 124, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/tests/t02-needs.js
  • lines changed 56, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/tests/t03-pw.js
  • lines changed 77, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/tests/t04-results.js
  • lines changed 35, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/uiTest
  • lines changed 416, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/uiTest/uiTest.mk
  • lines changed 76, context: html, text, full: html, text
    756582322f8a53fef19e8c24ba902e353ba3623f Thu Aug 27 14:38:34 2026 -0700
    uiTest: shared browser UI test harness, plus an hgTracks example, refs #38188 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/urlCommandCatalog/urlCommandCatalog.py
  • lines changed 44, context: html, text, full: html, text
    008d638cf77546cee712cda8412c13ed2782c838 Sat Sep 5 07:29:12 2026 -0700
    urlCommandCatalog: describe the recovery-email link and the Share dialog. --reconcile was reporting six names the tree reads that were in neither the catalog nor the baseline. Four are URL parameters and get rows; two are cart reads and go in the baseline, by the rule that a cgi* read is a URL parameter and a cart* read is not. recovEmail hgLogin.c:1391, cgiUsualString. The recovery address a confirmation link is claiming, on the hgLogin.do.confirmRecovEmail path. Same shape as newEmail and authorized by the same sig and exp. In hgLogin's excludeVars, so it does not persist. hgS_failIfExists hgSession.c:1096, cgiBoolean. hgS_snapshotType hgSession.c:1099, cgiOptionalString. hgS_doAnonName one more of the hgS_do* commands, so it is listed as a member of the hgS_* family rather than given a row of its own. hgLogin_newRecovEmail1, hgLogin_newRecovEmail2 cartUsualString reads at hgLogin.c:1532 and 1533, the two inputs on the change-recovery-address form. Baselined next to hgLogin_newEmail1 and 2, which are the same thing. While writing those, hgS_shareAnon turned out to be misfiled. It sat in urlNamesNotCataloged.txt as internal state, but hgSession.c:1095 reads it with cgiBoolean, which makes it a request parameter like the two that arrived beside it. It now has a row with them. All three are cartRemove'd in the handler, at hgSession.c:1107 to 1109, so none of them leaks into the saved session. The baseline also moves s and u from hgBlat.c to hgc.c. Neither name changed; hgc.c gained its own reads of the two old BLAT share parameters, and the annotation records one site. refs #37923
  • lines changed 19, context: html, text, full: html, text
    a88795fb31a7f51bef649eb6eedb626b771a8893 Sat Sep 5 08:20:34 2026 -0700
    urlCommandCatalog: the SNP "Set defaults" button persists in the session. Fallout from walking the cart baseline: snp125Defaults_coloring is read with cgiOptionalString at hgTrackUi.c:563, deliberately so, because only a click in this request should clear the SNP colour variables. Nothing then keeps the click out of the session. It is in one saved session and four live carts. Harmless in itself, since the reset only fires when the value arrives on the request and a stored copy does nothing, but a one-shot command that persists is the same defect whatever its blast radius, so it gets a row with leaks=True. Two things the row records because they are not obvious from the name. It looks track-scoped and is not: the prefix is the fixed string snp125Defaults. And listing it in hgTrackUi's excludeVars would not help as that array stands, because hgTrackUi.c:4718 has a stray NULL in the middle of it, so "ajax" and anything added after it are unreachable. That is latent rather than live: hgTrackUi reads ajax from the cart and removes it explicitly, and no cart in the three tables holds it. I sorted through every excludeVars[] in the tree and it is the only one with a NULL before the end. Not fixed here. refs #37923 #37979
• src/hg/utils/urlCommandCatalog/urlNamesNotCataloged.txt
  • lines changed 5, context: html, text, full: html, text
    008d638cf77546cee712cda8412c13ed2782c838 Sat Sep 5 07:29:12 2026 -0700
    urlCommandCatalog: describe the recovery-email link and the Share dialog. --reconcile was reporting six names the tree reads that were in neither the catalog nor the baseline. Four are URL parameters and get rows; two are cart reads and go in the baseline, by the rule that a cgi* read is a URL parameter and a cart* read is not. recovEmail hgLogin.c:1391, cgiUsualString. The recovery address a confirmation link is claiming, on the hgLogin.do.confirmRecovEmail path. Same shape as newEmail and authorized by the same sig and exp. In hgLogin's excludeVars, so it does not persist. hgS_failIfExists hgSession.c:1096, cgiBoolean. hgS_snapshotType hgSession.c:1099, cgiOptionalString. hgS_doAnonName one more of the hgS_do* commands, so it is listed as a member of the hgS_* family rather than given a row of its own. hgLogin_newRecovEmail1, hgLogin_newRecovEmail2 cartUsualString reads at hgLogin.c:1532 and 1533, the two inputs on the change-recovery-address form. Baselined next to hgLogin_newEmail1 and 2, which are the same thing. While writing those, hgS_shareAnon turned out to be misfiled. It sat in urlNamesNotCataloged.txt as internal state, but hgSession.c:1095 reads it with cgiBoolean, which makes it a request parameter like the two that arrived beside it. It now has a row with them. All three are cartRemove'd in the handler, at hgSession.c:1107 to 1109, so none of them leaks into the saved session. The baseline also moves s and u from hgBlat.c to hgc.c. Neither name changed; hgc.c gained its own reads of the two old BLAT share parameters, and the annotation records one site. refs #37923
• src/hg/visiGene/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/inc/htmlSanitize.h
  • lines changed 25, context: html, text, full: html, text
    4037b582757f86eed1c5559eca9ccc7af85c1496 Fri Aug 21 10:00:49 2026 -0700
    lib: add htmlSanitize, an allowlist filter for HTML written elsewhere, refs #38126 htmlSanitize() takes a piece of HTML and returns a copy holding only the elements, attributes and style properties on its lists. An element on the keep list survives with its allowed attributes. A short list of elements that carry nothing for a reader, script and style and form among them, is dropped along with its contents. Every other element loses its tag and keeps its text, so a whole document that somebody saved and pasted in comes out as the article it was meant to be. The lists come from a survey of all 5390 description pages reachable from the public hub list, so they are sized to what hubs actually write. The style attribute is filtered a property at a time, and href and src are checked for a scheme we do not print, after decoding entities and padding. An iframe is kept only when it plays a video from one of a few hosts, and then with a sandbox attribute. htmlSanitizeReport() returns the same copy plus a list of one-line messages naming what came out, for hubCheck to show a hub author. The tokenizer is hand written and forgiving. It never aborts and always returns something, because the HTML it will be handed is often broken. lib/htmlPage.c cannot be reused for this: its parser aborts on bad input.
  • lines changed 6, context: html, text, full: html, text
    fcf433a736a8670c9da0e2af3ac54db24b0aaa3b Mon Aug 24 14:47:47 2026 -0700
    htmlSanitize: rename the ids that come in with the HTML, refs #38126 The description HTML is printed inside a page of ours, and it is our own JavaScript that looks ids up. A page that carries an id we already use puts two elements of that name in one document, and getElementById returns whichever comes first. Two ids in the public hub pages collide with ours today, one of them "content". An id also becomes a property of that name on window, which reaches the guards we write as "typeof X !== 'undefined' && X". Every id, and every name on an anchor, now gets the prefix descPage- . The hyphen means the window property it makes can never be spelled as a JavaScript name, so it cannot stand in for one of our globals either. A link to a name on the same page, href="#x", is rewritten with the same prefix and keeps working. A link that names another page is left alone: it leaves our page for the hub's own file, where the names are unchanged. Measured over the 5390 description pages of the public hubs: 1643 ids and anchor names renamed, no id left without the prefix, and the number of same page links with nothing to point at is 22 before and 22 after, the same ones. No page loses reader-visible text and the hubCheck warnings are unchanged at 118 pages, since a reader sees nothing of this.
• src/isPcr/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/jkOwnLib/LICENSE
  • lines changed 1, context: html, text, full: html, text
    50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
    Fix licensing page ambiguities and LICENSE tag spelling, refs #37781 The license/index.html page had several passages easy to misread: a claim that liftOver was the "only" data exception directly contradicted by the clinical-genetics exceptions listed two paragraphs later, a header/body tension in the graphics-reuse section, a garbled sentence about entity type, a missing "than", and two same-named "Software:" headings covering opposite licensing outcomes (free vs. commercial-restricted). Also fixes the SPDX tag referenced on that page: the LICENSE files in restricted directories used the misspelled "LicenceRef-proprietary" rather than the standard "LicenseRef-proprietary", so grepping the tree for the tag as documented turned up almost nothing. Adds a link to the (currently sandboxed) list of tracks we cannot distribute, next to the clinical-genetics exception examples.
• src/lib/cheapcgi.c
  • lines changed 10, context: html, text, full: html, text
    ac3630ef55d95934c11c0aed992c9aeda1988ed4 Mon Aug 31 13:11:23 2026 -0700
    Fixing an enum type mismatch for osType vs browserType when parsing user agent strings, refs #37947
  • lines changed 20, context: html, text, full: html, text
    03583f9df6c2c04a7e1072b021bbdd73d5072926 Mon Aug 31 15:36:58 2026 -0700
    cheapcgi: skip an empty pair in a query string, refs #38185 cgiParseInputAbort and cgiParseNext end a value at the first separator after it, so an empty pair left the next variable named "&name". Nothing looks that name up, so the variable was lost with no warning. This is what broke the link in #38145: the URL had "&&" in front of hgS_doLoadUrl, so the saved session never loaded and the reporter saw a browser without the tracks he expected. The same empty pair at the end of a query string has no '=' after it, and aborted the CGI instead, so the behavior differed by position. Both parsers now skip the separators of an empty pair and carry on. cgiEncode escapes everything but alphanumerics, '.' and '_', so no encoded name can begin with a separator and none is ever eaten. Checked against the unpatched library over every string of "& ; = a b % +" up to length five, 19608 inputs per parser. Nothing that parsed before now aborts, no variable is ever lost, and every difference is the intended one. 484 inputs to cgiParseInputAbort and 62 to cgiParseNext used to abort and now parse.
• src/lib/filePath.c
  • lines changed 1, context: html, text, full: html, text
    21123cb5a9a1542b7ff01542f29078adc04764a0 Sun Sep 6 07:31:34 2026 -0700
    lib: check every component in isSafeRelativePath The loop that walks the '/'-separated components stepped to the terminator that replaced the slash instead of past it, so it ended after the first component and the rest of the path went unchecked. Callers are gfServer's dynamic mode, which takes genome and genomeDataDir from the client.
• src/lib/htmlSanitize.c
  • lines changed 766, context: html, text, full: html, text
    4037b582757f86eed1c5559eca9ccc7af85c1496 Fri Aug 21 10:00:49 2026 -0700
    lib: add htmlSanitize, an allowlist filter for HTML written elsewhere, refs #38126 htmlSanitize() takes a piece of HTML and returns a copy holding only the elements, attributes and style properties on its lists. An element on the keep list survives with its allowed attributes. A short list of elements that carry nothing for a reader, script and style and form among them, is dropped along with its contents. Every other element loses its tag and keeps its text, so a whole document that somebody saved and pasted in comes out as the article it was meant to be. The lists come from a survey of all 5390 description pages reachable from the public hub list, so they are sized to what hubs actually write. The style attribute is filtered a property at a time, and href and src are checked for a scheme we do not print, after decoding entities and padding. An iframe is kept only when it plays a video from one of a few hosts, and then with a sandbox attribute. htmlSanitizeReport() returns the same copy plus a list of one-line messages naming what came out, for hubCheck to show a hub author. The tokenizer is hand written and forgiving. It never aborts and always returns something, because the HTML it will be handed is often broken. lib/htmlPage.c cannot be reused for this: its parser aborts on bad input.
  • lines changed 118, context: html, text, full: html, text
    f83fd4fd350727eb3a42a709849bab0c4658ab84 Fri Aug 21 10:17:16 2026 -0700
    htmlSanitize: three fixes from a review of it, refs #38126 Read the scheme of an href or src the way a browser arrives at it. A browser turns a numeric character reference into its character before it decides what the scheme is, and it accepts one written with any number of leading zeros and with no closing semicolon at all. Decode those the same way, then insist that what stands in front of the first slash is either a plain scheme we allow or a plain path. A named entity, a backslash or a control character in that part of the value means we do not print the link, because those are the ways the check gets walked around. This keeps the 33 encoded mailto links two hubs write, and they were the only links in the public hubs the plainer rule would have lost. Treat a trailing slash on a kept element as the nothing that HTML says it is. Otherwise <div/> came out as an open div and the rest of our own page sat inside it. Remember when the search for a closing tag has run off the end of the input. Every later search for that same tag runs off the end too, so a page made of two hundred thousand unclosed tags no longer costs one pass over the page each. The unit test grows a case for each of the three.
  • lines changed 13, context: html, text, full: html, text
    71366eb92d89bbb2cf87e2a9293ea019b63981a4 Fri Aug 21 10:22:02 2026 -0700
    htmlSanitize: keep odd characters out of the messages we hand back, refs #38126 An attribute name in HTML can hold almost anything, and the name goes into a message that hubCheck prints to a terminal. Name the attribute only when it reads like a name.
  • lines changed 30, context: html, text, full: html, text
    fcf433a736a8670c9da0e2af3ac54db24b0aaa3b Mon Aug 24 14:47:47 2026 -0700
    htmlSanitize: rename the ids that come in with the HTML, refs #38126 The description HTML is printed inside a page of ours, and it is our own JavaScript that looks ids up. A page that carries an id we already use puts two elements of that name in one document, and getElementById returns whichever comes first. Two ids in the public hub pages collide with ours today, one of them "content". An id also becomes a property of that name on window, which reaches the guards we write as "typeof X !== 'undefined' && X". Every id, and every name on an anchor, now gets the prefix descPage- . The hyphen means the window property it makes can never be spelled as a JavaScript name, so it cannot stand in for one of our globals either. A link to a name on the same page, href="#x", is rewritten with the same prefix and keeps working. A link that names another page is left alone: it leaves our page for the hub's own file, where the names are unchanged. Measured over the 5390 description pages of the public hubs: 1643 ids and anchor names renamed, no id left without the prefix, and the number of same page links with nothing to point at is 22 before and 22 after, the same ones. No page loses reader-visible text and the hubCheck warnings are unchanged at 118 pages, since a reader sees nothing of this.
  • lines changed 70, context: html, text, full: html, text
    c01de718b2fb8e7706a0fede0e5ea4149ac6f378 Mon Aug 31 13:48:13 2026 -0700
    htmlSanitize: six fixes from a second review of it, refs #38126 Filter the style attribute on the text a browser will see. A browser turns a character reference into the character it names before the CSS parser runs, so a value spelled u&#114l( reached the page as url( and walked past the check that is there to stop it. The value is decoded before the check now, and the author is told which property lost its value. Say something when we drop the rest of the page. A tag that never ends, most often an attribute value whose quote is never closed, threw away everything after it and reported nothing, so hubCheck stayed quiet about it. The depth cap did the same. Write a less than sign that starts no tag as &lt;. A browser reads text like "</ " as a comment and swallows markup of ours up to the next '>', which could eat a closing tag we added. Write an attribute once. A browser keeps the first of a repeated attribute and drops the rest. We checked the first and then printed them all, which was only correct because of that rule. Do not stack a second descPage- prefix on an id we already renamed, and do not add a second noopener noreferrer to a rel that already has one. hgCustom hands the text we returned back to us when a custom track is edited and saved again, so both of these grew a little more on every save. The test now re-runs the filter over its own output and prints any case that changes, so a transform that compounds shows up in the diff.
• src/lib/makefile
  • lines changed 2, context: html, text, full: html, text
    4037b582757f86eed1c5559eca9ccc7af85c1496 Fri Aug 21 10:00:49 2026 -0700
    lib: add htmlSanitize, an allowlist filter for HTML written elsewhere, refs #38126 htmlSanitize() takes a piece of HTML and returns a copy holding only the elements, attributes and style properties on its lists. An element on the keep list survives with its allowed attributes. A short list of elements that carry nothing for a reader, script and style and form among them, is dropped along with its contents. Every other element loses its tag and keeps its text, so a whole document that somebody saved and pasted in comes out as the article it was meant to be. The lists come from a survey of all 5390 description pages reachable from the public hub list, so they are sized to what hubs actually write. The style attribute is filtered a property at a time, and href and src are checked for a scheme we do not print, after decoding entities and padding. An iframe is kept only when it plays a video from one of a few hosts, and then with a sandbox attribute. htmlSanitizeReport() returns the same copy plus a list of one-line messages naming what came out, for hubCheck to show a hub author. The tokenizer is hand written and forgiving. It never aborts and always returns something, because the HTML it will be handed is often broken. lib/htmlPage.c cannot be reused for this: its parser aborts on bad input.
  • lines changed 3, context: html, text, full: html, text
    646595ff1c14e1e575a832fc17ec8e238d7ba2e5 Mon Aug 31 14:01:32 2026 -0700
    Merge branch 'hubSanitize': filter the description HTML a hub sends us, refs #38126 A track hub, a custom track and an assembly description page all hand us HTML that we print inside a page of ours. htmlSanitize reduces that HTML to an allowlist of elements, attributes and style properties before we print it. An element that is not on the keep list loses its tag but keeps its text, so a whole pasted document still comes out as the article it was meant to be. Script, style, form and their kin go away with everything inside them. Every id, and every name on an anchor, is renamed with a fixed prefix, so a name the outside HTML chose cannot be one our own JavaScript looks up. A link to a name on the same page is renamed to match. hubCheck now names the parts of a description page that the Browser will not print, so a hub author hears it from us rather than from a page that comes out wrong.
• src/lib/tests/expected/htmlSanitizeTest
  • lines changed 53, context: html, text, full: html, text
    4037b582757f86eed1c5559eca9ccc7af85c1496 Fri Aug 21 10:00:49 2026 -0700
    lib: add htmlSanitize, an allowlist filter for HTML written elsewhere, refs #38126 htmlSanitize() takes a piece of HTML and returns a copy holding only the elements, attributes and style properties on its lists. An element on the keep list survives with its allowed attributes. A short list of elements that carry nothing for a reader, script and style and form among them, is dropped along with its contents. Every other element loses its tag and keeps its text, so a whole document that somebody saved and pasted in comes out as the article it was meant to be. The lists come from a survey of all 5390 description pages reachable from the public hub list, so they are sized to what hubs actually write. The style attribute is filtered a property at a time, and href and src are checked for a scheme we do not print, after decoding entities and padding. An iframe is kept only when it plays a video from one of a few hosts, and then with a sandbox attribute. htmlSanitizeReport() returns the same copy plus a list of one-line messages naming what came out, for hubCheck to show a hub author. The tokenizer is hand written and forgiving. It never aborts and always returns something, because the HTML it will be handed is often broken. lib/htmlPage.c cannot be reused for this: its parser aborts on bad input.
  • lines changed 19, context: html, text, full: html, text
    f83fd4fd350727eb3a42a709849bab0c4658ab84 Fri Aug 21 10:17:16 2026 -0700
    htmlSanitize: three fixes from a review of it, refs #38126 Read the scheme of an href or src the way a browser arrives at it. A browser turns a numeric character reference into its character before it decides what the scheme is, and it accepts one written with any number of leading zeros and with no closing semicolon at all. Decode those the same way, then insist that what stands in front of the first slash is either a plain scheme we allow or a plain path. A named entity, a backslash or a control character in that part of the value means we do not print the link, because those are the ways the check gets walked around. This keeps the 33 encoded mailto links two hubs write, and they were the only links in the public hubs the plainer rule would have lost. Treat a trailing slash on a kept element as the nothing that HTML says it is. Otherwise <div/> came out as an open div and the rest of our own page sat inside it. Remember when the search for a closing tag has run off the end of the input. Every later search for that same tag runs off the end too, so a page made of two hundred thousand unclosed tags no longer costs one pass over the page each. The unit test grows a case for each of the three.
  • lines changed 5, context: html, text, full: html, text
    fcf433a736a8670c9da0e2af3ac54db24b0aaa3b Mon Aug 24 14:47:47 2026 -0700
    htmlSanitize: rename the ids that come in with the HTML, refs #38126 The description HTML is printed inside a page of ours, and it is our own JavaScript that looks ids up. A page that carries an id we already use puts two elements of that name in one document, and getElementById returns whichever comes first. Two ids in the public hub pages collide with ours today, one of them "content". An id also becomes a property of that name on window, which reaches the guards we write as "typeof X !== 'undefined' && X". Every id, and every name on an anchor, now gets the prefix descPage- . The hyphen means the window property it makes can never be spelled as a JavaScript name, so it cannot stand in for one of our globals either. A link to a name on the same page, href="#x", is rewritten with the same prefix and keeps working. A link that names another page is left alone: it leaves our page for the hub's own file, where the names are unchanged. Measured over the 5390 description pages of the public hubs: 1643 ids and anchor names renamed, no id left without the prefix, and the number of same page links with nothing to point at is 22 before and 22 after, the same ones. No page loses reader-visible text and the hubCheck warnings are unchanged at 118 pages, since a reader sees nothing of this.
  • lines changed 21, context: html, text, full: html, text
    c01de718b2fb8e7706a0fede0e5ea4149ac6f378 Mon Aug 31 13:48:13 2026 -0700
    htmlSanitize: six fixes from a second review of it, refs #38126 Filter the style attribute on the text a browser will see. A browser turns a character reference into the character it names before the CSS parser runs, so a value spelled u&#114l( reached the page as url( and walked past the check that is there to stop it. The value is decoded before the check now, and the author is told which property lost its value. Say something when we drop the rest of the page. A tag that never ends, most often an attribute value whose quote is never closed, threw away everything after it and reported nothing, so hubCheck stayed quiet about it. The depth cap did the same. Write a less than sign that starts no tag as &lt;. A browser reads text like "</ " as a comment and swallows markup of ours up to the next '>', which could eat a closing tag we added. Write an attribute once. A browser keeps the first of a repeated attribute and drops the rest. We checked the first and then printed them all, which was only correct because of that rule. Do not stack a second descPage- prefix on an id we already renamed, and do not add a second noopener noreferrer to a rel that already has one. hgCustom hands the text we returned back to us when a custom track is edited and saved again, so both of these grew a little more on every save. The test now re-runs the filter over its own output and prints any case that changes, so a transform that compounds shows up in the diff.
• src/lib/tests/htmlSanitizeTest.c
  • lines changed 63, context: html, text, full: html, text
    4037b582757f86eed1c5559eca9ccc7af85c1496 Fri Aug 21 10:00:49 2026 -0700
    lib: add htmlSanitize, an allowlist filter for HTML written elsewhere, refs #38126 htmlSanitize() takes a piece of HTML and returns a copy holding only the elements, attributes and style properties on its lists. An element on the keep list survives with its allowed attributes. A short list of elements that carry nothing for a reader, script and style and form among them, is dropped along with its contents. Every other element loses its tag and keeps its text, so a whole document that somebody saved and pasted in comes out as the article it was meant to be. The lists come from a survey of all 5390 description pages reachable from the public hub list, so they are sized to what hubs actually write. The style attribute is filtered a property at a time, and href and src are checked for a scheme we do not print, after decoding entities and padding. An iframe is kept only when it plays a video from one of a few hosts, and then with a sandbox attribute. htmlSanitizeReport() returns the same copy plus a list of one-line messages naming what came out, for hubCheck to show a hub author. The tokenizer is hand written and forgiving. It never aborts and always returns something, because the HTML it will be handed is often broken. lib/htmlPage.c cannot be reused for this: its parser aborts on bad input.
  • lines changed 9, context: html, text, full: html, text
    f83fd4fd350727eb3a42a709849bab0c4658ab84 Fri Aug 21 10:17:16 2026 -0700
    htmlSanitize: three fixes from a review of it, refs #38126 Read the scheme of an href or src the way a browser arrives at it. A browser turns a numeric character reference into its character before it decides what the scheme is, and it accepts one written with any number of leading zeros and with no closing semicolon at all. Decode those the same way, then insist that what stands in front of the first slash is either a plain scheme we allow or a plain path. A named entity, a backslash or a control character in that part of the value means we do not print the link, because those are the ways the check gets walked around. This keeps the 33 encoded mailto links two hubs write, and they were the only links in the public hubs the plainer rule would have lost. Treat a trailing slash on a kept element as the nothing that HTML says it is. Otherwise <div/> came out as an open div and the rest of our own page sat inside it. Remember when the search for a closing tag has run off the end of the input. Every later search for that same tag runs off the end too, so a page made of two hundred thousand unclosed tags no longer costs one pass over the page each. The unit test grows a case for each of the three.
  • lines changed 4, context: html, text, full: html, text
    fcf433a736a8670c9da0e2af3ac54db24b0aaa3b Mon Aug 24 14:47:47 2026 -0700
    htmlSanitize: rename the ids that come in with the HTML, refs #38126 The description HTML is printed inside a page of ours, and it is our own JavaScript that looks ids up. A page that carries an id we already use puts two elements of that name in one document, and getElementById returns whichever comes first. Two ids in the public hub pages collide with ours today, one of them "content". An id also becomes a property of that name on window, which reaches the guards we write as "typeof X !== 'undefined' && X". Every id, and every name on an anchor, now gets the prefix descPage- . The hyphen means the window property it makes can never be spelled as a JavaScript name, so it cannot stand in for one of our globals either. A link to a name on the same page, href="#x", is rewritten with the same prefix and keeps working. A link that names another page is left alone: it leaves our page for the hub's own file, where the names are unchanged. Measured over the 5390 description pages of the public hubs: 1643 ids and anchor names renamed, no id left without the prefix, and the number of same page links with nothing to point at is 22 before and 22 after, the same ones. No page loses reader-visible text and the hubCheck warnings are unchanged at 118 pages, since a reader sees nothing of this.
  • lines changed 26, context: html, text, full: html, text
    c01de718b2fb8e7706a0fede0e5ea4149ac6f378 Mon Aug 31 13:48:13 2026 -0700
    htmlSanitize: six fixes from a second review of it, refs #38126 Filter the style attribute on the text a browser will see. A browser turns a character reference into the character it names before the CSS parser runs, so a value spelled u&#114l( reached the page as url( and walked past the check that is there to stop it. The value is decoded before the check now, and the author is told which property lost its value. Say something when we drop the rest of the page. A tag that never ends, most often an attribute value whose quote is never closed, threw away everything after it and reported nothing, so hubCheck stayed quiet about it. The depth cap did the same. Write a less than sign that starts no tag as &lt;. A browser reads text like "</ " as a comment and swallows markup of ours up to the next '>', which could eat a closing tag we added. Write an attribute once. A browser keeps the first of a repeated attribute and drops the rest. We checked the first and then printed them all, which was only correct because of that rule. Do not stack a second descPage- prefix on an id we already renamed, and do not add a second noopener noreferrer to a rel that already has one. hgCustom hands the text we returned back to us when a custom track is edited and saved again, so both of these grew a little more on every save. The test now re-runs the filter over its own output and prints any case that changes, so a transform that compounds shows up in the diff.
• src/lib/tests/makefile
  • lines changed 8, context: html, text, full: html, text
    4037b582757f86eed1c5559eca9ccc7af85c1496 Fri Aug 21 10:00:49 2026 -0700
    lib: add htmlSanitize, an allowlist filter for HTML written elsewhere, refs #38126 htmlSanitize() takes a piece of HTML and returns a copy holding only the elements, attributes and style properties on its lists. An element on the keep list survives with its allowed attributes. A short list of elements that carry nothing for a reader, script and style and form among them, is dropped along with its contents. Every other element loses its tag and keeps its text, so a whole document that somebody saved and pasted in comes out as the article it was meant to be. The lists come from a survey of all 5390 description pages reachable from the public hub list, so they are sized to what hubs actually write. The style attribute is filtered a property at a time, and href and src are checked for a scheme we do not print, after decoding entities and padding. An iframe is kept only when it plays a video from one of a few hosts, and then with a sandbox attribute. htmlSanitizeReport() returns the same copy plus a list of one-line messages naming what came out, for hubCheck to show a hub author. The tokenizer is hand written and forgiving. It never aborts and always returns something, because the HTML it will be handed is often broken. lib/htmlPage.c cannot be reused for this: its parser aborts on bad input.
• src/parasol/lib/paraMessage.c
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/parasol/parasol/parasol.c
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/product/README.txt
  • lines changed 9, context: html, text, full: html, text
    70ac948e82b41ef316523635c04e5e2c4a89c417 Fri Sep 4 16:57:54 2026 -0700
    trackLists: give the variant frequency projects their own table, move the page to goldenPath/help/mirrorTracks.html, add the otto cron line, refs #37781 Half the restricted list was national sequencing cohorts sitting under varFreqs and phasedVars, and in one alphabetical table they buried the tracks people actually write in about, OMIM, HGMD and DECIPHER. Those cohorts now get a table of their own below the rest. The split is read off the trackDb parent chain, so the next cohort added under varFreqs lands in the right table with no edit here. The page moves off the htdocs root to goldenPath/help/mirrorTracks.html, beside mirror.html, which now links to it. That link goes in src/product/README.txt, the pandoc source mirror.html is generated from. The licensing page link follows the move, and the page title changes with the file name. Also adds the weekly otto line, placed above the HGDB_CONF that would otherwise apply to it, and keeps the job from listing itself as a self-updating track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/product/ex.hg.conf
  • lines changed 6, context: html, text, full: html, text
    994528edb7d89068f371fef6d8a5fc72566f7334 Fri Sep 4 13:35:16 2026 -0700
    Adding an opt-in "Keep only last search" checkbox to the new BLAT search form. refs #36292 Behind a new hg.conf setting, blatOnlyLatestCheckbox (default off, documented in ex.hg.conf). When checked, each new BLAT search removes the user's earlier BLAT result tracks in hgc's buildBigPsl (cart variable blatOnlyLatest, remembered per browser in localStorage); unchecked keeps the current accumulate behavior, so site defaults are unchanged. Deletion requires both the hg.conf setting and the cart variable, so a stale cart cannot delete tracks where the feature is off. Independent of, and coexists with, the blatOldTracks hg.conf setting.
  • lines changed 6, context: html, text, full: html, text
    cc23b76a1f2c7dc6a29c227661bfb0ce4d20eb54 Fri Sep 4 13:43:48 2026 -0700
    BLAT page polish from QA: Load example button, corrected input limits, wording fixes. refs #37996 Load example is a chip-style button with a mouseover instead of a hyperlink. Removed the Show input limits modal; corrected the limits in hgTracksHelp.html, stale since 2013 (DNA is 75,000 bases per sequence and 187,500 combined, protein and translated 10,000 and 25,000), added a #blatLimits anchor and pointed the two over-limit BLAT warnings and the form at it. Genome selector: section title Target assembly, type-to-search hint in the dropdown, Shortcuts: label, input and Query type select sized to match. Similar tools sidebar uses house-style hyphens and corrects the findMotif name and description. Share and rename text now states link lifetimes accurately (snapshot links last years, only sessions are permanent). Alignment page buttons match .gbPill, summary strip spacing fixed, New BLAT search button demoted from primary. Documents blatResultsGroup in ex.hg.conf.
  • lines changed 5, context: html, text, full: html, text
    7a69a5535f1f66f23f5b1329ecbe508983ea01fa Fri Sep 4 14:44:10 2026 -0700
    BLAT classic form: announce the new-page switch date via hg.conf, fix stale input limits. refs #37996 New hg.conf setting blatNewFormSwitchDate (documented in ex.hg.conf): when set, the classic form's banner announces "We will be updating this BLAT page on <date>" with an opt-in link, a feedback mailto, and a pointer to the news announcement; unset keeps the previous "We are testing" banner. The date lives in hg.conf so a schedule slip is a config change, not a build patch. Also corrects the classic form's stated input limits (25,000/50,000 bases, stale since 2013) to the enforced 75,000 per sequence and 187,500 combined.
  • lines changed 4, context: html, text, full: html, text
    a9c71c94309db1f28fc29cbb2d9c062e79c7201f Fri Sep 4 14:58:18 2026 -0700
    BLAT dated banner: make the news announcement URL an hg.conf setting. refs #37996 blatNewFormNewsUrl, defaulting to the news archive page, so the banner can point at the announcement's anchor once it is published without a CGI rebuild. Documented in ex.hg.conf.
• src/product/installer/docker/Dockerfile
  • lines changed 1, context: html, text, full: html, text
    f8ce38bb1a25f654f4052f0e44b8328decd8f8ab Thu Sep 3 17:39:21 2026 -0700
    docker: add an OCI description label pointing at the docker help page Sets org.opencontainers.image.description on the base Dockerfile, so it shows up in `docker inspect` for every image we publish: the release amd64 base and arm64 images from buildReleaseDocker.sh, the amd64 beta overlay that FROMs the base, and the kent:tip QA image. The Docker Hub repo page has its own description fields, which are not read from this label; those are set through the Hub API and now carry the same text. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/product/scripts/trackDbCacheCleaner.sh
  • lines changed 32, context: html, text, full: html, text
    811d6fef1fb93efa979d039f4effb7ebe7d98780 Wed Sep 2 10:21:29 2026 -0700
    trackDbCacheCleaner: parse -n with getopts, and allow a cache at the top level, refs #37551 #38211 Code review asked why the option parsing compared $1 to "-n" by hand instead of using getopts, and pointed out that the path depth rule refused a cache directory a mirror put at the top level, such as /mirrorTrash. Use getopts for -n, and move the usage text into a function so that an unknown option prints it too. Drop the depth rule. It also did not do what its comment claimed, since /dev/shm is two levels deep and passed it. Refuse only the root directory, after stripping any trailing slash so that "/" and "//" are both caught. The check on name.txt below it is what decides whether a directory really is a trackDb cache.
• src/utils/bedScore/bedScore.c
  • lines changed 1, context: html, text, full: html, text
    90dc829b2d87b15c4fc4bbc1d814f83d1274ef52 Fri Sep 4 14:00:05 2026 -0700
    Pass element counts, not byte sizes, to the chop routines chopByWhite, chopString and chopByChar take their last argument as a count of elements in the output array. Fifteen call sites passed sizeof(array) instead. For an array of pointers that is eight times the real capacity on a 64-bit build, so the chop could write well past the end of the array. Switched each to ArraySize(). Behaviour is unchanged for any input that already fitted in the array. Built clean: lib, hg/lib, hg/hgTracks, hg/hgc, hg/utils/hubCheck, utils/bedScore, parasol/lib, parasol/parasol. Three directories do not build, but they fail the same way without this change: checkExp is missing htslib link flags, and cgapSageFind and affySplice have stale prototypes in files this does not touch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/utils/codeReviewAi.py
  • lines changed 48, context: html, text, full: html, text
    696f5424e3ce2c3ec88589d0836a3acc9b894d8b Mon Aug 31 12:14:05 2026 -0700
    Stop the daily code review emailing approved digests whose OVERALL STATUS line the model omitted. refs #38020 Since the 38020 fix landed, the model dropped the OVERALL STATUS block in about one digest in six, nearly always alongside the optional sections the prompt invites it to omit, and the fail-open email decision mailed every one of those as a possible FEEDBACK even when every per-commit verdict said APPROVED. Two changes. The prompt now says OVERALL STATUS is not one of the omittable sections and a review without it gets emailed. And when the status line is still unreadable, the per-commit Verdict lines now decide instead of sending unconditionally - but only on a full accounting: exactly one readable verdict per commit reviewed and every one APPROVED. A missing or extra verdict, or any FEEDBACK, still sends, so a real review can still never be dropped. Replayed against all 129 digests since 6 Aug: 17 of the 21 phantom emails stop, the 4 that carried real per-commit FEEDBACK still send, and nothing that skipped before sends now.
• src/utils/hubtools/hubtools
  • lines changed 230, context: html, text, full: html, text
    82747d60dcdd8207c9d3324fe6e8a1ba3417fd28 Thu Sep 3 12:17:58 2026 -0700
    hubtools import session: one hub.txt per session, and output that passes hubCheck, refs #34405 Converting Ian Donaldson's hg38 session (30 custom tracks) turned up five problems, all fixed here: - Custom track lines often say autoScale=OFF, and hubCheck rejects the uppercase spelling. The on/off settings are now lower-cased on the way out, which was 14 of the 15 problems hubCheck reported for that session. - hub.txt was written to outDir/<db>/hub.txt, one hub per assembly, so a session covering two assemblies needed two hubUrls. There is now a single outDir/hub.txt with one genome stanza per assembly and the data files in outDir/<db>/. Two consequences of that had to be handled: track names are not scoped per genome stanza, so the name counter runs across assemblies now; and stanzaKey() returned ".genome" for every genome stanza, which made 'tdb add' on such a hub silently drop all but the last assembly. - The tool said nothing about where the hub went or how to load it. It now prints the path, the track and assembly counts, the hubUrl to open and the hubCheck command. - The hub was labelled "Auto-generated hub". A short session link redirects to a URL that names the session and its owner, so the labels come from the session itself. A local archive is named after the file. An email= line in ~/.hubtools.conf sets the contact address. The 'hub' key in tracks.json was documented but ignored; it works now. - Added the missing hubDescription.html, with a note saying where the hub came from, which clears the last hubCheck warning. Also: outDir/<db>/ is only created when a file goes into it, and a session with no custom tracks aborts instead of writing an empty hub.
• src/utils/qa/assemblyStatsCron.py
  • lines changed 689, context: html, text, full: html, text
    4395aff08f55db6aea215e9717ceed746054840b Wed Sep 2 15:34:32 2026 -0700
    Fix and extend the monthly usage stats cron. refs #38232 Adds the GenArk and hubSpace reporting asked for in the ticket, and fixes several things that were quietly producing wrong numbers. New in the report: a summed GenArk row in the database usage table, a section ranking the top GenArk assemblies, and a closing section listing hubSpace usage with one row per user. The non-public hub list now drops GenArk and curated hubs, keeps only each hubSpace user's busiest hub, and reports track collections and ENCODE search hubs as one summed line each - previously a single person uploading a couple of hundred hubs took over most of the list. Fixes in generateUsageStats.py: a missing space meant db= was never passed to hgTracks, so every assembly was reported with hg38's default tracks. Only the first ~800 byte block of hgTracks' track list was read, so hg38 was analysed with 40 of its 91 defaults. Hub backed databases are now skipped, since hub ids differ between hgcentrals and their track names could never match the logs. Fixes in assemblyStatsCron.py: hgTracks was run from the wrong directory and died before logging anything, so the default track list was parsed out of timestamps and CGI_TIME lines - the "non-default track usage" section was really listing default tracks. The startup cleanup was deleting the hubStatus file that genome-asia copies over each month, which dropped asia-only hubs from the counts from February 2025 onward. The report label was computed as today minus 30 days, so the March run labelled itself January and overwrote it, and no February report has ever been published. Hub lookups no longer stop at the first 20 hubs, and are resolved by streaming each mirror's hubStatus once instead of grepping a 341MB file per hub. Counts for hubs are now keyed on the resolved hub URL rather than the track name, so unrelated hubs sharing a common track name are no longer added together. This moves some numbers: CADD reads 280 rather than 293, because three separate CADD download URLs were previously summed into one figure. Table columns, headers included, are now aligned in the emailed output. Warnings about an unreachable mirror or a missing asia file appear in the report body instead of a stderr stream that the cron discards.
  • lines changed 17, context: html, text, full: html, text
    32515a6797b6e68daa3aa94c663760216b24dd67 Thu Sep 3 13:30:26 2026 -0700
    Usage stats cron fixes from code review. refs #38232 Apply the same db mismatch guard to assemblyStatsCron.py that generateUsageStats.py already had. Both files ask hgTracks for one assembly's default tracks and parse the answer off stderr, but only one of them checked that hgTracks answered about the assembly it was asked about. Without it, a repeat of the db= bug this ticket fixed would quietly fill the default track filter with another assembly's tracks and say nothing. Drop any database name that does not look like one before interpolating it into a shell command. The trimmed logs only ever carry real assembly names, so this is defence in depth rather than a live hole. Fix a comment in resolveHub that still described the lastOkTime fallback, which was replaced by mirror order earlier in this ticket. No change to the report. The row counts for the database usage and non-public hub tables were cut from 15 to 10 in the previous commit, which its message did not mention; that was intentional, to make room for the new GenArk and hubSpace sections.
• src/utils/qa/weeklybld/authorEmail.pl
  • lines changed 1, context: html, text, full: html, text
    8cb00e70bc6a533345d30c0456cd7f43e53a6f40 Wed Sep 2 11:34:55 2026 -0700
    weeklybld: default the authorEmail.pl fallback to the current buildmeister Fall back to braney rather than hiram when a login is not found in the git-reports authors.html.
• src/utils/qa/weeklybld/buildEnv.csh
  • lines changed 2, context: html, text, full: html, text
    a8459150ae249f002ecf6d4213ca0ef8fd6a620e Mon Sep 7 10:01:14 2026 -0700
    v504 preview1 (automated)
• src/utils/redmineCli
  • lines changed 282, context: html, text, full: html, text
    0549a7574450f649cf2444a62876a2db1cca47fe Tue Sep 1 12:18:12 2026 -0700
    redmineCli: add a sql subcommand for read-only reporting queries Questions that span many tickets took hundreds of API calls. They are one join in SQL. The subcommand runs a query against the Redmine database and prints an aligned table, or TSV, or JSON. redmineCli sql --tables redmineCli sql --describe issues redmineCli sql "select id, subject from issues limit 5" It sends read-only statements only, and multi-statement support is off so a semicolon cannot smuggle in a second statement. Credential columns are never printed. --timeout sets max_statement_time, default 30 seconds, so a bad join does not sit on the server. --limit caps printed rows at 500. Credentials come from ~/.hg.conf. read_api_key now shares one config reader with the new code instead of parsing the file itself. No RM.
• src/utils/ts/README
  • lines changed 13, context: html, text, full: html, text
    cd0d053cdf8a7f53ee89a920eab46353797a90ed Mon Aug 31 15:33:37 2026 -0700
    ts: drop the cookie domain so a parked instance can hold a cart, refs #37867 The shared config sets central.domain=.ucsc.edu, and cartWriteCookie puts that on the cart cookie. A parked instance answers on loopback, so the browser dropped the cookie and every request got a fresh cart. Each track then came up at its trackDb default instead of the setting the run had asked for. Nothing errored, which is what made it worth fixing rather than documenting. The page still rendered, so a scripted check measured the defaults and reported a clean pass. Clicking through a park by hand hid it completely, because hgTracks puts the hgsid into the links on its own pages; only a run that navigates by absolute URL has nothing to carry. setCookieDomain writes an empty central.domain into the frozen hg.conf, which leaves the domain attribute off the cookie and makes it host-only. That works whether the instance is reached as localhost or as 127.0.0.1, so neither the tunnel line nor any existing script has to change. HTTPHOST would not do: it uses the request's own host, and an IP address has a dot in it, so 127.0.0.1 becomes a real Domain attribute and the browser rejects that too. The login cookies follow the same setting through getCookieDomainString in wikiLink.c. It is written the way setUdcDir is, idempotent on a marker comment and called from both freeze and conf, so "ts conf NNNNN" retrofits an instance frozen before this existed.
  • lines changed 22, context: html, text, full: html, text
    be830879c66213de693050b5e8697f5826f51ad1 Wed Sep 2 10:56:33 2026 -0700
    ts: serve each ticket sandbox over https as well as http, refs #37867 A parked instance answered only over plain http, so anything a CGI decides from the request scheme could not be exercised in one at all. Apache sets HTTPS=on for a TLS request and cgiServerHttpsIsOn() reads it, so a CGI that branches on it, such as one deciding whether to mark a cookie Secure, always took the same branch in a park no matter what was being tested. Each instance now listens twice: plain http on its registered port, as before, and https on that port plus 1000, from a self-signed certificate generated once and shared by every park on the account. Both listeners serve the same frozen code, so hitting the pair is the comparison. Only the http port is in the registry and the https port is derived from it, so nothing about the existing layout changes and "ts conf NNNNN" adds https to an instance frozen before this. http ports are now kept below the start of the https range so the two cannot overlap. "ts list" prints both, "ts tunnel" forwards both, and "ts port NNNNN ssl" gives the https one on its own, which is how the laptop wrapper asks, rather than repeating the offset in a second file.
• src/utils/ts/ts
  • lines changed 27, context: html, text, full: html, text
    cd0d053cdf8a7f53ee89a920eab46353797a90ed Mon Aug 31 15:33:37 2026 -0700
    ts: drop the cookie domain so a parked instance can hold a cart, refs #37867 The shared config sets central.domain=.ucsc.edu, and cartWriteCookie puts that on the cart cookie. A parked instance answers on loopback, so the browser dropped the cookie and every request got a fresh cart. Each track then came up at its trackDb default instead of the setting the run had asked for. Nothing errored, which is what made it worth fixing rather than documenting. The page still rendered, so a scripted check measured the defaults and reported a clean pass. Clicking through a park by hand hid it completely, because hgTracks puts the hgsid into the links on its own pages; only a run that navigates by absolute URL has nothing to carry. setCookieDomain writes an empty central.domain into the frozen hg.conf, which leaves the domain attribute off the cookie and makes it host-only. That works whether the instance is reached as localhost or as 127.0.0.1, so neither the tunnel line nor any existing script has to change. HTTPHOST would not do: it uses the request's own host, and an IP address has a dot in it, so 127.0.0.1 becomes a real Domain attribute and the browser rejects that too. The login cookies follow the same setting through getCookieDomainString in wikiLink.c. It is written the way setUdcDir is, idempotent on a marker comment and called from both freeze and conf, so "ts conf NNNNN" retrofits an instance frozen before this existed.
  • lines changed 93, context: html, text, full: html, text
    be830879c66213de693050b5e8697f5826f51ad1 Wed Sep 2 10:56:33 2026 -0700
    ts: serve each ticket sandbox over https as well as http, refs #37867 A parked instance answered only over plain http, so anything a CGI decides from the request scheme could not be exercised in one at all. Apache sets HTTPS=on for a TLS request and cgiServerHttpsIsOn() reads it, so a CGI that branches on it, such as one deciding whether to mark a cookie Secure, always took the same branch in a park no matter what was being tested. Each instance now listens twice: plain http on its registered port, as before, and https on that port plus 1000, from a self-signed certificate generated once and shared by every park on the account. Both listeners serve the same frozen code, so hitting the pair is the comparison. Only the http port is in the registry and the https port is derived from it, so nothing about the existing layout changes and "ts conf NNNNN" adds https to an instance frozen before this. http ports are now kept below the start of the https range so the two cannot overlap. "ts list" prints both, "ts tunnel" forwards both, and "ts port NNNNN ssl" gives the https one on its own, which is how the laptop wrapper asks, rather than repeating the offset in a second file.
• src/utils/ts/ts.mac
  • lines changed 10, context: html, text, full: html, text
    be830879c66213de693050b5e8697f5826f51ad1 Wed Sep 2 10:56:33 2026 -0700
    ts: serve each ticket sandbox over https as well as http, refs #37867 A parked instance answered only over plain http, so anything a CGI decides from the request scheme could not be exercised in one at all. Apache sets HTTPS=on for a TLS request and cgiServerHttpsIsOn() reads it, so a CGI that branches on it, such as one deciding whether to mark a cookie Secure, always took the same branch in a park no matter what was being tested. Each instance now listens twice: plain http on its registered port, as before, and https on that port plus 1000, from a self-signed certificate generated once and shared by every park on the account. Both listeners serve the same frozen code, so hitting the pair is the comparison. Only the http port is in the registry and the https port is derived from it, so nothing about the existing layout changes and "ts conf NNNNN" adds https to an instance frozen before this. http ports are now kept below the start of the https range so the two cannot overlap. "ts list" prints both, "ts tunnel" forwards both, and "ts port NNNNN ssl" gives the https one on its own, which is how the laptop wrapper asks, rather than repeating the offset in a second file.

• lines changed: 39022
• files changed: 564