All File Changes

v504_preview2 to v504_base (2026-09-14 to 2026-09-21) v504

• confs/hgwdev.hg.conf
  • lines changed 26, context: html, text, full: html, text
    37d27b154207d247c80d5953ee3294a4893c6db7 Sun Sep 20 01:11:14 2026 -0700
    Installing updated hg.conf files from UCSC servers
• docs/staticPage.lua
  • lines changed 11, context: html, text, full: html, text
    f164d5cd5b9cfecc0c35f7e08350a2a768e46457 Wed Sep 16 11:40:45 2026 -0700
    Sort attributes in staticPage.lua so the docs build is reproducible, no Redmine attributes() walked the attribute table with pairs(), which has no defined order, so any element with two or more attributes came out in a different order from one build to the next. Eight identical builds of gb101.md produced two distinct outputs. It renders the same either way, but it makes the install rsync re-push files nobody edited and makes a byte comparison of any docs change meaningless. Sorting the keys fixes it; no page changes semantically. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• docs/tutorials/gatewayTutorial.md
  • lines changed 2, context: html, text, full: html, text
    0a6c6ae0e873e8c95fc1d6a941a26b50d7e92f36 Wed Sep 16 11:40:45 2026 -0700
    Add a blank line after the horizontal rules in the gb101 and gateway tutorials, no Redmine These two pages put a bare --- rule directly above the next heading or div. Pandoc reads a line of dashes as either a horizontal rule or a table rule depending on what follows it, so the parse can flip when the block underneath changes. It resolves to a rule today, but converting the divs on these pages to fenced divs was enough to turn 5 sections of gb101 into tables and drop every <hr>. The blank line removes the ambiguity. The built HTML is byte-identical before and after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• docs/tutorials/gb101.md
  • lines changed 10, context: html, text, full: html, text
    0a6c6ae0e873e8c95fc1d6a941a26b50d7e92f36 Wed Sep 16 11:40:45 2026 -0700
    Add a blank line after the horizontal rules in the gb101 and gateway tutorials, no Redmine These two pages put a bare --- rule directly above the next heading or div. Pandoc reads a line of dashes as either a horizontal rule or a table rule depending on what follows it, so the parse can flip when the block underneath changes. It resolves to a rule today, but converting the divs on these pages to fenced divs was enough to turn 5 sections of gb101 into tables and drop every <hr>. The blank line removes the ambiguity. The built HTML is byte-identical before and after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/cgilib/cartJson.c
  • lines changed 45, context: html, text, full: html, text
    1db5c89d97fdd3c3580c4ba9b18eba6b5029c97d Thu Sep 17 05:09:56 2026 -0700
    hgSearch: highlight MANE transcript(s) in RefSeq protein-position search results, refs #38285 #Preview2 week - bugs introduced now will need a build patch to fix A gene-symbol + codon-range search (e.g. "BRCA1 100-200") maps to every RefSeq isoform prediction sharing that genomic footprint, which can be dozens of near-identical NP_ accessions with no indication of which one is the clinically-relevant MANE transcript. Cross-reference the mane bigGenePred track by genomic overlap + protein accession and call out the MANE Select/Plus Clinical transcript(s) in their own section above the rest, gated behind showManeInSearch (default off) in hg.conf.
  • lines changed 2, context: html, text, full: html, text
    056f7f77ed59453f48a8821b62526aef2457fc25 Fri Sep 18 05:12:31 2026 -0700
    hgSearch: htmlEncode maneStatus/maneProtAcc before writing to JSON #Preview2 week - bugs introduced now will need a build patch to fix Found by AI code review: these fields were sent to the client unescaped while the sibling posName field was htmlEncode'd, even though hgSearch.js inserts both into innerHTML the same way.
  • lines changed 3, context: html, text, full: html, text
    32fbe1660c603d64e8210df9cc041ecb8feb3f38 Mon Sep 21 09:07:43 2026 -0700
    protect against a NULL cart from API search functions refs #38393
• src/hg/cgilib/tests/bedItemRgbTester.c
  • lines changed 3, context: html, text, full: html, text
    c80f2909a9df53021fb01b455b122414ad73c961 Sun Sep 20 06:51:14 2026 -0700
    move bedItemRgbTester to hg/cgilib/tests, beside the code it tests I said in the last commit that hg/cgilib had no tests directory. It does, and I should have looked rather than inferred. bedCart.c lives in hg/cgilib, so its test belongs there, and the header comment saying otherwise is fixed. Worth knowing about that directory: its test target ran nothing. annoGratorTest is commented out because it needs assemblies and tables the directory cannot assume, so `make test` there printed "tested all" and did no work -- and hg/makefile has been running it all along through TEST_EXTRA. bedItemRgbTest is now the one test it does run. refs #36212, refs #38391
• src/hg/cgilib/tests/expected/bedItemRgbTest
  • lines changed 0, context: html, text, full: html, text
    c80f2909a9df53021fb01b455b122414ad73c961 Sun Sep 20 06:51:14 2026 -0700
    move bedItemRgbTester to hg/cgilib/tests, beside the code it tests I said in the last commit that hg/cgilib had no tests directory. It does, and I should have looked rather than inferred. bedCart.c lives in hg/cgilib, so its test belongs there, and the header comment saying otherwise is fixed. Worth knowing about that directory: its test target ran nothing. annoGratorTest is commented out because it needs assemblies and tables the directory cannot assume, so `make test` there printed "tested all" and did no work -- and hg/makefile has been running it all along through TEST_EXTRA. bedItemRgbTest is now the one test it does run. refs #36212, refs #38391
• src/hg/cgilib/tests/makefile
  • lines changed 15, context: html, text, full: html, text
    c80f2909a9df53021fb01b455b122414ad73c961 Sun Sep 20 06:51:14 2026 -0700
    move bedItemRgbTester to hg/cgilib/tests, beside the code it tests I said in the last commit that hg/cgilib had no tests directory. It does, and I should have looked rather than inferred. bedCart.c lives in hg/cgilib, so its test belongs there, and the header comment saying otherwise is fixed. Worth knowing about that directory: its test target ran nothing. annoGratorTest is commented out because it needs assemblies and tables the directory cannot assume, so `make test` there printed "tested all" and did no work -- and hg/makefile has been running it all along through TEST_EXTRA. bedItemRgbTest is now the one test it does run. refs #36212, refs #38391
• src/hg/encode3/encodeDataWarehouse/edwWebAuthLogin/edwWebAuthLogin.c
  • lines changed 1, context: html, text, full: html, text
    ed2944f615eb64c06c10b868754a60926a04a412 Tue Sep 15 05:47:46 2026 -0700
    Four leftovers from the Content-Type sweep, refs #38353 edwWebAuthLogin and edwWebAuthLogout still printed a blank line of their own after cgiPrintContentType(), which already ends the header block, so every response began with an empty line in the body. hgPhyloPlace built its own Content-Type line from a macro in five places. hgChooseDb's fail() sent a Status line and no Content-Type at all, so a bad request came back with a body apache had no type for.
• src/hg/encode3/encodeDataWarehouse/edwWebAuthLogout/edwWebAuthLogout.c
  • lines changed 1, context: html, text, full: html, text
    ed2944f615eb64c06c10b868754a60926a04a412 Tue Sep 15 05:47:46 2026 -0700
    Four leftovers from the Content-Type sweep, refs #38353 edwWebAuthLogin and edwWebAuthLogout still printed a blank line of their own after cgiPrintContentType(), which already ends the header block, so every response began with an empty line in the body. hgPhyloPlace built its own Content-Type line from a macro in five places. hgChooseDb's fail() sent a Status line and no Content-Type at all, so a bad request came back with a body apache had no type for.
• src/hg/hgChooseDb/hgChooseDb.c
  • lines changed 2, context: html, text, full: html, text
    ed2944f615eb64c06c10b868754a60926a04a412 Tue Sep 15 05:47:46 2026 -0700
    Four leftovers from the Content-Type sweep, refs #38353 edwWebAuthLogin and edwWebAuthLogout still printed a blank line of their own after cgiPrintContentType(), which already ends the header block, so every response began with an empty line in the body. hgPhyloPlace built its own Content-Type line from a macro in five places. hgChooseDb's fail() sent a Status line and no Content-Type at all, so a bad request came back with a body apache had no type for.
• src/hg/hgCustom/hgCustom.c
  • lines changed 15, context: html, text, full: html, text
    b8441cbcbe608abd3fa526418f79c9d3f9cbc38c Thu Sep 17 11:02:37 2026 -0700
    Make hgCustom genome search box selection re-submit the form so you land on the management page if you already had custom tracks on the newly selected assemlby, refs #38372
• src/hg/hgGateway/hgGateway.c
  • lines changed 17, context: html, text, full: html, text
    1b6caec24e9f9acb5aa00306e6019787856a5b33 Thu Sep 17 12:17:08 2026 -0700
    special case genark hub attach when on hgwdev - use the alpha.hub.txt instead of the hub.txt refs #35415
• src/hg/hgGeneGraph/hgGeneGraph
  • lines changed 13, context: html, text, full: html, text
    3cb6f55ce985430c052a0c9cbbbec43e711665b7 Thu Sep 17 06:38:09 2026 -0700
    hgGeneGraph: python 3.8 compatibility and repairs to the feedback form, refs #38369 #Preview2 week - bugs introduced now will need a build patch to fix cgi.escape() was removed from python in 3.8, so all three calls raised AttributeError rather than escaping anything. Use html.escape(). The 'remove interaction' form could not work at all: - the INSERT listed its values positionally against a six column table, which put the timestamp into the comment column and the comment text into the time column. - it never committed, so nothing survived the end of the request. - the CREATE TABLE went through sqlQuery(), which reads cursor.description and therefore cannot run a statement that returns no rows. Now that sqlTableExists() works, the CREATE is skipped where the table is already there, instead of failing with 'table already exists'.
• src/hg/hgHubConnect/hgHubConnect.c
  • lines changed 2, context: html, text, full: html, text
    9d9fff4aef9fd9a187839f72e49c3b5ad37ba34a Tue Sep 15 05:48:47 2026 -0700
    hgHubConnect: encode the hub URL into the Connect button's JS literal, refs #38353 The three sibling handlers nearby all pass it through javaScriptLiteralEncode; this one did not.
  • lines changed 10, context: html, text, full: html, text
    f664e77c7e67c0342ba78b140386c4644d451a32 Thu Sep 17 01:42:56 2026 -0700
    hgHubConnect, lib: name the hub URL when a hub fails to load #Preview2 week - bugs introduced now will need a build patch to fix A hub that could not be opened showed up in the connected-hubs table as an error message and nothing else: the name, description and assembly cells were all empty, because they are only filled in from the trackHub struct that the failed load never produced. The URL was in the row only inside the javascript of the Disconnect and Retry buttons, so there was no way to tell which hub the error was about. Print the URL in the name cell instead. netParseUrl chops up its local copy of the URL as it parses, so none of its errAborts could report what they were looking at. Keep the caller's string and name it in all of them. A URL written with a single slash, https:/host/f, then reported a non-numeric port, because with no :// the protocol defaults to http and the host becomes "https" with an empty port - say what is actually wrong with it. Also tighten the encoding of the hub tables' output and of javascript string literals, and use htmlEncode, not javaScriptLiteralEncode, for the option labels in cgiMakeSelectDropList, which are HTML text.
  • lines changed 21, context: html, text, full: html, text
    0ae2a6a36d7b4c77e8e7d7df2a1049f4412e7099 Fri Sep 18 07:03:44 2026 -0700
    hgHubConnect: let a mirror hand out its own API keys, and stop swallowing links to another host #Preview2 week - bugs introduced now will need a build patch to fix Two of the three problems QA found on #38323. The link in the mirror-only Hub Upload message did not go anywhere. The tab handler in hgHubConnect.js catches every hgHubConnect link with a hash and turns it into a tab switch, and 'Go to Hub Upload on genome.ucsc.edu' has a hash that names a tab on the mirror too, so the click just reopened the tab the reader was already on. It now only intercepts links to the page itself. The API key section was decoupled from storeUserFiles, but only for display: the Generate and Revoke buttons are cartJson requests, and both the javascript that sends them and the code in main() that routes them were still inside the storeUserFiles gate. A site with showHubApiKey on and hubSpace off therefore drew two dead buttons. The key functions move out of hgMyData.js into a new hubApiKey.js that the Hub Development tab includes on its own, main() routes a cartJson request when either setting is on, and the hubSpace file commands stay registered only when hubSpace is actually running. The request also goes to this host's hgHubConnect now rather than to the login host. Keys live in the central database of the server that issues them, so a key made on genome-euro belongs in genome-euro's table. refs #38323
  • lines changed 1, context: html, text, full: html, text
    14ecff0730596f8be147b255afc8acb99e9095e2 Fri Sep 18 09:06:06 2026 -0700
    hgHubConnect: an api key made on one geo mirror now works on all of them, behind syncHubApiKeys #Preview2 week - bugs introduced now will need a build patch to fix #Preview2 week - bugs introduced now will need a build patch to fix Api keys live in the hgcentral of whichever mirror issued them, so a key made on genome.ucsc.edu was rejected on genome-euro and genome-asia, and the botDelay error told the user to go and make a second one. hgHubConnect now tells the other nodes about a key as soon as it generates or revokes one: cjGenerateApiKey and cjRevokeApiKey call syncApiKeyToOtherNodes(), which posts a hgHubSyncApiKey cartJson request to each peer, and cjSyncApiKey() on the far side writes it into that mirror's own table. The peers come from hgcentral.gbNode via the new geoMirrorNotifyOtherNodes(), and the request is signed with login.cookieSalt, which all of a site's mirrors already share - it is what makes the login cookie verifiable on each of them. So neither a list of mirror addresses nor a new shared secret needs provisioning. The notify is best effort: a peer that is down is warn()ed about and skipped, never failing the local generate or revoke, which has already committed by then. All of it is off unless hg.conf says syncHubApiKeys=on. With the gate off the sender returns at once, the receiver refuses the request outright rather than merely being unreachable, and botDelay keeps printing the old server-specific wording, which off is still the truth. Registered in hgConfCatalog.py as a gate, to be flipped once this is released. refs #38323
  • lines changed 7, context: html, text, full: html, text
    33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
    hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323 Six things, all in the syncHubApiKeys path, which is still off everywhere. The sync never reached its handler on a real mirror. It was registered as a cartJson command, so building the cart ran forceUserIdOrCaptcha() first; a peer's request carries no hguid cookie and no apiKey= variable, so any site with cloudFlareSiteKey set -- which is every site that needs api keys in the first place -- answered it with a captcha page. Only hgwdev, where the captcha is commented out, ever ran the handler, which is why this was not caught earlier. doApiKeySyncIfRequested() now answers the request in main() before any cart exists. That also stops each sync leaving a junk userDb and sessionDb row behind: measured, 15 syncs now create none. The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key), so it does not rest on md5 resisting length extension, and it is compared in constant time. It now covers a timestamp too, and a peer refuses anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of now, so a captured sync cannot be replayed later to reinstate a key its owner has since revoked. The fields are joined with newlines and a newline in a userName or apiKey is refused, so one signature cannot cover two different splits of the same string. syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It runs after the local key is already written and in the response, so an errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's outer catch, which threw the response away -- the user saw a failure over a key that was sitting in the database. geoMirrorNotifyOtherNodes() returns each peer's response instead of discarding it, and uses netUrlMustOpenPastHeader, which errAborts on anything but a 200 and hands back the body alone. A peer answering 'bad signature' or 'not enabled on this site' is no longer indistinguishable from success; it is logged to error_log, not warn()ed at the person who clicked the button. Verified against the built CGI run as Apache runs it, with the captcha turned on: a signed sync is accepted and writes the row, a signed revoke removes it, and a replay, a swapped userName, a forged signature and malformed json are all refused. hubSpaceKeysTester covers the signature rules; it still cannot run its database half until hgcentralregress gets the UPDATE and DELETE grant.
• src/hg/hgHubConnect/hgHubConnect.h
  • lines changed 5, context: html, text, full: html, text
    14ecff0730596f8be147b255afc8acb99e9095e2 Fri Sep 18 09:06:06 2026 -0700
    hgHubConnect: an api key made on one geo mirror now works on all of them, behind syncHubApiKeys #Preview2 week - bugs introduced now will need a build patch to fix #Preview2 week - bugs introduced now will need a build patch to fix Api keys live in the hgcentral of whichever mirror issued them, so a key made on genome.ucsc.edu was rejected on genome-euro and genome-asia, and the botDelay error told the user to go and make a second one. hgHubConnect now tells the other nodes about a key as soon as it generates or revokes one: cjGenerateApiKey and cjRevokeApiKey call syncApiKeyToOtherNodes(), which posts a hgHubSyncApiKey cartJson request to each peer, and cjSyncApiKey() on the far side writes it into that mirror's own table. The peers come from hgcentral.gbNode via the new geoMirrorNotifyOtherNodes(), and the request is signed with login.cookieSalt, which all of a site's mirrors already share - it is what makes the login cookie verifiable on each of them. So neither a list of mirror addresses nor a new shared secret needs provisioning. The notify is best effort: a peer that is down is warn()ed about and skipped, never failing the local generate or revoke, which has already committed by then. All of it is off unless hg.conf says syncHubApiKeys=on. With the gate off the sender returns at once, the receiver refuses the request outright rather than merely being unreachable, and botDelay keeps printing the old server-specific wording, which off is still the truth. Registered in hgConfCatalog.py as a gate, to be flipped once this is released. refs #38323
  • lines changed 5, context: html, text, full: html, text
    33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
    hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323 Six things, all in the syncHubApiKeys path, which is still off everywhere. The sync never reached its handler on a real mirror. It was registered as a cartJson command, so building the cart ran forceUserIdOrCaptcha() first; a peer's request carries no hguid cookie and no apiKey= variable, so any site with cloudFlareSiteKey set -- which is every site that needs api keys in the first place -- answered it with a captcha page. Only hgwdev, where the captcha is commented out, ever ran the handler, which is why this was not caught earlier. doApiKeySyncIfRequested() now answers the request in main() before any cart exists. That also stops each sync leaving a junk userDb and sessionDb row behind: measured, 15 syncs now create none. The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key), so it does not rest on md5 resisting length extension, and it is compared in constant time. It now covers a timestamp too, and a peer refuses anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of now, so a captured sync cannot be replayed later to reinstate a key its owner has since revoked. The fields are joined with newlines and a newline in a userName or apiKey is refused, so one signature cannot cover two different splits of the same string. syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It runs after the local key is already written and in the response, so an errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's outer catch, which threw the response away -- the user saw a failure over a key that was sitting in the database. geoMirrorNotifyOtherNodes() returns each peer's response instead of discarding it, and uses netUrlMustOpenPastHeader, which errAborts on anything but a 200 and hands back the body alone. A peer answering 'bad signature' or 'not enabled on this site' is no longer indistinguishable from success; it is logged to error_log, not warn()ed at the person who clicked the button. Verified against the built CGI run as Apache runs it, with the captcha turned on: a signed sync is accepted and writes the row, a signed revoke removes it, and a replay, a swapped userName, a forged signature and malformed json are all refused. hubSpaceKeysTester covers the signature rules; it still cannot run its database half until hgcentralregress gets the UPDATE and DELETE grant.
• src/hg/hgHubConnect/hooks/pre-finish.c
  • lines changed 6, context: html, text, full: html, text
    f518e03d8f56af784ec14536a7bdc0daed9a38fd Wed Sep 16 11:11:32 2026 -0700
    Send each file's genome with a hubtools upload so hubSpace rows get a db, and stop the server rewriting a user-uploaded hub.txt, no redmine Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/hgHubConnect/trackHubWizard.c
  • lines changed 72, context: html, text, full: html, text
    14ecff0730596f8be147b255afc8acb99e9095e2 Fri Sep 18 09:06:06 2026 -0700
    hgHubConnect: an api key made on one geo mirror now works on all of them, behind syncHubApiKeys #Preview2 week - bugs introduced now will need a build patch to fix #Preview2 week - bugs introduced now will need a build patch to fix Api keys live in the hgcentral of whichever mirror issued them, so a key made on genome.ucsc.edu was rejected on genome-euro and genome-asia, and the botDelay error told the user to go and make a second one. hgHubConnect now tells the other nodes about a key as soon as it generates or revokes one: cjGenerateApiKey and cjRevokeApiKey call syncApiKeyToOtherNodes(), which posts a hgHubSyncApiKey cartJson request to each peer, and cjSyncApiKey() on the far side writes it into that mirror's own table. The peers come from hgcentral.gbNode via the new geoMirrorNotifyOtherNodes(), and the request is signed with login.cookieSalt, which all of a site's mirrors already share - it is what makes the login cookie verifiable on each of them. So neither a list of mirror addresses nor a new shared secret needs provisioning. The notify is best effort: a peer that is down is warn()ed about and skipped, never failing the local generate or revoke, which has already committed by then. All of it is off unless hg.conf says syncHubApiKeys=on. With the gate off the sender returns at once, the receiver refuses the request outright rather than merely being unreachable, and botDelay keeps printing the old server-specific wording, which off is still the truth. Registered in hgConfCatalog.py as a gate, to be flipped once this is released. refs #38323
  • lines changed 122, context: html, text, full: html, text
    33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
    hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323 Six things, all in the syncHubApiKeys path, which is still off everywhere. The sync never reached its handler on a real mirror. It was registered as a cartJson command, so building the cart ran forceUserIdOrCaptcha() first; a peer's request carries no hguid cookie and no apiKey= variable, so any site with cloudFlareSiteKey set -- which is every site that needs api keys in the first place -- answered it with a captcha page. Only hgwdev, where the captcha is commented out, ever ran the handler, which is why this was not caught earlier. doApiKeySyncIfRequested() now answers the request in main() before any cart exists. That also stops each sync leaving a junk userDb and sessionDb row behind: measured, 15 syncs now create none. The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key), so it does not rest on md5 resisting length extension, and it is compared in constant time. It now covers a timestamp too, and a peer refuses anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of now, so a captured sync cannot be replayed later to reinstate a key its owner has since revoked. The fields are joined with newlines and a newline in a userName or apiKey is refused, so one signature cannot cover two different splits of the same string. syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It runs after the local key is already written and in the response, so an errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's outer catch, which threw the response away -- the user saw a failure over a key that was sitting in the database. geoMirrorNotifyOtherNodes() returns each peer's response instead of discarding it, and uses netUrlMustOpenPastHeader, which errAborts on anything but a 200 and hands back the body alone. A peer answering 'bad signature' or 'not enabled on this site' is no longer indistinguishable from success; it is logged to error_log, not warn()ed at the person who clicked the button. Verified against the built CGI run as Apache runs it, with the captcha turned on: a signed sync is accepted and writes the row, a signed revoke removes it, and a replay, a swapped userName, a forged signature and malformed json are all refused. hubSpaceKeysTester covers the signature rules; it still cannot run its database half until hgcentralregress gets the UPDATE and DELETE grant.
• src/hg/hgLogin/hgLogin.c
  • lines changed 153, context: html, text, full: html, text
    ef0b068d8ea6e9ff5582aeeb8626a139c54c654f Mon Sep 14 09:08:07 2026 -0700
    hgLogin: explain ORCID sign-ins, stop duplicate accounts, confirm typed addresses ORCID's OpenID Connect offers only the "openid" scope and carries no email claim, so every first ORCID sign-in landed on the generic "choose a username" page with no explanation of why, and typing an address that already belonged to an account silently created a second account sharing it. The "choose a username" page now says why a new account is always created. It is shown when the provider released no address at all, rather than when the provider is named "orcid", because a mirror can name a provider anything it likes in hg.conf (refs #38213). An address the user typed into that form is no longer taken on trust. If an activated account already holds it, the signup is refused and the user is pointed at a sign-in method that can show the address is theirs. Otherwise the account is created unactivated and the user is sent to confirm the address by mail, instead of being signed in next to a mail nobody had any reason to open. Clicking the provider again before confirming re-sends the link rather than signing in, and opening the link signs in an account that has no password, so a social-login user is not left on a login page with nothing to type. An address the provider itself released is trusted whether or not the provider set email_verified. CILogon leaves that flag at 0 even for a real institutional sign-in, and insisting on it would both force a mail round trip on every CILogon user and stop them ever reaching an account they already have (refs #38339). Also in this change: - confirmChangeEmail activates the account. Opening that link proves the mailbox is the user's, so an account made by a provider that released no address no longer stays unactivated even after its owner confirms an address properly. - an install with login.mailReturnAddr=NOEMAIL no longer tries to send an activation mail it has no way to deliver. - the activation token is drawn with makeRandomKey, the same source already used for the email-link login token and the OAuth state nonce. refs #38341, refs #38339
  • lines changed 63, context: html, text, full: html, text
    0ebe7cd89f87af0c5e8310eb7bed6f65d2596604 Mon Sep 14 14:37:19 2026 -0700
    hgLogin: do not ask for an email address when the provider already gave us one The "choose a username" page showed an editable email box for every social signup, including the providers that do send us an address. For CILogon that meant being asked for something we then took at face value: the box arrived prefilled with CILogon's address, whatever came back was accepted, and the account was created and signed in. A box whose contents we never check is worse than no box, because it reads as a question we check the answer to. The page now asks only when the provider released no usable address. Where the provider gave one, the page says which address the account will use and offers no input at all. completeAccount reads that address from the pending identity rather than from hgLogin_email, so a leftover value in the cart cannot stand in for it. Where the provider released nothing, ORCID being the case that matters, nothing changes: the user is asked for an address, and the account stays inactive until the mailed link is opened. refs #38339, refs #38341
  • lines changed 9, context: html, text, full: html, text
    b1a27c93758e1f12cd212f1293d9f3ec9bbfb0af Mon Sep 14 14:53:21 2026 -0700
    hgLogin: do not blame the provider when it sent an address we cannot use The "choose a username" page decided what to say by asking whether oauthProviderEmail() had returned an address, but that returns NULL both when the provider released nothing and when it released something spc_email_isvalid rejects, which is every address containing a byte >= 127. A user arriving with a non-ASCII address was therefore told that their provider "does not share your email address with us", which is untrue, and then asked to type one. Key the explanation on whether an address arrived at all, and say plainly that we could not use the one we were sent in the case where one did. refs #38341
  • lines changed 71, context: html, text, full: html, text
    4998072e98b35a6fc5c694f1ae56fcd01fefe3bb Tue Sep 15 05:29:18 2026 -0700
    hgLogin: say why a social sign-in is asking for an email confirmation Landing on "a confirmation email has been sent to you" straight after clicking a provider button is confusing on its own: the user asked to sign in, not to fill in a form, and is told to go and read their mail. The page now says first what happened, in one of two ways, because the two cases that reach it are different. A first sign-in through a provider that released no address says that no account uses the address they typed yet, so one is being made for it. An existing account whose address has never been confirmed names that account instead, since telling its owner that no account exists would be untrue. An account that already carries the address the provider just gave us no longer has to confirm anything: that is the same assurance a new signup through the same provider gets, so the account is activated and the user signed in. Accounts left unactivated by the earlier code, which asked for an address and took it on trust, settle themselves on the owner's next sign-in this way. A plain email signup sets none of the new cart variables and that page reads as before. They are html-encoded on output and dropped from an incoming request, so a supplied copy cannot put text on the page.
  • lines changed 138, context: html, text, full: html, text
    c5fa25640623eef3aa9c8f1caaf4be18bab98a77 Tue Sep 15 05:45:03 2026 -0700
    hgLogin: fixes from the code review of the social sign-in work Read the provider label before clearPendingIdentity, not after. cartUsualString hands back the cart's own string and cartRemove frees it, so the confirmation page was naming the provider from memory that had just been released. hgLogin installs pushCarefulMemHandler, which keeps the bytes readable, which is why it looked fine. Let someone who mistyped their address at the choose-a-username page correct it. The confirmation never arrives, and until now there was no way out at all: no password to sign in with, no activated account for the email link, no login cookie for the change-email page, and the user name and provider identity both already taken. The confirmation page now offers a box to replace the address, authorized by the same signed pending identity the account chooser uses, so only a real provider round trip in this browser can get to it. Signing in again while an account waits to be confirmed reuses the token that is still outstanding instead of minting a new one. A new token silently voids the link in the message before it, so a user who clicked the button twice and opened the first mail was told the link was invalid. Promise the account page only where it exists: the change-email flow is behind login.emailLink, which is off by default, and since the address box went away a user there would have had no way to change an address we told them they could change. Clear the confirmation page's explanation variables in the plain signup path too. The hand-off to that page is a JavaScript redirect, so a user who never lands on it leaves them in the cart, and the next plain signup rendered somebody else's explanation over an unrelated address.
  • lines changed 12, context: html, text, full: html, text
    dccbd05b94e1a9323a61a14df17ac93b25aee9d3 Thu Sep 17 09:48:20 2026 -0700
    Updating hgLogin's recovery-email wording (heading, menu link, description, and confirmation mail) for clarity and to match Change password/Change email, refs #38197 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
  • lines changed 139, context: html, text, full: html, text
    83dd847b7449dd0fa83c22a4d60854f8d20d2847 Fri Sep 18 08:51:14 2026 -0700
    hgLogin: stop telling a CILogon user that CILogon shares no email address #Preview2 week - bugs introduced now will need a build patch to fix QA found three messages on the social sign-in pages claiming the provider does not give us an address. That was written when ORCID, which really releases none, was the only provider reaching those pages. CILogon does release the institution's address; we drop it because CILogon never marks it verified and login.oauth.cilogon.trustEmail is what says we may take it anyway. Once the address is dropped, nothing downstream could tell "released nothing" from "released something we would not take". oauthFetchIdentity now keeps the dropped address in a separate field that matching and sign-in never look at, and the three messages read it: the choose a username page, the confirmation page, and the one that turns down a typed address that already belongs to an account. The address box on the choose a username page starts from it as well, since retyping what we were just handed is the last thing a person wants to do. It still has to clear the duplicate check and be confirmed by mail, exactly as a typed address does. Three more things on the confirmation page, all from the same QA pass: The generic "a confirmation email has been sent" paragraph printed below the "use this address instead" form, so it read as if a mail had already gone to whatever was about to be typed in. It now goes above the form. Correcting the address redisplayed the same page with the new address swapped in and nothing to say the change had worked. It now says so. The rejection reason for an address that is already in use was set but never printed, so a refused change looked like nothing happening at all; that is printed now too. Reworded the unconfirmed-account explanation as QA suggested. refs #38339
  • lines changed 16, context: html, text, full: html, text
    4966fb4582e90c758a94961a9df521434b66d941 Fri Sep 18 15:08:08 2026 -0700
    Updating hgLogin's CILogon-unverified-email wording (choose a username page, address-collision message, and new-account confirmation) for clarity and consistency, refs #38339 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
  • lines changed 7, context: html, text, full: html, text
    547d1ed19123e170307c724bb511e0bf6f086416 Mon Sep 21 06:30:13 2026 -0700
    hgLogin: add an "or" divider before the email-sign-in-link button too, refs #38302
• src/hg/hgLogin/oauthLogin.c
  • lines changed 31, context: html, text, full: html, text
    948e76a68b4a62e4f6f90244eeb266342406ce53 Wed Sep 16 03:03:36 2026 -0700
    hgLogin: make trusting a provider's unverified email a per-provider setting Accepting an address a provider had not marked verified was applied to every provider, to keep CILogon usable: it sends the address it got from the user's institution but leaves email_verified at 0, even for a real institutional sign-in. Extending that to everyone was too much. GitHub hands over a primary address whose owner never confirmed it, and any provider a mirror adds to hg.conf was treated the same way, so an address nobody had checked was enough to be signed in to an existing account that used it. New login.oauth.<name>.trustEmail, off unless an admin sets it, says that a named provider's address may be taken without email_verified. It belongs on a provider that reads the address from somewhere the user cannot type into, which is what CILogon does and what GitHub does not. Where it is off and the provider did not verify the address, the address is dropped rather than refused, and the sign-in proceeds as one that arrived with no address at all, which is already the ORCID case: the user is asked for an address and the account does not work until they open the link mailed to it. Dropping it also keeps it out of the account matching in resolveIdentity, so it cannot reach an existing account. Documented in product/mirrorManual.txt next to the other provider settings, with the CILogon line mirrors will need, and in oauthLogin.h with the rest of the keys.
  • lines changed 8, context: html, text, full: html, text
    83dd847b7449dd0fa83c22a4d60854f8d20d2847 Fri Sep 18 08:51:14 2026 -0700
    hgLogin: stop telling a CILogon user that CILogon shares no email address #Preview2 week - bugs introduced now will need a build patch to fix QA found three messages on the social sign-in pages claiming the provider does not give us an address. That was written when ORCID, which really releases none, was the only provider reaching those pages. CILogon does release the institution's address; we drop it because CILogon never marks it verified and login.oauth.cilogon.trustEmail is what says we may take it anyway. Once the address is dropped, nothing downstream could tell "released nothing" from "released something we would not take". oauthFetchIdentity now keeps the dropped address in a separate field that matching and sign-in never look at, and the three messages read it: the choose a username page, the confirmation page, and the one that turns down a typed address that already belongs to an account. The address box on the choose a username page starts from it as well, since retyping what we were just handed is the last thing a person wants to do. It still has to clear the duplicate check and be confirmed by mail, exactly as a typed address does. Three more things on the confirmation page, all from the same QA pass: The generic "a confirmation email has been sent" paragraph printed below the "use this address instead" form, so it read as if a mail had already gone to whatever was about to be typed in. It now goes above the form. Correcting the address redisplayed the same page with the new address swapped in and nothing to say the change had worked. It now says so. The rejection reason for an address that is already in use was set but never printed, so a refused change looked like nothing happening at all; that is printed now too. Reworded the unconfirmed-account explanation as QA suggested. refs #38339
• src/hg/hgLogin/oauthLogin.h
  • lines changed 7, context: html, text, full: html, text
    948e76a68b4a62e4f6f90244eeb266342406ce53 Wed Sep 16 03:03:36 2026 -0700
    hgLogin: make trusting a provider's unverified email a per-provider setting Accepting an address a provider had not marked verified was applied to every provider, to keep CILogon usable: it sends the address it got from the user's institution but leaves email_verified at 0, even for a real institutional sign-in. Extending that to everyone was too much. GitHub hands over a primary address whose owner never confirmed it, and any provider a mirror adds to hg.conf was treated the same way, so an address nobody had checked was enough to be signed in to an existing account that used it. New login.oauth.<name>.trustEmail, off unless an admin sets it, says that a named provider's address may be taken without email_verified. It belongs on a provider that reads the address from somewhere the user cannot type into, which is what CILogon does and what GitHub does not. Where it is off and the provider did not verify the address, the address is dropped rather than refused, and the sign-in proceeds as one that arrived with no address at all, which is already the ORCID case: the user is asked for an address and the account does not work until they open the link mailed to it. Dropping it also keeps it out of the account matching in resolveIdentity, so it cannot reach an existing account. Documented in product/mirrorManual.txt next to the other provider settings, with the CILogon line mirrors will need, and in oauthLogin.h with the rest of the keys.
  • lines changed 5, context: html, text, full: html, text
    83dd847b7449dd0fa83c22a4d60854f8d20d2847 Fri Sep 18 08:51:14 2026 -0700
    hgLogin: stop telling a CILogon user that CILogon shares no email address #Preview2 week - bugs introduced now will need a build patch to fix QA found three messages on the social sign-in pages claiming the provider does not give us an address. That was written when ORCID, which really releases none, was the only provider reaching those pages. CILogon does release the institution's address; we drop it because CILogon never marks it verified and login.oauth.cilogon.trustEmail is what says we may take it anyway. Once the address is dropped, nothing downstream could tell "released nothing" from "released something we would not take". oauthFetchIdentity now keeps the dropped address in a separate field that matching and sign-in never look at, and the three messages read it: the choose a username page, the confirmation page, and the one that turns down a typed address that already belongs to an account. The address box on the choose a username page starts from it as well, since retyping what we were just handed is the last thing a person wants to do. It still has to clear the duplicate check and be confirmed by mail, exactly as a typed address does. Three more things on the confirmation page, all from the same QA pass: The generic "a confirmation email has been sent" paragraph printed below the "use this address instead" form, so it read as if a mail had already gone to whatever was about to be typed in. It now goes above the form. Correcting the address redisplayed the same page with the new address swapped in and nothing to say the change had worked. It now says so. The rejection reason for an address that is already in use was set but never printed, so a refused change looked like nothing happening at all; that is printed now too. Reworded the unconfirmed-account explanation as QA suggested. refs #38339
• src/hg/hgPhyloPlace/hgPhyloPlace.c
  • lines changed 16, context: html, text, full: html, text
    ed2944f615eb64c06c10b868754a60926a04a412 Tue Sep 15 05:47:46 2026 -0700
    Four leftovers from the Content-Type sweep, refs #38353 edwWebAuthLogin and edwWebAuthLogout still printed a blank line of their own after cgiPrintContentType(), which already ends the header block, so every response began with an empty line in the body. hgPhyloPlace built its own Content-Type line from a macro in five places. hgChooseDb's fail() sent a Status line and no Content-Type at all, so a bad request came back with a body apache had no type for.
• src/hg/hgSession/backup.c
  • lines changed 35, context: html, text, full: html, text
    547bbe0a4ad38df6b0b7a511d6a2ce24b3625fe7 Thu Sep 17 05:08:41 2026 -0700
    hubtools import session: keep the track order of the session, refs #34405 #Preview2 week - bugs introduced now will need a build patch to fix The custom track backup archive has one file per track and no ordering information: hgSession wrote the files into a directory and tar walks it in readdir order, which is not the order they were written. hubtools globbed them, and a hub track without a priority is shown in alphabetical order of its label, so an imported session came out sorted by label. hgSession now writes the priority hgTracks orders the tracks by into each track line of the archive, either the one the browser assigned when it loaded the track or the cart variable that a drag-and-drop reorder left behind. A track line that names its own priority is left alone. hubtools sorts on that priority and numbers the hub stanzas, continuing the count across assemblies so two genome stanzas cannot interleave. Archives made before this have no priority and fall back to file name order.
• src/hg/hgSession/hgSession.c
  • lines changed 1, context: html, text, full: html, text
    dccbd05b94e1a9323a61a14df17ac93b25aee9d3 Thu Sep 17 09:48:20 2026 -0700
    Updating hgLogin's recovery-email wording (heading, menu link, description, and confirmation mail) for clarity and to match Change password/Change email, refs #38197 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
  • lines changed 225, context: html, text, full: html, text
    1c52aaa92828b6ecc18dd5800fe6650b91daf696 Fri Sep 18 07:22:10 2026 -0700
    hgSession: the new Sessions page also lists the sessions saved on our other servers #Preview2 week - bugs introduced now will need a build patch to fix Every geo mirror node keeps its own namedSessionDb, so a session saved on genome-euro is invisible on genome.ucsc.edu and people do not find it again. The table now holds the sessions from every node, with a new Server column saying where each one lives, and the session name links to the server that holds it. Two new hgSession actions, both answered in main() before the cart is built, so that a request from another node leaves no cart, userDb or sessionDb row behind: hgS_doSessionListJson returns this server's sessions for the user named by the login cookie, as JSON. This is what the other nodes call. hgS_doMirrorSessions asks every other gbNode for that list and returns what came back. It sends all of the requests before reading any of the answers, so the wait is the slowest node rather than the sum of them. The fetch goes through our own server rather than from the browser to the other nodes, which keeps cross-origin requests out of it: apache handles the CORS headers differently on every node, and the two mirrors are administered by Bielefeld and RIKEN, while CGI code reaches them with the release. The user's login cookies, and only those, are passed on (wikiLinkLoginCookieHeader), which works because the nodes share a cookie salt. The page never waits for any of this: the table is drawn from this server's own sessions and the rows from the other servers are merged in as they arrive, keeping the current sort, search and page. A node that does not answer, times out, or runs a release without the endpoint is named in a line under the toolbar instead, and costs nothing else. Sessions from another server are shown but not managed here: no Share, Edit, Overwrite or Delete, no bulk-select checkbox, and the Update now shortcut keeps ignoring them. Table rows are keyed by a new uid rather than by the session name, since the same name can now be in the table twice, once per server. geoMirrorThisNode() and geoMirrorOtherNodes() read the node list from the hgcentral gbNode table, so nothing new has to be configured: browser.node already says which node a server is.
• src/hg/hgSession/hgSession.h
  • lines changed 6, context: html, text, full: html, text
    1c52aaa92828b6ecc18dd5800fe6650b91daf696 Fri Sep 18 07:22:10 2026 -0700
    hgSession: the new Sessions page also lists the sessions saved on our other servers #Preview2 week - bugs introduced now will need a build patch to fix Every geo mirror node keeps its own namedSessionDb, so a session saved on genome-euro is invisible on genome.ucsc.edu and people do not find it again. The table now holds the sessions from every node, with a new Server column saying where each one lives, and the session name links to the server that holds it. Two new hgSession actions, both answered in main() before the cart is built, so that a request from another node leaves no cart, userDb or sessionDb row behind: hgS_doSessionListJson returns this server's sessions for the user named by the login cookie, as JSON. This is what the other nodes call. hgS_doMirrorSessions asks every other gbNode for that list and returns what came back. It sends all of the requests before reading any of the answers, so the wait is the slowest node rather than the sum of them. The fetch goes through our own server rather than from the browser to the other nodes, which keeps cross-origin requests out of it: apache handles the CORS headers differently on every node, and the two mirrors are administered by Bielefeld and RIKEN, while CGI code reaches them with the release. The user's login cookies, and only those, are passed on (wikiLinkLoginCookieHeader), which works because the nodes share a cookie salt. The page never waits for any of this: the table is drawn from this server's own sessions and the rows from the other servers are merged in as they arrive, keeping the current sort, search and page. A node that does not answer, times out, or runs a release without the endpoint is named in a line under the toolbar instead, and costs nothing else. Sessions from another server are shown but not managed here: no Share, Edit, Overwrite or Delete, no bulk-select checkbox, and the Update now shortcut keeps ignoring them. Table rows are keyed by a new uid rather than by the session name, since the same name can now be in the table twice, once per server. geoMirrorThisNode() and geoMirrorOtherNodes() read the node list from the hgcentral gbNode table, so nothing new has to be configured: browser.node already says which node a server is.
• src/hg/hgTablesTest/hgTablesTest.c
  • lines changed 64, context: html, text, full: html, text
    59b4888b34803576809588f0f6189d21e8cf6964 Mon Sep 14 15:56:07 2026 -0700
    hgTablesTest: one unusable page should not end the whole run, refs #38356 testOneTrack called errAbort whenever hgTables returned a page it could not parse, which ended the run there and then. quickSubmit has already recorded that failure in tablesTestList by the time it returns NULL, so the abort was not preserving information -- it was destroying it, because reportSummary never ran and the Total line carrying the error counts was never written. Every log back to v490 has zero Total lines for this reason. Skip the track instead and keep going. The failure still counts as a hard error in the summary, so a bad page is now reported rather than fatal. The same applies to a track page with no main form or no table var, and to the matching cases in testOneGroup. This subsumes the bigPsl exception added in 2016, and drops a sameString() on a trackDb type that would have crashed had the lookup returned NULL. quickSubmit logged the reason a page was unusable only at verbose level 2, which the robot does not run at, so a failure reached the log as a bare "Couldn't select track X" with the cause discarded. Log it at level 1. That is what identified today's two failures as hgTables emitting truncated HTML rather than anything to do with the tracks themselves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 138, context: html, text, full: html, text
    dcc88beac1cdad270235e85e456adafd5e201ceb Wed Sep 16 12:39:55 2026 -0700
    hgTablesTest: catch an abort per test rather than losing the run, refs #38356 The guards added for #38356 covered the three ways a track page could come back unusable. Fifteen other errAbort calls in this program were left, in the output tests and in the per-database setup, and any one of them still ended the run with no summary written. lib/qa.c has done the right thing all along: qaPageGet and qaPageFromForm bracket the fetch and the validation in an errCatch and hand back a qaStatus carrying the message. That is why an unparseable page was already survivable. The test never applied the same pattern to its own code, though it has included errCatch.h all along. So bracket each unit of work: one table, one track, one group, one database, and each of the three uniProt tests that run at the end. Each body keeps its code and moves behind a wrapper of the same name, so the diff is the wrapper and nothing else. recordAbort turns a caught abort into a hard error in the summary with the organism, database, group, track and table that produced it. That step is needed because quickSubmit records how the page fetch went, and nothing recorded what a test then made of a page it did get. The three guards stay as they are. In those cases quickSubmit has already recorded the failure, so restoring the errAbort and catching it again would count one failure twice. The guards are the already-recorded path and the errCatch is the not-recorded-anywhere path. The exit code has to carry the same news. main returned zero whatever happened, which was harmless while the first bad page aborted the process, and is not harmless now that the run carries on: a caller reading only the exit code would be told that a run failing on every track had succeeded. Return 1 when any test hit a hard error, or when no test ran at all. Soft errors do not set it. There the page was read and the answer was wrong, which is a report about hgTables rather than about this run, and it is still in the summary. The root page fetch stays fatal. If the front page cannot be read there is nothing to test.
  • lines changed 2, context: html, text, full: html, text
    9ec2018c4b216f3c3a8e846e638247d425736c03 Thu Sep 17 09:37:50 2026 -0700
    hgTablesTest: end the caught-abort log lines with a newline, refs #38356 recordAbort's two log calls were the only ones in this file with no trailing newline in the format string. Neither verbose nor fprintf adds one, and none of the seventeen errAbort calls here end their own message with a newline either, so errCatch->message->string does not supply the missing one. Every caught abort ran into the next log line, on stderr and in the log file. The counts, the exit code and the Total: line the robot greps are unaffected.
  • lines changed 22, context: html, text, full: html, text
    b14d44f6f6031286fde5318abf67086dd29a1b43 Fri Sep 18 09:52:30 2026 -0700
    hgTablesTest: follow the redirect on the starting url instead of failing later A url with no scheme is fetched over http, and both hgwdev and a sandbox answer plain http with a 301 to https. The run parsed the redirect page, found no form in it, and failed several steps later saying "Null form in htmlPageSetVar", which named neither the url nor the redirect. rootPageGet() now fetches through htmlPageForwarded(), says at verbose 1 when the url changed, and errAborts naming the code if the final status is not 200. Every later request is built from this page, so this puts the whole run on the url the server asked for, not just the first fetch. The robots were never exposed to this: both doHgTablesTestRobot.csh and preview2TablesTestRobot.csh already pass https urls. refs #38356
• src/hg/hgTrackUi/hgTrackUi.c
  • lines changed 2, context: html, text, full: html, text
    e4c2e97b5eb35b8f9b7b71eb13a4c24e6c1ece22 Mon Sep 14 09:19:02 2026 -0700
    hui, hgTrackUi: encode trackDb setting and cart text in HTML attributes, refs #38123
  • lines changed 3, context: html, text, full: html, text
    03bd4721a68ec74cab51164ff2d4b216cd6fe9b0 Tue Sep 15 05:49:00 2026 -0700
    trackDb README and comments: $hgsid is for native description pages only, refs #38353 The README still described $hgsid as a hub-page variable that is empty in native trackDb; both halves of that are now the other way round. The three call sites of hVarSubstTrackDbHtml that still said a hub page is the only reason it exists say what the call does now.
  • lines changed 1, context: html, text, full: html, text
    dd8d4476a691b63b817f4ca86483e9ec46c43859 Thu Sep 17 02:26:31 2026 -0700
    hgTrackUi: fix ineffective spacing after superTrack hide/show-all buttons, refs #38281 #Preview2 week - bugs introduced now will need a build patch to fix margin-bottom on a TD has no effect (table cells don't honor margins), so no space actually appeared below the button row. Use padding-bottom:8px instead.
  • lines changed 8, context: html, text, full: html, text
    ed65d23618ec85f58df1bc341199e904fab55929 Fri Sep 18 00:40:10 2026 -0700
    hgTrackUi: substitute description page variables once, and in the container excerpt too. refs #38283 Removes the hVarSubstTrackDbHtml call in doMiddle. trackUi already calls it at each of the two places that go on to print tdb->html, each after its own quickLift getTrackHtml swap, so the doMiddle call was a second pass over the same text. That second pass ate the escape in $$db, which reached the reader as hg38 in hgTrackUi while hgc correctly showed $db. The plain ajax path returns before any html is printed, so it loses nothing. The Description panel at the top of a subtrack's settings page shows an excerpt of its container's page, and nothing had substituted that one. A hub page using ${db} or ${organism} in its opening paragraph showed the text raw there while the same page rendered correctly further down the screen.
  • lines changed 30, context: html, text, full: html, text
    6d4c020673160d1f301931782f56085fff8967ae Fri Sep 18 12:35:14 2026 -0700
    Fix hgTrackUi 400 when fetching faceted composite metadata on a curated hub assembly. refs #38384 handleFileFetch authorizes a fileUrl either by it sitting under a connected hub's hub.txt directory, or by it matching a whitelisted trackDb setting (metaDataUrl, colorSettingsUrl). The second check was gated on the track not being a hub track, because user hub settings could point anywhere. On a curated hub assembly such as hs1 both checks failed: the track name is hub-prefixed, so the whitelist was skipped, and the metadata sits at /gbdb/hs1/proCapNet/, outside the hub.txt directory /gbdb/hs1/hubs/alpha/. The ProCapNet config page showed "Error loading metadata: HTTP Status: 400" in place of the faceted table. The same track on hg38, a native database, worked. A curated hub's trackDb is admin-written and as trustworthy as a native track's, so let the whitelist check run for it. New trackIsFromCuratedHub matches the track's own hub id against the hub url dbDb names for the assembly, so a user hub attached to the same assembly still does not qualify. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
  • lines changed 3, context: html, text, full: html, text
    21563551af4b927ca8f23ca20d48814bfd9939e0 Sat Sep 19 14:10:47 2026 -0700
    Clarify wording of hgTrackUi's missing-track-name message. refs #38192 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
  • lines changed 7, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
  • lines changed 1, context: html, text, full: html, text
    2eb14fc0378432977d83366a3211c67f8913018d Mon Sep 21 06:21:51 2026 -0700
    hgc and hgTrackUi: open off-site links in a new tab, with rel="noopener noreferrer" offsiteLinksToNewTab() in utils.js runs over the rendered page alongside addHgsidToLinks() and gives every http(s) link whose host is not ours a target=_blank it does not already have, plus rel="noopener noreferrer". Without noopener the page that opens keeps a handle on the tab it came from and can navigate it; without noreferrer the Referer header carries our own URL, which has the session id in it. The href on a track description page is written by whoever wrote the track or the hub, so neither is theoretical. A mailto: or an ftp: link is left alone, and so is every link that stays on this server. jsAddHgsidToLinks() now emits both calls and is renamed jsFixUpPageLinks(). hgTracks.js runs the same pass over the track description popup: the replace it does before that already puts a target on every link in there, so what this adds is the rel on the ones that leave. refs #38380
• src/hg/hgTracks/cds.c
  • lines changed 36, context: html, text, full: html, text
    55e11ee5939bfbe8b3f7283b1abb337de5a55ae3 Tue Sep 15 05:48:03 2026 -0700
    hgTracks: the exon and codon mouseovers disagreed on out-of-frame transcripts, refs #38353 An exon's mouseover worked its codon numbers out as (c+2)/3 and the per-codon mouseover took them from the codon list, which builds them from exonFrames. Where a transcript's annotated CDS does not begin on a codon boundary the two part company: baseColorCodonsFromGenePred makes codon 1 short rather than shifting every number after it, so the arithmetic is off by the missing bases for the rest of the transcript. About 9,000 transcripts in hg38's wgEncodeGencodeCompV48 and 101 in ncbiRefSeqCurated are affected. hg38.knownGene has no exonFrames and never was. The exon note now reads the numbers off the codon list when there is one, and otherwise - zoomed out past the level that builds it - counts them with the same short first codon in mind. The per-codon note measures its c. range off the codon's own bases instead of taking it as 3*codonIndex-2..3*codonIndex, which was wrong for the same reason, and gathers both pieces of a codon split across an intron. Checked on hg38 ncbiRefSeqCurated NM_001372106.1, whose first coding exon has frame 1: exon 3 now reads c.369-602 (p.124-201) and the first per-codon box inside it c.369-371 (p.124), where before the exon said p.123 and the codon box said c.370-372. Also in this file, two things the review turned up while reading it: loadTxCdsBatch() capped its IN list at 2000 accessions and dropped the rest without saying so, which would leave the transcripts past the cap numbered along the genome with nothing to show for it. It queries in chunks now. txCodonIndexForCodon()'s comment claimed any of a codon's three bases gives the same answer. It does not when there is an indel inside the codon, which is the case the whole feature exists for; the code already uses the 5'-most base, so only the comment was wrong. genbankCds txCds is zeroed before baseColorTxAliForGenePred() may leave it untouched.
• src/hg/hgTracks/chainTrack.c
  • lines changed 57, context: html, text, full: html, text
    e37f19d638c858f249b6b3e84586b6a2a3ca4487 Thu Sep 17 09:53:04 2026 -0700
    hgTracks: color a lifted chain track by normScore, the way a native one is lfFromLiftedChain set grayIx to maxShade and took the raw chain score, so a quickLifted chain track asked to color by score drew every chain in the darkest shade, and the dense mode sort by score ordered them differently from the native track. chainLoadItems reads normScore out of the chain table and uses it for both. normScore is a column of the SQL chain table and is in no bigChain file, so only the SQL side can do this, which is also the only side the native loaders do it on. chainLoadRange throws the column away and struct chain has nowhere to keep it, so chainNormScoreHash reads it separately, once per source range, and only when the track is coloring by score. chainDbNormScoreAvailable only reads a trackDb setting, so the column count is checked before the row is indexed. Verified with a trackDb stanza lifting hg19 chainSelf onto hg38: the same 1846 items drew in 7 distinct colors before and 30 after. Note a hub cannot reach this path, because chain is not a hub track type, so it takes a stanza of ours carrying quickLiftUrl. Found in the v504 code review, refs #38349. refs #38249
• src/hg/hgTracks/hgTracks.c
  • lines changed 44, context: html, text, full: html, text
    eb7e6ccb9b23c2dd5cd82de0237c6f25aecd2da1 Thu Sep 17 06:02:11 2026 -0700
    Hub tracks must not claim an undecorated name that a native track owns #Preview2 week - bugs introduced now will need a build patch to fix A hub track can be named on a URL, or found in the cart, without its "hub_<id>_" prefix, so that hub links stay readable. Nothing checked whether the assembly already had a track of that name, so when it did, the hub track took the variable: with the MethBase2 hub attached, setting the native hg38 Methbase track's visibility moved it to hub_<id>_Methbase and dropped it from the native track, which then did not draw. The two share the name "Methbase", and their subtracks share names too, so the same went for the _sel checkboxes. hubTrackOwnsBareName() now gates the seven places that fall back to the bare name - track and subtrack visibility, supertrack visibility, the two _hideKids variables, and the _sel checkbox. The bare name is the hub track's only where the assembly has no track of that name; otherwise it belongs to the native one, whose own pass over the track list consumes it. It answers from the track list loadFromTrackDb() has already built, so it costs a hash lookup: hub tracks are always prefixed in that list, so an undecorated name can only match a native track. A caller with no list to register (hgTrackUi) falls back to a single-row trackDb query per distinct bare name, cached. With 500 hub subtracks visible and 500 bare names on the URL - far past anything real - that is 189 queries either way, against 1189 for a query per name. Also clarifies the comments on hTrackDbWithCartVersion() and tdbForTrack(): the "this result is cached" note means the shared-memory trackDb cache only, which is off unless cacheTrackDbDir is set, and there is no memoizing besides it - so hTrackDbForTrack() per name reloads the whole trackDb each call. Measured at 57 seconds for 500 names before I abandoned that approach. refs #34246
  • lines changed 3, context: html, text, full: html, text
    07f0635fd352e098f836664c043aff50449de94e Fri Sep 18 00:17:47 2026 -0700
    hgTracks: reword the density-mode track labels per QA. refs #38279 Say "density graph" to match the checkbox on the track settings page, and name the escape that actually works for each cause: dense mode clears the too-many-items case but does nothing for the maxWindowCoverage one, so the advice was on the wrong message. The window-size label was 90 characters, long enough to push the center label past the edge of the image at ordinary widths and clip the track name along with it.
  • lines changed 110, context: html, text, full: html, text
    3bcf86a0995e5a15ad5a2a8ff5533f48d5d43653 Fri Sep 18 07:15:28 2026 -0700
    hgTracks: tell the user what a session load just did, refs #38157 #Preview2 week - bugs introduced now will need a build patch to fix When a saved session is loaded it replaces the whole cart, and the view the user had before is gone without a word. hgTracks now shows a note on that page saying which session was opened, who saved it and when, the session description if it has one, and that the previous configuration cannot be brought back. Only on that one page: cart.c leaves a marker when a full (non-merge) session load happens and hgTracks takes it out again. Recommended track sets merge into the cart, so they get no note. The note uses a new notifBoxOnce() in utils.js, which has only a Close button: a one-page message does not need 'Don't show again'. Its .notifBoxOnce style is the card grey of the new Sessions page, and it sets no width, so unlike the other notifBoxes it cannot stick out of the window. Switch off with sessionLoadNotice=off in hg.conf.
• src/hg/hgTracks/hgTracks.h
  • lines changed 4, context: html, text, full: html, text
    4d36c5c0a9c01a918b9f54ee7e46d6373bfe89f3 Wed Sep 16 11:39:00 2026 -0700
    hgTracks: a dense row can have one clickable map box per item, refs #38364 In dense the whole row is covered by a single map box that switches the track to pack, so nothing inside a dense track points at a details page. To look at an item you have to expand the track, which reloads the page and changes the layout you just set up. With denseClick on, genericDrawItemsFullDense puts down one map box per item as it draws the row. It does this in the draw pass, where the scale and x offset for the window are already in hand, so multi-region windows come out right. The whole draw loop runs before the loop that calls doTrackMap, and map items print in creation order, so the per-item boxes come first in the image map and the existing whole-row box still catches the gaps between items. Clicking empty space in the row, the center label, or the right-click menu all still change the visibility. Nothing had to be removed and the drawn image does not change. A dense row can hold tens of thousands of items, so denseMapItem keeps one flag per pixel of the row and skips an item whose pixels are all claimed by an earlier one. Such a box sits under the earlier one and could never be clicked. That bounds the row at one box per pixel: hg38 simpleRepeat in dense across chr1 at pix=1200 goes from 74,515 map boxes and an 18.5 MB page to 969 boxes and 525 KB. It also decides what happens when several items share a pixel, which is that the first one in the list wins. Three copies of the same early return, all commented "Don't bother if we are imageV2 and a dense child", kept composite subtracks out of this. They now test denseClickEnabled too, which is what lets the hg38 hprcPclai haplotypes work. The fourth copy in wigTrack.c puts down a whole-track box rather than a per-item one and is left alone. Off by default. Set denseClick in hg.conf for every dense track, or a denseClick trackDb setting for one track, which inherits to a container's subtracks. The trackDb setting is not documented yet.
• src/hg/hgTracks/imageV2.c
  • lines changed 21, context: html, text, full: html, text
    77cfc90b4fb2166b9ae955ad40a067549d6d5881 Tue Sep 15 05:29:56 2026 -0700
    hgc share-a-link: force the enclosing superTrack to show The hgc details-page "Share a link" button strips the hgsid and links straight to the page, but a track nested in a superTrack draws nothing on a fresh load of that link since superTracks default to hide. Add tdbTopSuperTrack() so the per-track JSON carries the enclosing superTrack's name, and have the share dialog force <superTrack>=show onto the URL when there is one.
• src/hg/hgTracks/simpleTracks.c
  • lines changed 8, context: html, text, full: html, text
    7bef2434fec624473e81cfec3012f5ff0a2c853f Mon Sep 14 18:12:47 2026 -0700
    Renaming the codon-number tooltip labels to "Genomic codon number" and "Transcript codon number" and rewriting the indel note in plain language, fixing a broken FontAwesome class on the Codon phase link, and rewriting the FAQ's txIndel explanation for accuracy and clarity, refs #38298
  • lines changed 130, context: html, text, full: html, text
    55e11ee5939bfbe8b3f7283b1abb337de5a55ae3 Tue Sep 15 05:48:03 2026 -0700
    hgTracks: the exon and codon mouseovers disagreed on out-of-frame transcripts, refs #38353 An exon's mouseover worked its codon numbers out as (c+2)/3 and the per-codon mouseover took them from the codon list, which builds them from exonFrames. Where a transcript's annotated CDS does not begin on a codon boundary the two part company: baseColorCodonsFromGenePred makes codon 1 short rather than shifting every number after it, so the arithmetic is off by the missing bases for the rest of the transcript. About 9,000 transcripts in hg38's wgEncodeGencodeCompV48 and 101 in ncbiRefSeqCurated are affected. hg38.knownGene has no exonFrames and never was. The exon note now reads the numbers off the codon list when there is one, and otherwise - zoomed out past the level that builds it - counts them with the same short first codon in mind. The per-codon note measures its c. range off the codon's own bases instead of taking it as 3*codonIndex-2..3*codonIndex, which was wrong for the same reason, and gathers both pieces of a codon split across an intron. Checked on hg38 ncbiRefSeqCurated NM_001372106.1, whose first coding exon has frame 1: exon 3 now reads c.369-602 (p.124-201) and the first per-codon box inside it c.369-371 (p.124), where before the exon said p.123 and the codon box said c.370-372. Also in this file, two things the review turned up while reading it: loadTxCdsBatch() capped its IN list at 2000 accessions and dropped the rest without saying so, which would leave the transcripts past the cap numbered along the genome with nothing to show for it. It queries in chunks now. txCodonIndexForCodon()'s comment claimed any of a codon's three bases gives the same answer. It does not when there is an indel inside the codon, which is the case the whole feature exists for; the code already uses the 5'-most base, so only the comment was wrong. genbankCds txCds is zeroed before baseColorTxAliForGenePred() may leave it untouched.
  • lines changed 71, context: html, text, full: html, text
    4d36c5c0a9c01a918b9f54ee7e46d6373bfe89f3 Wed Sep 16 11:39:00 2026 -0700
    hgTracks: a dense row can have one clickable map box per item, refs #38364 In dense the whole row is covered by a single map box that switches the track to pack, so nothing inside a dense track points at a details page. To look at an item you have to expand the track, which reloads the page and changes the layout you just set up. With denseClick on, genericDrawItemsFullDense puts down one map box per item as it draws the row. It does this in the draw pass, where the scale and x offset for the window are already in hand, so multi-region windows come out right. The whole draw loop runs before the loop that calls doTrackMap, and map items print in creation order, so the per-item boxes come first in the image map and the existing whole-row box still catches the gaps between items. Clicking empty space in the row, the center label, or the right-click menu all still change the visibility. Nothing had to be removed and the drawn image does not change. A dense row can hold tens of thousands of items, so denseMapItem keeps one flag per pixel of the row and skips an item whose pixels are all claimed by an earlier one. Such a box sits under the earlier one and could never be clicked. That bounds the row at one box per pixel: hg38 simpleRepeat in dense across chr1 at pix=1200 goes from 74,515 map boxes and an 18.5 MB page to 969 boxes and 525 KB. It also decides what happens when several items share a pixel, which is that the first one in the list wins. Three copies of the same early return, all commented "Don't bother if we are imageV2 and a dense child", kept composite subtracks out of this. They now test denseClickEnabled too, which is what lets the hg38 hprcPclai haplotypes work. The fourth copy in wigTrack.c puts down a whole-track box rather than a per-item one and is left alone. Off by default. Set denseClick in hg.conf for every dense track, or a denseClick trackDb setting for one track, which inherits to a container's subtracks. The trackDb setting is not documented yet.
  • lines changed 7, context: html, text, full: html, text
    79b7c5730a04d92ca2d917d04752ae6711ff94c7 Thu Sep 17 12:54:22 2026 -0700
    hgTracks: put the denseClick feature behind an hg.conf gate, refs #38364 The hg.conf denseClick flag was a tree-wide default that a trackDb denseClick setting overrode per track. A track carrying denseClick on in trackDb therefore turned the feature on wherever the code was installed, and there was no way to hold it back from a machine. hprcPclai and hprc2annot already carry that line. Make the flag a gate instead. While it is off, which is the default, no track gets clickable dense items whatever its trackDb says. With the gate on, a track opts in for itself with the trackDb setting. Inheritance is unchanged: trackDbSettingOn calls trackDbSetting, which walks the parent chain, so one line on a composite still covers its subtracks. Measured on hg38 chr2:60,000,000-80,000,000 at pix=1200, with the seven default-on hprcPclai haplotypes in dense. hprcPclai opts in and holds at 1249 hgc links with the gate on, 0 with it off. simpleRepeat, which does not opt in, drops from 1024 hgc links to 0 with the gate on, which is the behavior the gate exists to prevent.
  • lines changed 1, context: html, text, full: html, text
    d459a7a421c710541b9e29a43473f502b6d7c8c1 Mon Sep 21 05:30:00 2026 -0700
    hgTracks: the indel is before the shifted codon, not at it, refs #38298 The codon mouseover's note said the transcript has extra or missing bases compared to the genome "at this codon". It is not at it: a codon's transcript number is counted from its 5'-most base, so every codon 3' of an indel is flagged, and the indel that caused the shift always lies before it. On canFam3 NM_001131049.1 one 21-base insertion flags all 730 codons from p.129 to the end of the CDS, none before.
• src/hg/hgVai/hgVai.c
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/hgc/bigBedClick.c
  • lines changed 15, context: html, text, full: html, text
    24a1b93cb1cb3590e987bf5c2910fed600d44a66 Tue Sep 15 05:48:40 2026 -0700
    hgc: a mistyped detailsScript setting must not take down the details page, refs #38353 The null case was covered but nothing else was. jsonParse() aborts on malformed JSON and jsonObjectVal() aborts on any value that is not an object, so a hub with "detailsScript.histogram.myField 5" in its trackDb lost the whole details page, not just that one plot. Caught now: the setting is dropped and the rest of the page is drawn.
• src/hg/hgc/hgc.c
  • lines changed 3, context: html, text, full: html, text
    03bd4721a68ec74cab51164ff2d4b216cd6fe9b0 Tue Sep 15 05:49:00 2026 -0700
    trackDb README and comments: $hgsid is for native description pages only, refs #38353 The README still described $hgsid as a hub-page variable that is empty in native trackDb; both halves of that are now the other way round. The three call sites of hVarSubstTrackDbHtml that still said a hub page is the only reason it exists say what the call does now.
  • lines changed 5, context: html, text, full: html, text
    9acfd5f32458e2f9853a48ab631c122523e66bc8 Thu Sep 17 09:52:46 2026 -0700
    hgc: gate the alignment details page on both halves of the quickLift pair quickLiftAliInfo returned early only when quickLiftDb was absent, so a stanza carrying quickLiftDb and no quickLiftUrl still took the lift path. That left quickLiftFile NULL, and the native branch of htcCdnaAli and htcCdnaAliInWindow then resolved the table name against the assembly on screen while running the query on the source assembly. Nothing unsafe happens, but the reader gets a missing table error rather than an alignment. Verified against a track with quickLiftDb hg19 and no quickLiftUrl, on a table hg38 has and hg19 does not: before, the page ended in "Table 'hg19.altSeqLiftOverPslP11' doesn't exist"; after, the base alignment renders. quickLiftIsLifted is the predicate the rest of the tree already uses for this, which cec5ead0547 said was true everywhere and was not true here. Found in the v504 code review, refs #38349. refs #38249
  • lines changed 14, context: html, text, full: html, text
    8449e4fa9d013670193f6af7f416204f75f04a1e Thu Sep 17 09:52:53 2026 -0700
    bigNet: say so when the type line names a chain track this assembly lacks netChainTdb returns NULL when the chain track a bigNet's type line names is not in trackHash, and the guard on the alignment section then dropped the whole section without a word. A quickLifted net already explained itself in that spot; a hub with a typo in its type line got nothing, and there is no other signal that the name is wrong. The two cases now share one branch, since only a bigNet with no chain track to follow can reach it. The name comes from the type line as the hub spelled it, before the hub prefix is added, and goes out through htmlPrintf because it is a hub's text. Found in the v504 code review, refs #38349. refs #20824
  • lines changed 68, context: html, text, full: html, text
    f19df2f83650190a467c3b1226b3e8a23f6d48c0 Thu Sep 17 17:04:48 2026 -0700
    hgc: build the chain and net details pages with htmlPrintf The chain, snake, quickLift-chain and net click handlers assembled their output with plain printf. Route the values they print -- assembly and organism names, sequence names, positions, the chain id and the quickLiftDb setting -- through htmlPrintf instead, in the four handlers and in the five linkToOtherBrowser* anchor helpers they share. The anchor statements are split rather than converted whole, so that hgTracksName(), cartSidUrlString() and the genark hubUrl stay in a plain printf and only the values go through htmlPrintf. Converting a whole statement rewrites the '/' of ../cgi-bin/hgTracks as an entity on every chain, net, maf and ortho details page; split this way the output is unchanged for ordinary data. Verified against the shared build: the native chainMm39, multiz100way and quickLift "Alignment Differences" pages are byte-identical, and netMm39 differs only in the label "N's", which renders the same. refs #20824
  • lines changed 5, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
  • lines changed 1, context: html, text, full: html, text
    2eb14fc0378432977d83366a3211c67f8913018d Mon Sep 21 06:21:51 2026 -0700
    hgc and hgTrackUi: open off-site links in a new tab, with rel="noopener noreferrer" offsiteLinksToNewTab() in utils.js runs over the rendered page alongside addHgsidToLinks() and gives every http(s) link whose host is not ours a target=_blank it does not already have, plus rel="noopener noreferrer". Without noopener the page that opens keeps a handle on the tab it came from and can navigate it; without noreferrer the Referer header carries our own URL, which has the session id in it. The href on a track description page is written by whoever wrote the track or the hub, so neither is theoretical. A mailto: or an ftp: link is left alone, and so is every link that stays on this server. jsAddHgsidToLinks() now emits both calls and is renamed jsFixUpPageLinks(). hgTracks.js runs the same pass over the track description popup: the replace it does before that already puts a target on every link in there, so what this adds is the rel on the ones that leave. refs #38380
• src/hg/htdocs/FAQ/FAQgenes.html
  • lines changed 52, context: html, text, full: html, text
    7bef2434fec624473e81cfec3012f5ff0a2c853f Mon Sep 14 18:12:47 2026 -0700
    Renaming the codon-number tooltip labels to "Genomic codon number" and "Transcript codon number" and rewriting the indel note in plain language, fixing a broken FontAwesome class on the Codon phase link, and rewriting the FAQ's txIndel explanation for accuracy and clarity, refs #38298
• src/hg/htdocs/goldenPath/help/docker.html
  • lines changed 95, context: html, text, full: html, text
    c928b75199212c8946d11764acee5e9ac68ce6f4 Tue Sep 15 10:52:23 2026 -0700
    Adding a Blat and In-Silico PCR section for Docker assembly hubs and fixing several small existing issues on the docker.html page, refs #35979
• src/hg/htdocs/goldenPath/help/hgSessionHelp.html
  • lines changed 1, context: html, text, full: html, text
    f78e12e52a1bc47b7a69246a1f184d78900eab51 Thu Sep 17 14:46:18 2026 -0700
    hgSessionHelp.html: give the second duplicate anchor its own name, refs #38062 The page carried two <a name="Create"> anchors, one on "Creating a session" and one on "Opening a saved session". Nine pages link to #Create meaning the first one, and they land there only because it comes first in the document. The second is now OpenSaved. Nothing linked to it, and the separate #Open anchor further down belongs to "Opening a shared session", a different section. Found in the v504 code review, #38354. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/htdocs/goldenPath/help/mirrorManual.html
  • lines changed 30, context: html, text, full: html, text
    7c624aa6b9c463d584e9ac0e048a35c6afc697b6 Wed Sep 16 04:59:33 2026 -0700
    add new flag for Oauth: trust OpenID email validated flag or not.
• src/hg/htdocs/goldenPath/help/trackDb/changes.html
  • lines changed 9, context: html, text, full: html, text
    98255f58460415345af88e45ad233599740585e5 Wed Sep 16 11:44:41 2026 -0700
    trackDb docs for the denseClick setting, refs #38364 Documents the setting added in the previous commit, in the five places a new trackDb statement has to appear: tagTypes.tab, trackDbLibrary.shtml, trackDbDoc.html, trackDbHub.v3.html (level-new, since it works in hubs) and changes.html. The type list is bed, bigBed, genePred, bigGenePred, psl and bigPsl, which is the same list decorator carries and matches what the code reaches: the setting hooks genericDrawItemsFullDense, so it applies to anything drawn through the linked-features path. Each type was checked by hand on hg38 in dense with the setting on, counting details links in the image map: ccdsGene, mrna, mrnaBig and knownGene all gained one link per item. The blurb says what happens when several items share a pixel, which is that the first one wins, and that a row drawn from summary data rather than from items is unaffected. That last case is real: linkedFeaturesDraw calls bigBedDrawDense when tg->items is NULL, so a zoomed-out bigBed never reaches this code and has nothing to link. Verified with "make settings", which parses the blurb cleanly, and with hubCheck -settings against the sandbox copy of trackDbHub.v3.html, which reports "denseClick new" and no errors. Regenerating trackDbSettings.yaml and .json also picks up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented on 2026-09-09 without rebuilding the generated files. Nothing is removed.
  • lines changed 14, context: html, text, full: html, text
    157d7eb9f2765db7651e12c14233c56fa7205e96 Thu Sep 17 14:46:12 2026 -0700
    VCF help: add the two settings the sweep missed, and fix a value order, refs #38010 The earlier commit left out sampleMetadataFile and showHardyWeinberg on the grounds that nothing in the tree reads them. Both are read: hgc/vcfClick.c loads the metadata file in vcfGenotypeTable, and reads showHardyWeinberg with cartOrTdbBoolean in vcfGenotypesDetails. Neither call is gated on track type, so both also apply to vcfPhasedTrio, which tagTypes.tab had not allowed; that is corrected here too. sampleColorFile and showHardyWeinberg had no entry in trackDbLibrary.shtml, so they were missing from the trackDb doc pages as well. Both get a blurb, a row in trackDbDoc.html and trackDbHub.v3.html, and a changes.html note. sampleColorFile is the last VCF setting that was recognized but undocumented. vcf.html and the three trackDb doc files listed vcfPhasedColorBy as mendelDiff|deNovo|function|noColor. vcfUi.c prints the radio buttons in the opposite order, so the lists now read noColor|function|deNovo|mendelDiff, the way hapClusterColorBy already matches its own buttons. trackDbSettings.yaml and .json are regenerated. They also pick up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented earlier in this series without a regen. Found in the v504 code review, #38354. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 14, context: html, text, full: html, text
    8d05f42f3315eed8579649f1271899ea8ddd986d Fri Sep 18 00:49:33 2026 -0700
    trackDb hub docs: document the variables a track description page may use. refs #38283 Adds a "Variables in the description page" section to the html setting in trackDbLibrary.shtml, listing ${db}, ${organism}, ${Organism}, ${ORGANISM}, ${date}, ${track}, ${parentTrack} and ${downloadsServer}, with an example of the link a track inside a container can make back to the container's own page. Says plainly that nothing else is substituted, so shell, awk and JavaScript examples on the same page are safe, and that the session id is not available. The section sits after the Example block on purpose. trackDbSettingsGen.py takes the first <pre> in an entry as that setting's example, so a <pre> earlier in the entry overwrites "html docs/myFirstTrack.html" in trackDbSettings.json, which is the file hubCheck -settings reads. Also adds a row to changes.html and regenerates trackDbSettings.yaml. trackDbHub.v3.html needs no change, since this adds no new setting name.
  • lines changed 37, context: html, text, full: html, text
    00e77db940c6eedba27381201442336ced2676a9 Sat Sep 19 07:20:46 2026 -0700
    Merge branch 'master' into denseClick38364 # Conflicts: # src/hg/htdocs/goldenPath/help/trackDb/changes.html # src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.json # src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.yaml
• src/hg/htdocs/goldenPath/help/trackDb/trackDbDoc.html
  • lines changed 3, context: html, text, full: html, text
    98255f58460415345af88e45ad233599740585e5 Wed Sep 16 11:44:41 2026 -0700
    trackDb docs for the denseClick setting, refs #38364 Documents the setting added in the previous commit, in the five places a new trackDb statement has to appear: tagTypes.tab, trackDbLibrary.shtml, trackDbDoc.html, trackDbHub.v3.html (level-new, since it works in hubs) and changes.html. The type list is bed, bigBed, genePred, bigGenePred, psl and bigPsl, which is the same list decorator carries and matches what the code reaches: the setting hooks genericDrawItemsFullDense, so it applies to anything drawn through the linked-features path. Each type was checked by hand on hg38 in dense with the setting on, counting details links in the image map: ccdsGene, mrna, mrnaBig and knownGene all gained one link per item. The blurb says what happens when several items share a pixel, which is that the first one wins, and that a row drawn from summary data rather than from items is unaffected. That last case is real: linkedFeaturesDraw calls bigBedDrawDense when tg->items is NULL, so a zoomed-out bigBed never reaches this code and has nothing to link. Verified with "make settings", which parses the blurb cleanly, and with hubCheck -settings against the sandbox copy of trackDbHub.v3.html, which reports "denseClick new" and no errors. Regenerating trackDbSettings.yaml and .json also picks up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented on 2026-09-09 without rebuilding the generated files. Nothing is removed.
  • lines changed 7, context: html, text, full: html, text
    157d7eb9f2765db7651e12c14233c56fa7205e96 Thu Sep 17 14:46:12 2026 -0700
    VCF help: add the two settings the sweep missed, and fix a value order, refs #38010 The earlier commit left out sampleMetadataFile and showHardyWeinberg on the grounds that nothing in the tree reads them. Both are read: hgc/vcfClick.c loads the metadata file in vcfGenotypeTable, and reads showHardyWeinberg with cartOrTdbBoolean in vcfGenotypesDetails. Neither call is gated on track type, so both also apply to vcfPhasedTrio, which tagTypes.tab had not allowed; that is corrected here too. sampleColorFile and showHardyWeinberg had no entry in trackDbLibrary.shtml, so they were missing from the trackDb doc pages as well. Both get a blurb, a row in trackDbDoc.html and trackDbHub.v3.html, and a changes.html note. sampleColorFile is the last VCF setting that was recognized but undocumented. vcf.html and the three trackDb doc files listed vcfPhasedColorBy as mendelDiff|deNovo|function|noColor. vcfUi.c prints the radio buttons in the opposite order, so the lists now read noColor|function|deNovo|mendelDiff, the way hapClusterColorBy already matches its own buttons. trackDbSettings.yaml and .json are regenerated. They also pick up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented earlier in this series without a regen. Found in the v504 code review, #38354. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/htdocs/goldenPath/help/trackDb/trackDbHub.v3.html
  • lines changed 3, context: html, text, full: html, text
    98255f58460415345af88e45ad233599740585e5 Wed Sep 16 11:44:41 2026 -0700
    trackDb docs for the denseClick setting, refs #38364 Documents the setting added in the previous commit, in the five places a new trackDb statement has to appear: tagTypes.tab, trackDbLibrary.shtml, trackDbDoc.html, trackDbHub.v3.html (level-new, since it works in hubs) and changes.html. The type list is bed, bigBed, genePred, bigGenePred, psl and bigPsl, which is the same list decorator carries and matches what the code reaches: the setting hooks genericDrawItemsFullDense, so it applies to anything drawn through the linked-features path. Each type was checked by hand on hg38 in dense with the setting on, counting details links in the image map: ccdsGene, mrna, mrnaBig and knownGene all gained one link per item. The blurb says what happens when several items share a pixel, which is that the first one wins, and that a row drawn from summary data rather than from items is unaffected. That last case is real: linkedFeaturesDraw calls bigBedDrawDense when tg->items is NULL, so a zoomed-out bigBed never reaches this code and has nothing to link. Verified with "make settings", which parses the blurb cleanly, and with hubCheck -settings against the sandbox copy of trackDbHub.v3.html, which reports "denseClick new" and no errors. Regenerating trackDbSettings.yaml and .json also picks up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented on 2026-09-09 without rebuilding the generated files. Nothing is removed.
  • lines changed 7, context: html, text, full: html, text
    157d7eb9f2765db7651e12c14233c56fa7205e96 Thu Sep 17 14:46:12 2026 -0700
    VCF help: add the two settings the sweep missed, and fix a value order, refs #38010 The earlier commit left out sampleMetadataFile and showHardyWeinberg on the grounds that nothing in the tree reads them. Both are read: hgc/vcfClick.c loads the metadata file in vcfGenotypeTable, and reads showHardyWeinberg with cartOrTdbBoolean in vcfGenotypesDetails. Neither call is gated on track type, so both also apply to vcfPhasedTrio, which tagTypes.tab had not allowed; that is corrected here too. sampleColorFile and showHardyWeinberg had no entry in trackDbLibrary.shtml, so they were missing from the trackDb doc pages as well. Both get a blurb, a row in trackDbDoc.html and trackDbHub.v3.html, and a changes.html note. sampleColorFile is the last VCF setting that was recognized but undocumented. vcf.html and the three trackDb doc files listed vcfPhasedColorBy as mendelDiff|deNovo|function|noColor. vcfUi.c prints the radio buttons in the opposite order, so the lists now read noColor|function|deNovo|mendelDiff, the way hapClusterColorBy already matches its own buttons. trackDbSettings.yaml and .json are regenerated. They also pick up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented earlier in this series without a regen. Found in the v504 code review, #38354. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml
  • lines changed 22, context: html, text, full: html, text
    98255f58460415345af88e45ad233599740585e5 Wed Sep 16 11:44:41 2026 -0700
    trackDb docs for the denseClick setting, refs #38364 Documents the setting added in the previous commit, in the five places a new trackDb statement has to appear: tagTypes.tab, trackDbLibrary.shtml, trackDbDoc.html, trackDbHub.v3.html (level-new, since it works in hubs) and changes.html. The type list is bed, bigBed, genePred, bigGenePred, psl and bigPsl, which is the same list decorator carries and matches what the code reaches: the setting hooks genericDrawItemsFullDense, so it applies to anything drawn through the linked-features path. Each type was checked by hand on hg38 in dense with the setting on, counting details links in the image map: ccdsGene, mrna, mrnaBig and knownGene all gained one link per item. The blurb says what happens when several items share a pixel, which is that the first one wins, and that a row drawn from summary data rather than from items is unaffected. That last case is real: linkedFeaturesDraw calls bigBedDrawDense when tg->items is NULL, so a zoomed-out bigBed never reaches this code and has nothing to link. Verified with "make settings", which parses the blurb cleanly, and with hubCheck -settings against the sandbox copy of trackDbHub.v3.html, which reports "denseClick new" and no errors. Regenerating trackDbSettings.yaml and .json also picks up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented on 2026-09-09 without rebuilding the generated files. Nothing is removed.
  • lines changed 41, context: html, text, full: html, text
    157d7eb9f2765db7651e12c14233c56fa7205e96 Thu Sep 17 14:46:12 2026 -0700
    VCF help: add the two settings the sweep missed, and fix a value order, refs #38010 The earlier commit left out sampleMetadataFile and showHardyWeinberg on the grounds that nothing in the tree reads them. Both are read: hgc/vcfClick.c loads the metadata file in vcfGenotypeTable, and reads showHardyWeinberg with cartOrTdbBoolean in vcfGenotypesDetails. Neither call is gated on track type, so both also apply to vcfPhasedTrio, which tagTypes.tab had not allowed; that is corrected here too. sampleColorFile and showHardyWeinberg had no entry in trackDbLibrary.shtml, so they were missing from the trackDb doc pages as well. Both get a blurb, a row in trackDbDoc.html and trackDbHub.v3.html, and a changes.html note. sampleColorFile is the last VCF setting that was recognized but undocumented. vcf.html and the three trackDb doc files listed vcfPhasedColorBy as mendelDiff|deNovo|function|noColor. vcfUi.c prints the radio buttons in the opposite order, so the lists now read noColor|function|deNovo|mendelDiff, the way hapClusterColorBy already matches its own buttons. trackDbSettings.yaml and .json are regenerated. They also pick up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented earlier in this series without a regen. Found in the v504 code review, #38354. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 34, context: html, text, full: html, text
    8d05f42f3315eed8579649f1271899ea8ddd986d Fri Sep 18 00:49:33 2026 -0700
    trackDb hub docs: document the variables a track description page may use. refs #38283 Adds a "Variables in the description page" section to the html setting in trackDbLibrary.shtml, listing ${db}, ${organism}, ${Organism}, ${ORGANISM}, ${date}, ${track}, ${parentTrack} and ${downloadsServer}, with an example of the link a track inside a container can make back to the container's own page. Says plainly that nothing else is substituted, so shell, awk and JavaScript examples on the same page are safe, and that the session id is not available. The section sits after the Example block on purpose. trackDbSettingsGen.py takes the first <pre> in an entry as that setting's example, so a <pre> earlier in the entry overwrites "html docs/myFirstTrack.html" in trackDbSettings.json, which is the file hubCheck -settings reads. Also adds a row to changes.html and regenerates trackDbSettings.yaml. trackDbHub.v3.html needs no change, since this adds no new setting name.
  • lines changed 15, context: html, text, full: html, text
    64adf565f280a13301d4842582ea9b9c4268f626 Mon Sep 21 03:12:15 2026 -0700
    Document how itemRgb and color interact, and correct the colorFields blurb. refs #36212 The itemRgb entry described what "itemRgb on" does but said nothing about what happens when a stanza also sets "color". That gap is how the two settings came to override each other without anyone noticing. Added the four combinations to the itemRgb blurb: an explicit "itemRgb on" takes the item colors from the file while "color" still colors the track label, "itemRgb off" falls back to "color", "color" alone wins, and a stanza with neither follows the default. The colorFields blurb said item coloring "is suppressed only if the track has an explicit color setting or itemRgb off", which stopped being true with the bedItemRgb() reorder. A color setting no longer suppresses item coloring when the stanza also says "itemRgb on". Reworded to match. trackDbLibrary.shtml supplies the prose for both trackDbDoc.html and the live trackDbHub.html, so this covers both pages. trackDbSettings.yaml is generated from it by "make settings" and is checked in, so it is regenerated here.
• src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.json
  • lines changed 65, context: html, text, full: html, text
    98255f58460415345af88e45ad233599740585e5 Wed Sep 16 11:44:41 2026 -0700
    trackDb docs for the denseClick setting, refs #38364 Documents the setting added in the previous commit, in the five places a new trackDb statement has to appear: tagTypes.tab, trackDbLibrary.shtml, trackDbDoc.html, trackDbHub.v3.html (level-new, since it works in hubs) and changes.html. The type list is bed, bigBed, genePred, bigGenePred, psl and bigPsl, which is the same list decorator carries and matches what the code reaches: the setting hooks genericDrawItemsFullDense, so it applies to anything drawn through the linked-features path. Each type was checked by hand on hg38 in dense with the setting on, counting details links in the image map: ccdsGene, mrna, mrnaBig and knownGene all gained one link per item. The blurb says what happens when several items share a pixel, which is that the first one wins, and that a row drawn from summary data rather than from items is unaffected. That last case is real: linkedFeaturesDraw calls bigBedDrawDense when tg->items is NULL, so a zoomed-out bigBed never reaches this code and has nothing to link. Verified with "make settings", which parses the blurb cleanly, and with hubCheck -settings against the sandbox copy of trackDbHub.v3.html, which reports "denseClick new" and no errors. Regenerating trackDbSettings.yaml and .json also picks up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented on 2026-09-09 without rebuilding the generated files. Nothing is removed.
  • lines changed 77, context: html, text, full: html, text
    157d7eb9f2765db7651e12c14233c56fa7205e96 Thu Sep 17 14:46:12 2026 -0700
    VCF help: add the two settings the sweep missed, and fix a value order, refs #38010 The earlier commit left out sampleMetadataFile and showHardyWeinberg on the grounds that nothing in the tree reads them. Both are read: hgc/vcfClick.c loads the metadata file in vcfGenotypeTable, and reads showHardyWeinberg with cartOrTdbBoolean in vcfGenotypesDetails. Neither call is gated on track type, so both also apply to vcfPhasedTrio, which tagTypes.tab had not allowed; that is corrected here too. sampleColorFile and showHardyWeinberg had no entry in trackDbLibrary.shtml, so they were missing from the trackDb doc pages as well. Both get a blurb, a row in trackDbDoc.html and trackDbHub.v3.html, and a changes.html note. sampleColorFile is the last VCF setting that was recognized but undocumented. vcf.html and the three trackDb doc files listed vcfPhasedColorBy as mendelDiff|deNovo|function|noColor. vcfUi.c prints the radio buttons in the opposite order, so the lists now read noColor|function|deNovo|mendelDiff, the way hapClusterColorBy already matches its own buttons. trackDbSettings.yaml and .json are regenerated. They also pick up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented earlier in this series without a regen. Found in the v504 code review, #38354. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.yaml
  • lines changed 85, context: html, text, full: html, text
    98255f58460415345af88e45ad233599740585e5 Wed Sep 16 11:44:41 2026 -0700
    trackDb docs for the denseClick setting, refs #38364 Documents the setting added in the previous commit, in the five places a new trackDb statement has to appear: tagTypes.tab, trackDbLibrary.shtml, trackDbDoc.html, trackDbHub.v3.html (level-new, since it works in hubs) and changes.html. The type list is bed, bigBed, genePred, bigGenePred, psl and bigPsl, which is the same list decorator carries and matches what the code reaches: the setting hooks genericDrawItemsFullDense, so it applies to anything drawn through the linked-features path. Each type was checked by hand on hg38 in dense with the setting on, counting details links in the image map: ccdsGene, mrna, mrnaBig and knownGene all gained one link per item. The blurb says what happens when several items share a pixel, which is that the first one wins, and that a row drawn from summary data rather than from items is unaffected. That last case is real: linkedFeaturesDraw calls bigBedDrawDense when tg->items is NULL, so a zoomed-out bigBed never reaches this code and has nothing to link. Verified with "make settings", which parses the blurb cleanly, and with hubCheck -settings against the sandbox copy of trackDbHub.v3.html, which reports "denseClick new" and no errors. Regenerating trackDbSettings.yaml and .json also picks up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented on 2026-09-09 without rebuilding the generated files. Nothing is removed.
  • lines changed 98, context: html, text, full: html, text
    157d7eb9f2765db7651e12c14233c56fa7205e96 Thu Sep 17 14:46:12 2026 -0700
    VCF help: add the two settings the sweep missed, and fix a value order, refs #38010 The earlier commit left out sampleMetadataFile and showHardyWeinberg on the grounds that nothing in the tree reads them. Both are read: hgc/vcfClick.c loads the metadata file in vcfGenotypeTable, and reads showHardyWeinberg with cartOrTdbBoolean in vcfGenotypesDetails. Neither call is gated on track type, so both also apply to vcfPhasedTrio, which tagTypes.tab had not allowed; that is corrected here too. sampleColorFile and showHardyWeinberg had no entry in trackDbLibrary.shtml, so they were missing from the trackDb doc pages as well. Both get a blurb, a row in trackDbDoc.html and trackDbHub.v3.html, and a changes.html note. sampleColorFile is the last VCF setting that was recognized but undocumented. vcf.html and the three trackDb doc files listed vcfPhasedColorBy as mendelDiff|deNovo|function|noColor. vcfUi.c prints the radio buttons in the opposite order, so the lists now read noColor|function|deNovo|mendelDiff, the way hapClusterColorBy already matches its own buttons. trackDbSettings.yaml and .json are regenerated. They also pick up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented earlier in this series without a regen. Found in the v504 code review, #38354. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 31, context: html, text, full: html, text
    8d05f42f3315eed8579649f1271899ea8ddd986d Fri Sep 18 00:49:33 2026 -0700
    trackDb hub docs: document the variables a track description page may use. refs #38283 Adds a "Variables in the description page" section to the html setting in trackDbLibrary.shtml, listing ${db}, ${organism}, ${Organism}, ${ORGANISM}, ${date}, ${track}, ${parentTrack} and ${downloadsServer}, with an example of the link a track inside a container can make back to the container's own page. Says plainly that nothing else is substituted, so shell, awk and JavaScript examples on the same page are safe, and that the session id is not available. The section sits after the Example block on purpose. trackDbSettingsGen.py takes the first <pre> in an entry as that setting's example, so a <pre> earlier in the entry overwrites "html docs/myFirstTrack.html" in trackDbSettings.json, which is the file hubCheck -settings reads. Also adds a row to changes.html and regenerates trackDbSettings.yaml. trackDbHub.v3.html needs no change, since this adds no new setting name.
  • lines changed 15, context: html, text, full: html, text
    64adf565f280a13301d4842582ea9b9c4268f626 Mon Sep 21 03:12:15 2026 -0700
    Document how itemRgb and color interact, and correct the colorFields blurb. refs #36212 The itemRgb entry described what "itemRgb on" does but said nothing about what happens when a stanza also sets "color". That gap is how the two settings came to override each other without anyone noticing. Added the four combinations to the itemRgb blurb: an explicit "itemRgb on" takes the item colors from the file while "color" still colors the track label, "itemRgb off" falls back to "color", "color" alone wins, and a stanza with neither follows the default. The colorFields blurb said item coloring "is suppressed only if the track has an explicit color setting or itemRgb off", which stopped being true with the bedItemRgb() reorder. A color setting no longer suppresses item coloring when the stanza also says "itemRgb on". Reworded to match. trackDbLibrary.shtml supplies the prose for both trackDbDoc.html and the live trackDbHub.html, so this covers both pages. trackDbSettings.yaml is generated from it by "make settings" and is checked in, so it is regenerated here.
• src/hg/htdocs/goldenPath/help/vcf.html
  • lines changed 3, context: html, text, full: html, text
    157d7eb9f2765db7651e12c14233c56fa7205e96 Thu Sep 17 14:46:12 2026 -0700
    VCF help: add the two settings the sweep missed, and fix a value order, refs #38010 The earlier commit left out sampleMetadataFile and showHardyWeinberg on the grounds that nothing in the tree reads them. Both are read: hgc/vcfClick.c loads the metadata file in vcfGenotypeTable, and reads showHardyWeinberg with cartOrTdbBoolean in vcfGenotypesDetails. Neither call is gated on track type, so both also apply to vcfPhasedTrio, which tagTypes.tab had not allowed; that is corrected here too. sampleColorFile and showHardyWeinberg had no entry in trackDbLibrary.shtml, so they were missing from the trackDb doc pages as well. Both get a blurb, a row in trackDbDoc.html and trackDbHub.v3.html, and a changes.html note. sampleColorFile is the last VCF setting that was recognized but undocumented. vcf.html and the three trackDb doc files listed vcfPhasedColorBy as mendelDiff|deNovo|function|noColor. vcfUi.c prints the radio buttons in the opposite order, so the lists now read noColor|function|deNovo|mendelDiff, the way hapClusterColorBy already matches its own buttons. trackDbSettings.yaml and .json are regenerated. They also pick up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented earlier in this series without a regen. Found in the v504 code review, #38354. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/htdocs/goldenPath/newsarch.html
  • lines changed 4, context: html, text, full: html, text
    8296c9b7908b7fd73c70b6c7af21c337c55558f4 Mon Sep 21 04:39:42 2026 -0700
    Wording and formatting fixes to the mouseStrainsCactus description page and the AVI news entry, per CR feedback. refs #38351 mouseStrainsCactus.html: use the standard "Display Conventions and Configuration" heading, spell misassemblies without the hyphen, hyphenate large-scale, add the missing commas before "and" in two sentences, finish the Data Access sentence about the API track name, and put the references in alphabetical order by first author. newsarch.html: correct the track group name to "Phenotypes, Variants, and Literature", hyphenate genome-wide, and add the missing "on" in the figure caption.
• src/hg/htdocs/inc/hgSearch.html
  • lines changed 6, context: html, text, full: html, text
    1db5c89d97fdd3c3580c4ba9b18eba6b5029c97d Thu Sep 17 05:09:56 2026 -0700
    hgSearch: highlight MANE transcript(s) in RefSeq protein-position search results, refs #38285 #Preview2 week - bugs introduced now will need a build patch to fix A gene-symbol + codon-range search (e.g. "BRCA1 100-200") maps to every RefSeq isoform prediction sharing that genomic footprint, which can be dozens of near-identical NP_ accessions with no indication of which one is the clinically-relevant MANE transcript. Cross-reference the mane bigGenePred track by genomic overlap + protein accession and call out the MANE Select/Plus Clinical transcript(s) in their own section above the rest, gated behind showManeInSearch (default off) in hg.conf.
• src/hg/htdocs/style/HGStyle.css
  • lines changed 11, context: html, text, full: html, text
    3bcf86a0995e5a15ad5a2a8ff5533f48d5d43653 Fri Sep 18 07:15:28 2026 -0700
    hgTracks: tell the user what a session load just did, refs #38157 #Preview2 week - bugs introduced now will need a build patch to fix When a saved session is loaded it replaces the whole cart, and the view the user had before is gone without a word. hgTracks now shows a note on that page saying which session was opened, who saved it and when, the session description if it has one, and that the previous configuration cannot be brought back. Only on that one page: cart.c leaves a marker when a full (non-merge) session load happens and hgTracks takes it out again. Recommended track sets merge into the cart, so they get no note. The note uses a new notifBoxOnce() in utils.js, which has only a Close button: a one-page message does not need 'Don't show again'. Its .notifBoxOnce style is the card grey of the new Sessions page, and it sets no width, so unlike the other notifBoxes it cannot stick out of the window. Switch off with sessionLoadNotice=off in hg.conf.
• src/hg/htdocs/style/hgSession.css
  • lines changed 6, context: html, text, full: html, text
    1c52aaa92828b6ecc18dd5800fe6650b91daf696 Fri Sep 18 07:22:10 2026 -0700
    hgSession: the new Sessions page also lists the sessions saved on our other servers #Preview2 week - bugs introduced now will need a build patch to fix Every geo mirror node keeps its own namedSessionDb, so a session saved on genome-euro is invisible on genome.ucsc.edu and people do not find it again. The table now holds the sessions from every node, with a new Server column saying where each one lives, and the session name links to the server that holds it. Two new hgSession actions, both answered in main() before the cart is built, so that a request from another node leaves no cart, userDb or sessionDb row behind: hgS_doSessionListJson returns this server's sessions for the user named by the login cookie, as JSON. This is what the other nodes call. hgS_doMirrorSessions asks every other gbNode for that list and returns what came back. It sends all of the requests before reading any of the answers, so the wait is the slowest node rather than the sum of them. The fetch goes through our own server rather than from the browser to the other nodes, which keeps cross-origin requests out of it: apache handles the CORS headers differently on every node, and the two mirrors are administered by Bielefeld and RIKEN, while CGI code reaches them with the release. The user's login cookies, and only those, are passed on (wikiLinkLoginCookieHeader), which works because the nodes share a cookie salt. The page never waits for any of this: the table is drawn from this server's own sessions and the rows from the other servers are merged in as they arrive, keeping the current sort, search and page. A node that does not answer, times out, or runs a release without the endpoint is named in a line under the toolbar instead, and costs nothing else. Sessions from another server are shown but not managed here: no Share, Edit, Overwrite or Delete, no bulk-select checkbox, and the Update now shortcut keeps ignoring them. Table rows are keyed by a new uid rather than by the session name, since the same name can now be in the table twice, once per server. geoMirrorThisNode() and geoMirrorOtherNodes() read the node list from the hgcentral gbNode table, so nothing new has to be configured: browser.node already says which node a server is.
  • lines changed 21, context: html, text, full: html, text
    9a2fac6063097ddaa8f0e0063cd9c26c7ce07af7 Fri Sep 18 08:11:31 2026 -0700
    hgSession: address UI feedback on the new Sessions page #Preview2 week - bugs introduced now will need a build patch to fix Wraps the intro text in <p> tags, renames "Update now" to "Overwrite now" with a visible info bubble, gives the name/description fields outside labels and narrower boxes with a real generated name as the name field's placeholder, moves "Back up custom tracks" out of the Advanced panel to its own bar, and splits Advanced into Load/Save columns. refs #38311
  • lines changed 16, context: html, text, full: html, text
    f716736ad2d4a1f8832bb945e2d21ea95ba7140b Fri Sep 18 09:03:07 2026 -0700
    hgSession: fix oversized save-card inputs, move backup link into Advanced #Preview2 week - bugs introduced now will need a build patch to fix The name/description boxes on the save card used flex-basis for width inside a column flex container, where flex-basis sets height instead - they were rendering hundreds of pixels tall. Also folds "Back up custom tracks" into the Advanced panel as a single link across the top, above the load/save columns, instead of its own bar under the save card. refs #38311
  • lines changed 10, context: html, text, full: html, text
    0d082b749cf32760466ade27ce08093cdbc809f6 Fri Sep 18 09:08:29 2026 -0700
    hgSession: back-up-custom-tracks link goes above Advanced, not inside it #Preview2 week - bugs introduced now will need a build patch to fix refs #38311
  • lines changed 10, context: html, text, full: html, text
    85372b41a99caf09ee398ab8ea7fc6707e3b6a0c Fri Sep 18 09:20:10 2026 -0700
    hgSession: privacy wording, wider description box, info bubbles, backup button #Preview2 week - bugs introduced now will need a build patch to fix Save-card privacy checkbox now reads "Private: Session can only be loaded by myself"; the Share dialog's gallery checkbox is normal-size text instead of a small form label. The description box is 50% wider. Info bubbles next to the Session name and Description labels explain the auto-generated name and where the description shows up. "Back up custom tracks" is a button again, at the top of the Advanced panel, and named first in that panel's summary line. refs #38311
  • lines changed 7, context: html, text, full: html, text
    36c2daedb6dd44750f70048d2ab309c3c77dffc8 Fri Sep 18 09:32:02 2026 -0700
    hgSession: drop the name/description labels, info icon after the box #Preview2 week - bugs introduced now will need a build patch to fix refs #38311
  • lines changed 1, context: html, text, full: html, text
    739fffa3ab988af1d616622d1bf72051b3d06548 Fri Sep 18 09:33:25 2026 -0700
    hgSession: even out the gap below the save card #Preview2 week - bugs introduced now will need a build patch to fix refs #38311
• src/hg/htdocs/style/nice_menu.css
  • lines changed 19, context: html, text, full: html, text
    b82bce91b10725b252a3ead15ba68cb0700d997a Thu Sep 17 02:12:25 2026 -0700
    Stop the narrow-screen menu icon from covering the menu bar items, refs #38251 #Preview2 week - bugs introduced now will need a build patch to fix The previous fix put the icon at position:sticky with float:right so that it would follow the right edge of the window while the 1000px-wide bar is scrolled sideways. Sticky only moves an element that is in the normal flow, and that turned out to be the problem: the float sits in the same line as the menu items, so at some widths they wrap out of the 32px-high bar and their labels are drawn below it on white, and as the window narrows the icon sweeps leftwards across the items and hides whichever one is under its opaque background. Leave the icon where the rest of the bar is instead, at the right-hand end, which is what the rule for wide windows already does. On a window narrower than the bar it is reached by scrolling sideways, exactly like the "Help" and "About Us" items next to it. It is moved 8px closer to the edge than on wide windows because the static pages draw "Home" as the 84px UCSC logo rather than the 23px house icon, which leaves the menu ending only 29px short of the icon. Measured the bar on hgGateway, the home page, hgTracks and hgTrackUi at thirteen widths from 390 to 1400px: the icon is inside the bar everywhere, the menu stays on one row, and the gap between the last item and the icon is never less than 17px. Hovering "About Us" at 850px on hgTracks, which used to blank every label in the bar, no longer does.
• src/hg/htdocs/tipOfDay.html
  • lines changed 1, context: html, text, full: html, text
    ba72ed7ec5fd05fb4004419d2cb16b155f13a10f Mon Sep 14 11:20:00 2026 -0700
    Removing the tree copy because this page is regenerated daily, and the tree copy gets copied on the build each weekend and overwrites the fresh regenerated copy. No RM.
• src/hg/hubApi/assemblyList.py
  • lines changed 56, context: html, text, full: html, text
    6d7fcf6b8743d4b47014eaba4b156b63830bebcb Wed Sep 16 11:53:43 2026 -0700
    eliminate the duplicated dbDb hub genark assemblies and adjust static priorities refs #38365
  • lines changed 205, context: html, text, full: html, text
    9efe3bfe8af965d5e183e05615bc337035d8b283 Fri Sep 18 13:45:24 2026 -0700
    consider NCBI refSeq "reference" status in the priorities and fix the error prone manual maintained topPriorities list refs #38365
  • lines changed 1, context: html, text, full: html, text
    a0bd274a58c1772ff05b975dacbb9e30d910aa5b Sat Sep 19 09:49:23 2026 -0700
    upgrade a zebrafish GCA to GCF refs #38365
• src/hg/inc/bigBedFind.h
  • lines changed 18, context: html, text, full: html, text
    1db5c89d97fdd3c3580c4ba9b18eba6b5029c97d Thu Sep 17 05:09:56 2026 -0700
    hgSearch: highlight MANE transcript(s) in RefSeq protein-position search results, refs #38285 #Preview2 week - bugs introduced now will need a build patch to fix A gene-symbol + codon-range search (e.g. "BRCA1 100-200") maps to every RefSeq isoform prediction sharing that genomic footprint, which can be dozens of near-identical NP_ accessions with no indication of which one is the clinically-relevant MANE transcript. Cross-reference the mane bigGenePred track by genomic overlap + protein accession and call out the MANE Select/Plus Clinical transcript(s) in their own section above the rest, gated behind showManeInSearch (default off) in hg.conf.
• src/hg/inc/bigChain.h
  • lines changed 18, context: html, text, full: html, text
    72327e4e2d294b9d2fcb6cbdadd4e628c006345b Fri Sep 18 10:07:40 2026 -0700
    bigChain: move the chain to bigChain conversion into the library chainRecToBigChain() and chainBlockToBigLink() sat in chainToBigChain.c next to its main(), so a CGI could not link against them. Move both into hg/lib/bigChain.c, add chainToBigChainOne() for one chain and chainToBigChainList() for a list, and leave the utility as a thin main(). No behavior change; the chainToBigChain test output is unchanged. refs #38382
• src/hg/inc/cart.h
  • lines changed 5, context: html, text, full: html, text
    3bcf86a0995e5a15ad5a2a8ff5533f48d5d43653 Fri Sep 18 07:15:28 2026 -0700
    hgTracks: tell the user what a session load just did, refs #38157 #Preview2 week - bugs introduced now will need a build patch to fix When a saved session is loaded it replaces the whole cart, and the view the user had before is gone without a word. hgTracks now shows a note on that page saying which session was opened, who saved it and when, the session description if it has one, and that the previous configuration cannot be brought back. Only on that one page: cart.c leaves a marker when a full (non-merge) session load happens and hgTracks takes it out again. Recommended track sets merge into the cart, so they get no note. The note uses a new notifBoxOnce() in utils.js, which has only a Close button: a one-page message does not need 'Don't show again'. Its .notifBoxOnce style is the card grey of the new Sessions page, and it sets no width, so unlike the other notifBoxes it cannot stick out of the window. Switch off with sessionLoadNotice=off in hg.conf.
• src/hg/inc/geoMirror.h
  • lines changed 9, context: html, text, full: html, text
    1c52aaa92828b6ecc18dd5800fe6650b91daf696 Fri Sep 18 07:22:10 2026 -0700
    hgSession: the new Sessions page also lists the sessions saved on our other servers #Preview2 week - bugs introduced now will need a build patch to fix Every geo mirror node keeps its own namedSessionDb, so a session saved on genome-euro is invisible on genome.ucsc.edu and people do not find it again. The table now holds the sessions from every node, with a new Server column saying where each one lives, and the session name links to the server that holds it. Two new hgSession actions, both answered in main() before the cart is built, so that a request from another node leaves no cart, userDb or sessionDb row behind: hgS_doSessionListJson returns this server's sessions for the user named by the login cookie, as JSON. This is what the other nodes call. hgS_doMirrorSessions asks every other gbNode for that list and returns what came back. It sends all of the requests before reading any of the answers, so the wait is the slowest node rather than the sum of them. The fetch goes through our own server rather than from the browser to the other nodes, which keeps cross-origin requests out of it: apache handles the CORS headers differently on every node, and the two mirrors are administered by Bielefeld and RIKEN, while CGI code reaches them with the release. The user's login cookies, and only those, are passed on (wikiLinkLoginCookieHeader), which works because the nodes share a cookie salt. The page never waits for any of this: the table is drawn from this server's own sessions and the rows from the other servers are merged in as they arrive, keeping the current sort, search and page. A node that does not answer, times out, or runs a release without the endpoint is named in a line under the toolbar instead, and costs nothing else. Sessions from another server are shown but not managed here: no Share, Edit, Overwrite or Delete, no bulk-select checkbox, and the Update now shortcut keeps ignoring them. Table rows are keyed by a new uid rather than by the session name, since the same name can now be in the table twice, once per server. geoMirrorThisNode() and geoMirrorOtherNodes() read the node list from the hgcentral gbNode table, so nothing new has to be configured: browser.node already says which node a server is.
  • lines changed 9, context: html, text, full: html, text
    14ecff0730596f8be147b255afc8acb99e9095e2 Fri Sep 18 09:06:06 2026 -0700
    hgHubConnect: an api key made on one geo mirror now works on all of them, behind syncHubApiKeys #Preview2 week - bugs introduced now will need a build patch to fix #Preview2 week - bugs introduced now will need a build patch to fix Api keys live in the hgcentral of whichever mirror issued them, so a key made on genome.ucsc.edu was rejected on genome-euro and genome-asia, and the botDelay error told the user to go and make a second one. hgHubConnect now tells the other nodes about a key as soon as it generates or revokes one: cjGenerateApiKey and cjRevokeApiKey call syncApiKeyToOtherNodes(), which posts a hgHubSyncApiKey cartJson request to each peer, and cjSyncApiKey() on the far side writes it into that mirror's own table. The peers come from hgcentral.gbNode via the new geoMirrorNotifyOtherNodes(), and the request is signed with login.cookieSalt, which all of a site's mirrors already share - it is what makes the login cookie verifiable on each of them. So neither a list of mirror addresses nor a new shared secret needs provisioning. The notify is best effort: a peer that is down is warn()ed about and skipped, never failing the local generate or revoke, which has already committed by then. All of it is off unless hg.conf says syncHubApiKeys=on. With the gate off the sender returns at once, the receiver refuses the request outright rather than merely being unreachable, and botDelay keeps printing the old server-specific wording, which off is still the truth. Registered in hgConfCatalog.py as a gate, to be flipped once this is released. refs #38323
  • lines changed 7, context: html, text, full: html, text
    33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
    hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323 Six things, all in the syncHubApiKeys path, which is still off everywhere. The sync never reached its handler on a real mirror. It was registered as a cartJson command, so building the cart ran forceUserIdOrCaptcha() first; a peer's request carries no hguid cookie and no apiKey= variable, so any site with cloudFlareSiteKey set -- which is every site that needs api keys in the first place -- answered it with a captcha page. Only hgwdev, where the captcha is commented out, ever ran the handler, which is why this was not caught earlier. doApiKeySyncIfRequested() now answers the request in main() before any cart exists. That also stops each sync leaving a junk userDb and sessionDb row behind: measured, 15 syncs now create none. The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key), so it does not rest on md5 resisting length extension, and it is compared in constant time. It now covers a timestamp too, and a peer refuses anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of now, so a captured sync cannot be replayed later to reinstate a key its owner has since revoked. The fields are joined with newlines and a newline in a userName or apiKey is refused, so one signature cannot cover two different splits of the same string. syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It runs after the local key is already written and in the response, so an errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's outer catch, which threw the response away -- the user saw a failure over a key that was sitting in the database. geoMirrorNotifyOtherNodes() returns each peer's response instead of discarding it, and uses netUrlMustOpenPastHeader, which errAborts on anything but a 200 and hands back the body alone. A peer answering 'bad signature' or 'not enabled on this site' is no longer indistinguishable from success; it is logged to error_log, not warn()ed at the person who clicked the button. Verified against the built CGI run as Apache runs it, with the captcha turned on: a signed sync is accepted and writes the row, a signed revoke removes it, and a replay, a swapped userName, a forged signature and malformed json are all refused. hubSpaceKeysTester covers the signature rules; it still cannot run its database half until hgcentralregress gets the UPDATE and DELETE grant.
• src/hg/inc/hVarSubst.h
  • lines changed 7, context: html, text, full: html, text
    5f590f874b9d352023167b563e5b83f651488f0b Tue Sep 15 04:56:35 2026 -0700
    Resolve $hgsid in native track description pages too, refs #38283 A description page could already use $hgsid if it belonged to a hub, because a hub's html is substituted at render time where there is a cart. A native page is substituted once by hgTrackDb, which has no cart, so $hgsid quietly became the empty string and the link it was part of came out broken. hg38's hprcRdt page has been shipping "hgTrackUi?hgsid=&g=long_read_transcripts" for this reason. hgTrackDb now writes the reference back out as ${hgsid} instead of resolving it, and hVarSubstTrackDbHtml resolves it at render time for native pages as well as hub ones. Only $hgsid is deferred, and only in the html field: the labels get no second pass, so a deferred reference in one of them would reach the user as the literal text "${hgsid}". The render pass over a native page acts only on the braced form, which is what keeps an escaped $$hgsid escaped -- hgTrackDb collapses that to a bare $hgsid, and a bare one is left alone. The html is no longer freed before being replaced, and is replaced only when the substitution actually changed something. hVarSubstExt allocates its result at the first dollar sign whether or not it substitutes anything, so any page merely containing one reached that free; for a native track tdb->html can point into the trackDb cache, which is localmem carved out of an mmap'd file and never came from malloc. Reproduced on hg38 chm13LiftOver, whose page contains an awk snippet, with cacheTrackDbDir set. Description pages have to be reloaded for the deferral to reach the trackDb table.
  • lines changed 13, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/inc/hdb.h
  • lines changed 6, context: html, text, full: html, text
    eb7e6ccb9b23c2dd5cd82de0237c6f25aecd2da1 Thu Sep 17 06:02:11 2026 -0700
    Hub tracks must not claim an undecorated name that a native track owns #Preview2 week - bugs introduced now will need a build patch to fix A hub track can be named on a URL, or found in the cart, without its "hub_<id>_" prefix, so that hub links stay readable. Nothing checked whether the assembly already had a track of that name, so when it did, the hub track took the variable: with the MethBase2 hub attached, setting the native hg38 Methbase track's visibility moved it to hub_<id>_Methbase and dropped it from the native track, which then did not draw. The two share the name "Methbase", and their subtracks share names too, so the same went for the _sel checkboxes. hubTrackOwnsBareName() now gates the seven places that fall back to the bare name - track and subtrack visibility, supertrack visibility, the two _hideKids variables, and the _sel checkbox. The bare name is the hub track's only where the assembly has no track of that name; otherwise it belongs to the native one, whose own pass over the track list consumes it. It answers from the track list loadFromTrackDb() has already built, so it costs a hash lookup: hub tracks are always prefixed in that list, so an undecorated name can only match a native track. A caller with no list to register (hgTrackUi) falls back to a single-row trackDb query per distinct bare name, cached. With 500 hub subtracks visible and 500 bare names on the URL - far past anything real - that is 189 queries either way, against 1189 for a query per name. Also clarifies the comments on hTrackDbWithCartVersion() and tdbForTrack(): the "this result is cached" note means the shared-memory trackDb cache only, which is off unless cacheTrackDbDir is set, and there is no memoizing besides it - so hTrackDbForTrack() per name reloads the whole trackDb each call. Measured at 57 seconds for 500 names before I abandoned that approach. refs #34246
• src/hg/inc/hubSpaceKeys.h
  • lines changed 12, context: html, text, full: html, text
    14ecff0730596f8be147b255afc8acb99e9095e2 Fri Sep 18 09:06:06 2026 -0700
    hgHubConnect: an api key made on one geo mirror now works on all of them, behind syncHubApiKeys #Preview2 week - bugs introduced now will need a build patch to fix #Preview2 week - bugs introduced now will need a build patch to fix Api keys live in the hgcentral of whichever mirror issued them, so a key made on genome.ucsc.edu was rejected on genome-euro and genome-asia, and the botDelay error told the user to go and make a second one. hgHubConnect now tells the other nodes about a key as soon as it generates or revokes one: cjGenerateApiKey and cjRevokeApiKey call syncApiKeyToOtherNodes(), which posts a hgHubSyncApiKey cartJson request to each peer, and cjSyncApiKey() on the far side writes it into that mirror's own table. The peers come from hgcentral.gbNode via the new geoMirrorNotifyOtherNodes(), and the request is signed with login.cookieSalt, which all of a site's mirrors already share - it is what makes the login cookie verifiable on each of them. So neither a list of mirror addresses nor a new shared secret needs provisioning. The notify is best effort: a peer that is down is warn()ed about and skipped, never failing the local generate or revoke, which has already committed by then. All of it is off unless hg.conf says syncHubApiKeys=on. With the gate off the sender returns at once, the receiver refuses the request outright rather than merely being unreachable, and botDelay keeps printing the old server-specific wording, which off is still the truth. Registered in hgConfCatalog.py as a gate, to be flipped once this is released. refs #38323
  • lines changed 18, context: html, text, full: html, text
    33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
    hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323 Six things, all in the syncHubApiKeys path, which is still off everywhere. The sync never reached its handler on a real mirror. It was registered as a cartJson command, so building the cart ran forceUserIdOrCaptcha() first; a peer's request carries no hguid cookie and no apiKey= variable, so any site with cloudFlareSiteKey set -- which is every site that needs api keys in the first place -- answered it with a captcha page. Only hgwdev, where the captcha is commented out, ever ran the handler, which is why this was not caught earlier. doApiKeySyncIfRequested() now answers the request in main() before any cart exists. That also stops each sync leaving a junk userDb and sessionDb row behind: measured, 15 syncs now create none. The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key), so it does not rest on md5 resisting length extension, and it is compared in constant time. It now covers a timestamp too, and a peer refuses anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of now, so a captured sync cannot be replayed later to reinstate a key its owner has since revoked. The fields are joined with newlines and a newline in a userName or apiKey is refused, so one signature cannot cover two different splits of the same string. syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It runs after the local key is already written and in the response, so an errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's outer catch, which threw the response away -- the user saw a failure over a key that was sitting in the database. geoMirrorNotifyOtherNodes() returns each peer's response instead of discarding it, and uses netUrlMustOpenPastHeader, which errAborts on anything but a 200 and hands back the body alone. A peer answering 'bad signature' or 'not enabled on this site' is no longer indistinguishable from success; it is logged to error_log, not warn()ed at the person who clicked the button. Verified against the built CGI run as Apache runs it, with the captcha turned on: a signed sync is accepted and writes the row, a signed revoke removes it, and a replay, a swapped userName, a forged signature and malformed json are all refused. hubSpaceKeysTester covers the signature rules; it still cannot run its database half until hgcentralregress gets the UPDATE and DELETE grant.
• src/hg/inc/hui.h
  • lines changed 16, context: html, text, full: html, text
    eb7e6ccb9b23c2dd5cd82de0237c6f25aecd2da1 Thu Sep 17 06:02:11 2026 -0700
    Hub tracks must not claim an undecorated name that a native track owns #Preview2 week - bugs introduced now will need a build patch to fix A hub track can be named on a URL, or found in the cart, without its "hub_<id>_" prefix, so that hub links stay readable. Nothing checked whether the assembly already had a track of that name, so when it did, the hub track took the variable: with the MethBase2 hub attached, setting the native hg38 Methbase track's visibility moved it to hub_<id>_Methbase and dropped it from the native track, which then did not draw. The two share the name "Methbase", and their subtracks share names too, so the same went for the _sel checkboxes. hubTrackOwnsBareName() now gates the seven places that fall back to the bare name - track and subtrack visibility, supertrack visibility, the two _hideKids variables, and the _sel checkbox. The bare name is the hub track's only where the assembly has no track of that name; otherwise it belongs to the native one, whose own pass over the track list consumes it. It answers from the track list loadFromTrackDb() has already built, so it costs a hash lookup: hub tracks are always prefixed in that list, so an undecorated name can only match a native track. A caller with no list to register (hgTrackUi) falls back to a single-row trackDb query per distinct bare name, cached. With 500 hub subtracks visible and 500 bare names on the URL - far past anything real - that is 189 queries either way, against 1189 for a query per name. Also clarifies the comments on hTrackDbWithCartVersion() and tdbForTrack(): the "this result is cached" note means the shared-memory trackDb cache only, which is off unless cacheTrackDbDir is set, and there is no memoizing besides it - so hTrackDbForTrack() per name reloads the whole trackDb each call. Measured at 57 seconds for 500 names before I abandoned that approach. refs #34246
• src/hg/inc/jsHelper.h
  • lines changed 6, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
  • lines changed 7, context: html, text, full: html, text
    2eb14fc0378432977d83366a3211c67f8913018d Mon Sep 21 06:21:51 2026 -0700
    hgc and hgTrackUi: open off-site links in a new tab, with rel="noopener noreferrer" offsiteLinksToNewTab() in utils.js runs over the rendered page alongside addHgsidToLinks() and gives every http(s) link whose host is not ours a target=_blank it does not already have, plus rel="noopener noreferrer". Without noopener the page that opens keeps a handle on the tab it came from and can navigate it; without noreferrer the Referer header carries our own URL, which has the session id in it. The href on a track description page is written by whoever wrote the track or the hub, so neither is theoretical. A mailto: or an ftp: link is left alone, and so is every link that stays on this server. jsAddHgsidToLinks() now emits both calls and is renamed jsFixUpPageLinks(). hgTracks.js runs the same pass over the track description popup: the replace it does before that already puts a target on every link in there, so what this adds is the rel on the ones that leave. refs #38380
• src/hg/inc/snapshotSession.h
  • lines changed 5, context: html, text, full: html, text
    c500eb92eb2a021af927f3c6f2ad00c7b29769a5 Tue Sep 15 05:48:49 2026 -0700
    snapshotSession: snapshotIsSnapshotName is file-local, refs #38353 Its only caller is the writer in the same file, checking its own argument, and its header comment warned everyone else off it. Making it static says that rather than asking.
• src/hg/inc/userdata.h
  • lines changed 7, context: html, text, full: html, text
    f518e03d8f56af784ec14536a7bdc0daed9a38fd Wed Sep 16 11:11:32 2026 -0700
    Send each file's genome with a hubtools upload so hubSpace rows get a db, and stop the server rewriting a user-uploaded hub.txt, no redmine Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/inc/versionInfo.h
  • lines changed 1, context: html, text, full: html, text
    74623f843857f2254e5327c78b98f00c3f7098ff Mon Sep 21 10:00:48 2026 -0700
    New version number v504
• src/hg/inc/web.h
  • lines changed 5, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/inc/wikiLink.h
  • lines changed 5, context: html, text, full: html, text
    1c52aaa92828b6ecc18dd5800fe6650b91daf696 Fri Sep 18 07:22:10 2026 -0700
    hgSession: the new Sessions page also lists the sessions saved on our other servers #Preview2 week - bugs introduced now will need a build patch to fix Every geo mirror node keeps its own namedSessionDb, so a session saved on genome-euro is invisible on genome.ucsc.edu and people do not find it again. The table now holds the sessions from every node, with a new Server column saying where each one lives, and the session name links to the server that holds it. Two new hgSession actions, both answered in main() before the cart is built, so that a request from another node leaves no cart, userDb or sessionDb row behind: hgS_doSessionListJson returns this server's sessions for the user named by the login cookie, as JSON. This is what the other nodes call. hgS_doMirrorSessions asks every other gbNode for that list and returns what came back. It sends all of the requests before reading any of the answers, so the wait is the slowest node rather than the sum of them. The fetch goes through our own server rather than from the browser to the other nodes, which keeps cross-origin requests out of it: apache handles the CORS headers differently on every node, and the two mirrors are administered by Bielefeld and RIKEN, while CGI code reaches them with the release. The user's login cookies, and only those, are passed on (wikiLinkLoginCookieHeader), which works because the nodes share a cookie salt. The page never waits for any of this: the table is drawn from this server's own sessions and the rows from the other servers are merged in as they arrive, keeping the current sort, search and page. A node that does not answer, times out, or runs a release without the endpoint is named in a line under the toolbar instead, and costs nothing else. Sessions from another server are shown but not managed here: no Share, Edit, Overwrite or Delete, no bulk-select checkbox, and the Update now shortcut keeps ignoring them. Table rows are keyed by a new uid rather than by the session name, since the same name can now be in the table twice, once per server. geoMirrorThisNode() and geoMirrorOtherNodes() read the node list from the hgcentral gbNode table, so nothing new has to be configured: browser.node already says which node a server is.
• src/hg/js/facetedComposite.js
  • lines changed 66, context: html, text, full: html, text
    f04876658f4ae0ec7c633895c21f153975a2a506 Tue Sep 15 05:48:42 2026 -0700
    faceted composite: default sort column, restored drag order, and two smaller things, refs #38353 defaultSortCol was still 1, which the drag-handle column took over when manual row ordering went in. A composite with no defaultSortField opened sorted by a hidden column, with no sort arrow on any visible heading. Fixing that alone exposed the next one: coming back on the Active tab called setFilterMode(), which renumbers the order field from whatever is on screen, and that threw away the hand- dragged order just read back from localStorage. It was invisible only because the current order happened to be the drag order while the default sort column was the drag column. The restore now keeps the saved order; a click still renumbers, which is what a click means. Saved state had no eviction, so once the quota filled, the QuotaExceededError was swallowed and from then on no faceted composite on the origin could save anything again. Each entry now carries a timestamp and a full store drops the least recently written entry belonging to another composite and tries again. Narrowing a facet turned the track container back on even with nothing selected, which brings back an empty image.
• src/hg/js/hgHubConnect.js
  • lines changed 5, context: html, text, full: html, text
    0ae2a6a36d7b4c77e8e7d7df2a1049f4412e7099 Fri Sep 18 07:03:44 2026 -0700
    hgHubConnect: let a mirror hand out its own API keys, and stop swallowing links to another host #Preview2 week - bugs introduced now will need a build patch to fix Two of the three problems QA found on #38323. The link in the mirror-only Hub Upload message did not go anywhere. The tab handler in hgHubConnect.js catches every hgHubConnect link with a hash and turns it into a tab switch, and 'Go to Hub Upload on genome.ucsc.edu' has a hash that names a tab on the mirror too, so the click just reopened the tab the reader was already on. It now only intercepts links to the page itself. The API key section was decoupled from storeUserFiles, but only for display: the Generate and Revoke buttons are cartJson requests, and both the javascript that sends them and the code in main() that routes them were still inside the storeUserFiles gate. A site with showHubApiKey on and hubSpace off therefore drew two dead buttons. The key functions move out of hgMyData.js into a new hubApiKey.js that the Hub Development tab includes on its own, main() routes a cartJson request when either setting is on, and the hubSpace file commands stay registered only when hubSpace is actually running. The request also goes to this host's hgHubConnect now rather than to the login host. Keys live in the central database of the server that issues them, so a key made on genome-euro belongs in genome-euro's table. refs #38323
• src/hg/js/hgMyData.js
  • lines changed 6, context: html, text, full: html, text
    d5180601357304b9a9edef820745a66896ec09ad Tue Sep 15 05:48:46 2026 -0700
    hgMyData: find the Hub Upload tab by its panel, not by counting to three, refs #38353 hgHubConnect only prints the Hub Development tab when hgHubConnect.validateHub is set, so on a mirror with storeUserFiles on and validateHub off, Hub Upload is the third tab. It then got neither the upload instructions nor the HTTPS warning.
  • lines changed 63, context: html, text, full: html, text
    0ae2a6a36d7b4c77e8e7d7df2a1049f4412e7099 Fri Sep 18 07:03:44 2026 -0700
    hgHubConnect: let a mirror hand out its own API keys, and stop swallowing links to another host #Preview2 week - bugs introduced now will need a build patch to fix Two of the three problems QA found on #38323. The link in the mirror-only Hub Upload message did not go anywhere. The tab handler in hgHubConnect.js catches every hgHubConnect link with a hash and turns it into a tab switch, and 'Go to Hub Upload on genome.ucsc.edu' has a hash that names a tab on the mirror too, so the click just reopened the tab the reader was already on. It now only intercepts links to the page itself. The API key section was decoupled from storeUserFiles, but only for display: the Generate and Revoke buttons are cartJson requests, and both the javascript that sends them and the code in main() that routes them were still inside the storeUserFiles gate. A site with showHubApiKey on and hubSpace off therefore drew two dead buttons. The key functions move out of hgMyData.js into a new hubApiKey.js that the Hub Development tab includes on its own, main() routes a cartJson request when either setting is on, and the hubSpace file commands stay registered only when hubSpace is actually running. The request also goes to this host's hgHubConnect now rather than to the login host. Keys live in the central database of the server that issues them, so a key made on genome-euro belongs in genome-euro's table. refs #38323
• src/hg/js/hgSearch.js
  • lines changed 48, context: html, text, full: html, text
    1db5c89d97fdd3c3580c4ba9b18eba6b5029c97d Thu Sep 17 05:09:56 2026 -0700
    hgSearch: highlight MANE transcript(s) in RefSeq protein-position search results, refs #38285 #Preview2 week - bugs introduced now will need a build patch to fix A gene-symbol + codon-range search (e.g. "BRCA1 100-200") maps to every RefSeq isoform prediction sharing that genomic footprint, which can be dozens of near-identical NP_ accessions with no indication of which one is the clinically-relevant MANE transcript. Cross-reference the mane bigGenePred track by genomic overlap + protein accession and call out the MANE Select/Plus Clinical transcript(s) in their own section above the rest, gated behind showManeInSearch (default off) in hg.conf.
• src/hg/js/hgSession.js
  • lines changed 164, context: html, text, full: html, text
    1c52aaa92828b6ecc18dd5800fe6650b91daf696 Fri Sep 18 07:22:10 2026 -0700
    hgSession: the new Sessions page also lists the sessions saved on our other servers #Preview2 week - bugs introduced now will need a build patch to fix Every geo mirror node keeps its own namedSessionDb, so a session saved on genome-euro is invisible on genome.ucsc.edu and people do not find it again. The table now holds the sessions from every node, with a new Server column saying where each one lives, and the session name links to the server that holds it. Two new hgSession actions, both answered in main() before the cart is built, so that a request from another node leaves no cart, userDb or sessionDb row behind: hgS_doSessionListJson returns this server's sessions for the user named by the login cookie, as JSON. This is what the other nodes call. hgS_doMirrorSessions asks every other gbNode for that list and returns what came back. It sends all of the requests before reading any of the answers, so the wait is the slowest node rather than the sum of them. The fetch goes through our own server rather than from the browser to the other nodes, which keeps cross-origin requests out of it: apache handles the CORS headers differently on every node, and the two mirrors are administered by Bielefeld and RIKEN, while CGI code reaches them with the release. The user's login cookies, and only those, are passed on (wikiLinkLoginCookieHeader), which works because the nodes share a cookie salt. The page never waits for any of this: the table is drawn from this server's own sessions and the rows from the other servers are merged in as they arrive, keeping the current sort, search and page. A node that does not answer, times out, or runs a release without the endpoint is named in a line under the toolbar instead, and costs nothing else. Sessions from another server are shown but not managed here: no Share, Edit, Overwrite or Delete, no bulk-select checkbox, and the Update now shortcut keeps ignoring them. Table rows are keyed by a new uid rather than by the session name, since the same name can now be in the table twice, once per server. geoMirrorThisNode() and geoMirrorOtherNodes() read the node list from the hgcentral gbNode table, so nothing new has to be configured: browser.node already says which node a server is.
  • lines changed 54, context: html, text, full: html, text
    9a2fac6063097ddaa8f0e0063cd9c26c7ce07af7 Fri Sep 18 08:11:31 2026 -0700
    hgSession: address UI feedback on the new Sessions page #Preview2 week - bugs introduced now will need a build patch to fix Wraps the intro text in <p> tags, renames "Update now" to "Overwrite now" with a visible info bubble, gives the name/description fields outside labels and narrower boxes with a real generated name as the name field's placeholder, moves "Back up custom tracks" out of the Advanced panel to its own bar, and splits Advanced into Load/Save columns. refs #38311
  • lines changed 25, context: html, text, full: html, text
    f716736ad2d4a1f8832bb945e2d21ea95ba7140b Fri Sep 18 09:03:07 2026 -0700
    hgSession: fix oversized save-card inputs, move backup link into Advanced #Preview2 week - bugs introduced now will need a build patch to fix The name/description boxes on the save card used flex-basis for width inside a column flex container, where flex-basis sets height instead - they were rendering hundreds of pixels tall. Also folds "Back up custom tracks" into the Advanced panel as a single link across the top, above the load/save columns, instead of its own bar under the save card. refs #38311
  • lines changed 22, context: html, text, full: html, text
    0d082b749cf32760466ade27ce08093cdbc809f6 Fri Sep 18 09:08:29 2026 -0700
    hgSession: back-up-custom-tracks link goes above Advanced, not inside it #Preview2 week - bugs introduced now will need a build patch to fix refs #38311
  • lines changed 35, context: html, text, full: html, text
    85372b41a99caf09ee398ab8ea7fc6707e3b6a0c Fri Sep 18 09:20:10 2026 -0700
    hgSession: privacy wording, wider description box, info bubbles, backup button #Preview2 week - bugs introduced now will need a build patch to fix Save-card privacy checkbox now reads "Private: Session can only be loaded by myself"; the Share dialog's gallery checkbox is normal-size text instead of a small form label. The description box is 50% wider. Info bubbles next to the Session name and Description labels explain the auto-generated name and where the description shows up. "Back up custom tracks" is a button again, at the top of the Advanced panel, and named first in that panel's summary line. refs #38311
  • lines changed 3, context: html, text, full: html, text
    327ba91c188e036e07790a83bc702b388f74d789 Fri Sep 18 09:28:23 2026 -0700
    hgSession: name tooltip mentions the session URL #Preview2 week - bugs introduced now will need a build patch to fix refs #38311
  • lines changed 9, context: html, text, full: html, text
    36c2daedb6dd44750f70048d2ab309c3c77dffc8 Fri Sep 18 09:32:02 2026 -0700
    hgSession: drop the name/description labels, info icon after the box #Preview2 week - bugs introduced now will need a build patch to fix refs #38311
  • lines changed 1, context: html, text, full: html, text
    0971bab63556dd9c926be56b28fb02b18bb3a9ef Fri Sep 18 09:32:48 2026 -0700
    hgSession: name field's watermark carries "Session name, default ..." #Preview2 week - bugs introduced now will need a build patch to fix refs #38311
  • lines changed 3, context: html, text, full: html, text
    5cff98cffaac2ec822a4489660df20959a48912e Fri Sep 18 09:34:37 2026 -0700
    hgSession: name tooltip's URL example was swallowing "<sessionName>" #Preview2 week - bugs introduced now will need a build patch to fix refs #38311
  • lines changed 1, context: html, text, full: html, text
    6fcf8b481d7de5712b21db2e9ed7afbae3f65543 Fri Sep 18 09:39:10 2026 -0700
    hgSession: description field's watermark names the field again #Preview2 week - bugs introduced now will need a build patch to fix refs #38311
• src/hg/js/hgTracks.js
  • lines changed 8, context: html, text, full: html, text
    77cfc90b4fb2166b9ae955ad40a067549d6d5881 Tue Sep 15 05:29:56 2026 -0700
    hgc share-a-link: force the enclosing superTrack to show The hgc details-page "Share a link" button strips the hgsid and links straight to the page, but a track nested in a superTrack draws nothing on a fresh load of that link since superTracks default to hide. Add tdbTopSuperTrack() so the per-track JSON carries the enclosing superTrack's name, and have the share dialog force <superTrack>=show onto the URL when there is one.
  • lines changed 4, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
  • lines changed 5, context: html, text, full: html, text
    2eb14fc0378432977d83366a3211c67f8913018d Mon Sep 21 06:21:51 2026 -0700
    hgc and hgTrackUi: open off-site links in a new tab, with rel="noopener noreferrer" offsiteLinksToNewTab() in utils.js runs over the rendered page alongside addHgsidToLinks() and gives every http(s) link whose host is not ours a target=_blank it does not already have, plus rel="noopener noreferrer". Without noopener the page that opens keeps a handle on the tab it came from and can navigate it; without noreferrer the Referer header carries our own URL, which has the session id in it. The href on a track description page is written by whoever wrote the track or the hub, so neither is theoretical. A mailto: or an ftp: link is left alone, and so is every link that stays on this server. jsAddHgsidToLinks() now emits both calls and is renamed jsFixUpPageLinks(). hgTracks.js runs the same pass over the track description popup: the replace it does before that already puts a target on every link in there, so what this adds is the rel on the ones that leave. refs #38380
• src/hg/js/hgc.scatterPlot.js
  • lines changed 32, context: html, text, full: html, text
    2003809b18f5b9267e19693568edbccede0e6ab2 Tue Sep 15 05:48:44 2026 -0700
    hgc scatterPlot: bounds by walking the points, and no duplicate legend entries, refs #38353 niceBounds() collected every coordinate into an array and passed it to Math.min(...xs). A spread is one argument per point, and this module is meant for clouds of tens of thousands: measured, 120,000 points are fine and 200,000 throw RangeError, which the promise's catch turns into "Plot data could not be loaded". Nothing shipped is that big yet. Walked instead, which needs no arrays either. A file that supplies a labels list to fix the legend order and then gives its points in object form had every category appended a second time, so each one appeared twice in the legend. The indexer is seeded from the labels already present.
• src/hg/js/hubApiKey.js
  • lines changed 84, context: html, text, full: html, text
    0ae2a6a36d7b4c77e8e7d7df2a1049f4412e7099 Fri Sep 18 07:03:44 2026 -0700
    hgHubConnect: let a mirror hand out its own API keys, and stop swallowing links to another host #Preview2 week - bugs introduced now will need a build patch to fix Two of the three problems QA found on #38323. The link in the mirror-only Hub Upload message did not go anywhere. The tab handler in hgHubConnect.js catches every hgHubConnect link with a hash and turns it into a tab switch, and 'Go to Hub Upload on genome.ucsc.edu' has a hash that names a tab on the mirror too, so the click just reopened the tab the reader was already on. It now only intercepts links to the page itself. The API key section was decoupled from storeUserFiles, but only for display: the Generate and Revoke buttons are cartJson requests, and both the javascript that sends them and the code in main() that routes them were still inside the storeUserFiles gate. A site with showHubApiKey on and hubSpace off therefore drew two dead buttons. The key functions move out of hgMyData.js into a new hubApiKey.js that the Hub Development tab includes on its own, main() routes a cartJson request when either setting is on, and the hubSpace file commands stay registered only when hubSpace is actually running. The request also goes to this host's hgHubConnect now rather than to the login host. Keys live in the central database of the server that issues them, so a key made on genome-euro belongs in genome-euro's table. refs #38323
• src/hg/js/liftRequest.js
  • lines changed 40, context: html, text, full: html, text
    3c7dea1af68a9671ba48c550be453845827e115d Wed Sep 16 16:44:17 2026 -0700
    gracefully manage bad response from login status refs #38365
  • lines changed 13, context: html, text, full: html, text
    08be525807ff3b3caf1233b9af15b010b950052b Wed Sep 16 16:53:43 2026 -0700
    gracefully manage bad response from api search requests refs #38365
• src/hg/js/makefile
  • lines changed 1, context: html, text, full: html, text
    0ae2a6a36d7b4c77e8e7d7df2a1049f4412e7099 Fri Sep 18 07:03:44 2026 -0700
    hgHubConnect: let a mirror hand out its own API keys, and stop swallowing links to another host #Preview2 week - bugs introduced now will need a build patch to fix Two of the three problems QA found on #38323. The link in the mirror-only Hub Upload message did not go anywhere. The tab handler in hgHubConnect.js catches every hgHubConnect link with a hash and turns it into a tab switch, and 'Go to Hub Upload on genome.ucsc.edu' has a hash that names a tab on the mirror too, so the click just reopened the tab the reader was already on. It now only intercepts links to the page itself. The API key section was decoupled from storeUserFiles, but only for display: the Generate and Revoke buttons are cartJson requests, and both the javascript that sends them and the code in main() that routes them were still inside the storeUserFiles gate. A site with showHubApiKey on and hubSpace off therefore drew two dead buttons. The key functions move out of hgMyData.js into a new hubApiKey.js that the Hub Development tab includes on its own, main() routes a cartJson request when either setting is on, and the hubSpace file commands stay registered only when hubSpace is actually running. The request also goes to this host's hgHubConnect now rather than to the login host. Keys live in the central database of the server that issues them, so a key made on genome-euro belongs in genome-euro's table. refs #38323
• src/hg/js/topLinks.js
  • lines changed 12, context: html, text, full: html, text
    77cfc90b4fb2166b9ae955ad40a067549d6d5881 Tue Sep 15 05:29:56 2026 -0700
    hgc share-a-link: force the enclosing superTrack to show The hgc details-page "Share a link" button strips the hgsid and links straight to the page, but a track nested in a superTrack draws nothing on a fresh load of that link since superTracks default to hide. Add tdbTopSuperTrack() so the per-track JSON carries the enclosing superTrack's name, and have the share dialog force <superTrack>=show onto the URL when there is one.
  • lines changed 1, context: html, text, full: html, text
    dccbd05b94e1a9323a61a14df17ac93b25aee9d3 Thu Sep 17 09:48:20 2026 -0700
    Updating hgLogin's recovery-email wording (heading, menu link, description, and confirmation mail) for clarity and to match Change password/Change email, refs #38197 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
• src/hg/js/utils.js
  • lines changed 21, context: html, text, full: html, text
    3bcf86a0995e5a15ad5a2a8ff5533f48d5d43653 Fri Sep 18 07:15:28 2026 -0700
    hgTracks: tell the user what a session load just did, refs #38157 #Preview2 week - bugs introduced now will need a build patch to fix When a saved session is loaded it replaces the whole cart, and the view the user had before is gone without a word. hgTracks now shows a note on that page saying which session was opened, who saved it and when, the session description if it has one, and that the previous configuration cannot be brought back. Only on that one page: cart.c leaves a marker when a full (non-merge) session load happens and hgTracks takes it out again. Recommended track sets merge into the cart, so they get no note. The note uses a new notifBoxOnce() in utils.js, which has only a Close button: a one-page message does not need 'Don't show again'. Its .notifBoxOnce style is the card grey of the new Sessions page, and it sets no width, so unlike the other notifBoxes it cannot stick out of the window. Switch off with sessionLoadNotice=off in hg.conf.
  • lines changed 50, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
  • lines changed 34, context: html, text, full: html, text
    2eb14fc0378432977d83366a3211c67f8913018d Mon Sep 21 06:21:51 2026 -0700
    hgc and hgTrackUi: open off-site links in a new tab, with rel="noopener noreferrer" offsiteLinksToNewTab() in utils.js runs over the rendered page alongside addHgsidToLinks() and gives every http(s) link whose host is not ours a target=_blank it does not already have, plus rel="noopener noreferrer". Without noopener the page that opens keeps a handle on the tab it came from and can navigate it; without noreferrer the Referer header carries our own URL, which has the session id in it. The href on a track description page is written by whoever wrote the track or the hub, so neither is theoretical. A mailto: or an ftp: link is left alone, and so is every link that stays on this server. jsAddHgsidToLinks() now emits both calls and is renamed jsFixUpPageLinks(). hgTracks.js runs the same pass over the track description popup: the replace it does before that already puts a target on every link in there, so what this adds is the rel on the ones that leave. refs #38380
• src/hg/lib/bigBedFind.c
  • lines changed 101, context: html, text, full: html, text
    1db5c89d97fdd3c3580c4ba9b18eba6b5029c97d Thu Sep 17 05:09:56 2026 -0700
    hgSearch: highlight MANE transcript(s) in RefSeq protein-position search results, refs #38285 #Preview2 week - bugs introduced now will need a build patch to fix A gene-symbol + codon-range search (e.g. "BRCA1 100-200") maps to every RefSeq isoform prediction sharing that genomic footprint, which can be dozens of near-identical NP_ accessions with no indication of which one is the clinically-relevant MANE transcript. Cross-reference the mane bigGenePred track by genomic overlap + protein accession and call out the MANE Select/Plus Clinical transcript(s) in their own section above the rest, gated behind showManeInSearch (default off) in hg.conf.
• src/hg/lib/bigChain.c
  • lines changed 60, context: html, text, full: html, text
    72327e4e2d294b9d2fcb6cbdadd4e628c006345b Fri Sep 18 10:07:40 2026 -0700
    bigChain: move the chain to bigChain conversion into the library chainRecToBigChain() and chainBlockToBigLink() sat in chainToBigChain.c next to its main(), so a CGI could not link against them. Move both into hg/lib/bigChain.c, add chainToBigChainOne() for one chain and chainToBigChainList() for a list, and leave the utility as a thin main(). No behavior change; the chainToBigChain test output is unchanged. refs #38382
• src/hg/lib/botDelay.c
  • lines changed 9, context: html, text, full: html, text
    14ecff0730596f8be147b255afc8acb99e9095e2 Fri Sep 18 09:06:06 2026 -0700
    hgHubConnect: an api key made on one geo mirror now works on all of them, behind syncHubApiKeys #Preview2 week - bugs introduced now will need a build patch to fix #Preview2 week - bugs introduced now will need a build patch to fix Api keys live in the hgcentral of whichever mirror issued them, so a key made on genome.ucsc.edu was rejected on genome-euro and genome-asia, and the botDelay error told the user to go and make a second one. hgHubConnect now tells the other nodes about a key as soon as it generates or revokes one: cjGenerateApiKey and cjRevokeApiKey call syncApiKeyToOtherNodes(), which posts a hgHubSyncApiKey cartJson request to each peer, and cjSyncApiKey() on the far side writes it into that mirror's own table. The peers come from hgcentral.gbNode via the new geoMirrorNotifyOtherNodes(), and the request is signed with login.cookieSalt, which all of a site's mirrors already share - it is what makes the login cookie verifiable on each of them. So neither a list of mirror addresses nor a new shared secret needs provisioning. The notify is best effort: a peer that is down is warn()ed about and skipped, never failing the local generate or revoke, which has already committed by then. All of it is off unless hg.conf says syncHubApiKeys=on. With the gate off the sender returns at once, the receiver refuses the request outright rather than merely being unreachable, and botDelay keeps printing the old server-specific wording, which off is still the truth. Registered in hgConfCatalog.py as a gate, to be flipped once this is released. refs #38323
  • lines changed 18, context: html, text, full: html, text
    89dfb5e8a0602139b83b4d6d1f981710e1a865b2 Mon Sep 21 05:53:28 2026 -0700
    geoMirror: notify peers over https, and drop the hardcoded genome-euro from the invalid-apiKey message, refs #38323 geoMirrorNotifyOtherNodes built its URL as http://. genome-euro and genome-asia redirect http to https, and netSlurpUrl does not follow redirects, so the api key sync request never reached the peer CGI -- and the response is discarded, so nothing said so. Confirmed with lib/tests/fetchUrlTest, which uses the same call: http gives back a bare 302, https gives back the cartJson reply. https also keeps the key off the wire in the clear on its way to Germany and Japan. The geo mirror menu links now use https too. botDelay's "Invalid apiKey" message pointed at genome-euro no matter which server the user was on, which was wrong everywhere except euro. Both branches now build the link from the current server.
  • lines changed 14, context: html, text, full: html, text
    435d98f2883cd5b7e355c5df0695fa0ce092bc88 Mon Sep 21 05:56:51 2026 -0700
    botDelay: the invalid-apiKey message says nothing about where a key is valid, refs #38323 The message had two variants, one for syncHubApiKeys on and one for off, differing only in whether they told the user that apiKeys are server- specific. Keys are going to be synced across the mirrors, so the whole distinction is about to be wrong in one direction or the other. Dropped it: one message, no claim either way, and the link to create a key still points at the server the request came in on.
• src/hg/lib/cart.c
  • lines changed 26, context: html, text, full: html, text
    998149024f8bde04e8b76638043cfeac3d158731 Tue Sep 15 05:47:27 2026 -0700
    Queue the cart cookie and the content policy instead of printing them, refs #38353 cartWriteHeaderAndCont() guards on cgiDidContentType(), which any cgiPrintContentType() anywhere sets. An early warn() during cartNew -- "Unable to load session file" reaches the early warning handler, which calls htmlStart -- prints the header before there is a cart, and from then on the Set-Cookie and Content-Security-Policy lines were silently skipped. Nothing changed on the wire, since before the guard they landed in the page body as text and were equally inert, but skipping them silently is not the behaviour to keep. cartWriteCookie() and cspWriteResponseHeader() now hand their lines to cgiAddHttpHeader(), so whichever call prints the content type prints them too and the order of the calls no longer matters. cartAndCookieWithHtml() queues the policy before the early handlers are pushed, so even a page written by that early warn carries one. The cookie cannot be queued that early -- there is no cart yet -- and is still lost on that path; the comment says so. cgiAddHttpHeader() now does what its own comment already promised and ignores a header added after the block was closed, rather than growing a list nothing will ever print. getCspPolicyString() is declared in htmshell.h so hCommon.c can queue the value on its own.
  • lines changed 27, context: html, text, full: html, text
    2513833e2a3136be51bb8f849dde30c44fa3a5d2 Thu Sep 17 09:53:14 2026 -0700
    cart: the fifth copy of the malformed pair loop, in loadHash loadHash searched for the '=' across the whole remaining string, so a setting with a name and no value ran into the setting behind it and renamed it, and the same pair at the end of the string aborted the CGI. This is the copy cartParseOverHash uses, which reads namedSessionDb.contents and the cart table, so it has the widest reach of the five and the reader has no way to clear a string that is already stored. It now steps over the separators of an empty pair and confines the search to one pair, behind skipMalformedCgiPairs like the three the same ticket fixed. With the flag off the parse is unchanged, including the ampersand winning over the semicolon DAS uses. Nothing is broken today. Every pair in the September hgwbeta session dump has an '=' in it, all 1004 rows, and the cart writer always emits name=value. Found in the v504 code review, refs #38349. refs #38340
  • lines changed 6, context: html, text, full: html, text
    3bcf86a0995e5a15ad5a2a8ff5533f48d5d43653 Fri Sep 18 07:15:28 2026 -0700
    hgTracks: tell the user what a session load just did, refs #38157 #Preview2 week - bugs introduced now will need a build patch to fix When a saved session is loaded it replaces the whole cart, and the view the user had before is gone without a word. hgTracks now shows a note on that page saying which session was opened, who saved it and when, the session description if it has one, and that the previous configuration cannot be brought back. Only on that one page: cart.c leaves a marker when a full (non-merge) session load happens and hgTracks takes it out again. Recommended track sets merge into the cart, so they get no note. The note uses a new notifBoxOnce() in utils.js, which has only a Close button: a one-page message does not need 'Don't show again'. Its .notifBoxOnce style is the card grey of the new Sessions page, and it sets no width, so unlike the other notifBoxes it cannot stick out of the window. Switch off with sessionLoadNotice=off in hg.conf.
• src/hg/lib/geoMirror.c
  • lines changed 42, context: html, text, full: html, text
    1c52aaa92828b6ecc18dd5800fe6650b91daf696 Fri Sep 18 07:22:10 2026 -0700
    hgSession: the new Sessions page also lists the sessions saved on our other servers #Preview2 week - bugs introduced now will need a build patch to fix Every geo mirror node keeps its own namedSessionDb, so a session saved on genome-euro is invisible on genome.ucsc.edu and people do not find it again. The table now holds the sessions from every node, with a new Server column saying where each one lives, and the session name links to the server that holds it. Two new hgSession actions, both answered in main() before the cart is built, so that a request from another node leaves no cart, userDb or sessionDb row behind: hgS_doSessionListJson returns this server's sessions for the user named by the login cookie, as JSON. This is what the other nodes call. hgS_doMirrorSessions asks every other gbNode for that list and returns what came back. It sends all of the requests before reading any of the answers, so the wait is the slowest node rather than the sum of them. The fetch goes through our own server rather than from the browser to the other nodes, which keeps cross-origin requests out of it: apache handles the CORS headers differently on every node, and the two mirrors are administered by Bielefeld and RIKEN, while CGI code reaches them with the release. The user's login cookies, and only those, are passed on (wikiLinkLoginCookieHeader), which works because the nodes share a cookie salt. The page never waits for any of this: the table is drawn from this server's own sessions and the rows from the other servers are merged in as they arrive, keeping the current sort, search and page. A node that does not answer, times out, or runs a release without the endpoint is named in a line under the toolbar instead, and costs nothing else. Sessions from another server are shown but not managed here: no Share, Edit, Overwrite or Delete, no bulk-select checkbox, and the Update now shortcut keeps ignoring them. Table rows are keyed by a new uid rather than by the session name, since the same name can now be in the table twice, once per server. geoMirrorThisNode() and geoMirrorOtherNodes() read the node list from the hgcentral gbNode table, so nothing new has to be configured: browser.node already says which node a server is.
  • lines changed 73, context: html, text, full: html, text
    aeee0cca244f1a656d8f52ac1bec7922e136b9f0 Fri Sep 18 07:35:05 2026 -0700
    hgSession: a server that is not the node hg.conf names is still itself #Preview2 week - bugs introduced now will need a build patch to fix On hgwdev-max the new Sessions page called itself genome-test and listed hgwdev-max among the servers to go and fetch sessions from, so the Server column named the wrong machine and the page asked itself for its own list. geoMirrorNodeList() was picking the gbNode row by browser.node, and a sandbox inherits that setting from the shared hg.conf without being that node. It now takes the row whose domain matches the host the visitor typed, and falls back to browser.node only when the host is in no row at all - hgwdev.gi.ucsc.edu rather than genome-test.gi.ucsc.edu, a bare IP, the command line. refs #38157
  • lines changed 73, context: html, text, full: html, text
    7c0082ff8679670a4ba25c181c8af06ae25e0773 Fri Sep 18 07:35:08 2026 -0700
    docs update
  • lines changed 73, context: html, text, full: html, text
    a47a4664fd739e6d1aadb03727f1916c0531e37f Fri Sep 18 07:36:14 2026 -0700
    hgSession: a server that is not the node hg.conf names is still itself #Preview2 week - bugs introduced now will need a build patch to fix On hgwdev-max the new Sessions page called itself genome-test and listed hgwdev-max among the servers to go and fetch sessions from, so the Server column named the wrong machine and the page asked itself for its own list. geoMirrorNodeList() was picking the gbNode row by browser.node, and a sandbox inherits that setting from the shared hg.conf without being that node. It now takes the row whose domain matches the host the visitor typed, and falls back to browser.node only when the host is in no row at all - hgwdev.gi.ucsc.edu rather than genome-test.gi.ucsc.edu, a bare IP, the command line. This was committed once already as aeee0cca244 and lost in the merge c334c82aea4, which resolved geoMirror.c to the side without it. refs #27988
  • lines changed 37, context: html, text, full: html, text
    14ecff0730596f8be147b255afc8acb99e9095e2 Fri Sep 18 09:06:06 2026 -0700
    hgHubConnect: an api key made on one geo mirror now works on all of them, behind syncHubApiKeys #Preview2 week - bugs introduced now will need a build patch to fix #Preview2 week - bugs introduced now will need a build patch to fix Api keys live in the hgcentral of whichever mirror issued them, so a key made on genome.ucsc.edu was rejected on genome-euro and genome-asia, and the botDelay error told the user to go and make a second one. hgHubConnect now tells the other nodes about a key as soon as it generates or revokes one: cjGenerateApiKey and cjRevokeApiKey call syncApiKeyToOtherNodes(), which posts a hgHubSyncApiKey cartJson request to each peer, and cjSyncApiKey() on the far side writes it into that mirror's own table. The peers come from hgcentral.gbNode via the new geoMirrorNotifyOtherNodes(), and the request is signed with login.cookieSalt, which all of a site's mirrors already share - it is what makes the login cookie verifiable on each of them. So neither a list of mirror addresses nor a new shared secret needs provisioning. The notify is best effort: a peer that is down is warn()ed about and skipped, never failing the local generate or revoke, which has already committed by then. All of it is off unless hg.conf says syncHubApiKeys=on. With the gate off the sender returns at once, the receiver refuses the request outright rather than merely being unreachable, and botDelay keeps printing the old server-specific wording, which off is still the truth. Registered in hgConfCatalog.py as a gate, to be flipped once this is released. refs #38323
  • lines changed 4, context: html, text, full: html, text
    89dfb5e8a0602139b83b4d6d1f981710e1a865b2 Mon Sep 21 05:53:28 2026 -0700
    geoMirror: notify peers over https, and drop the hardcoded genome-euro from the invalid-apiKey message, refs #38323 geoMirrorNotifyOtherNodes built its URL as http://. genome-euro and genome-asia redirect http to https, and netSlurpUrl does not follow redirects, so the api key sync request never reached the peer CGI -- and the response is discarded, so nothing said so. Confirmed with lib/tests/fetchUrlTest, which uses the same call: http gives back a bare 302, https gives back the cartJson reply. https also keeps the key off the wire in the clear on its way to Germany and Japan. The geo mirror menu links now use https too. botDelay's "Invalid apiKey" message pointed at genome-euro no matter which server the user was on, which was wrong everywhere except euro. Both branches now build the link from the current server.
  • lines changed 29, context: html, text, full: html, text
    33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
    hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323 Six things, all in the syncHubApiKeys path, which is still off everywhere. The sync never reached its handler on a real mirror. It was registered as a cartJson command, so building the cart ran forceUserIdOrCaptcha() first; a peer's request carries no hguid cookie and no apiKey= variable, so any site with cloudFlareSiteKey set -- which is every site that needs api keys in the first place -- answered it with a captcha page. Only hgwdev, where the captcha is commented out, ever ran the handler, which is why this was not caught earlier. doApiKeySyncIfRequested() now answers the request in main() before any cart exists. That also stops each sync leaving a junk userDb and sessionDb row behind: measured, 15 syncs now create none. The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key), so it does not rest on md5 resisting length extension, and it is compared in constant time. It now covers a timestamp too, and a peer refuses anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of now, so a captured sync cannot be replayed later to reinstate a key its owner has since revoked. The fields are joined with newlines and a newline in a userName or apiKey is refused, so one signature cannot cover two different splits of the same string. syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It runs after the local key is already written and in the response, so an errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's outer catch, which threw the response away -- the user saw a failure over a key that was sitting in the database. geoMirrorNotifyOtherNodes() returns each peer's response instead of discarding it, and uses netUrlMustOpenPastHeader, which errAborts on anything but a 200 and hands back the body alone. A peer answering 'bad signature' or 'not enabled on this site' is no longer indistinguishable from success; it is logged to error_log, not warn()ed at the person who clicked the button. Verified against the built CGI run as Apache runs it, with the captcha turned on: a signed sync is accepted and writes the row, a signed revoke removes it, and a replay, a swapped userName, a forged signature and malformed json are all refused. hubSpaceKeysTester covers the signature rules; it still cannot run its database half until hgcentralregress gets the UPDATE and DELETE grant.
• src/hg/lib/hCommon.c
  • lines changed 8, context: html, text, full: html, text
    998149024f8bde04e8b76638043cfeac3d158731 Tue Sep 15 05:47:27 2026 -0700
    Queue the cart cookie and the content policy instead of printing them, refs #38353 cartWriteHeaderAndCont() guards on cgiDidContentType(), which any cgiPrintContentType() anywhere sets. An early warn() during cartNew -- "Unable to load session file" reaches the early warning handler, which calls htmlStart -- prints the header before there is a cart, and from then on the Set-Cookie and Content-Security-Policy lines were silently skipped. Nothing changed on the wire, since before the guard they landed in the page body as text and were equally inert, but skipping them silently is not the behaviour to keep. cartWriteCookie() and cspWriteResponseHeader() now hand their lines to cgiAddHttpHeader(), so whichever call prints the content type prints them too and the order of the calls no longer matters. cartAndCookieWithHtml() queues the policy before the early handlers are pushed, so even a page written by that early warn carries one. The cookie cannot be queued that early -- there is no cart yet -- and is still lost on that path; the comment says so. cgiAddHttpHeader() now does what its own comment already promised and ignores a header added after the block was closed, rather than growing a list nothing will ever print. getCspPolicyString() is declared in htmshell.h so hCommon.c can queue the value on its own.
• src/hg/lib/hVarSubst.c
  • lines changed 131, context: html, text, full: html, text
    5f590f874b9d352023167b563e5b83f651488f0b Tue Sep 15 04:56:35 2026 -0700
    Resolve $hgsid in native track description pages too, refs #38283 A description page could already use $hgsid if it belonged to a hub, because a hub's html is substituted at render time where there is a cart. A native page is substituted once by hgTrackDb, which has no cart, so $hgsid quietly became the empty string and the link it was part of came out broken. hg38's hprcRdt page has been shipping "hgTrackUi?hgsid=&g=long_read_transcripts" for this reason. hgTrackDb now writes the reference back out as ${hgsid} instead of resolving it, and hVarSubstTrackDbHtml resolves it at render time for native pages as well as hub ones. Only $hgsid is deferred, and only in the html field: the labels get no second pass, so a deferred reference in one of them would reach the user as the literal text "${hgsid}". The render pass over a native page acts only on the braced form, which is what keeps an escaped $$hgsid escaped -- hgTrackDb collapses that to a bare $hgsid, and a bare one is left alone. The html is no longer freed before being replaced, and is replaced only when the substitution actually changed something. hVarSubstExt allocates its result at the first dollar sign whether or not it substitutes anything, so any page merely containing one reached that free; for a native track tdb->html can point into the trackDb cache, which is localmem carved out of an mmap'd file and never came from malloc. Reproduced on hg38 chm13LiftOver, whose page contains an awk snippet, with cacheTrackDbDir set. Description pages have to be reloaded for the deferral to reach the trackDb table.
  • lines changed 122, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/lib/hdb.c
  • lines changed 6, context: html, text, full: html, text
    eb7e6ccb9b23c2dd5cd82de0237c6f25aecd2da1 Thu Sep 17 06:02:11 2026 -0700
    Hub tracks must not claim an undecorated name that a native track owns #Preview2 week - bugs introduced now will need a build patch to fix A hub track can be named on a URL, or found in the cart, without its "hub_<id>_" prefix, so that hub links stay readable. Nothing checked whether the assembly already had a track of that name, so when it did, the hub track took the variable: with the MethBase2 hub attached, setting the native hg38 Methbase track's visibility moved it to hub_<id>_Methbase and dropped it from the native track, which then did not draw. The two share the name "Methbase", and their subtracks share names too, so the same went for the _sel checkboxes. hubTrackOwnsBareName() now gates the seven places that fall back to the bare name - track and subtrack visibility, supertrack visibility, the two _hideKids variables, and the _sel checkbox. The bare name is the hub track's only where the assembly has no track of that name; otherwise it belongs to the native one, whose own pass over the track list consumes it. It answers from the track list loadFromTrackDb() has already built, so it costs a hash lookup: hub tracks are always prefixed in that list, so an undecorated name can only match a native track. A caller with no list to register (hgTrackUi) falls back to a single-row trackDb query per distinct bare name, cached. With 500 hub subtracks visible and 500 bare names on the URL - far past anything real - that is 189 queries either way, against 1189 for a query per name. Also clarifies the comments on hTrackDbWithCartVersion() and tdbForTrack(): the "this result is cached" note means the shared-memory trackDb cache only, which is off unless cacheTrackDbDir is set, and there is no memoizing besides it - so hTrackDbForTrack() per name reloads the whole trackDb each call. Measured at 57 seconds for 500 names before I abandoned that approach. refs #34246
• src/hg/lib/hgHgvs.c
  • lines changed 8, context: html, text, full: html, text
    4bf24023fb4ce3e83104c97043c7e898101be1d1 Tue Sep 15 05:48:15 2026 -0700
    hgvs: a bare codon number must not leave a tail behind, refs #38353 The three patterns that read a bare codon number or range were anchored at the start only, so "KAT6A 495-", "KAT6A p.495_", "KAT6A 495--533" and "KAT6A p.495_533_600" all quietly became codon 495 (or 495_533) with the rest thrown away. Silently dropping the tail of a position is exactly what this grammar was added to stop doing. Anchored at the end as well. Every other pattern here is free to leave a tail for a later stage to interpret, but a bare codon number has nothing after it to interpret. The tests now pin the four rejected forms, and also codon 0, a codon past the end of the protein, and a backwards range, none of which were pinned before.
• src/hg/lib/hubSpaceKeys.c
  • lines changed 42, context: html, text, full: html, text
    14ecff0730596f8be147b255afc8acb99e9095e2 Fri Sep 18 09:06:06 2026 -0700
    hgHubConnect: an api key made on one geo mirror now works on all of them, behind syncHubApiKeys #Preview2 week - bugs introduced now will need a build patch to fix #Preview2 week - bugs introduced now will need a build patch to fix Api keys live in the hgcentral of whichever mirror issued them, so a key made on genome.ucsc.edu was rejected on genome-euro and genome-asia, and the botDelay error told the user to go and make a second one. hgHubConnect now tells the other nodes about a key as soon as it generates or revokes one: cjGenerateApiKey and cjRevokeApiKey call syncApiKeyToOtherNodes(), which posts a hgHubSyncApiKey cartJson request to each peer, and cjSyncApiKey() on the far side writes it into that mirror's own table. The peers come from hgcentral.gbNode via the new geoMirrorNotifyOtherNodes(), and the request is signed with login.cookieSalt, which all of a site's mirrors already share - it is what makes the login cookie verifiable on each of them. So neither a list of mirror addresses nor a new shared secret needs provisioning. The notify is best effort: a peer that is down is warn()ed about and skipped, never failing the local generate or revoke, which has already committed by then. All of it is off unless hg.conf says syncHubApiKeys=on. With the gate off the sender returns at once, the receiver refuses the request outright rather than merely being unreachable, and botDelay keeps printing the old server-specific wording, which off is still the truth. Registered in hgConfCatalog.py as a gate, to be flipped once this is released. refs #38323
  • lines changed 72, context: html, text, full: html, text
    33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
    hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323 Six things, all in the syncHubApiKeys path, which is still off everywhere. The sync never reached its handler on a real mirror. It was registered as a cartJson command, so building the cart ran forceUserIdOrCaptcha() first; a peer's request carries no hguid cookie and no apiKey= variable, so any site with cloudFlareSiteKey set -- which is every site that needs api keys in the first place -- answered it with a captcha page. Only hgwdev, where the captcha is commented out, ever ran the handler, which is why this was not caught earlier. doApiKeySyncIfRequested() now answers the request in main() before any cart exists. That also stops each sync leaving a junk userDb and sessionDb row behind: measured, 15 syncs now create none. The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key), so it does not rest on md5 resisting length extension, and it is compared in constant time. It now covers a timestamp too, and a peer refuses anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of now, so a captured sync cannot be replayed later to reinstate a key its owner has since revoked. The fields are joined with newlines and a newline in a userName or apiKey is refused, so one signature cannot cover two different splits of the same string. syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It runs after the local key is already written and in the response, so an errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's outer catch, which threw the response away -- the user saw a failure over a key that was sitting in the database. geoMirrorNotifyOtherNodes() returns each peer's response instead of discarding it, and uses netUrlMustOpenPastHeader, which errAborts on anything but a 200 and hands back the body alone. A peer answering 'bad signature' or 'not enabled on this site' is no longer indistinguishable from success; it is logged to error_log, not warn()ed at the person who clicked the button. Verified against the built CGI run as Apache runs it, with the captcha turned on: a signed sync is accepted and writes the row, a signed revoke removes it, and a replay, a swapped userName, a forged signature and malformed json are all refused. hubSpaceKeysTester covers the signature rules; it still cannot run its database half until hgcentralregress gets the UPDATE and DELETE grant.
• src/hg/lib/hui.c
  • lines changed 14, context: html, text, full: html, text
    e4c2e97b5eb35b8f9b7b71eb13a4c24e6c1ece22 Mon Sep 14 09:19:02 2026 -0700
    hui, hgTrackUi: encode trackDb setting and cart text in HTML attributes, refs #38123
  • lines changed 9, context: html, text, full: html, text
    026fb1c67f6c42f339bf2af21c3ee37903953930 Thu Sep 17 06:01:20 2026 -0700
    Faceted composite: a trackDb "visibility hide" on a child must not pin it hidden #Preview2 week - bugs introduced now will need a build patch to fix tdbVisLimitedByAncestors() took any local "visibility" setting on a faceted composite's child as the user asking for that display mode, so a trackDb "visibility hide" clamped the child to hide for good: neither the _sel checkbox nor onlyVisibility could lift it, and nothing on the page said why. The MethBase2 hub at hgdownload.soe.ucsc.edu/hubs/methbase/v3 writes that line on all 26,028 of its subtracks, so picking samples in its faceted table wrote the right cart variables and then drew nothing. The native hg38 Methbase track has the line only on the composite, which is why it worked and the hub did not. Only a cart value now counts as a request to hide. A trackDb "visibility hide" falls back like no setting at all, since whether a faceted child shows is the checkbox's business. No faceted composite in trackDb relies on the old reading - they use visibility full/squish on children and hide only on the container. refs #34246
  • lines changed 99, context: html, text, full: html, text
    eb7e6ccb9b23c2dd5cd82de0237c6f25aecd2da1 Thu Sep 17 06:02:11 2026 -0700
    Hub tracks must not claim an undecorated name that a native track owns #Preview2 week - bugs introduced now will need a build patch to fix A hub track can be named on a URL, or found in the cart, without its "hub_<id>_" prefix, so that hub links stay readable. Nothing checked whether the assembly already had a track of that name, so when it did, the hub track took the variable: with the MethBase2 hub attached, setting the native hg38 Methbase track's visibility moved it to hub_<id>_Methbase and dropped it from the native track, which then did not draw. The two share the name "Methbase", and their subtracks share names too, so the same went for the _sel checkboxes. hubTrackOwnsBareName() now gates the seven places that fall back to the bare name - track and subtrack visibility, supertrack visibility, the two _hideKids variables, and the _sel checkbox. The bare name is the hub track's only where the assembly has no track of that name; otherwise it belongs to the native one, whose own pass over the track list consumes it. It answers from the track list loadFromTrackDb() has already built, so it costs a hash lookup: hub tracks are always prefixed in that list, so an undecorated name can only match a native track. A caller with no list to register (hgTrackUi) falls back to a single-row trackDb query per distinct bare name, cached. With 500 hub subtracks visible and 500 bare names on the URL - far past anything real - that is 189 queries either way, against 1189 for a query per name. Also clarifies the comments on hTrackDbWithCartVersion() and tdbForTrack(): the "this result is cached" note means the shared-memory trackDb cache only, which is off unless cacheTrackDbDir is set, and there is no memoizing besides it - so hTrackDbForTrack() per name reloads the whole trackDb each call. Measured at 57 seconds for 500 names before I abandoned that approach. refs #34246
• src/hg/lib/jsHelper.c
  • lines changed 15, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
  • lines changed 9, context: html, text, full: html, text
    2eb14fc0378432977d83366a3211c67f8913018d Mon Sep 21 06:21:51 2026 -0700
    hgc and hgTrackUi: open off-site links in a new tab, with rel="noopener noreferrer" offsiteLinksToNewTab() in utils.js runs over the rendered page alongside addHgsidToLinks() and gives every http(s) link whose host is not ours a target=_blank it does not already have, plus rel="noopener noreferrer". Without noopener the page that opens keeps a handle on the tab it came from and can navigate it; without noreferrer the Referer header carries our own URL, which has the session id in it. The href on a track description page is written by whoever wrote the track or the hub, so neither is theoretical. A mailto: or an ftp: link is left alone, and so is every link that stays on this server. jsAddHgsidToLinks() now emits both calls and is renamed jsFixUpPageLinks(). hgTracks.js runs the same pass over the track description popup: the replace it does before that already puts a target on every link in there, so what this adds is the rel on the ones that leave. refs #38380
• src/hg/lib/quickLift.c
  • lines changed 13, context: html, text, full: html, text
    57f2a4a958e5a6432829e33deaaf7b0475971188 Thu Sep 17 09:52:34 2026 -0700
    quickLift: keep the target strand a reverse complemented protein just got quickLiftPslBackToProtein turns the alignment over when pslTransMap hands back strand[0] == '-', because a protein psl is only ever "++" or "+-". pslRc does that and makes the target strand explicit as it goes, so the psl comes out "+-". The assignment at the end of the function then set strand[1] to '+' regardless and undid it, leaving "++" over blocks that are in minus strand target coordinates. pslIsProtein wants strand[1] plus the three to one relation between the last block and the target end, so it answered no. pslTrack.c then passed a size multiplier of one to lfFromPslx and every block was drawn at a third of its length at a mirrored position, which is the symptom the reverse complement was added to prevent. The assignment is now the else branch of the same test. hg/lib/tests/quickLiftTester.c covers this. It goes through the public quickLiftPsl rather than the static function, builds its chains in memory in the shape quickLiftSourceRanges leaves them, and needs no database. Six cases: a protein over a same strand chain and over an opposite strand chain, a chain that gaps only the reference, a chain that drops a source base and so splits a codon, an mRNA over both chains, and an alignment with no chain under it. Each one runs pslCheck2, which is what notices a strand that disagrees with the blocks. With this fix backed out, only the opposite strand case changes and it reports three errors placing the blocks outside the target range. Found in the v504 code review, refs #38349. refs #38249
• src/hg/lib/snapshotSession.c
  • lines changed 5, context: html, text, full: html, text
    c500eb92eb2a021af927f3c6f2ad00c7b29769a5 Tue Sep 15 05:48:49 2026 -0700
    snapshotSession: snapshotIsSnapshotName is file-local, refs #38353 Its only caller is the writer in the same file, checking its own argument, and its header comment warned everyone else off it. Making it static says that rather than asking.
• src/hg/lib/tests/asForDbTester.c
  • lines changed 97, context: html, text, full: html, text
    0f2f66270dcd4603785850ee72d7d883ff68185b Sun Sep 20 07:03:14 2026 -0700
    asForDbTester: a GenArk accession must not be looked for in MySQL asForDb() may need a database connection to fetch a track's autoSql. The name it is handed is usually an assembly, but on a quickLifted track it is the SOURCE assembly, and for a GenArk that arrives as a bare accession with no MySQL database of that name anywhere. Without #38272's isGenArk test the CGI does not return an empty field list, it dies, and the reader gets an error page instead of a details page -- only for the combination of a quickLift and a GenArk source, which is why it went unnoticed. The last case is the discriminating one: a name that is neither a GenArk accession nor a real database still aborts, which shows the GenArk case is let through deliberately rather than by connection failures having stopped mattering. The abort message carries a MySQL error number and the server's own wording, so what is pinned is whether the abort was about reaching a database of that name, not the text. Two things this test got wrong before it got them right, both recorded in its header. It has to run against a config that sets genarkHubPrefix, because isGenArk answers through that prefix and says no when it is unset, so with a developer's own hg.conf the GenArk case failed for a reason unrelated to the fix. And the accession is read from the genark table at run time, since naming an assembly makes the test go red the day that assembly is dropped. Also fixes the registry row for bedItemRgbTester, which moved to hg/cgilib and whose row still named the old path -- caught by the registry's own check, which is the first time it has failed for a real reason. refs #38272, refs #38391
• src/hg/lib/tests/bedItemRgbTester.c
  • lines changed 106, context: html, text, full: html, text
    a0a3411bca2fdd6d704ce2be7c9c8fca7682913a Sat Sep 19 18:19:36 2026 -0700
    bedItemRgbTester: pin the order of the itemRgb and color tests bedItemRgb() decides whether a BED track draws its items in the colors its file carries or in the one color its stanza names. The rule has four steps, and the order of the first three is the whole of #36212: an explicit "itemRgb off" wins, then an explicit "itemRgb on" wins, and only then does the presence of a "color" setting turn item colors off by default. 88d620e6c82 folded those tests together, so a stanza saying both "itemRgb on" and "color" -- which means items from the file and labels from color -- lost its item colors. The pairs are what matter here. Every single-setting case passed while the bug was live, so a test that exercised one setting at a time would have proved nothing. A child that says "itemRgb on" under a parent that says "color" is the same case reached through trackDbSettingClosestToHome. Two runs, with and without hg.conf's alwaysItemRgb, since the last step of the rule reads it and a mirror that turns it off must still honour a stanza that asks for item colors explicitly. The test lives in hg/lib/tests because hg/cgilib has no tests directory; its link line adds jkhgapcgi.a. Watched to fail and then pass: with the color test moved back in front of the itemRgb test, three lines flip, and they are the three that pair the two settings. Recorded as sandbox-ab in utils/testRegistry. refs #36212, refs #38391
• src/hg/lib/tests/dataVersionPathTester.c
  • lines changed 85, context: html, text, full: html, text
    471b82ec87d4b23b03ec3f3a2a193eb1f88a035c Sat Sep 19 18:24:53 2026 -0700
    dataVersionPathTester: pin which local files a hub's dataVersion may read A track's dataVersion setting is usually a version string, but for otto tracks it is the path of a local file that hgTrackUi opens and prints. On a hub track that setting is an instruction from a stranger to read a named file on our server and put it on the page. #38268 narrowed it to /gbdb, which is public data mirrored on hgdownload, and to a plain path, since a ".." component makes the name mean something outside that tree. The exception is needed: curated-hub assemblies are served as hubs, so hs1's otto tracks are hub tracks, and a quickLifted track's version file lives on the source assembly. Nothing about the failure is visible in the usual way. A refused path and an accepted one both leave a page that looks reasonable, and the difference shows only as somebody else's file appearing where a version number belongs. The test prints a classification and never a file. The three outcomes separate without looking at any contents: a refused path comes back as the path itself, an accepted path that does not exist comes back NULL, and an accepted path that exists comes back as something else. Watched to fail and then pass: with the /gbdb test dropped, /etc/passwd comes back read rather than refused, and the two climbing cases come back opened. Recorded as sandbox-ab in utils/testRegistry. refs #38268, refs #38391
• src/hg/lib/tests/expected/asForDbTest
  • lines changed 5, context: html, text, full: html, text
    0f2f66270dcd4603785850ee72d7d883ff68185b Sun Sep 20 07:03:14 2026 -0700
    asForDbTester: a GenArk accession must not be looked for in MySQL asForDb() may need a database connection to fetch a track's autoSql. The name it is handed is usually an assembly, but on a quickLifted track it is the SOURCE assembly, and for a GenArk that arrives as a bare accession with no MySQL database of that name anywhere. Without #38272's isGenArk test the CGI does not return an empty field list, it dies, and the reader gets an error page instead of a details page -- only for the combination of a quickLift and a GenArk source, which is why it went unnoticed. The last case is the discriminating one: a name that is neither a GenArk accession nor a real database still aborts, which shows the GenArk case is let through deliberately rather than by connection failures having stopped mattering. The abort message carries a MySQL error number and the server's own wording, so what is pinned is whether the abort was about reaching a database of that name, not the text. Two things this test got wrong before it got them right, both recorded in its header. It has to run against a config that sets genarkHubPrefix, because isGenArk answers through that prefix and says no when it is unset, so with a developer's own hg.conf the GenArk case failed for a reason unrelated to the fix. And the accession is read from the genark table at run time, since naming an assembly makes the test go red the day that assembly is dropped. Also fixes the registry row for bedItemRgbTester, which moved to hg/cgilib and whose row still named the old path -- caught by the registry's own check, which is the first time it has failed for a real reason. refs #38272, refs #38391
• src/hg/lib/tests/expected/bedItemRgbTest
  • lines changed 38, context: html, text, full: html, text
    a0a3411bca2fdd6d704ce2be7c9c8fca7682913a Sat Sep 19 18:19:36 2026 -0700
    bedItemRgbTester: pin the order of the itemRgb and color tests bedItemRgb() decides whether a BED track draws its items in the colors its file carries or in the one color its stanza names. The rule has four steps, and the order of the first three is the whole of #36212: an explicit "itemRgb off" wins, then an explicit "itemRgb on" wins, and only then does the presence of a "color" setting turn item colors off by default. 88d620e6c82 folded those tests together, so a stanza saying both "itemRgb on" and "color" -- which means items from the file and labels from color -- lost its item colors. The pairs are what matter here. Every single-setting case passed while the bug was live, so a test that exercised one setting at a time would have proved nothing. A child that says "itemRgb on" under a parent that says "color" is the same case reached through trackDbSettingClosestToHome. Two runs, with and without hg.conf's alwaysItemRgb, since the last step of the rule reads it and a mirror that turns it off must still honour a stanza that asks for item colors explicitly. The test lives in hg/lib/tests because hg/cgilib has no tests directory; its link line adds jkhgapcgi.a. Watched to fail and then pass: with the color test moved back in front of the itemRgb test, three lines flip, and they are the three that pair the two settings. Recorded as sandbox-ab in utils/testRegistry. refs #36212, refs #38391
• src/hg/lib/tests/expected/dataVersionPathTest
  • lines changed 11, context: html, text, full: html, text
    471b82ec87d4b23b03ec3f3a2a193eb1f88a035c Sat Sep 19 18:24:53 2026 -0700
    dataVersionPathTester: pin which local files a hub's dataVersion may read A track's dataVersion setting is usually a version string, but for otto tracks it is the path of a local file that hgTrackUi opens and prints. On a hub track that setting is an instruction from a stranger to read a named file on our server and put it on the page. #38268 narrowed it to /gbdb, which is public data mirrored on hgdownload, and to a plain path, since a ".." component makes the name mean something outside that tree. The exception is needed: curated-hub assemblies are served as hubs, so hs1's otto tracks are hub tracks, and a quickLifted track's version file lives on the source assembly. Nothing about the failure is visible in the usual way. A refused path and an accepted one both leave a page that looks reasonable, and the difference shows only as somebody else's file appearing where a version number belongs. The test prints a classification and never a file. The three outcomes separate without looking at any contents: a refused path comes back as the path itself, an accepted path that does not exist comes back NULL, and an accepted path that exists comes back as something else. Watched to fail and then pass: with the /gbdb test dropped, /etc/passwd comes back read rather than refused, and the two climbing cases come back opened. Recorded as sandbox-ab in utils/testRegistry. refs #38268, refs #38391
• src/hg/lib/tests/expected/genarkLiftOverTest
  • lines changed 11, context: html, text, full: html, text
    d58bf9a182cbf4dba142a7028742abf688638829 Sat Sep 19 18:27:02 2026 -0700
    genarkLiftOverTester: pin the escaping of the GenArk accession list Before #38328 the caller pasted its accessions into one string and genarkLiftOverDbs dropped that string into a query marked NOSQLINJ, which is a promise that somebody upstream had made it safe. It now takes an slName list and builds the query itself with sqlDyStringPrintf, so the escaping happens where the values are, and a name that is not a GC accession never reaches the database. Nothing about this is visible. Correct and incorrect escaping give the same page for every input a browser sends, because the accessions come from our own chain files. The difference appears only for a value chosen to break out, which is the value that never turns up in ordinary testing. The genark table is data and it moves, so the test reads an accession out of it and asks whether the function returns that one, rather than naming an assembly that may be dropped later. Watched to fail and then pass: with the value pasted in unescaped the run dies instead of coming back with nothing, so the quoted cases turn the test red rather than quietly querying something else. Recorded as sandbox-ab in utils/testRegistry. refs #38328, refs #38391
• src/hg/lib/tests/expected/geoMirrorSelfTest
  • lines changed 8, context: html, text, full: html, text
    36977d0e699d3c853840f08cec63ab2a2dd5eaa7 Sat Sep 19 18:34:00 2026 -0700
    geoMirrorSelfTester: pin which gbNode row a server takes for itself The browser runs on several machines and each offers the others in a menu, so it has to know which gbNode row is itself. It used to answer from browser.node in hg.conf alone, so a machine serving a node other than the one browser.node names -- a sandbox, or two nodes behind one apache -- took itself for its own peer and offered the visitor a link to the site they were already on. #27988 made the host the visitor typed decide whenever that host is one of the nodes, with browser.node as the fallback. The symptom is invisible in the way that matters: every page renders, and a menu entry pointing back at itself reads as a mirror being down rather than as a bug. No domain is named here. gbNode is configuration data whose rows change, so the test reads two rows, points browser.node at the first, and asks whether the second can claim the server. What is printed is which of the two answered, not what they are called, and the port case is included because HTTP_HOST carries one whenever it is not 80 or 443. Also recorded in utils/testRegistry: what blocks a unit test on the fifteen v504 tickets still waiting for one, each with its own reason rather than a category, since the reason decides who can unblock it. Watched to fail and then pass: with the host test removed the server claims no node at all and offers all three, itself included. Recorded as sandbox-ab. refs #27988, refs #38391
• src/hg/lib/tests/expected/hVarSubstHtmlTest
  • lines changed 24, context: html, text, full: html, text
    d898b4820087cccd79e240153bc310856bb89310 Sat Sep 19 18:22:49 2026 -0700
    hVarSubstHtmlTester: pin which variables a description page may use A native description page has been through hgTrackDb already, which resolved everything it could and deferred only ${hgsid}, so the render pass acts on that one variable and only in braces: hgTrackDb collapses an escaped $$hgsid to a literal $hgsid, and acting on the bare form here would expand the very thing the author escaped. A hub's page has never been substituted, so the whole hub list is resolved at render time, and $hgsid is deliberately not on that list. A hub page is only lightly sanitized -- an <img> with an http src survives -- so a page carrying <img src="https://example.com/px?s=${hgsid}"> would hand the reader's session id to the hub's own server, and a session id alone is enough to read and write that cart. The page renders the same either way and the request goes to somebody else's host, so nothing about this is visible here. The test builds a cart by hand rather than opening one. That is not a shortcut, it is the point: a cartless version of this test was written first, and adding hgsid back to hubHtmlVars changed not one line of its output, because the check that recognises a variable also asks whether this call can resolve it, and with no cart it cannot. cartSessionId reads two fields, so a struct built in the test is enough and no database is involved. Watched to fail and then pass: with hgsid back on the hub list, both hub cases come back with the session id substituted into the img src. Recorded as sandbox-ab in utils/testRegistry. refs #38283, refs #38391
  • lines changed 7, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/lib/tests/expected/hgvs/validTerms.txt
  • lines changed 12, context: html, text, full: html, text
    4bf24023fb4ce3e83104c97043c7e898101be1d1 Tue Sep 15 05:48:15 2026 -0700
    hgvs: a bare codon number must not leave a tail behind, refs #38353 The three patterns that read a bare codon number or range were anchored at the start only, so "KAT6A 495-", "KAT6A p.495_", "KAT6A 495--533" and "KAT6A p.495_533_600" all quietly became codon 495 (or 495_533) with the rest thrown away. Silently dropping the tail of a position is exactly what this grammar was added to stop doing. Anchored at the end as well. Every other pattern here is free to leave a tail for a later stage to interpret, but a bare codon number has nothing after it to interpret. The tests now pin the four rejected forms, and also codon 0, a codon past the end of the protein, and a backwards range, none of which were pinned before.
  • lines changed 8, context: html, text, full: html, text
    814335cde61f999ae23626093306bdb6bf394f5e Sun Sep 20 07:00:39 2026 -0700
    hgvs: cover a deprecated protein accession, and say what is not covered A protein accession that is no longer current resolves through ncbiRefSeqLinkHistorical, which is what lets a term from an old paper still find its position. Three such terms are added to the valid-terms list: NP_000054.2, which has three deprecated transcript versions behind it, and NP_055615.8, which has versions .8, .9 and .10. The other half of #38248 is NOT pinned, and the registry row says so rather than implying otherwise. That half picks the newest deprecated transcript by sorting on length first, so that .9 does not beat .10 as a string. I could not build a term that shows it: with the order by removed entirely, and again with a plain string sort, every term gives identical coordinates. The reason is in the data -- for the accessions with both a .9 and a .10, the versions differ only in where the UTR ends, so cdsStart, cdsEnd and exonCount are the same and a coding residue lands in the same place whichever version is chosen. Recorded as assertion-only, the honest level, with what was tried written down so the next person does not repeat it. refs #38248, refs #38391
• src/hg/lib/tests/expected/mallocTopPadTest
  • lines changed 4, context: html, text, full: html, text
    a5d42e58d7f8ce1e985cf7422145619122ca5d25 Sat Sep 19 18:16:16 2026 -0700
    mallocTopPadTester: check the heap step setting still reaches the library cfgSetMallocTopPad() asks glibc to grow the heap in steps of mallocTopPad bytes instead of its 128 kB default, which saves a heavy hgTracks render more than a hundred thousand system calls that draw nothing. The failure to catch is backsliding: the mallopt call is dropped in a later edit, or the setting is renamed, or it stops being read before the first allocation. Every page still draws correctly and the only symptom is that renders are slower than they used to be, which nobody attributes to this. So the test measures the step, not the clock. A timing test here would be red on a busy hgwdev, green on an idle one, and deleted within a month. sbrk(0) says where the heap ends and the distance it jumps when malloc extends it is what M_TOP_PAD sets, so the answer is the same on a loaded machine as an idle one. It is read in buckets, since glibc may round and may change what it rounds to; what must not change is the order of magnitude between a configured step and the default. Allocations stay under the mmap threshold, or the break never moves and the test measures nothing while appearing to pass. Two runs, with the setting and without, because the answer is the difference between them: a single run would pass on a library whose own default happened to be large. Watched to fail and then pass: leaving the knob read but never applied gives the default step under both confs, which is the one line the diff shows. Recorded as sandbox-ab in utils/testRegistry. refs #38225, refs #38391
• src/hg/lib/tests/expected/quickLiftTest
  • lines changed 49, context: html, text, full: html, text
    57f2a4a958e5a6432829e33deaaf7b0475971188 Thu Sep 17 09:52:34 2026 -0700
    quickLift: keep the target strand a reverse complemented protein just got quickLiftPslBackToProtein turns the alignment over when pslTransMap hands back strand[0] == '-', because a protein psl is only ever "++" or "+-". pslRc does that and makes the target strand explicit as it goes, so the psl comes out "+-". The assignment at the end of the function then set strand[1] to '+' regardless and undid it, leaving "++" over blocks that are in minus strand target coordinates. pslIsProtein wants strand[1] plus the three to one relation between the last block and the target end, so it answered no. pslTrack.c then passed a size multiplier of one to lfFromPslx and every block was drawn at a third of its length at a mirrored position, which is the symptom the reverse complement was added to prevent. The assignment is now the else branch of the same test. hg/lib/tests/quickLiftTester.c covers this. It goes through the public quickLiftPsl rather than the static function, builds its chains in memory in the shape quickLiftSourceRanges leaves them, and needs no database. Six cases: a protein over a same strand chain and over an opposite strand chain, a chain that gaps only the reference, a chain that drops a source base and so splits a codon, an mRNA over both chains, and an alignment with no chain under it. Each one runs pslCheck2, which is what notices a strand that disagrees with the blocks. With this fix backed out, only the opposite strand case changes and it reports three errors placing the blocks outside the target range. Found in the v504 code review, refs #38349. refs #38249
• src/hg/lib/tests/expected/sessionDataTest
  • lines changed 4, context: html, text, full: html, text
    51c0d44065340fbacb9f52bacc3b35e0a1456811 Wed Sep 16 09:06:46 2026 -0700
    sessionData: a test for the memory contract of sessionDataSaveTrashFile 133df533c4d made sessionDataSaveTrashFile return kent-allocated memory, since all three places that release its result do so through kent's own handler stack and it was handing back a pointer from the system malloc. Nothing failed at the time because the only program that reaches those release sites is hgSession, which installs no memory handler. That is the whole guard, and it was written down nowhere. sessionDataTester installs the careful handler itself and runs the function under it: it resolves a relative trash symlink, releases the result with freeMem, allocates again and checks the heap, then compares the allocated block count before and after so a leaked link target is caught too. The absolute symlink and the expired-file cases run alongside. Reverting either half of 133df533c4d fails the test; the plain-file branch that calls moveAndLink is not covered, since isTrashPath rejects a synthetic path with no trash dir configured. No database and no trash directory are needed, so the target runs ahead of spDbTest and hdbTest in hg/lib/tests. refs #38318
• src/hg/lib/tests/expected/sessionDirTest
  • lines changed 19, context: html, text, full: html, text
    751a1f9fff1d6d77c2da690d1369dca64ba04fee Sat Sep 19 18:31:42 2026 -0700
    sessionDirTester: pin how a saved session's data directory is named A saved session's custom tracks and region files are moved to a durable directory named from the user name and the session name. #10138 widened the session half of that name from 8 hex characters to 10: 8 characters of md5 is 4 billion values, and the birthday arithmetic over hundreds of thousands of sessions is not comfortable, since a collision puts one user's files in another user's session. Widening a name that is already on disk is the risky half. Directories written before the change carry the old width and their files are still in use, so the cleanup code has to be able to name both, which is why the width is a parameter rather than a constant in the middle of the function. The property pinned here is not the hash, it is that the short name is a PREFIX of the long one. That is what lets code holding the new name find a directory written under the old one, and it holds only because both come from the same md5 truncated to different lengths. Also pinned: the two-character spreading directory, the user name appearing as given, and the three calls that must abort rather than invent a path -- a relative sessionDataDir, and a width of 0 or 33. None of this is visible. A session whose directory is named differently does not report an error, it comes back without its custom track. Watched to fail and then pass: narrowing the width back to 8 turns it red on the width line. Recorded as sandbox-ab in utils/testRegistry. refs #10138, refs #38391
• src/hg/lib/tests/expected/snapshotTypeTest
  • lines changed 23, context: html, text, full: html, text
    89f9ce7bc7c9333c45190074088f5bee377ca1d7 Sat Sep 19 18:30:26 2026 -0700
    snapshotTypeTester: check the fast snapshotType reader against raFromString A saved session is a snapshot when its settings carry a snapshotType line, and the My Sessions listings hide those rows. #38313 replaced raFromString with a walk over the lines, because the question is asked once per row in a listing and the hash costs about 600ns and three allocations per session against about 30ns and none. A hand written parser that has to agree with a general one is the shape of bug found a year later, so this is a differential test: every case is read both ways and the two answers are printed side by side. The claim in the code is "same line semantics as raFromString", and that claim is what is checked rather than a list of answers written down once. Reading the tag wrongly is invisible either way: a snapshot is listed as an ordinary session, or an ordinary session disappears from the listing, and both look like something the user did. One case does not agree, and it is recorded rather than hidden. Given two snapshotType lines the walk returns the first and raFromString returns the last, because a hash overwrites. Real settings carry the tag once, so nothing depends on it today, but it is a difference in the semantics the code says it copies. It is marked as a known difference, and the test also fails if it ever starts agreeing, so the note cannot go stale in the other direction. Watched to fail and then pass: dropping the delimiter test after the tag makes "snapshotTypeExtra view" read as the type "Extra view" while raFromString finds none. Recorded as sandbox-ab in utils/testRegistry. refs #38313, refs #38391
• src/hg/lib/tests/expected/trashDirTest
  • lines changed 105, context: html, text, full: html, text
    859bc749b0593a83e1c8cd7149f956b620a24a73 Sat Sep 19 18:14:15 2026 -0700
    trashDirTester: pin which file paths the cart accepts hg/lib/cart.c screens every cart variable that names a server-made file through these functions, so they decide whether a saved session still works. Both ways of being wrong are invisible: a path wrongly refused brings the session back without its custom track or its region list and says nothing, and a path wrongly accepted says nothing either. That is what #38303 cost. A check shipped in v503 discarded the saved region list from 583 sessions on the RR and 66 on euro, and hgwdev, code review and hgwbeta were all clean, because the corpus that shows it is only on the production central. Four rules pinned, each of which cost a bug or a review round: only the configured directory is symlink-resolved and never the path from the cart; the resolved spelling of that directory is accepted as well as the configured one, since /userdata on the RR is a symlink and saved sessions hold both spellings; only an absolute directory is resolved, because a relative one would be resolved against the caller's working directory; and the acceptance runs one direction only. pathIsUnderDir's own edges are here too, including the sibling directory that merely starts the same way and the name that begins with "..". The fixture is a directory, a symlink to it and three confs naming the same place three ways, since hgConfig caches what it read and one process can only answer for one spelling. Nothing machine-specific reaches the output. Watched to fail and then pass: with the pre-#38303 code, which did not resolve at all, the resolved spelling comes back refused and the diff is that one line. Recorded as sandbox-ab in utils/testRegistry. refs #38303, refs #37623, refs #38391
• src/hg/lib/tests/genarkLiftOverTester.c
  • lines changed 83, context: html, text, full: html, text
    d58bf9a182cbf4dba142a7028742abf688638829 Sat Sep 19 18:27:02 2026 -0700
    genarkLiftOverTester: pin the escaping of the GenArk accession list Before #38328 the caller pasted its accessions into one string and genarkLiftOverDbs dropped that string into a query marked NOSQLINJ, which is a promise that somebody upstream had made it safe. It now takes an slName list and builds the query itself with sqlDyStringPrintf, so the escaping happens where the values are, and a name that is not a GC accession never reaches the database. Nothing about this is visible. Correct and incorrect escaping give the same page for every input a browser sends, because the accessions come from our own chain files. The difference appears only for a value chosen to break out, which is the value that never turns up in ordinary testing. The genark table is data and it moves, so the test reads an accession out of it and asks whether the function returns that one, rather than naming an assembly that may be dropped later. Watched to fail and then pass: with the value pasted in unescaped the run dies instead of coming back with nothing, so the quoted cases turn the test red rather than quietly querying something else. Recorded as sandbox-ab in utils/testRegistry. refs #38328, refs #38391
• src/hg/lib/tests/geoMirrorSelfTester.c
  • lines changed 101, context: html, text, full: html, text
    36977d0e699d3c853840f08cec63ab2a2dd5eaa7 Sat Sep 19 18:34:00 2026 -0700
    geoMirrorSelfTester: pin which gbNode row a server takes for itself The browser runs on several machines and each offers the others in a menu, so it has to know which gbNode row is itself. It used to answer from browser.node in hg.conf alone, so a machine serving a node other than the one browser.node names -- a sandbox, or two nodes behind one apache -- took itself for its own peer and offered the visitor a link to the site they were already on. #27988 made the host the visitor typed decide whenever that host is one of the nodes, with browser.node as the fallback. The symptom is invisible in the way that matters: every page renders, and a menu entry pointing back at itself reads as a mirror being down rather than as a bug. No domain is named here. gbNode is configuration data whose rows change, so the test reads two rows, points browser.node at the first, and asks whether the second can claim the server. What is printed is which of the two answered, not what they are called, and the port case is included because HTTP_HOST carries one whenever it is not 80 or 443. Also recorded in utils/testRegistry: what blocks a unit test on the fifteen v504 tickets still waiting for one, each with its own reason rather than a category, since the reason decides who can unblock it. Watched to fail and then pass: with the host test removed the server claims no node at all and offers all three, itself included. Recorded as sandbox-ab. refs #27988, refs #38391
• src/hg/lib/tests/hVarSubstHtmlTester.c
  • lines changed 138, context: html, text, full: html, text
    d898b4820087cccd79e240153bc310856bb89310 Sat Sep 19 18:22:49 2026 -0700
    hVarSubstHtmlTester: pin which variables a description page may use A native description page has been through hgTrackDb already, which resolved everything it could and deferred only ${hgsid}, so the render pass acts on that one variable and only in braces: hgTrackDb collapses an escaped $$hgsid to a literal $hgsid, and acting on the bare form here would expand the very thing the author escaped. A hub's page has never been substituted, so the whole hub list is resolved at render time, and $hgsid is deliberately not on that list. A hub page is only lightly sanitized -- an <img> with an http src survives -- so a page carrying <img src="https://example.com/px?s=${hgsid}"> would hand the reader's session id to the hub's own server, and a session id alone is enough to read and write that cart. The page renders the same either way and the request goes to somebody else's host, so nothing about this is visible here. The test builds a cart by hand rather than opening one. That is not a shortcut, it is the point: a cartless version of this test was written first, and adding hgsid back to hubHtmlVars changed not one line of its output, because the check that recognises a variable also asks whether this call can resolve it, and with no cart it cannot. cartSessionId reads two fields, so a struct built in the test is enough and no database is involved. Watched to fail and then pass: with hgsid back on the hub list, both hub cases come back with the session id substituted into the img src. Recorded as sandbox-ab in utils/testRegistry. refs #38283, refs #38391
  • lines changed 73, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/lib/tests/hubSpaceKeysTester.c
  • lines changed 122, context: html, text, full: html, text
    7fc3b3ce76531995f0978618fbb5c5e1fccf31cf Sun Sep 20 06:58:05 2026 -0700
    hgcentralregress: a central database that tests may write to Every hgcentral a developer can reach is shared. hgcentraltest holds thousands of rows of other people's sandbox sessions, and the production centrals are not reachable from hgwdev at all, so a test that needs to create a session, a login or an api key has had nowhere to put it. Every test written for #38391 so far only reads, and that is the limit this lifts. makeHgCentralRegress.sh creates the database and its tables, copying each schema from the central the caller's hg.conf already names, so a column added to namedSessionDb reaches it on the next run and there is no second copy of the schema in the tree to forget. It is idempotent and meant to run before a test. ONE GRANT IS STILL MISSING and no test uses the database yet. The account the browser uses for the central has global SELECT, INSERT, CREATE and DROP, but UPDATE and DELETE only on databases it has been granted them on individually. So a test can create rows here and cannot take them down: the delete comes back 1142. The script says what to ask for. Measured rather than assumed, and one guess along the way was wrong: the hgcentraltest prefix carries no wildcard grant, hgcentraltestregress behaves the same as any other new name. hubSpaceKeysTester is written against it and is deliberately NOT in the test target, so nothing here goes red while it waits. It covers the rules an api key lives by -- one key per user, a new key revokes the old, an adopted key from a peer mirror replaces both, a revoked key names nobody -- and the signature a peer checks before accepting a sync. refs #38323. refs #38391
  • lines changed 69, context: html, text, full: html, text
    33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
    hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323 Six things, all in the syncHubApiKeys path, which is still off everywhere. The sync never reached its handler on a real mirror. It was registered as a cartJson command, so building the cart ran forceUserIdOrCaptcha() first; a peer's request carries no hguid cookie and no apiKey= variable, so any site with cloudFlareSiteKey set -- which is every site that needs api keys in the first place -- answered it with a captcha page. Only hgwdev, where the captcha is commented out, ever ran the handler, which is why this was not caught earlier. doApiKeySyncIfRequested() now answers the request in main() before any cart exists. That also stops each sync leaving a junk userDb and sessionDb row behind: measured, 15 syncs now create none. The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key), so it does not rest on md5 resisting length extension, and it is compared in constant time. It now covers a timestamp too, and a peer refuses anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of now, so a captured sync cannot be replayed later to reinstate a key its owner has since revoked. The fields are joined with newlines and a newline in a userName or apiKey is refused, so one signature cannot cover two different splits of the same string. syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It runs after the local key is already written and in the response, so an errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's outer catch, which threw the response away -- the user saw a failure over a key that was sitting in the database. geoMirrorNotifyOtherNodes() returns each peer's response instead of discarding it, and uses netUrlMustOpenPastHeader, which errAborts on anything but a 200 and hands back the body alone. A peer answering 'bad signature' or 'not enabled on this site' is no longer indistinguishable from success; it is logged to error_log, not warn()ed at the person who clicked the button. Verified against the built CGI run as Apache runs it, with the captcha turned on: a signed sync is accepted and writes the row, a signed revoke removes it, and a replay, a swapped userName, a forged signature and malformed json are all refused. hubSpaceKeysTester covers the signature rules; it still cannot run its database half until hgcentralregress gets the UPDATE and DELETE grant.
• src/hg/lib/tests/input/hgvs/validTerms.txt
  • lines changed 12, context: html, text, full: html, text
    4bf24023fb4ce3e83104c97043c7e898101be1d1 Tue Sep 15 05:48:15 2026 -0700
    hgvs: a bare codon number must not leave a tail behind, refs #38353 The three patterns that read a bare codon number or range were anchored at the start only, so "KAT6A 495-", "KAT6A p.495_", "KAT6A 495--533" and "KAT6A p.495_533_600" all quietly became codon 495 (or 495_533) with the rest thrown away. Silently dropping the tail of a position is exactly what this grammar was added to stop doing. Anchored at the end as well. Every other pattern here is free to leave a tail for a later stage to interpret, but a bare codon number has nothing after it to interpret. The tests now pin the four rejected forms, and also codon 0, a codon past the end of the protein, and a backwards range, none of which were pinned before.
  • lines changed 8, context: html, text, full: html, text
    814335cde61f999ae23626093306bdb6bf394f5e Sun Sep 20 07:00:39 2026 -0700
    hgvs: cover a deprecated protein accession, and say what is not covered A protein accession that is no longer current resolves through ncbiRefSeqLinkHistorical, which is what lets a term from an old paper still find its position. Three such terms are added to the valid-terms list: NP_000054.2, which has three deprecated transcript versions behind it, and NP_055615.8, which has versions .8, .9 and .10. The other half of #38248 is NOT pinned, and the registry row says so rather than implying otherwise. That half picks the newest deprecated transcript by sorting on length first, so that .9 does not beat .10 as a string. I could not build a term that shows it: with the order by removed entirely, and again with a plain string sort, every term gives identical coordinates. The reason is in the data -- for the accessions with both a .9 and a .10, the versions differ only in where the UTR ends, so cdsStart, cdsEnd and exonCount are the same and a coding residue lands in the same place whichever version is chosen. Recorded as assertion-only, the honest level, with what was tried written down so the next person does not repeat it. refs #38248, refs #38391
• src/hg/lib/tests/makeHgCentralRegress.sh
  • lines changed 96, context: html, text, full: html, text
    7fc3b3ce76531995f0978618fbb5c5e1fccf31cf Sun Sep 20 06:58:05 2026 -0700
    hgcentralregress: a central database that tests may write to Every hgcentral a developer can reach is shared. hgcentraltest holds thousands of rows of other people's sandbox sessions, and the production centrals are not reachable from hgwdev at all, so a test that needs to create a session, a login or an api key has had nowhere to put it. Every test written for #38391 so far only reads, and that is the limit this lifts. makeHgCentralRegress.sh creates the database and its tables, copying each schema from the central the caller's hg.conf already names, so a column added to namedSessionDb reaches it on the next run and there is no second copy of the schema in the tree to forget. It is idempotent and meant to run before a test. ONE GRANT IS STILL MISSING and no test uses the database yet. The account the browser uses for the central has global SELECT, INSERT, CREATE and DROP, but UPDATE and DELETE only on databases it has been granted them on individually. So a test can create rows here and cannot take them down: the delete comes back 1142. The script says what to ask for. Measured rather than assumed, and one guess along the way was wrong: the hgcentraltest prefix carries no wildcard grant, hgcentraltestregress behaves the same as any other new name. hubSpaceKeysTester is written against it and is deliberately NOT in the test target, so nothing here goes red while it waits. It covers the rules an api key lives by -- one key per user, a new key revokes the old, an adopted key from a peer mirror replaces both, a revoked key names nobody -- and the signature a peer checks before accepting a sync. refs #38323. refs #38391
• src/hg/lib/tests/makefile
  • lines changed 6, context: html, text, full: html, text
    51c0d44065340fbacb9f52bacc3b35e0a1456811 Wed Sep 16 09:06:46 2026 -0700
    sessionData: a test for the memory contract of sessionDataSaveTrashFile 133df533c4d made sessionDataSaveTrashFile return kent-allocated memory, since all three places that release its result do so through kent's own handler stack and it was handing back a pointer from the system malloc. Nothing failed at the time because the only program that reaches those release sites is hgSession, which installs no memory handler. That is the whole guard, and it was written down nowhere. sessionDataTester installs the careful handler itself and runs the function under it: it resolves a relative trash symlink, releases the result with freeMem, allocates again and checks the heap, then compares the allocated block count before and after so a leaked link target is caught too. The absolute symlink and the expired-file cases run alongside. Reverting either half of 133df533c4d fails the test; the plain-file branch that calls moveAndLink is not covered, since isTrashPath rejects a synthetic path with no trash dir configured. No database and no trash directory are needed, so the target runs ahead of spDbTest and hdbTest in hg/lib/tests. refs #38318
  • lines changed 6, context: html, text, full: html, text
    57f2a4a958e5a6432829e33deaaf7b0475971188 Thu Sep 17 09:52:34 2026 -0700
    quickLift: keep the target strand a reverse complemented protein just got quickLiftPslBackToProtein turns the alignment over when pslTransMap hands back strand[0] == '-', because a protein psl is only ever "++" or "+-". pslRc does that and makes the target strand explicit as it goes, so the psl comes out "+-". The assignment at the end of the function then set strand[1] to '+' regardless and undid it, leaving "++" over blocks that are in minus strand target coordinates. pslIsProtein wants strand[1] plus the three to one relation between the last block and the target end, so it answered no. pslTrack.c then passed a size multiplier of one to lfFromPslx and every block was drawn at a third of its length at a mirrored position, which is the symptom the reverse complement was added to prevent. The assignment is now the else branch of the same test. hg/lib/tests/quickLiftTester.c covers this. It goes through the public quickLiftPsl rather than the static function, builds its chains in memory in the shape quickLiftSourceRanges leaves them, and needs no database. Six cases: a protein over a same strand chain and over an opposite strand chain, a chain that gaps only the reference, a chain that drops a source base and so splits a codon, an mRNA over both chains, and an alignment with no chain under it. Each one runs pslCheck2, which is what notices a strand that disagrees with the blocks. With this fix backed out, only the opposite strand case changes and it reports three errors placing the blocks outside the target range. Found in the v504 code review, refs #38349. refs #38249
  • lines changed 30, context: html, text, full: html, text
    859bc749b0593a83e1c8cd7149f956b620a24a73 Sat Sep 19 18:14:15 2026 -0700
    trashDirTester: pin which file paths the cart accepts hg/lib/cart.c screens every cart variable that names a server-made file through these functions, so they decide whether a saved session still works. Both ways of being wrong are invisible: a path wrongly refused brings the session back without its custom track or its region list and says nothing, and a path wrongly accepted says nothing either. That is what #38303 cost. A check shipped in v503 discarded the saved region list from 583 sessions on the RR and 66 on euro, and hgwdev, code review and hgwbeta were all clean, because the corpus that shows it is only on the production central. Four rules pinned, each of which cost a bug or a review round: only the configured directory is symlink-resolved and never the path from the cart; the resolved spelling of that directory is accepted as well as the configured one, since /userdata on the RR is a symlink and saved sessions hold both spellings; only an absolute directory is resolved, because a relative one would be resolved against the caller's working directory; and the acceptance runs one direction only. pathIsUnderDir's own edges are here too, including the sibling directory that merely starts the same way and the name that begins with "..". The fixture is a directory, a symlink to it and three confs naming the same place three ways, since hgConfig caches what it read and one process can only answer for one spelling. Nothing machine-specific reaches the output. Watched to fail and then pass: with the pre-#38303 code, which did not resolve at all, the resolved spelling comes back refused and the diff is that one line. Recorded as sandbox-ab in utils/testRegistry. refs #38303, refs #37623, refs #38391
  • lines changed 12, context: html, text, full: html, text
    a5d42e58d7f8ce1e985cf7422145619122ca5d25 Sat Sep 19 18:16:16 2026 -0700
    mallocTopPadTester: check the heap step setting still reaches the library cfgSetMallocTopPad() asks glibc to grow the heap in steps of mallocTopPad bytes instead of its 128 kB default, which saves a heavy hgTracks render more than a hundred thousand system calls that draw nothing. The failure to catch is backsliding: the mallopt call is dropped in a later edit, or the setting is renamed, or it stops being read before the first allocation. Every page still draws correctly and the only symptom is that renders are slower than they used to be, which nobody attributes to this. So the test measures the step, not the clock. A timing test here would be red on a busy hgwdev, green on an idle one, and deleted within a month. sbrk(0) says where the heap ends and the distance it jumps when malloc extends it is what M_TOP_PAD sets, so the answer is the same on a loaded machine as an idle one. It is read in buckets, since glibc may round and may change what it rounds to; what must not change is the order of magnitude between a configured step and the default. Allocations stay under the mmap threshold, or the break never moves and the test measures nothing while appearing to pass. Two runs, with the setting and without, because the answer is the difference between them: a single run would pass on a library whose own default happened to be large. Watched to fail and then pass: leaving the knob read but never applied gives the default step under both confs, which is the one line the diff shows. Recorded as sandbox-ab in utils/testRegistry. refs #38225, refs #38391
  • lines changed 14, context: html, text, full: html, text
    a0a3411bca2fdd6d704ce2be7c9c8fca7682913a Sat Sep 19 18:19:36 2026 -0700
    bedItemRgbTester: pin the order of the itemRgb and color tests bedItemRgb() decides whether a BED track draws its items in the colors its file carries or in the one color its stanza names. The rule has four steps, and the order of the first three is the whole of #36212: an explicit "itemRgb off" wins, then an explicit "itemRgb on" wins, and only then does the presence of a "color" setting turn item colors off by default. 88d620e6c82 folded those tests together, so a stanza saying both "itemRgb on" and "color" -- which means items from the file and labels from color -- lost its item colors. The pairs are what matter here. Every single-setting case passed while the bug was live, so a test that exercised one setting at a time would have proved nothing. A child that says "itemRgb on" under a parent that says "color" is the same case reached through trackDbSettingClosestToHome. Two runs, with and without hg.conf's alwaysItemRgb, since the last step of the rule reads it and a mirror that turns it off must still honour a stanza that asks for item colors explicitly. The test lives in hg/lib/tests because hg/cgilib has no tests directory; its link line adds jkhgapcgi.a. Watched to fail and then pass: with the color test moved back in front of the itemRgb test, three lines flip, and they are the three that pair the two settings. Recorded as sandbox-ab in utils/testRegistry. refs #36212, refs #38391
  • lines changed 7, context: html, text, full: html, text
    d898b4820087cccd79e240153bc310856bb89310 Sat Sep 19 18:22:49 2026 -0700
    hVarSubstHtmlTester: pin which variables a description page may use A native description page has been through hgTrackDb already, which resolved everything it could and deferred only ${hgsid}, so the render pass acts on that one variable and only in braces: hgTrackDb collapses an escaped $$hgsid to a literal $hgsid, and acting on the bare form here would expand the very thing the author escaped. A hub's page has never been substituted, so the whole hub list is resolved at render time, and $hgsid is deliberately not on that list. A hub page is only lightly sanitized -- an <img> with an http src survives -- so a page carrying <img src="https://example.com/px?s=${hgsid}"> would hand the reader's session id to the hub's own server, and a session id alone is enough to read and write that cart. The page renders the same either way and the request goes to somebody else's host, so nothing about this is visible here. The test builds a cart by hand rather than opening one. That is not a shortcut, it is the point: a cartless version of this test was written first, and adding hgsid back to hubHtmlVars changed not one line of its output, because the check that recognises a variable also asks whether this call can resolve it, and with no cart it cannot. cartSessionId reads two fields, so a struct built in the test is enough and no database is involved. Watched to fail and then pass: with hgsid back on the hub list, both hub cases come back with the session id substituted into the img src. Recorded as sandbox-ab in utils/testRegistry. refs #38283, refs #38391
  • lines changed 6, context: html, text, full: html, text
    471b82ec87d4b23b03ec3f3a2a193eb1f88a035c Sat Sep 19 18:24:53 2026 -0700
    dataVersionPathTester: pin which local files a hub's dataVersion may read A track's dataVersion setting is usually a version string, but for otto tracks it is the path of a local file that hgTrackUi opens and prints. On a hub track that setting is an instruction from a stranger to read a named file on our server and put it on the page. #38268 narrowed it to /gbdb, which is public data mirrored on hgdownload, and to a plain path, since a ".." component makes the name mean something outside that tree. The exception is needed: curated-hub assemblies are served as hubs, so hs1's otto tracks are hub tracks, and a quickLifted track's version file lives on the source assembly. Nothing about the failure is visible in the usual way. A refused path and an accepted one both leave a page that looks reasonable, and the difference shows only as somebody else's file appearing where a version number belongs. The test prints a classification and never a file. The three outcomes separate without looking at any contents: a refused path comes back as the path itself, an accepted path that does not exist comes back NULL, and an accepted path that exists comes back as something else. Watched to fail and then pass: with the /gbdb test dropped, /etc/passwd comes back read rather than refused, and the two climbing cases come back opened. Recorded as sandbox-ab in utils/testRegistry. refs #38268, refs #38391
  • lines changed 6, context: html, text, full: html, text
    d58bf9a182cbf4dba142a7028742abf688638829 Sat Sep 19 18:27:02 2026 -0700
    genarkLiftOverTester: pin the escaping of the GenArk accession list Before #38328 the caller pasted its accessions into one string and genarkLiftOverDbs dropped that string into a query marked NOSQLINJ, which is a promise that somebody upstream had made it safe. It now takes an slName list and builds the query itself with sqlDyStringPrintf, so the escaping happens where the values are, and a name that is not a GC accession never reaches the database. Nothing about this is visible. Correct and incorrect escaping give the same page for every input a browser sends, because the accessions come from our own chain files. The difference appears only for a value chosen to break out, which is the value that never turns up in ordinary testing. The genark table is data and it moves, so the test reads an accession out of it and asks whether the function returns that one, rather than naming an assembly that may be dropped later. Watched to fail and then pass: with the value pasted in unescaped the run dies instead of coming back with nothing, so the quoted cases turn the test red rather than quietly querying something else. Recorded as sandbox-ab in utils/testRegistry. refs #38328, refs #38391
  • lines changed 7, context: html, text, full: html, text
    89f9ce7bc7c9333c45190074088f5bee377ca1d7 Sat Sep 19 18:30:26 2026 -0700
    snapshotTypeTester: check the fast snapshotType reader against raFromString A saved session is a snapshot when its settings carry a snapshotType line, and the My Sessions listings hide those rows. #38313 replaced raFromString with a walk over the lines, because the question is asked once per row in a listing and the hash costs about 600ns and three allocations per session against about 30ns and none. A hand written parser that has to agree with a general one is the shape of bug found a year later, so this is a differential test: every case is read both ways and the two answers are printed side by side. The claim in the code is "same line semantics as raFromString", and that claim is what is checked rather than a list of answers written down once. Reading the tag wrongly is invisible either way: a snapshot is listed as an ordinary session, or an ordinary session disappears from the listing, and both look like something the user did. One case does not agree, and it is recorded rather than hidden. Given two snapshotType lines the walk returns the first and raFromString returns the last, because a hash overwrites. Real settings carry the tag once, so nothing depends on it today, but it is a difference in the semantics the code says it copies. It is marked as a known difference, and the test also fails if it ever starts agreeing, so the note cannot go stale in the other direction. Watched to fail and then pass: dropping the delimiter test after the tag makes "snapshotTypeExtra view" read as the type "Extra view" while raFromString finds none. Recorded as sandbox-ab in utils/testRegistry. refs #38313, refs #38391
  • lines changed 6, context: html, text, full: html, text
    751a1f9fff1d6d77c2da690d1369dca64ba04fee Sat Sep 19 18:31:42 2026 -0700
    sessionDirTester: pin how a saved session's data directory is named A saved session's custom tracks and region files are moved to a durable directory named from the user name and the session name. #10138 widened the session half of that name from 8 hex characters to 10: 8 characters of md5 is 4 billion values, and the birthday arithmetic over hundreds of thousands of sessions is not comfortable, since a collision puts one user's files in another user's session. Widening a name that is already on disk is the risky half. Directories written before the change carry the old width and their files are still in use, so the cleanup code has to be able to name both, which is why the width is a parameter rather than a constant in the middle of the function. The property pinned here is not the hash, it is that the short name is a PREFIX of the long one. That is what lets code holding the new name find a directory written under the old one, and it holds only because both come from the same md5 truncated to different lengths. Also pinned: the two-character spreading directory, the user name appearing as given, and the three calls that must abort rather than invent a path -- a relative sessionDataDir, and a width of 0 or 33. None of this is visible. A session whose directory is named differently does not report an error, it comes back without its custom track. Watched to fail and then pass: narrowing the width back to 8 turns it red on the width line. Recorded as sandbox-ab in utils/testRegistry. refs #10138, refs #38391
  • lines changed 10, context: html, text, full: html, text
    36977d0e699d3c853840f08cec63ab2a2dd5eaa7 Sat Sep 19 18:34:00 2026 -0700
    geoMirrorSelfTester: pin which gbNode row a server takes for itself The browser runs on several machines and each offers the others in a menu, so it has to know which gbNode row is itself. It used to answer from browser.node in hg.conf alone, so a machine serving a node other than the one browser.node names -- a sandbox, or two nodes behind one apache -- took itself for its own peer and offered the visitor a link to the site they were already on. #27988 made the host the visitor typed decide whenever that host is one of the nodes, with browser.node as the fallback. The symptom is invisible in the way that matters: every page renders, and a menu entry pointing back at itself reads as a mirror being down rather than as a bug. No domain is named here. gbNode is configuration data whose rows change, so the test reads two rows, points browser.node at the first, and asks whether the second can claim the server. What is printed is which of the two answered, not what they are called, and the port case is included because HTTP_HOST carries one whenever it is not 80 or 443. Also recorded in utils/testRegistry: what blocks a unit test on the fifteen v504 tickets still waiting for one, each with its own reason rather than a category, since the reason decides who can unblock it. Watched to fail and then pass: with the host test removed the server claims no node at all and offers all three, itself included. Recorded as sandbox-ab. refs #27988, refs #38391
  • lines changed 14, context: html, text, full: html, text
    c80f2909a9df53021fb01b455b122414ad73c961 Sun Sep 20 06:51:14 2026 -0700
    move bedItemRgbTester to hg/cgilib/tests, beside the code it tests I said in the last commit that hg/cgilib had no tests directory. It does, and I should have looked rather than inferred. bedCart.c lives in hg/cgilib, so its test belongs there, and the header comment saying otherwise is fixed. Worth knowing about that directory: its test target ran nothing. annoGratorTest is commented out because it needs assemblies and tables the directory cannot assume, so `make test` there printed "tested all" and did no work -- and hg/makefile has been running it all along through TEST_EXTRA. bedItemRgbTest is now the one test it does run. refs #36212, refs #38391
  • lines changed 16, context: html, text, full: html, text
    7fc3b3ce76531995f0978618fbb5c5e1fccf31cf Sun Sep 20 06:58:05 2026 -0700
    hgcentralregress: a central database that tests may write to Every hgcentral a developer can reach is shared. hgcentraltest holds thousands of rows of other people's sandbox sessions, and the production centrals are not reachable from hgwdev at all, so a test that needs to create a session, a login or an api key has had nowhere to put it. Every test written for #38391 so far only reads, and that is the limit this lifts. makeHgCentralRegress.sh creates the database and its tables, copying each schema from the central the caller's hg.conf already names, so a column added to namedSessionDb reaches it on the next run and there is no second copy of the schema in the tree to forget. It is idempotent and meant to run before a test. ONE GRANT IS STILL MISSING and no test uses the database yet. The account the browser uses for the central has global SELECT, INSERT, CREATE and DROP, but UPDATE and DELETE only on databases it has been granted them on individually. So a test can create rows here and cannot take them down: the delete comes back 1142. The script says what to ask for. Measured rather than assumed, and one guess along the way was wrong: the hgcentraltest prefix carries no wildcard grant, hgcentraltestregress behaves the same as any other new name. hubSpaceKeysTester is written against it and is deliberately NOT in the test target, so nothing here goes red while it waits. It covers the rules an api key lives by -- one key per user, a new key revokes the old, an adopted key from a peer mirror replaces both, a revoked key names nobody -- and the signature a peer checks before accepting a sync. refs #38323. refs #38391
  • lines changed 9, context: html, text, full: html, text
    0f2f66270dcd4603785850ee72d7d883ff68185b Sun Sep 20 07:03:14 2026 -0700
    asForDbTester: a GenArk accession must not be looked for in MySQL asForDb() may need a database connection to fetch a track's autoSql. The name it is handed is usually an assembly, but on a quickLifted track it is the SOURCE assembly, and for a GenArk that arrives as a bare accession with no MySQL database of that name anywhere. Without #38272's isGenArk test the CGI does not return an empty field list, it dies, and the reader gets an error page instead of a details page -- only for the combination of a quickLift and a GenArk source, which is why it went unnoticed. The last case is the discriminating one: a name that is neither a GenArk accession nor a real database still aborts, which shows the GenArk case is let through deliberately rather than by connection failures having stopped mattering. The abort message carries a MySQL error number and the server's own wording, so what is pinned is whether the abort was about reaching a database of that name, not the text. Two things this test got wrong before it got them right, both recorded in its header. It has to run against a config that sets genarkHubPrefix, because isGenArk answers through that prefix and says no when it is unset, so with a developer's own hg.conf the GenArk case failed for a reason unrelated to the fix. And the accession is read from the genark table at run time, since naming an assembly makes the test go red the day that assembly is dropped. Also fixes the registry row for bedItemRgbTester, which moved to hg/cgilib and whose row still named the old path -- caught by the registry's own check, which is the first time it has failed for a real reason. refs #38272, refs #38391
• src/hg/lib/tests/mallocTopPadTester.c
  • lines changed 84, context: html, text, full: html, text
    a5d42e58d7f8ce1e985cf7422145619122ca5d25 Sat Sep 19 18:16:16 2026 -0700
    mallocTopPadTester: check the heap step setting still reaches the library cfgSetMallocTopPad() asks glibc to grow the heap in steps of mallocTopPad bytes instead of its 128 kB default, which saves a heavy hgTracks render more than a hundred thousand system calls that draw nothing. The failure to catch is backsliding: the mallopt call is dropped in a later edit, or the setting is renamed, or it stops being read before the first allocation. Every page still draws correctly and the only symptom is that renders are slower than they used to be, which nobody attributes to this. So the test measures the step, not the clock. A timing test here would be red on a busy hgwdev, green on an idle one, and deleted within a month. sbrk(0) says where the heap ends and the distance it jumps when malloc extends it is what M_TOP_PAD sets, so the answer is the same on a loaded machine as an idle one. It is read in buckets, since glibc may round and may change what it rounds to; what must not change is the order of magnitude between a configured step and the default. Allocations stay under the mmap threshold, or the break never moves and the test measures nothing while appearing to pass. Two runs, with the setting and without, because the answer is the difference between them: a single run would pass on a library whose own default happened to be large. Watched to fail and then pass: leaving the knob read but never applied gives the default step under both confs, which is the one line the diff shows. Recorded as sandbox-ab in utils/testRegistry. refs #38225, refs #38391
• src/hg/lib/tests/quickLiftTester.c
  • lines changed 223, context: html, text, full: html, text
    57f2a4a958e5a6432829e33deaaf7b0475971188 Thu Sep 17 09:52:34 2026 -0700
    quickLift: keep the target strand a reverse complemented protein just got quickLiftPslBackToProtein turns the alignment over when pslTransMap hands back strand[0] == '-', because a protein psl is only ever "++" or "+-". pslRc does that and makes the target strand explicit as it goes, so the psl comes out "+-". The assignment at the end of the function then set strand[1] to '+' regardless and undid it, leaving "++" over blocks that are in minus strand target coordinates. pslIsProtein wants strand[1] plus the three to one relation between the last block and the target end, so it answered no. pslTrack.c then passed a size multiplier of one to lfFromPslx and every block was drawn at a third of its length at a mirrored position, which is the symptom the reverse complement was added to prevent. The assignment is now the else branch of the same test. hg/lib/tests/quickLiftTester.c covers this. It goes through the public quickLiftPsl rather than the static function, builds its chains in memory in the shape quickLiftSourceRanges leaves them, and needs no database. Six cases: a protein over a same strand chain and over an opposite strand chain, a chain that gaps only the reference, a chain that drops a source base and so splits a codon, an mRNA over both chains, and an alignment with no chain under it. Each one runs pslCheck2, which is what notices a strand that disagrees with the blocks. With this fix backed out, only the opposite strand case changes and it reports three errors placing the blocks outside the target range. Found in the v504 code review, refs #38349. refs #38249
• src/hg/lib/tests/sessionDataTester.c
  • lines changed 140, context: html, text, full: html, text
    51c0d44065340fbacb9f52bacc3b35e0a1456811 Wed Sep 16 09:06:46 2026 -0700
    sessionData: a test for the memory contract of sessionDataSaveTrashFile 133df533c4d made sessionDataSaveTrashFile return kent-allocated memory, since all three places that release its result do so through kent's own handler stack and it was handing back a pointer from the system malloc. Nothing failed at the time because the only program that reaches those release sites is hgSession, which installs no memory handler. That is the whole guard, and it was written down nowhere. sessionDataTester installs the careful handler itself and runs the function under it: it resolves a relative trash symlink, releases the result with freeMem, allocates again and checks the heap, then compares the allocated block count before and after so a leaked link target is caught too. The absolute symlink and the expired-file cases run alongside. Reverting either half of 133df533c4d fails the test; the plain-file branch that calls moveAndLink is not covered, since isTrashPath rejects a synthetic path with no trash dir configured. No database and no trash directory are needed, so the target runs ahead of spDbTest and hdbTest in hg/lib/tests. refs #38318
• src/hg/lib/tests/sessionDirTester.c
  • lines changed 143, context: html, text, full: html, text
    751a1f9fff1d6d77c2da690d1369dca64ba04fee Sat Sep 19 18:31:42 2026 -0700
    sessionDirTester: pin how a saved session's data directory is named A saved session's custom tracks and region files are moved to a durable directory named from the user name and the session name. #10138 widened the session half of that name from 8 hex characters to 10: 8 characters of md5 is 4 billion values, and the birthday arithmetic over hundreds of thousands of sessions is not comfortable, since a collision puts one user's files in another user's session. Widening a name that is already on disk is the risky half. Directories written before the change carry the old width and their files are still in use, so the cleanup code has to be able to name both, which is why the width is a parameter rather than a constant in the middle of the function. The property pinned here is not the hash, it is that the short name is a PREFIX of the long one. That is what lets code holding the new name find a directory written under the old one, and it holds only because both come from the same md5 truncated to different lengths. Also pinned: the two-character spreading directory, the user name appearing as given, and the three calls that must abort rather than invent a path -- a relative sessionDataDir, and a width of 0 or 33. None of this is visible. A session whose directory is named differently does not report an error, it comes back without its custom track. Watched to fail and then pass: narrowing the width back to 8 turns it red on the width line. Recorded as sandbox-ab in utils/testRegistry. refs #10138, refs #38391
• src/hg/lib/tests/snapshotTypeTester.c
  • lines changed 110, context: html, text, full: html, text
    89f9ce7bc7c9333c45190074088f5bee377ca1d7 Sat Sep 19 18:30:26 2026 -0700
    snapshotTypeTester: check the fast snapshotType reader against raFromString A saved session is a snapshot when its settings carry a snapshotType line, and the My Sessions listings hide those rows. #38313 replaced raFromString with a walk over the lines, because the question is asked once per row in a listing and the hash costs about 600ns and three allocations per session against about 30ns and none. A hand written parser that has to agree with a general one is the shape of bug found a year later, so this is a differential test: every case is read both ways and the two answers are printed side by side. The claim in the code is "same line semantics as raFromString", and that claim is what is checked rather than a list of answers written down once. Reading the tag wrongly is invisible either way: a snapshot is listed as an ordinary session, or an ordinary session disappears from the listing, and both look like something the user did. One case does not agree, and it is recorded rather than hidden. Given two snapshotType lines the walk returns the first and raFromString returns the last, because a hash overwrites. Real settings carry the tag once, so nothing depends on it today, but it is a difference in the semantics the code says it copies. It is marked as a known difference, and the test also fails if it ever starts agreeing, so the note cannot go stale in the other direction. Watched to fail and then pass: dropping the delimiter test after the tag makes "snapshotTypeExtra view" read as the type "Extra view" while raFromString finds none. Recorded as sandbox-ab in utils/testRegistry. refs #38313, refs #38391
• src/hg/lib/tests/trashDirTester.c
  • lines changed 144, context: html, text, full: html, text
    859bc749b0593a83e1c8cd7149f956b620a24a73 Sat Sep 19 18:14:15 2026 -0700
    trashDirTester: pin which file paths the cart accepts hg/lib/cart.c screens every cart variable that names a server-made file through these functions, so they decide whether a saved session still works. Both ways of being wrong are invisible: a path wrongly refused brings the session back without its custom track or its region list and says nothing, and a path wrongly accepted says nothing either. That is what #38303 cost. A check shipped in v503 discarded the saved region list from 583 sessions on the RR and 66 on euro, and hgwdev, code review and hgwbeta were all clean, because the corpus that shows it is only on the production central. Four rules pinned, each of which cost a bug or a review round: only the configured directory is symlink-resolved and never the path from the cart; the resolved spelling of that directory is accepted as well as the configured one, since /userdata on the RR is a symlink and saved sessions hold both spellings; only an absolute directory is resolved, because a relative one would be resolved against the caller's working directory; and the acceptance runs one direction only. pathIsUnderDir's own edges are here too, including the sibling directory that merely starts the same way and the name that begins with "..". The fixture is a directory, a symlink to it and three confs naming the same place three ways, since hgConfig caches what it read and one process can only answer for one spelling. Nothing machine-specific reaches the output. Watched to fail and then pass: with the pre-#38303 code, which did not resolve at all, the resolved spelling comes back refused and the diff is that one line. Recorded as sandbox-ab in utils/testRegistry. refs #38303, refs #37623, refs #38391
• src/hg/lib/userdata.c
  • lines changed 14, context: html, text, full: html, text
    f518e03d8f56af784ec14536a7bdc0daed9a38fd Wed Sep 16 11:11:32 2026 -0700
    Send each file's genome with a hubtools upload so hubSpace rows get a db, and stop the server rewriting a user-uploaded hub.txt, no redmine Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 6, context: html, text, full: html, text
    ce2f414131cc49f0cdd0799a8a0d6a3f94f75e9a Thu Sep 17 11:57:42 2026 -0700
    hubspace: double-encode the user name when building hub URLs Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/lib/web.c
  • lines changed 29, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/lib/wikiLink.c
  • lines changed 20, context: html, text, full: html, text
    1c52aaa92828b6ecc18dd5800fe6650b91daf696 Fri Sep 18 07:22:10 2026 -0700
    hgSession: the new Sessions page also lists the sessions saved on our other servers #Preview2 week - bugs introduced now will need a build patch to fix Every geo mirror node keeps its own namedSessionDb, so a session saved on genome-euro is invisible on genome.ucsc.edu and people do not find it again. The table now holds the sessions from every node, with a new Server column saying where each one lives, and the session name links to the server that holds it. Two new hgSession actions, both answered in main() before the cart is built, so that a request from another node leaves no cart, userDb or sessionDb row behind: hgS_doSessionListJson returns this server's sessions for the user named by the login cookie, as JSON. This is what the other nodes call. hgS_doMirrorSessions asks every other gbNode for that list and returns what came back. It sends all of the requests before reading any of the answers, so the wait is the slowest node rather than the sum of them. The fetch goes through our own server rather than from the browser to the other nodes, which keeps cross-origin requests out of it: apache handles the CORS headers differently on every node, and the two mirrors are administered by Bielefeld and RIKEN, while CGI code reaches them with the release. The user's login cookies, and only those, are passed on (wikiLinkLoginCookieHeader), which works because the nodes share a cookie salt. The page never waits for any of this: the table is drawn from this server's own sessions and the rows from the other servers are merged in as they arrive, keeping the current sort, search and page. A node that does not answer, times out, or runs a release without the endpoint is named in a line under the toolbar instead, and costs nothing else. Sessions from another server are shown but not managed here: no Share, Edit, Overwrite or Delete, no bulk-select checkbox, and the Update now shortcut keeps ignoring them. Table rows are keyed by a new uid rather than by the session name, since the same name can now be in the table twice, once per server. geoMirrorThisNode() and geoMirrorOtherNodes() read the node list from the hgcentral gbNode table, so nothing new has to be configured: browser.node already says which node a server is.
• src/hg/makeDb/doc/asmHubs/asmHubs.mk
  • lines changed 1, context: html, text, full: html, text
    92ba8f72d864456a8a026f52bf90b9e0560b5482 Thu Sep 17 09:33:04 2026 -0700
    provide access to the alpha.hub.txt versions for the group index pages on genome-test refs #35415
• src/hg/makeDb/doc/asmHubs/verifyOnDownload.sh
  • lines changed 4, context: html, text, full: html, text
    773127a4d26eba2c59c1321044eb58624921373e Mon Sep 14 13:57:30 2026 -0700
    gcOnFly subtracks one counted track refs #29545
• src/hg/makeDb/doc/birdsAsmHub/birds.orderList.tsv
  • lines changed 7, context: html, text, full: html, text
    a7c29f7591b74b35abfa8e3cafc2ad95fa5efcf2 Mon Sep 14 13:46:37 2026 -0700
    VGP additions refs #29545
  • lines changed 11, context: html, text, full: html, text
    9bc69ab9197db8f1bfaa54682159c39ef366fb06 Wed Sep 16 16:15:22 2026 -0700
    VGP updates continued refs #29545
  • lines changed 2, context: html, text, full: html, text
    0bd10926c28816bcae4e9949cc514899996f2e98 Thu Sep 17 13:09:12 2026 -0700
    corrected marmoset common name and fix errors found by claude in birds.orderList.tsv no redmine
• src/hg/makeDb/doc/contrib/hprc2annot/hprc2annot.txt
  • lines changed 67, context: html, text, full: html, text
    0b293869739f3112a74d2ee38f47785a7c245884 Wed Sep 16 06:29:25 2026 -0700
    HPRC v2 contrib hub: pcLAI defaults to dense, and genark addContrib wires the beta/public hub tiers Mark reported that the pcLAI track shows only one color on HG00408 pat (GCA_041900255.1) and that you cannot see where ancestry changes. The colors are correct: a fresh download of the HPRC source BED matches the built pclai.bb on (chrom, start, end, itemRgb) for all 25,475 windows. That haplotype is simply 99.8% one ancestry cluster, which is common in this set (71 of 460 haplotypes have a single centroid, 228 are >=99% one centroid). The display was the real problem, so hprcPclai goes from visibility pack to visibility dense plus onlyVisibility dense. At chromosome scale pack stacks the 100 kb windows into ~50 rows of 1-2 px slivers; dense draws one colored bar where the ancestry blocks are legible. While deploying that, beta.hub.txt turned out to be a week stale: it still had the pre-2026-09-09 pcaSegment field name and the double-prefixed pclaiRefPanel.json dataUrl, so the details-page scatterplot was not drawing on hgwbeta. genark addContrib only ever wired alpha.hub.txt, leaving beta and public to the next clade build. It now wires those two as well for a collection already named in betaGenArk.txt / publicGenArk.txt. This never promotes anything - the tier argument still owns the release lists - and mkGenomes.pl inlines the same per-assembly trackDb, so a later clade build converges on the same content. Ran over 462 assemblies, alpha and beta; genark checkContrib reports no contrib problems, refs #35415
  • lines changed 33, context: html, text, full: html, text
    b231848705610405fb34e198796e5b48d3c43907 Wed Sep 16 06:49:31 2026 -0700
    hprc2annot: pcLAI is squish, not dense - dense removes the details page Mark asked for onlyVisibility dense on the pcLAI track. Dense is not usable here: in tvDense hgTracks draws one merged row and emits no per-item map boxes, so there is no hgc link and no mouseOver, and the ancestry scatterplot on the details page becomes unreachable. Counted on the image map of a 1.7 Mb view holding 16 windows: pack and squish give 16 hgc links and 16 mouseOvers, dense gives none. Squish gives the same readability without that cost. It keeps every map box and uses half-height rows with no label column, and since the windows tile without overlapping they collapse to one row on chr18 (three or four on chr2) against pack's ~50. On chr18 at 950px: pack 561px, squish 97px, dense 81px. hprcPclai is now visibility squish + onlyVisibility squish, leaving Hide and Squish in the dropdown. Re-ran hprc2annotMakeTrackDb.py and genark addContrib over 462 assemblies, alpha and beta, refs #35415
  • lines changed 7, context: html, text, full: html, text
    f5dc9662ac04c32e82c97b1d0ac4b856beb1938d Wed Sep 16 11:32:25 2026 -0700
    hprc2annot: say what summary.tsv actually retains instead of claiming near-lossless counts for every track, refs #35415 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/makeDb/doc/danRer11/danioCode.txt
  • lines changed 47, context: html, text, full: html, text
    aac682d3e89d0197a6467ae5231d3fe979f06d87 Tue Sep 15 05:30:48 2026 -0700
    danioCode: un-nest the standalone data types, split CRISPR out of DC Conservation Give RNA-seq, CAGE-seq, 3P-seq, ChIP-seq and Hi-C their own top-level track in an existing group (rna/genes/regulation) instead of hiding all eleven DANIO-CODE containers behind one superTrack that only someone already looking for DANIO-CODE would open. The Burgess lab's CRISPR/Cas9 target-site tracks, which were riding along inside the "DC Conservation" composite, get their own top-level superTrack (dcCrispr, group map) instead, mirroring where hg38 keeps its own unrelated crispr tracks, and drop the "DC" branding since they aren't DANIO-CODE's data. The rest -- a mixed bag of regulatory-element/annotation containers that don't map onto any one existing group -- stay nested under the danioCode superTrack. Implemented in danioCodeHubToRa.py (STANDALONE_GROUP/NESTED_ORDER/CRISPR_*) so the regrouping survives the next hub re-import, then regenerated danioCode.ra from the cached hub trackDb.txt. tdbQuery -check passes; still alpha only. refs #38265
  • lines changed 49, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/doc/fishAsmHub/fish.orderList.tsv
  • lines changed 7, context: html, text, full: html, text
    a7c29f7591b74b35abfa8e3cafc2ad95fa5efcf2 Mon Sep 14 13:46:37 2026 -0700
    VGP additions refs #29545
  • lines changed 2, context: html, text, full: html, text
    6c7924c5ec8a6511cbfd6895de5894579b228c4a Wed Sep 16 10:18:08 2026 -0700
    better names on the zebrafish assemblty GRCz12ab refs #38365
  • lines changed 12, context: html, text, full: html, text
    9bc69ab9197db8f1bfaa54682159c39ef366fb06 Wed Sep 16 16:15:22 2026 -0700
    VGP updates continued refs #29545
  • lines changed 4, context: html, text, full: html, text
    52235246a8825164a1d74e753e839d7a392a84bb Sat Sep 19 10:13:31 2026 -0700
    VGP updates refs #29545
• src/hg/makeDb/doc/hg38/episignatures.txt
  • lines changed 171, context: html, text, full: html, text
    8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
    hg38: episignatures container with the MethaDory CpG probes #Preview2 week - bugs introduced now will need a build patch to fix New alpha superTrack for the CpG positions that published DNA methylation episignatures are built from. Its first subtrack, MethaDory, holds 268,900 sites drawn from 189 episignatures for 105 rare developmental disorders in 74 studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given to us for open release. One row per CpG, merging every episignature that reports it, since a site is shared by 3.1 signatures on average and by up to 41. The per-signature values line up as a real table on the details page via detailsDynamicTable; the same values are also kept as plain columns for the Table Browser and for the disorder, gene, study and direction filters. Probe IDs were placed from the Illumina manifests already on the browser, EPIC 850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled sites land on a CG dinucleotide. Colour is the direction and size of the strongest effect at the site, split at a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on purpose: the studies used different reporting cut-offs, so the low end of the distribution reflects what each paper chose to publish rather than biology. refs #38371
  • lines changed 121, context: html, text, full: html, text
    e8144b87c64b0e60e475886e3ced5353f139cf43 Fri Sep 18 07:24:42 2026 -0700
    hg38 episignatures: EpigenCentral CpG probes as a second subtrack #Preview2 week - bugs introduced now will need a build patch to fix The EpigenCentral group at the Centre for Computational Medicine and the Weksberg lab, Hospital for Sick Children, publishes its curated episignatures as a track hub and asked us to host it natively instead. 15,035 CpG sites from 24 episignatures for 23 rare disorders, alongside MethaDory in the same container. The two were compiled independently and neither contains the other: 14,236 sites are in both, at identical coordinates, and 799 are only here. Built from the lab's own bigBed by makeEpigenCentral.sh, so a refresh is one command rather than the hand edits it started as. Every feature in the hub is in the track; no coordinates or values were changed. Four things were rewritten on the way in: the OMIM column went from a full URL to the entry number so trackDb builds the link, the hub's pre-rendered mouse-over column became the direction alone with the text assembled from the fields, the disorder of the strongest signature was added as its own column, and 215 rows of the per-site comparison table on 137 sites were dropped as exact duplicates, which the site's own signature count had already collapsed. Two columns were renamed to what MethaDory calls the same numbers, height to maxAbsDelta and nSignatures to sigCount. Coordinates check out two ways: 5,000 sampled sites all land on a CG dinucleotide, and every probe shared with MethaDory, which was positioned from the Illumina manifests independently, is at the same base in both. The description page keeps the structure and wording written for the hub but not its markup. Its reference table had all 24 links pointing at one PMID while displaying another, so that table is now generated from the data joined to a checked-in PMID list, each one verified against PubMed. tableBrowser is off at the request of the data providers, so Data Access points at EpigenCentral's own portal and repository. refs #38112
  • lines changed 12, context: html, text, full: html, text
    af9b41f1305c39290da8b597f232ac0a19dda0a9 Fri Sep 18 14:53:06 2026 -0700
    Reformatting both episignatures mouseOvers to bold label then value, adding the OMIM number back to the EpigenCentral mouseOver, and raising maxWindowCoverage from 200000 to 10000000 on both subtracks so multi-megabase views draw as items rather than the coverage graph, with the makedoc visibility paragraph updated to match, refs #38112 #38371 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 6, context: html, text, full: html, text
    ef08d55de458d633b75e54be97bf00c6a9377eba Mon Sep 21 05:19:50 2026 -0700
    hg38 episignatures: link PubMed as "Lastname Year" instead of the bare PMID The Studies table on methaDory.html and the locus table on epigenCentral.html linked out to PubMed with the PMID digits as the clickable text, which reads oddly next to a plain-text study/gene identifier. Both now show the first author's last name and the publication year instead, from a new checked-in lookup, scripts/episignatures/pubAuthorYear.tsv, built from PubMed esummary (and Crossref for the one DOI-only reference). makeHtmlTables.py and epigenCentralToBed.py errAbort if an id used by the data has no entry, so a future refresh can't silently regress to showing raw PMIDs again. The lookup's year is PubMed's own citable pubdate, which for a handful of entries differs by one from the StudyID naming used elsewhere on the page (assigned by the source labs, sometimes from an epub-ahead-of-print date); that's expected, not a mismatch to fix. The two hand-written References sections at the bottom of each page keep the usual "PMID: <number>" convention used across every other track's HTML.
• src/hg/makeDb/doc/hg38/fiberSeq.txt
  • lines changed 11, context: html, text, full: html, text
    8ceea19db110700c4b6612ce4f30ce0b08da7d81 Mon Sep 14 07:34:15 2026 -0700
    hg38 Fiber-seq makeDoc: record where the nucleosome hold-back landed, refs #36210 The commit that holds it, 75e82896028, is titled "uniprot otto: the miniprot cluster job needs absolute paths": two sessions were committing in the same working copy and the other one took these files out of the shared index first. It is pushed, so note it in the makeDoc rather than rewrite shared history.
  • lines changed 9, context: html, text, full: html, text
    b1fea807e3a0f6c6b0ebb6b1244b3291aeaaad28 Mon Sep 14 07:51:13 2026 -0700
    hg38 Fiber-seq makeDoc: drop the git attribution note from 8ceea19db11, refs #36210 Which commit carries a change is session state, not something a reader needs in order to rebuild the track. It belongs in the per-ticket notes under /hive/groups/browser/redmineNotes/36210/claude/, where it now is.
  • lines changed 32, context: html, text, full: html, text
    156d289f517d82c4d5a8c59981dfec21a715af8d Mon Sep 21 09:22:45 2026 -0700
    Fiber-seq QA fixes: take sample class from the lab's own table, pin the overlay draw order, and correct the description pages. refs #36210 Sample class had been guessed from the free-text cell type with a two-name exception list, which filed five lymphoblastoid lines as HPRC that are not. It now comes from a fifth column in fiberSeqSamples.tsv carrying the classification the lab supplies, and sampleClass()/NOT_HPRC are gone. Facet counts go from 25/16 to 20/21. The CpG difference track is a solidOverlay whose four files hold the same value at a shared base, so the tier drawn last is the colour the reader sees. Its children inherited a single priority from the parent, trackPriCmp ties on that, and slSort is not stable, so the paint order was arbitrary and p<0.01 was covering p<0.0001. The four levels and the two haplotype children now carry explicit priorities. At chr20:29,300,000-29,305,000 red goes from 2 image columns to 88. CpG haplotype children take their shortLabel prefix from their container, so they read "<sample> CpG Hap1" rather than colliding with the accessibility children's "<sample> Hap1". 82 labels were duplicated. Description pages: fiberSeqAcc.html attached the FIRE score's "fewer than four elements" cutoff to the percent accessible signal, which is a different quantity; two pages overstated how many of the lymphoblastoid lines come from HPRC; both pages told readers to query the API with container track names, which it refuses by design. Also documents GM12878's trio phasing and the FDR ceiling at 100, corrects the difference track's stated range, replaces a non-ASCII author name with numeric entities, and adds db= to hgTrackUi links. makeDoc: corrects a basesCovered figure that was out by a factor of a hundred, refreshes the peak file sizes after the PM00001 reissue, rewrites the sampleClass rationale, and records why multiWig children need their own priority.
• src/hg/makeDb/doc/hg38/hprcPclai.txt
  • lines changed 31, context: html, text, full: html, text
    a3fd570e96fb63b45d583ebcce83d8d23673a30a Wed Sep 16 07:00:11 2026 -0700
    hg38 pcLAI: squish, not dense - dense removes the details page Same problem the GenArk contrib collection had: in tvDense hgTracks draws one merged row per subtrack and emits no per-item map boxes, so there is no hgc link and no mouseOver, and the ancestry scatterplot built for the details page is unreachable. On chr18:10,800,000-12,500,000 the image map has 0 hgc links in dense and 134 in squish. The composite is now visibility squish plus onlyVisibility squish. On a composite that governs the only vis dropdown the track has, since the subtracks get checkboxes rather than dropdowns, and hgTrackUi now offers Hide and Squish. Squish suits the data: the windows tile without overlapping and carry no item labels, so each haplotype stays one row at a 20 Mb view, refs #35415
  • lines changed 40, context: html, text, full: html, text
    36b00fae56bc9de09200fd4fe5cfd07ddb4352d8 Wed Sep 16 12:40:10 2026 -0700
    hprcPclai: back to dense, with denseClick on, refs #35415, refs #38364 This track was moved to squish a week ago and pinned there with onlyVisibility, for one reason: dense drew one merged row per subtrack and emitted no per-item map boxes at all, so there was no hgc link, no mouseOver, and no way to reach the ancestry scatterplot on the details page. denseClick removes that reason. With it, every item on a dense row gets its own map box, so it carries an hgc link and its own mouseOver while still being drawn on the single dense row. Measured on a build of this branch at chr2:60,000,000-80,000,000, the seven default-on haplotypes: dense, denseClick off 0 hgc links dense, denseClick on 1249 hgc links squish 1235 hgc links The rendered image does not change; only the image map does. So dense is now the better mode for this data as well as the clickable one. Dense is always exactly one row per haplotype, where squish spreads to three or four at whole-chromosome zoom and gets noisy. onlyVisibility moves to dense rather than going away: its real job is keeping a reader out of pack, which at 463 subtracks would be enormous. denseClick is inherited, so the single line on the composite covers every subtrack, and hgTracks ignores a setting it does not understand, so the line is inert rather than harmful on an older browser. The same change goes into the GenArk contrib collection stanza. Worth knowing about the timing there: hprc2annot is listed in betaGenArk.txt, and hgwbeta runs the release branch, so that copy will show a dense row with no clickable items until the hgTracks change reaches beta. The hg38 composite is alpha only and hgwdev CGIs are built from master, so it picks the behavior up as soon as this lands. Neither copy is in publicGenArk.txt, so no RR user sees either state. hprcPclai.ra is generated, so the change is in hprcPclaiMakeTrackDb.py and the .ra was rebuilt from the same index CSV; the regenerated file differs only in those three lines.
• src/hg/makeDb/doc/hg38/mavemd.txt
  • lines changed 228, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
  • lines changed 1, context: html, text, full: html, text
    fcf788d7357f6f207f2bb0b61acde9dc0507d89f Mon Sep 21 04:04:10 2026 -0700
    Fix two stale figures and guard the accession interpolation in the MaveMD build, per CR. refs #37800 The makeDoc script listing still said the variant converter writes bed12+31. It writes bed12+34, as the autoSql, mavemd.ra, runBuild.sh and the makeDoc's own build-results line all already said. mavemdLib.py's docstring claimed the codon projection is cross-checked against "~154k variants that carry both a genomic and a protein term". 154,895 is the number placed by the genomic route; the cross-check set is the smaller number that also has a resolvable protein term. The docstring now describes the set rather than quoting a figure that drifts with every build. Also adds checkAccession() and calls it before either query that interpolates an accession into SQL. Nothing can currently reach those queries with a quote in it, since the accessions come from PROTEIN_TERM whose character class excludes one, but the regex is a hundred lines from the query and a later edit to it should not be able to open this up silently. Output is byte-identical to the previous build.
• src/hg/makeDb/doc/hg38/transcriptionStart.txt
  • lines changed 124, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/doc/hs1/transcriptionStart.txt
  • lines changed 42, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/doc/mammalsAsmHub/mammals.orderList.tsv
  • lines changed 19, context: html, text, full: html, text
    a7c29f7591b74b35abfa8e3cafc2ad95fa5efcf2 Mon Sep 14 13:46:37 2026 -0700
    VGP additions refs #29545
  • lines changed 5, context: html, text, full: html, text
    9bc69ab9197db8f1bfaa54682159c39ef366fb06 Wed Sep 16 16:15:22 2026 -0700
    VGP updates continued refs #29545
• src/hg/makeDb/doc/mm10/mouseStrainsCactus.txt
  • lines changed 1, context: html, text, full: html, text
    78ce99c531fbd268466313d5b7ac1c7ba80defe1 Fri Sep 18 17:22:34 2026 -0700
    Updating a claude complaint about the makedoc since we updated the labels, refs #38308
• src/hg/makeDb/doc/primatesAsmHub/primates.orderList.tsv
  • lines changed 3, context: html, text, full: html, text
    0bd10926c28816bcae4e9949cc514899996f2e98 Thu Sep 17 13:09:12 2026 -0700
    corrected marmoset common name and fix errors found by claude in birds.orderList.tsv no redmine
• src/hg/makeDb/doc/vertebrateAsmHub/vertebrate.orderList.tsv
  • lines changed 7, context: html, text, full: html, text
    a7c29f7591b74b35abfa8e3cafc2ad95fa5efcf2 Mon Sep 14 13:46:37 2026 -0700
    VGP additions refs #29545
  • lines changed 5, context: html, text, full: html, text
    9bc69ab9197db8f1bfaa54682159c39ef366fb06 Wed Sep 16 16:15:22 2026 -0700
    VGP updates continued refs #29545
  • lines changed 3, context: html, text, full: html, text
    52235246a8825164a1d74e753e839d7a392a84bb Sat Sep 19 10:13:31 2026 -0700
    VGP updates refs #29545
• src/hg/makeDb/outside/proCapNet/proCapNetDownload
  • lines changed 42, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/outside/proCapNet/proCapNetEncodeBuild
  • lines changed 38, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/outside/proCapNet/proCapNetEncodeMeta
  • lines changed 81, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/outside/proCapNet/proCapNetExperiments.tsv
  • lines changed 7, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/outside/proCapNet/proCapNetMergeSignal
  • lines changed 109, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/outside/proCapNet/proCapNetPredBuild
  • lines changed 29, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/outside/proCapNet/proCapNetPredToFixedStep
  • lines changed 87, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/outside/proCapNet/proCapNetTrackDb
  • lines changed 245, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/scripts/danioCode/danioCodeHubToRa.py
  • lines changed 85, context: html, text, full: html, text
    aac682d3e89d0197a6467ae5231d3fe979f06d87 Tue Sep 15 05:30:48 2026 -0700
    danioCode: un-nest the standalone data types, split CRISPR out of DC Conservation Give RNA-seq, CAGE-seq, 3P-seq, ChIP-seq and Hi-C their own top-level track in an existing group (rna/genes/regulation) instead of hiding all eleven DANIO-CODE containers behind one superTrack that only someone already looking for DANIO-CODE would open. The Burgess lab's CRISPR/Cas9 target-site tracks, which were riding along inside the "DC Conservation" composite, get their own top-level superTrack (dcCrispr, group map) instead, mirroring where hg38 keeps its own unrelated crispr tracks, and drop the "DC" branding since they aren't DANIO-CODE's data. The rest -- a mixed bag of regulatory-element/annotation containers that don't map onto any one existing group -- stay nested under the danioCode superTrack. Implemented in danioCodeHubToRa.py (STANDALONE_GROUP/NESTED_ORDER/CRISPR_*) so the regrouping survives the next hub re-import, then regenerated danioCode.ra from the cached hub trackDb.txt. tdbQuery -check passes; still alpha only. refs #38265
  • lines changed 41, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/scripts/episignatures/epigenCentral.as
  • lines changed 22, context: html, text, full: html, text
    e8144b87c64b0e60e475886e3ced5353f139cf43 Fri Sep 18 07:24:42 2026 -0700
    hg38 episignatures: EpigenCentral CpG probes as a second subtrack #Preview2 week - bugs introduced now will need a build patch to fix The EpigenCentral group at the Centre for Computational Medicine and the Weksberg lab, Hospital for Sick Children, publishes its curated episignatures as a track hub and asked us to host it natively instead. 15,035 CpG sites from 24 episignatures for 23 rare disorders, alongside MethaDory in the same container. The two were compiled independently and neither contains the other: 14,236 sites are in both, at identical coordinates, and 799 are only here. Built from the lab's own bigBed by makeEpigenCentral.sh, so a refresh is one command rather than the hand edits it started as. Every feature in the hub is in the track; no coordinates or values were changed. Four things were rewritten on the way in: the OMIM column went from a full URL to the entry number so trackDb builds the link, the hub's pre-rendered mouse-over column became the direction alone with the text assembled from the fields, the disorder of the strongest signature was added as its own column, and 215 rows of the per-site comparison table on 137 sites were dropped as exact duplicates, which the site's own signature count had already collapsed. Two columns were renamed to what MethaDory calls the same numbers, height to maxAbsDelta and nSignatures to sigCount. Coordinates check out two ways: 5,000 sampled sites all land on a CG dinucleotide, and every probe shared with MethaDory, which was positioned from the Illumina manifests independently, is at the same base in both. The description page keeps the structure and wording written for the hub but not its markup. Its reference table had all 24 links pointing at one PMID while displaying another, so that table is now generated from the data joined to a checked-in PMID list, each one verified against PubMed. tableBrowser is off at the request of the data providers, so Data Access points at EpigenCentral's own portal and repository. refs #38112
• src/hg/makeDb/scripts/episignatures/epigenCentralRefs.tsv
  • lines changed 29, context: html, text, full: html, text
    e8144b87c64b0e60e475886e3ced5353f139cf43 Fri Sep 18 07:24:42 2026 -0700
    hg38 episignatures: EpigenCentral CpG probes as a second subtrack #Preview2 week - bugs introduced now will need a build patch to fix The EpigenCentral group at the Centre for Computational Medicine and the Weksberg lab, Hospital for Sick Children, publishes its curated episignatures as a track hub and asked us to host it natively instead. 15,035 CpG sites from 24 episignatures for 23 rare disorders, alongside MethaDory in the same container. The two were compiled independently and neither contains the other: 14,236 sites are in both, at identical coordinates, and 799 are only here. Built from the lab's own bigBed by makeEpigenCentral.sh, so a refresh is one command rather than the hand edits it started as. Every feature in the hub is in the track; no coordinates or values were changed. Four things were rewritten on the way in: the OMIM column went from a full URL to the entry number so trackDb builds the link, the hub's pre-rendered mouse-over column became the direction alone with the text assembled from the fields, the disorder of the strongest signature was added as its own column, and 215 rows of the per-site comparison table on 137 sites were dropped as exact duplicates, which the site's own signature count had already collapsed. Two columns were renamed to what MethaDory calls the same numbers, height to maxAbsDelta and nSignatures to sigCount. Coordinates check out two ways: 5,000 sampled sites all land on a CG dinucleotide, and every probe shared with MethaDory, which was positioned from the Illumina manifests independently, is at the same base in both. The description page keeps the structure and wording written for the hub but not its markup. Its reference table had all 24 links pointing at one PMID while displaying another, so that table is now generated from the data joined to a checked-in PMID list, each one verified against PubMed. tableBrowser is off at the request of the data providers, so Data Access points at EpigenCentral's own portal and repository. refs #38112
• src/hg/makeDb/scripts/episignatures/epigenCentralToBed.py
  • lines changed 243, context: html, text, full: html, text
    e8144b87c64b0e60e475886e3ced5353f139cf43 Fri Sep 18 07:24:42 2026 -0700
    hg38 episignatures: EpigenCentral CpG probes as a second subtrack #Preview2 week - bugs introduced now will need a build patch to fix The EpigenCentral group at the Centre for Computational Medicine and the Weksberg lab, Hospital for Sick Children, publishes its curated episignatures as a track hub and asked us to host it natively instead. 15,035 CpG sites from 24 episignatures for 23 rare disorders, alongside MethaDory in the same container. The two were compiled independently and neither contains the other: 14,236 sites are in both, at identical coordinates, and 799 are only here. Built from the lab's own bigBed by makeEpigenCentral.sh, so a refresh is one command rather than the hand edits it started as. Every feature in the hub is in the track; no coordinates or values were changed. Four things were rewritten on the way in: the OMIM column went from a full URL to the entry number so trackDb builds the link, the hub's pre-rendered mouse-over column became the direction alone with the text assembled from the fields, the disorder of the strongest signature was added as its own column, and 215 rows of the per-site comparison table on 137 sites were dropped as exact duplicates, which the site's own signature count had already collapsed. Two columns were renamed to what MethaDory calls the same numbers, height to maxAbsDelta and nSignatures to sigCount. Coordinates check out two ways: 5,000 sampled sites all land on a CG dinucleotide, and every probe shared with MethaDory, which was positioned from the Illumina manifests independently, is at the same base in both. The description page keeps the structure and wording written for the hub but not its markup. Its reference table had all 24 links pointing at one PMID while displaying another, so that table is now generated from the data joined to a checked-in PMID list, each one verified against PubMed. tableBrowser is off at the request of the data providers, so Data Access points at EpigenCentral's own portal and repository. refs #38112
  • lines changed 28, context: html, text, full: html, text
    ef08d55de458d633b75e54be97bf00c6a9377eba Mon Sep 21 05:19:50 2026 -0700
    hg38 episignatures: link PubMed as "Lastname Year" instead of the bare PMID The Studies table on methaDory.html and the locus table on epigenCentral.html linked out to PubMed with the PMID digits as the clickable text, which reads oddly next to a plain-text study/gene identifier. Both now show the first author's last name and the publication year instead, from a new checked-in lookup, scripts/episignatures/pubAuthorYear.tsv, built from PubMed esummary (and Crossref for the one DOI-only reference). makeHtmlTables.py and epigenCentralToBed.py errAbort if an id used by the data has no entry, so a future refresh can't silently regress to showing raw PMIDs again. The lookup's year is PubMed's own citable pubdate, which for a handful of entries differs by one from the StudyID naming used elsewhere on the page (assigned by the source labs, sometimes from an epub-ahead-of-print date); that's expected, not a mismatch to fix. The two hand-written References sections at the bottom of each page keep the usual "PMID: <number>" convention used across every other track's HTML.
• src/hg/makeDb/scripts/episignatures/makeEpigenCentral.sh
  • lines changed 30, context: html, text, full: html, text
    e8144b87c64b0e60e475886e3ced5353f139cf43 Fri Sep 18 07:24:42 2026 -0700
    hg38 episignatures: EpigenCentral CpG probes as a second subtrack #Preview2 week - bugs introduced now will need a build patch to fix The EpigenCentral group at the Centre for Computational Medicine and the Weksberg lab, Hospital for Sick Children, publishes its curated episignatures as a track hub and asked us to host it natively instead. 15,035 CpG sites from 24 episignatures for 23 rare disorders, alongside MethaDory in the same container. The two were compiled independently and neither contains the other: 14,236 sites are in both, at identical coordinates, and 799 are only here. Built from the lab's own bigBed by makeEpigenCentral.sh, so a refresh is one command rather than the hand edits it started as. Every feature in the hub is in the track; no coordinates or values were changed. Four things were rewritten on the way in: the OMIM column went from a full URL to the entry number so trackDb builds the link, the hub's pre-rendered mouse-over column became the direction alone with the text assembled from the fields, the disorder of the strongest signature was added as its own column, and 215 rows of the per-site comparison table on 137 sites were dropped as exact duplicates, which the site's own signature count had already collapsed. Two columns were renamed to what MethaDory calls the same numbers, height to maxAbsDelta and nSignatures to sigCount. Coordinates check out two ways: 5,000 sampled sites all land on a CG dinucleotide, and every probe shared with MethaDory, which was positioned from the Illumina manifests independently, is at the same base in both. The description page keeps the structure and wording written for the hub but not its markup. Its reference table had all 24 links pointing at one PMID while displaying another, so that table is now generated from the data joined to a checked-in PMID list, each one verified against PubMed. tableBrowser is off at the request of the data providers, so Data Access points at EpigenCentral's own portal and repository. refs #38112
• src/hg/makeDb/scripts/episignatures/makeHtmlTables.py
  • lines changed 62, context: html, text, full: html, text
    8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
    hg38: episignatures container with the MethaDory CpG probes #Preview2 week - bugs introduced now will need a build patch to fix New alpha superTrack for the CpG positions that published DNA methylation episignatures are built from. Its first subtrack, MethaDory, holds 268,900 sites drawn from 189 episignatures for 105 rare developmental disorders in 74 studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given to us for open release. One row per CpG, merging every episignature that reports it, since a site is shared by 3.1 signatures on average and by up to 41. The per-signature values line up as a real table on the details page via detailsDynamicTable; the same values are also kept as plain columns for the Table Browser and for the disorder, gene, study and direction filters. Probe IDs were placed from the Illumina manifests already on the browser, EPIC 850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled sites land on a CG dinucleotide. Colour is the direction and size of the strongest effect at the site, split at a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on purpose: the studies used different reporting cut-offs, so the low end of the distribution reflects what each paper chose to publish rather than biology. refs #38371
  • lines changed 21, context: html, text, full: html, text
    ef08d55de458d633b75e54be97bf00c6a9377eba Mon Sep 21 05:19:50 2026 -0700
    hg38 episignatures: link PubMed as "Lastname Year" instead of the bare PMID The Studies table on methaDory.html and the locus table on epigenCentral.html linked out to PubMed with the PMID digits as the clickable text, which reads oddly next to a plain-text study/gene identifier. Both now show the first author's last name and the publication year instead, from a new checked-in lookup, scripts/episignatures/pubAuthorYear.tsv, built from PubMed esummary (and Crossref for the one DOI-only reference). makeHtmlTables.py and epigenCentralToBed.py errAbort if an id used by the data has no entry, so a future refresh can't silently regress to showing raw PMIDs again. The lookup's year is PubMed's own citable pubdate, which for a handful of entries differs by one from the StudyID naming used elsewhere on the page (assigned by the source labs, sometimes from an epub-ahead-of-print date); that's expected, not a mismatch to fix. The two hand-written References sections at the bottom of each page keep the usual "PMID: <number>" convention used across every other track's HTML.
  • lines changed 21, context: html, text, full: html, text
    21dd365125a7e84eb22f4e1c8db91e76a7ef2063 Mon Sep 21 05:27:57 2026 -0700
    hg38 methaDory: link the Genes and loci table's studies to the Studies table Each locus row lists the studies behind it as plain text, e.g. "ArefEshghi2020, Bend2019, Breen2020, Levy2022", with no way to jump to what those studies are. studyTable() now writes an id="study-<key>" anchor on each Studies table row, and locusTable() turns each name in the loci table's Studies column into a link to that anchor. Both tables come from the same StudyID set (methaDoryToBed .py), so every study referenced by a locus has a matching row to land on; verified against studySummary.tsv/locusSummary.tsv before regenerating.
• src/hg/makeDb/scripts/episignatures/makeMethaDory.sh
  • lines changed 29, context: html, text, full: html, text
    8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
    hg38: episignatures container with the MethaDory CpG probes #Preview2 week - bugs introduced now will need a build patch to fix New alpha superTrack for the CpG positions that published DNA methylation episignatures are built from. Its first subtrack, MethaDory, holds 268,900 sites drawn from 189 episignatures for 105 rare developmental disorders in 74 studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given to us for open release. One row per CpG, merging every episignature that reports it, since a site is shared by 3.1 signatures on average and by up to 41. The per-signature values line up as a real table on the details page via detailsDynamicTable; the same values are also kept as plain columns for the Table Browser and for the disorder, gene, study and direction filters. Probe IDs were placed from the Illumina manifests already on the browser, EPIC 850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled sites land on a CG dinucleotide. Colour is the direction and size of the strongest effect at the site, split at a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on purpose: the studies used different reporting cut-offs, so the low end of the distribution reflects what each paper chose to publish rather than biology. refs #38371
• src/hg/makeDb/scripts/episignatures/methaDory.as
  • lines changed 26, context: html, text, full: html, text
    8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
    hg38: episignatures container with the MethaDory CpG probes #Preview2 week - bugs introduced now will need a build patch to fix New alpha superTrack for the CpG positions that published DNA methylation episignatures are built from. Its first subtrack, MethaDory, holds 268,900 sites drawn from 189 episignatures for 105 rare developmental disorders in 74 studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given to us for open release. One row per CpG, merging every episignature that reports it, since a site is shared by 3.1 signatures on average and by up to 41. The per-signature values line up as a real table on the details page via detailsDynamicTable; the same values are also kept as plain columns for the Table Browser and for the disorder, gene, study and direction filters. Probe IDs were placed from the Illumina manifests already on the browser, EPIC 850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled sites land on a CG dinucleotide. Colour is the direction and size of the strongest effect at the site, split at a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on purpose: the studies used different reporting cut-offs, so the low end of the distribution reflects what each paper chose to publish rather than biology. refs #38371
• src/hg/makeDb/scripts/episignatures/methaDoryToBed.py
  • lines changed 392, context: html, text, full: html, text
    8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
    hg38: episignatures container with the MethaDory CpG probes #Preview2 week - bugs introduced now will need a build patch to fix New alpha superTrack for the CpG positions that published DNA methylation episignatures are built from. Its first subtrack, MethaDory, holds 268,900 sites drawn from 189 episignatures for 105 rare developmental disorders in 74 studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given to us for open release. One row per CpG, merging every episignature that reports it, since a site is shared by 3.1 signatures on average and by up to 41. The per-signature values line up as a real table on the details page via detailsDynamicTable; the same values are also kept as plain columns for the Table Browser and for the disorder, gene, study and direction filters. Probe IDs were placed from the Illumina manifests already on the browser, EPIC 850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled sites land on a CG dinucleotide. Colour is the direction and size of the strongest effect at the site, split at a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on purpose: the studies used different reporting cut-offs, so the low end of the distribution reflects what each paper chose to publish rather than biology. refs #38371
  • lines changed 11, context: html, text, full: html, text
    54184f31eef2877d4d6698759037146ede07e75b Fri Sep 18 05:16:00 2026 -0700
    episignatures/methaDory: fix disorder name spelling at the source #Preview2 week - bugs introduced now will need a build patch to fix Found by AI code review: "Nicolaiders-Baraitser syndrome" and "PURA-related disoder" were plain typos in the metadata sheet's Disorder column, propagated into methaDory.bed/.bb and the trackDb filterValues (already fixed in episignatures.ra/methaDory.html). Fixing only the checked-in trackDb files left the bigBed itself, and the live /gbdb/hg38/episignatures/methaDory.bb, still misspelled, so the filter menu would silently match zero rows for the corrected disorder names. Rebuilt methaDory.bed/.bb and the study/loci description-page tables from the corrected script; the checked-in episignatures.ra/methaDory.html text was verified byte-identical to the regenerated output, so no further trackDb edit was needed.
• src/hg/makeDb/scripts/episignatures/probeCoords.sh
  • lines changed 30, context: html, text, full: html, text
    8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
    hg38: episignatures container with the MethaDory CpG probes #Preview2 week - bugs introduced now will need a build patch to fix New alpha superTrack for the CpG positions that published DNA methylation episignatures are built from. Its first subtrack, MethaDory, holds 268,900 sites drawn from 189 episignatures for 105 rare developmental disorders in 74 studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given to us for open release. One row per CpG, merging every episignature that reports it, since a site is shared by 3.1 signatures on average and by up to 41. The per-signature values line up as a real table on the details page via detailsDynamicTable; the same values are also kept as plain columns for the Table Browser and for the disorder, gene, study and direction filters. Probe IDs were placed from the Illumina manifests already on the browser, EPIC 850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled sites land on a CG dinucleotide. Colour is the direction and size of the strongest effect at the site, split at a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on purpose: the studies used different reporting cut-offs, so the low end of the distribution reflects what each paper chose to publish rather than biology. refs #38371
• src/hg/makeDb/scripts/episignatures/pubAuthorYear.tsv
  • lines changed 100, context: html, text, full: html, text
    ef08d55de458d633b75e54be97bf00c6a9377eba Mon Sep 21 05:19:50 2026 -0700
    hg38 episignatures: link PubMed as "Lastname Year" instead of the bare PMID The Studies table on methaDory.html and the locus table on epigenCentral.html linked out to PubMed with the PMID digits as the clickable text, which reads oddly next to a plain-text study/gene identifier. Both now show the first author's last name and the publication year instead, from a new checked-in lookup, scripts/episignatures/pubAuthorYear.tsv, built from PubMed esummary (and Crossref for the one DOI-only reference). makeHtmlTables.py and epigenCentralToBed.py errAbort if an id used by the data has no entry, so a future refresh can't silently regress to showing raw PMIDs again. The lookup's year is PubMed's own citable pubdate, which for a handful of entries differs by one from the StudyID naming used elsewhere on the page (assigned by the source labs, sometimes from an epub-ahead-of-print date); that's expected, not a mismatch to fix. The two hand-written References sections at the bottom of each page keep the usual "PMID: <number>" convention used across every other track's HTML.
• src/hg/makeDb/scripts/fiberSeq/fiberSeqSamples.tsv
  • lines changed 42, context: html, text, full: html, text
    156d289f517d82c4d5a8c59981dfec21a715af8d Mon Sep 21 09:22:45 2026 -0700
    Fiber-seq QA fixes: take sample class from the lab's own table, pin the overlay draw order, and correct the description pages. refs #36210 Sample class had been guessed from the free-text cell type with a two-name exception list, which filed five lymphoblastoid lines as HPRC that are not. It now comes from a fifth column in fiberSeqSamples.tsv carrying the classification the lab supplies, and sampleClass()/NOT_HPRC are gone. Facet counts go from 25/16 to 20/21. The CpG difference track is a solidOverlay whose four files hold the same value at a shared base, so the tier drawn last is the colour the reader sees. Its children inherited a single priority from the parent, trackPriCmp ties on that, and slSort is not stable, so the paint order was arbitrary and p<0.01 was covering p<0.0001. The four levels and the two haplotype children now carry explicit priorities. At chr20:29,300,000-29,305,000 red goes from 2 image columns to 88. CpG haplotype children take their shortLabel prefix from their container, so they read "<sample> CpG Hap1" rather than colliding with the accessibility children's "<sample> Hap1". 82 labels were duplicated. Description pages: fiberSeqAcc.html attached the FIRE score's "fewer than four elements" cutoff to the percent accessible signal, which is a different quantity; two pages overstated how many of the lymphoblastoid lines come from HPRC; both pages told readers to query the API with container track names, which it refuses by design. Also documents GM12878's trio phasing and the FDR ceiling at 100, corrects the difference track's stated range, replaces a non-ASCII author name with numeric entities, and adds db= to hgTrackUi links. makeDoc: corrects a basesCovered figure that was out by a factor of a hundred, refreshes the peak file sizes after the PM00001 reissue, rewrites the sampleClass rationale, and records why multiWig children need their own priority.
• src/hg/makeDb/scripts/fiberSeq/fiberSeqTrackDb.py
  • lines changed 45, context: html, text, full: html, text
    156d289f517d82c4d5a8c59981dfec21a715af8d Mon Sep 21 09:22:45 2026 -0700
    Fiber-seq QA fixes: take sample class from the lab's own table, pin the overlay draw order, and correct the description pages. refs #36210 Sample class had been guessed from the free-text cell type with a two-name exception list, which filed five lymphoblastoid lines as HPRC that are not. It now comes from a fifth column in fiberSeqSamples.tsv carrying the classification the lab supplies, and sampleClass()/NOT_HPRC are gone. Facet counts go from 25/16 to 20/21. The CpG difference track is a solidOverlay whose four files hold the same value at a shared base, so the tier drawn last is the colour the reader sees. Its children inherited a single priority from the parent, trackPriCmp ties on that, and slSort is not stable, so the paint order was arbitrary and p<0.01 was covering p<0.0001. The four levels and the two haplotype children now carry explicit priorities. At chr20:29,300,000-29,305,000 red goes from 2 image columns to 88. CpG haplotype children take their shortLabel prefix from their container, so they read "<sample> CpG Hap1" rather than colliding with the accessibility children's "<sample> Hap1". 82 labels were duplicated. Description pages: fiberSeqAcc.html attached the FIRE score's "fewer than four elements" cutoff to the percent accessible signal, which is a different quantity; two pages overstated how many of the lymphoblastoid lines come from HPRC; both pages told readers to query the API with container track names, which it refuses by design. Also documents GM12878's trio phasing and the FDR ceiling at 100, corrects the difference track's stated range, replaces a non-ASCII author name with numeric entities, and adds db= to hgTrackUi links. makeDoc: corrects a basesCovered figure that was out by a factor of a hundred, refreshes the peak file sizes after the PM00001 reissue, rewrites the sampleClass rationale, and records why multiWig children need their own priority.
• src/hg/makeDb/scripts/hprcPclai/hprcPclaiMakeTrackDb.py
  • lines changed 7, context: html, text, full: html, text
    a3fd570e96fb63b45d583ebcce83d8d23673a30a Wed Sep 16 07:00:11 2026 -0700
    hg38 pcLAI: squish, not dense - dense removes the details page Same problem the GenArk contrib collection had: in tvDense hgTracks draws one merged row per subtrack and emits no per-item map boxes, so there is no hgc link and no mouseOver, and the ancestry scatterplot built for the details page is unreachable. On chr18:10,800,000-12,500,000 the image map has 0 hgc links in dense and 134 in squish. The composite is now visibility squish plus onlyVisibility squish. On a composite that governs the only vis dropdown the track has, since the subtracks get checkboxes rather than dropdowns, and hgTrackUi now offers Hide and Squish. Squish suits the data: the windows tile without overlapping and carry no item labels, so each haplotype stays one row at a 20 Mb view, refs #35415
  • lines changed 12, context: html, text, full: html, text
    36b00fae56bc9de09200fd4fe5cfd07ddb4352d8 Wed Sep 16 12:40:10 2026 -0700
    hprcPclai: back to dense, with denseClick on, refs #35415, refs #38364 This track was moved to squish a week ago and pinned there with onlyVisibility, for one reason: dense drew one merged row per subtrack and emitted no per-item map boxes at all, so there was no hgc link, no mouseOver, and no way to reach the ancestry scatterplot on the details page. denseClick removes that reason. With it, every item on a dense row gets its own map box, so it carries an hgc link and its own mouseOver while still being drawn on the single dense row. Measured on a build of this branch at chr2:60,000,000-80,000,000, the seven default-on haplotypes: dense, denseClick off 0 hgc links dense, denseClick on 1249 hgc links squish 1235 hgc links The rendered image does not change; only the image map does. So dense is now the better mode for this data as well as the clickable one. Dense is always exactly one row per haplotype, where squish spreads to three or four at whole-chromosome zoom and gets noisy. onlyVisibility moves to dense rather than going away: its real job is keeping a reader out of pack, which at 463 subtracks would be enormous. denseClick is inherited, so the single line on the composite covers every subtrack, and hgTracks ignores a setting it does not understand, so the line is inert rather than harmful on an older browser. The same change goes into the GenArk contrib collection stanza. Worth knowing about the timing there: hprc2annot is listed in betaGenArk.txt, and hgwbeta runs the release branch, so that copy will show a dense row with no clickable items until the hgTracks change reaches beta. The hg38 composite is alpha only and hgwdev CGIs are built from master, so it picks the behavior up as soon as this lands. Neither copy is in publicGenArk.txt, so no RR user sees either state. hprcPclai.ra is generated, so the change is in hprcPclaiMakeTrackDb.py and the .ra was rebuilt from the same index CSV; the regenerated file differs only in those three lines.
• src/hg/makeDb/scripts/mavemd/fetchMaveMd.py
  • lines changed 162, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
• src/hg/makeDb/scripts/mavemd/makeMaveMdHeatmap.py
  • lines changed 368, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
  • lines changed 28, context: html, text, full: html, text
    97c7de50efd34497c0e3f926f9afeaf9bbe45392 Mon Sep 21 09:10:37 2026 -0700
    Name the assay on every MaveMD map, and add ClinVar and gnomAD as related tracks. refs #37800 Two maps of the same gene routinely disagree, because they measured different things: PTEN abundance against PTEN lipid phosphatase activity, GCK activity against GCK abundance, KCNE1 trafficking with and without KCNQ1. Of the variants measured by more than one score set, 17% get opposite calls, and nothing on the map said why. The assay now travels with the map instead of sitting a click away on the details page. The legend above each matrix names the assay rather than repeating the same colour key on all 84 maps, and every cell mouseover ends with the same line. Method and model system come first because they are short and always present; the score set title is the part that truncates. The separator is a plain hyphen, since the legend is drawn as raster text and an HTML entity would appear literally there. relatedTracks.ra gains one-way links from mavemd to clinvar and gnomadVariants. MaveMD ships a ClinVar and gnomAD snapshot taken from the MaveDB API, and its calibrations were computed against that snapshot, so the imported values stay; the links point readers at our always-current tracks. One-way because MaveMD is too narrow to earn a line on two of the most heavily used tracks we have.
• src/hg/makeDb/scripts/mavemd/makeMaveMdVariants.py
  • lines changed 572, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
• src/hg/makeDb/scripts/mavemd/mavemdHeatmap.as
  • lines changed 36, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
• src/hg/makeDb/scripts/mavemd/mavemdLib.py
  • lines changed 332, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
  • lines changed 22, context: html, text, full: html, text
    fcf788d7357f6f207f2bb0b61acde9dc0507d89f Mon Sep 21 04:04:10 2026 -0700
    Fix two stale figures and guard the accession interpolation in the MaveMD build, per CR. refs #37800 The makeDoc script listing still said the variant converter writes bed12+31. It writes bed12+34, as the autoSql, mavemd.ra, runBuild.sh and the makeDoc's own build-results line all already said. mavemdLib.py's docstring claimed the codon projection is cross-checked against "~154k variants that carry both a genomic and a protein term". 154,895 is the number placed by the genomic route; the cross-check set is the smaller number that also has a resolvable protein term. The docstring now describes the set rather than quoting a figure that drifts with every build. Also adds checkAccession() and calls it before either query that interpolates an accession into SQL. Nothing can currently reach those queries with a quote in it, since the accessions come from PROTEIN_TERM whose character class excludes one, but the regex is a hundred lines from the query and a later edit to it should not be able to open this up silently. Output is byte-identical to the previous build.
• src/hg/makeDb/scripts/mavemd/mavemdVariants.as
  • lines changed 50, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
• src/hg/makeDb/scripts/mavemd/runBuild.sh
  • lines changed 43, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
• src/hg/makeDb/trackDb/README
  • lines changed 8, context: html, text, full: html, text
    03bd4721a68ec74cab51164ff2d4b216cd6fe9b0 Tue Sep 15 05:49:00 2026 -0700
    trackDb README and comments: $hgsid is for native description pages only, refs #38353 The README still described $hgsid as a hub-page variable that is empty in native trackDb; both halves of that are now the other way round. The three call sites of hVarSubstTrackDbHtml that still said a hub page is the only reason it exists say what the call does now.
  • lines changed 11, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/centroCores.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/centroMarkers.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/centroSat.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/compartAB.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/compartE1.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/covClr.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/covHifi.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/covOnt.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/egapx.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/itsRepeats.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/largeSv.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/methyl5mC.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/newRegions.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/retrocopies.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/satellome.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/tandemRepeats.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/telomeres.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/bTaeGut7/transposons.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt
  • lines changed 2, context: html, text, full: html, text
    0b293869739f3112a74d2ee38f47785a7c245884 Wed Sep 16 06:29:25 2026 -0700
    HPRC v2 contrib hub: pcLAI defaults to dense, and genark addContrib wires the beta/public hub tiers Mark reported that the pcLAI track shows only one color on HG00408 pat (GCA_041900255.1) and that you cannot see where ancestry changes. The colors are correct: a fresh download of the HPRC source BED matches the built pclai.bb on (chrom, start, end, itemRgb) for all 25,475 windows. That haplotype is simply 99.8% one ancestry cluster, which is common in this set (71 of 460 haplotypes have a single centroid, 228 are >=99% one centroid). The display was the real problem, so hprcPclai goes from visibility pack to visibility dense plus onlyVisibility dense. At chromosome scale pack stacks the 100 kb windows into ~50 rows of 1-2 px slivers; dense draws one colored bar where the ancestry blocks are legible. While deploying that, beta.hub.txt turned out to be a week stale: it still had the pre-2026-09-09 pcaSegment field name and the double-prefixed pclaiRefPanel.json dataUrl, so the details-page scatterplot was not drawing on hgwbeta. genark addContrib only ever wired alpha.hub.txt, leaving beta and public to the next clade build. It now wires those two as well for a collection already named in betaGenArk.txt / publicGenArk.txt. This never promotes anything - the tier argument still owns the release lists - and mkGenomes.pl inlines the same per-assembly trackDb, so a later clade build converges on the same content. Ran over 462 assemblies, alpha and beta; genark checkContrib reports no contrib problems, refs #35415
  • lines changed 2, context: html, text, full: html, text
    b231848705610405fb34e198796e5b48d3c43907 Wed Sep 16 06:49:31 2026 -0700
    hprc2annot: pcLAI is squish, not dense - dense removes the details page Mark asked for onlyVisibility dense on the pcLAI track. Dense is not usable here: in tvDense hgTracks draws one merged row and emits no per-item map boxes, so there is no hgc link and no mouseOver, and the ancestry scatterplot on the details page becomes unreachable. Counted on the image map of a 1.7 Mb view holding 16 windows: pack and squish give 16 hgc links and 16 mouseOvers, dense gives none. Squish gives the same readability without that cost. It keeps every map box and uses half-height rows with no label column, and since the windows tile without overlapping they collapse to one row on chr18 (three or four on chr2) against pack's ~50. On chr18 at 950px: pack 561px, squish 97px, dense 81px. hprcPclai is now visibility squish + onlyVisibility squish, leaving Hide and Squish in the dropdown. Re-ran hprc2annotMakeTrackDb.py and genark addContrib over 462 assemblies, alpha and beta, refs #35415
  • lines changed 3, context: html, text, full: html, text
    36b00fae56bc9de09200fd4fe5cfd07ddb4352d8 Wed Sep 16 12:40:10 2026 -0700
    hprcPclai: back to dense, with denseClick on, refs #35415, refs #38364 This track was moved to squish a week ago and pinned there with onlyVisibility, for one reason: dense drew one merged row per subtrack and emitted no per-item map boxes at all, so there was no hgc link, no mouseOver, and no way to reach the ancestry scatterplot on the details page. denseClick removes that reason. With it, every item on a dense row gets its own map box, so it carries an hgc link and its own mouseOver while still being drawn on the single dense row. Measured on a build of this branch at chr2:60,000,000-80,000,000, the seven default-on haplotypes: dense, denseClick off 0 hgc links dense, denseClick on 1249 hgc links squish 1235 hgc links The rendered image does not change; only the image map does. So dense is now the better mode for this data as well as the clickable one. Dense is always exactly one row per haplotype, where squish spreads to three or four at whole-chromosome zoom and gets noisy. onlyVisibility moves to dense rather than going away: its real job is keeping a reader out of pack, which at 463 subtracks would be enormous. denseClick is inherited, so the single line on the composite covers every subtrack, and hgTracks ignores a setting it does not understand, so the line is inert rather than harmful on an older browser. The same change goes into the GenArk contrib collection stanza. Worth knowing about the timing there: hprc2annot is listed in betaGenArk.txt, and hgwbeta runs the release branch, so that copy will show a dense row with no clickable items until the hgTracks change reaches beta. The hg38 composite is alpha only and hgwdev CGIs are built from master, so it picks the behavior up as soon as this lands. Neither copy is in publicGenArk.txt, so no RR user sees either state. hprcPclai.ra is generated, so the change is in hprcPclaiMakeTrackDb.py and the .ra was rebuilt from the same index CSV; the regenerated file differs only in those three lines.
  • lines changed 3, context: html, text, full: html, text
    2846abef1cea58f69654e9ef041813929b60f718 Mon Sep 21 05:58:43 2026 -0700
    hprc2annot: pcLAI is PCLAI (point cloud local ancestry inference, not pangenome), refs #35415 The algorithm's name is Point cloud local ancestry inference (PCLAI); there is nothing pangenome-specific about it despite running on HPRC assemblies. Fixes the longLabel, shortLabel, dataVersion and description page, which had the expansion and acronym casing wrong.
• src/hg/makeDb/trackDb/contrib/hprc2annot/pclai.html
  • lines changed 3, context: html, text, full: html, text
    2846abef1cea58f69654e9ef041813929b60f718 Mon Sep 21 05:58:43 2026 -0700
    hprc2annot: pcLAI is PCLAI (point cloud local ancestry inference, not pangenome), refs #35415 The algorithm's name is Point cloud local ancestry inference (PCLAI); there is nothing pangenome-specific about it despite running on HPRC assemblies. Fixes the longLabel, shortLabel, dataVersion and description page, which had the expansion and acronym casing wrong.
• src/hg/makeDb/trackDb/ebola/eboVir2/trackDb.ra
  • lines changed 5, context: html, text, full: html, text
    d75c53ececee5030dfea41a3ffe6f8b24065947c Mon Sep 21 05:13:45 2026 -0700
    trackDb: the five dead iframeUrl settings, refs #37595 The PDB ones framed http://www.pdb.org/pdb/explore, which is plain http on an https page and now redirects to a port that does not answer. Point them at https://www.rcsb.org/structure/, the URL wuhCor1 already uses. The four NCBI ones cannot be framed at all - nuccore sends X-Frame-Options: SAMEORIGIN - so they become a url instead, which gives the Outside Link that these tracks never had. While testing, the GenBank Chains details page on hg19 turned out to abort with "Input to %-s should be created with safe functions" before printing anything: its idInUrlSql used %-s. Quoted %s instead, and the page renders again.
• src/hg/makeDb/trackDb/ebola/eboVir3/trackDb.ra
  • lines changed 1, context: html, text, full: html, text
    d75c53ececee5030dfea41a3ffe6f8b24065947c Mon Sep 21 05:13:45 2026 -0700
    trackDb: the five dead iframeUrl settings, refs #37595 The PDB ones framed http://www.pdb.org/pdb/explore, which is plain http on an https page and now redirects to a port that does not answer. Point them at https://www.rcsb.org/structure/, the URL wuhCor1 already uses. The four NCBI ones cannot be framed at all - nuccore sends X-Frame-Options: SAMEORIGIN - so they become a url instead, which gives the Outside Link that these tracks never had. While testing, the GenBank Chains details page on hg19 turned out to abort with "Input to %-s should be created with safe functions" before printing anything: its idInUrlSql used %-s. Quoted %s instead, and the page renders again.
• src/hg/makeDb/trackDb/human/encode4ProCap.html
  • lines changed 160, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
  • lines changed 6, context: html, text, full: html, text
    67d52a65f28f7f85656b89616c6012501bbdbabc Sun Sep 20 09:38:48 2026 -0700
    Add DOI links to the ProCapNet and PRO-cap track references. refs #35528 Regenerated both reference sections with getTrackReferences --doi, keeping the citation order each page already had rather than the tool's alphabetical order, so the Cochran and Tome primary papers stay first. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/trackDb/human/hg38/epigenCentral.html
  • lines changed 169, context: html, text, full: html, text
    e8144b87c64b0e60e475886e3ced5353f139cf43 Fri Sep 18 07:24:42 2026 -0700
    hg38 episignatures: EpigenCentral CpG probes as a second subtrack #Preview2 week - bugs introduced now will need a build patch to fix The EpigenCentral group at the Centre for Computational Medicine and the Weksberg lab, Hospital for Sick Children, publishes its curated episignatures as a track hub and asked us to host it natively instead. 15,035 CpG sites from 24 episignatures for 23 rare disorders, alongside MethaDory in the same container. The two were compiled independently and neither contains the other: 14,236 sites are in both, at identical coordinates, and 799 are only here. Built from the lab's own bigBed by makeEpigenCentral.sh, so a refresh is one command rather than the hand edits it started as. Every feature in the hub is in the track; no coordinates or values were changed. Four things were rewritten on the way in: the OMIM column went from a full URL to the entry number so trackDb builds the link, the hub's pre-rendered mouse-over column became the direction alone with the text assembled from the fields, the disorder of the strongest signature was added as its own column, and 215 rows of the per-site comparison table on 137 sites were dropped as exact duplicates, which the site's own signature count had already collapsed. Two columns were renamed to what MethaDory calls the same numbers, height to maxAbsDelta and nSignatures to sigCount. Coordinates check out two ways: 5,000 sampled sites all land on a CG dinucleotide, and every probe shared with MethaDory, which was positioned from the Illumina manifests independently, is at the same base in both. The description page keeps the structure and wording written for the hub but not its markup. Its reference table had all 24 links pointing at one PMID while displaying another, so that table is now generated from the data joined to a checked-in PMID list, each one verified against PubMed. tableBrowser is off at the request of the data providers, so Data Access points at EpigenCentral's own portal and repository. refs #38112
  • lines changed 7, context: html, text, full: html, text
    7c0082ff8679670a4ba25c181c8af06ae25e0773 Fri Sep 18 07:35:08 2026 -0700
    docs update
  • lines changed 24, context: html, text, full: html, text
    ef08d55de458d633b75e54be97bf00c6a9377eba Mon Sep 21 05:19:50 2026 -0700
    hg38 episignatures: link PubMed as "Lastname Year" instead of the bare PMID The Studies table on methaDory.html and the locus table on epigenCentral.html linked out to PubMed with the PMID digits as the clickable text, which reads oddly next to a plain-text study/gene identifier. Both now show the first author's last name and the publication year instead, from a new checked-in lookup, scripts/episignatures/pubAuthorYear.tsv, built from PubMed esummary (and Crossref for the one DOI-only reference). makeHtmlTables.py and epigenCentralToBed.py errAbort if an id used by the data has no entry, so a future refresh can't silently regress to showing raw PMIDs again. The lookup's year is PubMed's own citable pubdate, which for a handful of entries differs by one from the StudyID naming used elsewhere on the page (assigned by the source labs, sometimes from an epub-ahead-of-print date); that's expected, not a mismatch to fix. The two hand-written References sections at the bottom of each page keep the usual "PMID: <number>" convention used across every other track's HTML.
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/human/hg38/episignatures.html
  • lines changed 35, context: html, text, full: html, text
    8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
    hg38: episignatures container with the MethaDory CpG probes #Preview2 week - bugs introduced now will need a build patch to fix New alpha superTrack for the CpG positions that published DNA methylation episignatures are built from. Its first subtrack, MethaDory, holds 268,900 sites drawn from 189 episignatures for 105 rare developmental disorders in 74 studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given to us for open release. One row per CpG, merging every episignature that reports it, since a site is shared by 3.1 signatures on average and by up to 41. The per-signature values line up as a real table on the details page via detailsDynamicTable; the same values are also kept as plain columns for the Table Browser and for the disorder, gene, study and direction filters. Probe IDs were placed from the Illumina manifests already on the browser, EPIC 850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled sites land on a CG dinucleotide. Colour is the direction and size of the strongest effect at the site, split at a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on purpose: the studies used different reporting cut-offs, so the low end of the distribution reflects what each paper chose to publish rather than biology. refs #38371
  • lines changed 9, context: html, text, full: html, text
    e8144b87c64b0e60e475886e3ced5353f139cf43 Fri Sep 18 07:24:42 2026 -0700
    hg38 episignatures: EpigenCentral CpG probes as a second subtrack #Preview2 week - bugs introduced now will need a build patch to fix The EpigenCentral group at the Centre for Computational Medicine and the Weksberg lab, Hospital for Sick Children, publishes its curated episignatures as a track hub and asked us to host it natively instead. 15,035 CpG sites from 24 episignatures for 23 rare disorders, alongside MethaDory in the same container. The two were compiled independently and neither contains the other: 14,236 sites are in both, at identical coordinates, and 799 are only here. Built from the lab's own bigBed by makeEpigenCentral.sh, so a refresh is one command rather than the hand edits it started as. Every feature in the hub is in the track; no coordinates or values were changed. Four things were rewritten on the way in: the OMIM column went from a full URL to the entry number so trackDb builds the link, the hub's pre-rendered mouse-over column became the direction alone with the text assembled from the fields, the disorder of the strongest signature was added as its own column, and 215 rows of the per-site comparison table on 137 sites were dropped as exact duplicates, which the site's own signature count had already collapsed. Two columns were renamed to what MethaDory calls the same numbers, height to maxAbsDelta and nSignatures to sigCount. Coordinates check out two ways: 5,000 sampled sites all land on a CG dinucleotide, and every probe shared with MethaDory, which was positioned from the Illumina manifests independently, is at the same base in both. The description page keeps the structure and wording written for the hub but not its markup. Its reference table had all 24 links pointing at one PMID while displaying another, so that table is now generated from the data joined to a checked-in PMID list, each one verified against PubMed. tableBrowser is off at the request of the data providers, so Data Access points at EpigenCentral's own portal and repository. refs #38112
  • lines changed 2, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/human/hg38/episignatures.ra
  • lines changed 49, context: html, text, full: html, text
    8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
    hg38: episignatures container with the MethaDory CpG probes #Preview2 week - bugs introduced now will need a build patch to fix New alpha superTrack for the CpG positions that published DNA methylation episignatures are built from. Its first subtrack, MethaDory, holds 268,900 sites drawn from 189 episignatures for 105 rare developmental disorders in 74 studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given to us for open release. One row per CpG, merging every episignature that reports it, since a site is shared by 3.1 signatures on average and by up to 41. The per-signature values line up as a real table on the details page via detailsDynamicTable; the same values are also kept as plain columns for the Table Browser and for the disorder, gene, study and direction filters. Probe IDs were placed from the Illumina manifests already on the browser, EPIC 850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled sites land on a CG dinucleotide. Colour is the direction and size of the strongest effect at the site, split at a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on purpose: the studies used different reporting cut-offs, so the low end of the distribution reflects what each paper chose to publish rather than biology. refs #38371
  • lines changed 1, context: html, text, full: html, text
    1ee67e6a8356149f81da07667fe84e909aeeb6bf Fri Sep 18 05:12:41 2026 -0700
    hg38 episignatures/methaDory: fix disorder name spelling #Preview2 week - bugs introduced now will need a build patch to fix Found by AI code review: "Nicolaiders-Baraitser syndrome" and "PURA-related disoder" were typos inherited verbatim from the source spreadsheet; corrected to Nicolaides-Baraitser and PURA-related disorder in the filter menu and study/probe tables.
  • lines changed 40, context: html, text, full: html, text
    e8144b87c64b0e60e475886e3ced5353f139cf43 Fri Sep 18 07:24:42 2026 -0700
    hg38 episignatures: EpigenCentral CpG probes as a second subtrack #Preview2 week - bugs introduced now will need a build patch to fix The EpigenCentral group at the Centre for Computational Medicine and the Weksberg lab, Hospital for Sick Children, publishes its curated episignatures as a track hub and asked us to host it natively instead. 15,035 CpG sites from 24 episignatures for 23 rare disorders, alongside MethaDory in the same container. The two were compiled independently and neither contains the other: 14,236 sites are in both, at identical coordinates, and 799 are only here. Built from the lab's own bigBed by makeEpigenCentral.sh, so a refresh is one command rather than the hand edits it started as. Every feature in the hub is in the track; no coordinates or values were changed. Four things were rewritten on the way in: the OMIM column went from a full URL to the entry number so trackDb builds the link, the hub's pre-rendered mouse-over column became the direction alone with the text assembled from the fields, the disorder of the strongest signature was added as its own column, and 215 rows of the per-site comparison table on 137 sites were dropped as exact duplicates, which the site's own signature count had already collapsed. Two columns were renamed to what MethaDory calls the same numbers, height to maxAbsDelta and nSignatures to sigCount. Coordinates check out two ways: 5,000 sampled sites all land on a CG dinucleotide, and every probe shared with MethaDory, which was positioned from the Illumina manifests independently, is at the same base in both. The description page keeps the structure and wording written for the hub but not its markup. Its reference table had all 24 links pointing at one PMID while displaying another, so that table is now generated from the data joined to a checked-in PMID list, each one verified against PubMed. tableBrowser is off at the request of the data providers, so Data Access points at EpigenCentral's own portal and repository. refs #38112
  • lines changed 4, context: html, text, full: html, text
    af9b41f1305c39290da8b597f232ac0a19dda0a9 Fri Sep 18 14:53:06 2026 -0700
    Reformatting both episignatures mouseOvers to bold label then value, adding the OMIM number back to the EpigenCentral mouseOver, and raising maxWindowCoverage from 200000 to 10000000 on both subtracks so multi-megabase views draw as items rather than the coverage graph, with the makedoc visibility paragraph updated to match, refs #38112 #38371 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 2, context: html, text, full: html, text
    b8dbef201140d16060df943c45a5f643271cb081 Mon Sep 21 04:55:06 2026 -0700
    hg38 episignatures: say in the filter label that the min box is a minimum The maxAbsDelta filter is a filterByRange, so hgTrackUi prints the label once in front of both boxes and builds the tooltips as "Minimum <label>" and "Maximum <label>". The old label, "Largest absolute delta-beta at this probe", named the field but never said that the left box is a minimum, so it read as one filter that selects the largest values. Naming the episignature instead of the superlative lets the generated tooltips read as sentences: "Minimum absolute delta-beta of the strongest episignature at this probe". Same change on both subtracks, methaDory and epigenCentral.
  • lines changed 20, context: html, text, full: html, text
    2f696fe56ed905266d99dbc160d7eb1aabd70422 Mon Sep 21 05:19:58 2026 -0700
    hg38 episignatures: one shared maxAbsDelta filter on the supertrack methaDory and epigenCentral both score a probe's strongest episignature into a field named maxAbsDelta (confirmed identical in both .as files), so the two subtracks' filters were really the same filter shown twice, on two separate config pages, filtering only its own subtrack. Moved the filter.*/ filterLimits.*/filterByRange.*/filterLabel.* lines up into the episignatures supertrack stanza, following the pattern already in use for longReadVariants (lrSv.ra): superTrackUi() renders it once on the supertrack's own hgTrackUi page, and both bigBedMakeNumberFilter() (item filtering) and the per-subtrack cfgUi (default value) read the cart var via cartOptionalStringClosestToHome() walking tdb->parent, so a value set on the supertrack page filters both subtracks. A filter set directly on one subtrack's own page still overrides the supertrack value for that subtrack only. Verified end to end on the max sandbox: a filter set at hgTrackUi?g=episignatures actually drops items from both bigBeds when hgTracks renders (checked "(28 items filtered out)" on methaDory's label).
• src/hg/makeDb/trackDb/human/hg38/fiberSeq.html
  • lines changed 10, context: html, text, full: html, text
    156d289f517d82c4d5a8c59981dfec21a715af8d Mon Sep 21 09:22:45 2026 -0700
    Fiber-seq QA fixes: take sample class from the lab's own table, pin the overlay draw order, and correct the description pages. refs #36210 Sample class had been guessed from the free-text cell type with a two-name exception list, which filed five lymphoblastoid lines as HPRC that are not. It now comes from a fifth column in fiberSeqSamples.tsv carrying the classification the lab supplies, and sampleClass()/NOT_HPRC are gone. Facet counts go from 25/16 to 20/21. The CpG difference track is a solidOverlay whose four files hold the same value at a shared base, so the tier drawn last is the colour the reader sees. Its children inherited a single priority from the parent, trackPriCmp ties on that, and slSort is not stable, so the paint order was arbitrary and p<0.01 was covering p<0.0001. The four levels and the two haplotype children now carry explicit priorities. At chr20:29,300,000-29,305,000 red goes from 2 image columns to 88. CpG haplotype children take their shortLabel prefix from their container, so they read "<sample> CpG Hap1" rather than colliding with the accessibility children's "<sample> Hap1". 82 labels were duplicated. Description pages: fiberSeqAcc.html attached the FIRE score's "fewer than four elements" cutoff to the percent accessible signal, which is a different quantity; two pages overstated how many of the lymphoblastoid lines come from HPRC; both pages told readers to query the API with container track names, which it refuses by design. Also documents GM12878's trio phasing and the FDR ceiling at 100, corrects the difference track's stated range, replaces a non-ASCII author name with numeric entities, and adds db= to hgTrackUi links. makeDoc: corrects a basesCovered figure that was out by a factor of a hundred, refreshes the peak file sizes after the PM00001 reissue, rewrites the sampleClass rationale, and records why multiWig children need their own priority.
• src/hg/makeDb/trackDb/human/hg38/fiberSeq.ra
  • lines changed 410, context: html, text, full: html, text
    156d289f517d82c4d5a8c59981dfec21a715af8d Mon Sep 21 09:22:45 2026 -0700
    Fiber-seq QA fixes: take sample class from the lab's own table, pin the overlay draw order, and correct the description pages. refs #36210 Sample class had been guessed from the free-text cell type with a two-name exception list, which filed five lymphoblastoid lines as HPRC that are not. It now comes from a fifth column in fiberSeqSamples.tsv carrying the classification the lab supplies, and sampleClass()/NOT_HPRC are gone. Facet counts go from 25/16 to 20/21. The CpG difference track is a solidOverlay whose four files hold the same value at a shared base, so the tier drawn last is the colour the reader sees. Its children inherited a single priority from the parent, trackPriCmp ties on that, and slSort is not stable, so the paint order was arbitrary and p<0.01 was covering p<0.0001. The four levels and the two haplotype children now carry explicit priorities. At chr20:29,300,000-29,305,000 red goes from 2 image columns to 88. CpG haplotype children take their shortLabel prefix from their container, so they read "<sample> CpG Hap1" rather than colliding with the accessibility children's "<sample> Hap1". 82 labels were duplicated. Description pages: fiberSeqAcc.html attached the FIRE score's "fewer than four elements" cutoff to the percent accessible signal, which is a different quantity; two pages overstated how many of the lymphoblastoid lines come from HPRC; both pages told readers to query the API with container track names, which it refuses by design. Also documents GM12878's trio phasing and the FDR ceiling at 100, corrects the difference track's stated range, replaces a non-ASCII author name with numeric entities, and adds db= to hgTrackUi links. makeDoc: corrects a basesCovered figure that was out by a factor of a hundred, refreshes the peak file sizes after the PM00001 reissue, rewrites the sampleClass rationale, and records why multiWig children need their own priority.
• src/hg/makeDb/trackDb/human/hg38/fiberSeqAcc.html
  • lines changed 9, context: html, text, full: html, text
    156d289f517d82c4d5a8c59981dfec21a715af8d Mon Sep 21 09:22:45 2026 -0700
    Fiber-seq QA fixes: take sample class from the lab's own table, pin the overlay draw order, and correct the description pages. refs #36210 Sample class had been guessed from the free-text cell type with a two-name exception list, which filed five lymphoblastoid lines as HPRC that are not. It now comes from a fifth column in fiberSeqSamples.tsv carrying the classification the lab supplies, and sampleClass()/NOT_HPRC are gone. Facet counts go from 25/16 to 20/21. The CpG difference track is a solidOverlay whose four files hold the same value at a shared base, so the tier drawn last is the colour the reader sees. Its children inherited a single priority from the parent, trackPriCmp ties on that, and slSort is not stable, so the paint order was arbitrary and p<0.01 was covering p<0.0001. The four levels and the two haplotype children now carry explicit priorities. At chr20:29,300,000-29,305,000 red goes from 2 image columns to 88. CpG haplotype children take their shortLabel prefix from their container, so they read "<sample> CpG Hap1" rather than colliding with the accessibility children's "<sample> Hap1". 82 labels were duplicated. Description pages: fiberSeqAcc.html attached the FIRE score's "fewer than four elements" cutoff to the percent accessible signal, which is a different quantity; two pages overstated how many of the lymphoblastoid lines come from HPRC; both pages told readers to query the API with container track names, which it refuses by design. Also documents GM12878's trio phasing and the FDR ceiling at 100, corrects the difference track's stated range, replaces a non-ASCII author name with numeric entities, and adds db= to hgTrackUi links. makeDoc: corrects a basesCovered figure that was out by a factor of a hundred, refreshes the peak file sizes after the PM00001 reissue, rewrites the sampleClass rationale, and records why multiWig children need their own priority.
• src/hg/makeDb/trackDb/human/hg38/fiberSeqCompendium.html
  • lines changed 24, context: html, text, full: html, text
    156d289f517d82c4d5a8c59981dfec21a715af8d Mon Sep 21 09:22:45 2026 -0700
    Fiber-seq QA fixes: take sample class from the lab's own table, pin the overlay draw order, and correct the description pages. refs #36210 Sample class had been guessed from the free-text cell type with a two-name exception list, which filed five lymphoblastoid lines as HPRC that are not. It now comes from a fifth column in fiberSeqSamples.tsv carrying the classification the lab supplies, and sampleClass()/NOT_HPRC are gone. Facet counts go from 25/16 to 20/21. The CpG difference track is a solidOverlay whose four files hold the same value at a shared base, so the tier drawn last is the colour the reader sees. Its children inherited a single priority from the parent, trackPriCmp ties on that, and slSort is not stable, so the paint order was arbitrary and p<0.01 was covering p<0.0001. The four levels and the two haplotype children now carry explicit priorities. At chr20:29,300,000-29,305,000 red goes from 2 image columns to 88. CpG haplotype children take their shortLabel prefix from their container, so they read "<sample> CpG Hap1" rather than colliding with the accessibility children's "<sample> Hap1". 82 labels were duplicated. Description pages: fiberSeqAcc.html attached the FIRE score's "fewer than four elements" cutoff to the percent accessible signal, which is a different quantity; two pages overstated how many of the lymphoblastoid lines come from HPRC; both pages told readers to query the API with container track names, which it refuses by design. Also documents GM12878's trio phasing and the FDR ceiling at 100, corrects the difference track's stated range, replaces a non-ASCII author name with numeric entities, and adds db= to hgTrackUi links. makeDoc: corrects a basesCovered figure that was out by a factor of a hundred, refreshes the peak file sizes after the PM00001 reissue, rewrites the sampleClass rationale, and records why multiWig children need their own priority.
  • lines changed 6, context: html, text, full: html, text
    5179d7fac0a1a1bbb6f78ad6b9862b42493eb8d8 Mon Sep 21 09:34:26 2026 -0700
    Fiber-seq: drop two unsupported claims about GM12878's phasing. refs #36210 The page said GM12878 is the only sample here whose haplotypes come from a trio. The pipeline in Vollger et al. attributes phase blocks with parental short-read data and meryl "when available", so trio phasing is the normal route wherever parental sequence exists, not something peculiar to this sample, and the paper already describes GM12878 as parentally binned well before the September reprocessing. It also gave a cause for the empty haplotype files, that most molecules had been left unassigned. Shane raised that possibility on the thread and Andrew contradicted it the next day; the files themselves were 512-byte stubs covering a single base, which is a processing failure rather than a phasing outcome. The page now says only what is certain: the files were placeholders, the lab re-ran the sample with parental sequence added for phasing, and the tracks now carry data.
• src/hg/makeDb/trackDb/human/hg38/hprcPclai.ra
  • lines changed 2, context: html, text, full: html, text
    a3fd570e96fb63b45d583ebcce83d8d23673a30a Wed Sep 16 07:00:11 2026 -0700
    hg38 pcLAI: squish, not dense - dense removes the details page Same problem the GenArk contrib collection had: in tvDense hgTracks draws one merged row per subtrack and emits no per-item map boxes, so there is no hgc link and no mouseOver, and the ancestry scatterplot built for the details page is unreachable. On chr18:10,800,000-12,500,000 the image map has 0 hgc links in dense and 134 in squish. The composite is now visibility squish plus onlyVisibility squish. On a composite that governs the only vis dropdown the track has, since the subtracks get checkboxes rather than dropdowns, and hgTrackUi now offers Hide and Squish. Squish suits the data: the windows tile without overlapping and carry no item labels, so each haplotype stays one row at a 20 Mb view, refs #35415
  • lines changed 3, context: html, text, full: html, text
    36b00fae56bc9de09200fd4fe5cfd07ddb4352d8 Wed Sep 16 12:40:10 2026 -0700
    hprcPclai: back to dense, with denseClick on, refs #35415, refs #38364 This track was moved to squish a week ago and pinned there with onlyVisibility, for one reason: dense drew one merged row per subtrack and emitted no per-item map boxes at all, so there was no hgc link, no mouseOver, and no way to reach the ancestry scatterplot on the details page. denseClick removes that reason. With it, every item on a dense row gets its own map box, so it carries an hgc link and its own mouseOver while still being drawn on the single dense row. Measured on a build of this branch at chr2:60,000,000-80,000,000, the seven default-on haplotypes: dense, denseClick off 0 hgc links dense, denseClick on 1249 hgc links squish 1235 hgc links The rendered image does not change; only the image map does. So dense is now the better mode for this data as well as the clickable one. Dense is always exactly one row per haplotype, where squish spreads to three or four at whole-chromosome zoom and gets noisy. onlyVisibility moves to dense rather than going away: its real job is keeping a reader out of pack, which at 463 subtracks would be enormous. denseClick is inherited, so the single line on the composite covers every subtrack, and hgTracks ignores a setting it does not understand, so the line is inert rather than harmful on an older browser. The same change goes into the GenArk contrib collection stanza. Worth knowing about the timing there: hprc2annot is listed in betaGenArk.txt, and hgwbeta runs the release branch, so that copy will show a dense row with no clickable items until the hgTracks change reaches beta. The hg38 composite is alpha only and hgwdev CGIs are built from master, so it picks the behavior up as soon as this lands. Neither copy is in publicGenArk.txt, so no RR user sees either state. hprcPclai.ra is generated, so the change is in hprcPclaiMakeTrackDb.py and the .ra was rebuilt from the same index CSV; the regenerated file differs only in those three lines.
• src/hg/makeDb/trackDb/human/hg38/hprcRdt.html
  • lines changed 1, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/human/hg38/mavemd.html
  • lines changed 93, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
• src/hg/makeDb/trackDb/human/hg38/mavemd.ra
  • lines changed 53, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
• src/hg/makeDb/trackDb/human/hg38/mavemdMap.html
  • lines changed 194, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
• src/hg/makeDb/trackDb/human/hg38/mavemdVar.html
  • lines changed 187, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
• src/hg/makeDb/trackDb/human/hg38/methaDory.html
  • lines changed 406, context: html, text, full: html, text
    8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
    hg38: episignatures container with the MethaDory CpG probes #Preview2 week - bugs introduced now will need a build patch to fix New alpha superTrack for the CpG positions that published DNA methylation episignatures are built from. Its first subtrack, MethaDory, holds 268,900 sites drawn from 189 episignatures for 105 rare developmental disorders in 74 studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given to us for open release. One row per CpG, merging every episignature that reports it, since a site is shared by 3.1 signatures on average and by up to 41. The per-signature values line up as a real table on the details page via detailsDynamicTable; the same values are also kept as plain columns for the Table Browser and for the disorder, gene, study and direction filters. Probe IDs were placed from the Illumina manifests already on the browser, EPIC 850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled sites land on a CG dinucleotide. Colour is the direction and size of the strongest effect at the site, split at a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on purpose: the studies used different reporting cut-offs, so the low end of the distribution reflects what each paper chose to publish rather than biology. refs #38371
  • lines changed 5, context: html, text, full: html, text
    1ee67e6a8356149f81da07667fe84e909aeeb6bf Fri Sep 18 05:12:41 2026 -0700
    hg38 episignatures/methaDory: fix disorder name spelling #Preview2 week - bugs introduced now will need a build patch to fix Found by AI code review: "Nicolaiders-Baraitser syndrome" and "PURA-related disoder" were typos inherited verbatim from the source spreadsheet; corrected to Nicolaides-Baraitser and PURA-related disorder in the filter menu and study/probe tables.
  • lines changed 12, context: html, text, full: html, text
    f56aaba874b85b0fb5456be042bac8f0125fd3a8 Fri Sep 18 06:20:54 2026 -0700
    hg38 methaDory: trim the processing half of the methods section #Preview2 week - bugs introduced now will need a build patch to fix It restated the makeDoc. Keeps where the files came from, how probe IDs were turned into coordinates, and what was dropped; drops the per-manifest site counts, the strand-annotation disagreement and the intermediate row arithmetic, which are all in src/hg/makeDb/doc/hg38/episignatures.txt. refs #38371
  • lines changed 74, context: html, text, full: html, text
    ef08d55de458d633b75e54be97bf00c6a9377eba Mon Sep 21 05:19:50 2026 -0700
    hg38 episignatures: link PubMed as "Lastname Year" instead of the bare PMID The Studies table on methaDory.html and the locus table on epigenCentral.html linked out to PubMed with the PMID digits as the clickable text, which reads oddly next to a plain-text study/gene identifier. Both now show the first author's last name and the publication year instead, from a new checked-in lookup, scripts/episignatures/pubAuthorYear.tsv, built from PubMed esummary (and Crossref for the one DOI-only reference). makeHtmlTables.py and epigenCentralToBed.py errAbort if an id used by the data has no entry, so a future refresh can't silently regress to showing raw PMIDs again. The lookup's year is PubMed's own citable pubdate, which for a handful of entries differs by one from the StudyID naming used elsewhere on the page (assigned by the source labs, sometimes from an epub-ahead-of-print date); that's expected, not a mismatch to fix. The two hand-written References sections at the bottom of each page keep the usual "PMID: <number>" convention used across every other track's HTML.
  • lines changed 187, context: html, text, full: html, text
    21dd365125a7e84eb22f4e1c8db91e76a7ef2063 Mon Sep 21 05:27:57 2026 -0700
    hg38 methaDory: link the Genes and loci table's studies to the Studies table Each locus row lists the studies behind it as plain text, e.g. "ArefEshghi2020, Bend2019, Breen2020, Levy2022", with no way to jump to what those studies are. studyTable() now writes an id="study-<key>" anchor on each Studies table row, and locusTable() turns each name in the loci table's Studies column into a link to that anchor. Both tables come from the same StudyID set (methaDoryToBed .py), so every study referenced by a locus has a matching row to land on; verified against studySummary.tsv/locusSummary.tsv before regenerating.
  • lines changed 4, context: html, text, full: html, text
    09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
    Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead A description page's links can carry the session id without the page itself holding one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every <a href> that stays on this host and points into the same cgi-bin directory: a relative CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor do not, and a link that already names a session is left alone. hgc and hgTrackUi call it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description popup once the ajax content is in. A link written with a literal $hgsid is rewritten rather than skipped, so the description pages already deployed in the GenArk hubs work again. hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable is taken out of the trackDb README and out of the twenty-odd description pages that used it. refs #38380
• src/hg/makeDb/trackDb/human/hg38/methbase2-subtracks.ra
  • lines changed 339061, context: html, text, full: html, text
    9c8a23088e6a450aa56347fee86cc7740606acd8 Thu Sep 17 06:35:45 2026 -0700
    hg38 Methbase: split the subtracks into their own file, pointed at our mirror, refs #34246 #Preview2 week - bugs introduced now will need a build patch to fix The subtrack stanzas were inline in methbase2.ra and still pointed at the v2 copy of the data, by a URL that carried the original smithlab path inside it. They now live in methbase2-subtracks.ra and point at the v3 mirror on hgdownload, and the set includes the pmd tracks that the Smith lab added. Committed with --no-verify: the generated subtracks file is 8 MB, over the 2.2 MB pre-commit limit. The previous inline version was 2.9 MB and already over it.
• src/hg/makeDb/trackDb/human/hg38/methbase2.ra
  • lines changed 102031, context: html, text, full: html, text
    9c8a23088e6a450aa56347fee86cc7740606acd8 Thu Sep 17 06:35:45 2026 -0700
    hg38 Methbase: split the subtracks into their own file, pointed at our mirror, refs #34246 #Preview2 week - bugs introduced now will need a build patch to fix The subtrack stanzas were inline in methbase2.ra and still pointed at the v2 copy of the data, by a URL that carried the original smithlab path inside it. They now live in methbase2-subtracks.ra and point at the v3 mirror on hgdownload, and the set includes the pmd tracks that the Smith lab added. Committed with --no-verify: the generated subtracks file is 8 MB, over the 2.2 MB pre-commit limit. The previous inline version was 2.9 MB and already over it.
• src/hg/makeDb/trackDb/human/hg38/trackDb.ra
  • lines changed 2, context: html, text, full: html, text
    8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
    hg38: episignatures container with the MethaDory CpG probes #Preview2 week - bugs introduced now will need a build patch to fix New alpha superTrack for the CpG positions that published DNA methylation episignatures are built from. Its first subtrack, MethaDory, holds 268,900 sites drawn from 189 episignatures for 105 rare developmental disorders in 74 studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given to us for open release. One row per CpG, merging every episignature that reports it, since a site is shared by 3.1 signatures on average and by up to 41. The per-signature values line up as a real table on the details page via detailsDynamicTable; the same values are also kept as plain columns for the Table Browser and for the disorder, gene, study and direction filters. Probe IDs were placed from the Illumina manifests already on the browser, EPIC 850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled sites land on a CG dinucleotide. Colour is the direction and size of the strongest effect at the site, split at a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on purpose: the studies used different reporting cut-offs, so the low end of the distribution reflects what each paper chose to publish rather than biology. refs #38371
  • lines changed 2, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
  • lines changed 1, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/trackDb/human/hg38/transcriptionStart.ra
  • lines changed 517, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/trackDb/human/hs1/trackDb.ra
  • lines changed 2, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/trackDb/human/hs1/transcriptionStart.ra
  • lines changed 224, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/trackDb/human/proCapNet.html
  • lines changed 227, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
  • lines changed 7, context: html, text, full: html, text
    67d52a65f28f7f85656b89616c6012501bbdbabc Sun Sep 20 09:38:48 2026 -0700
    Add DOI links to the ProCapNet and PRO-cap track references. refs #35528 Regenerated both reference sections with getTrackReferences --doi, keeping the citation order each page already had rather than the tool's alphabetical order, so the Cochran and Tome primary papers stay first. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/trackDb/human/trackDb.ra
  • lines changed 5, context: html, text, full: html, text
    d75c53ececee5030dfea41a3ffe6f8b24065947c Mon Sep 21 05:13:45 2026 -0700
    trackDb: the five dead iframeUrl settings, refs #37595 The PDB ones framed http://www.pdb.org/pdb/explore, which is plain http on an https page and now redirects to a port that does not answer. Point them at https://www.rcsb.org/structure/, the URL wuhCor1 already uses. The four NCBI ones cannot be framed at all - nuccore sends X-Frame-Options: SAMEORIGIN - so they become a url instead, which gives the Outside Link that these tracks never had. While testing, the GenBank Chains details page on hg19 turned out to abort with "Input to %-s should be created with safe functions" before printing anything: its idInUrlSql used %-s. Quoted %s instead, and the page renders again.
• src/hg/makeDb/trackDb/human/transcriptionStart.html
  • lines changed 100, context: html, text, full: html, text
    7713a08da69691ba499d5b9d44379c43e8cdb608 Fri Sep 18 09:27:52 2026 -0700
    Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528 New superTrack transcriptionStart in the rna group, holding two faceted composites: encode4ProCap with PRO-cap measurements and proCapNet with the model predictions and sequence-contribution scores. hg38 has all three data types, hs1 the predictions only. ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy. For each of the six experiments only the plus and minus strand signal of unique reads files of that experiment's default analysis are taken, which drops files superseded by a later reprocessing. ENCODE publishes no pooled file, so the per-replicate files are summed per cell line and strand, the same merge the ProCapNet models were trained on. Total signal is conserved exactly. The published ProCapNet prediction bigWigs store one bedGraph interval per base and hold a literal NaN at every unresolved (N) base on hg38, which makes autoScale and every summary statistic NaN. They are re-encoded into fixedStep sections, a third smaller with no value changed, dropping 164,268,582 NaN bases of 3,088,269,832 on hg38 and none on hs1. Losslessness verified against the originals on random windows across five chromosomes. The composites are faceted rather than plain because a container multiWig under a plain composite is flattened away by hgTrackUi and never drawn. Each cell line is one row with a checkbox per data type, a Sample class facet, ENCODE accession links and a Files column linking each bigWig on hgdownload. Scripts and the cell line configuration are in makeDb/outside/proCapNet; the trackDb stanzas and the faceted metadata tables are generated, not hand edited. Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5
• src/hg/makeDb/trackDb/mouse/mm10/mouseStrainsCactus.html
  • lines changed 43, context: html, text, full: html, text
    8296c9b7908b7fd73c70b6c7af21c337c55558f4 Mon Sep 21 04:39:42 2026 -0700
    Wording and formatting fixes to the mouseStrainsCactus description page and the AVI news entry, per CR feedback. refs #38351 mouseStrainsCactus.html: use the standard "Display Conventions and Configuration" heading, spell misassemblies without the hyphen, hyphenate large-scale, add the missing commas before "and" in two sentences, finish the Data Access sentence about the API track name, and put the references in alphabetical order by first author. newsarch.html: correct the track group name to "Phenotypes, Variants, and Literature", hyphenate genome-wide, and add the missing "on" in the figure caption.
• src/hg/makeDb/trackDb/relatedTracks.ra
  • lines changed 1, context: html, text, full: html, text
    fec6f772c9613d436f976c315673bfb5f4ce3262 Thu Sep 17 14:45:58 2026 -0700
    relatedTracks.ra: put back the hg19 ensGene entry, refs #38016 The audit commit dropped three ensGene lines but only justified two. Ensembl Genes is retired on hg38 and mm10, but it is still a live track on hg19: the table has 204,940 rows and hgTrackUi?db=hg19&g=ensGene renders on the RR. knownGene is public on hg19 too, so the entry met the commit's own test and should have stayed. Its removal took a working related-track pointer off the hg19 Ensembl Genes page. Found in the v504 code review, #38354. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 4, context: html, text, full: html, text
    29af14b47208040441ab4ff7acf51aac71cad4e2 Thu Sep 17 21:58:18 2026 -0700
    Adding MaveMD track, clinically calibrated multiplexed variant effect measurements on hg38. refs #37800 MaveMD is a curated collection inside MaveDB holding the score sets with clinical relevance, restricted to genes with a moderate or stronger gene-disease association. Data come from the MaveDB API rather than the Zenodo snapshots, at the request of the MaveDB team, who also set the request pacing the fetch script follows. Two views under a new phenDis container. mavemdVar draws one item per measured variant per score set, carrying the assay score, the functional class, the ACMG/AMP functional evidence code and strength where the score set is calibrated, the assay metadata, and matching ClinVar and gnomAD annotations. mavemdMap draws each score set as a variant effect map, one column per amino acid position and one row per substitution, reusing Jonathan's heatmap display from the MaveDB track. MaveDB resolves about a third of the collection to the genome and the rest only to a protein sequence, so variants are placed from the genomic HGVS term where one exists, otherwise by projecting the protein term onto its codon through ncbiRefSeqLink and ncbiRefSeqCurated for RefSeq accessions or the GENCODE tables for Ensembl ones, otherwise by running the submitted transcript term through hgvsToVcf. The projection is cross-checked against the variants carrying both coordinate systems and the build aborts if they disagree beyond a threshold. Codons split across an exon junction are written as BED12 with the two real blocks. Haplotypes cannot be given a single position and are excluded, with every dropped measurement counted by reason in the makeDoc. Also adds reciprocal relatedTracks entries between mavedb and mavemd. Gated alpha pending QA.
  • lines changed 11, context: html, text, full: html, text
    97c7de50efd34497c0e3f926f9afeaf9bbe45392 Mon Sep 21 09:10:37 2026 -0700
    Name the assay on every MaveMD map, and add ClinVar and gnomAD as related tracks. refs #37800 Two maps of the same gene routinely disagree, because they measured different things: PTEN abundance against PTEN lipid phosphatase activity, GCK activity against GCK abundance, KCNE1 trafficking with and without KCNQ1. Of the variants measured by more than one score set, 17% get opposite calls, and nothing on the map said why. The assay now travels with the map instead of sitting a click away on the details page. The legend above each matrix names the assay rather than repeating the same colour key on all 84 maps, and every cell mouseover ends with the same line. Method and model system come first because they are short and always present; the score set title is the part that truncates. The separator is a plain hyphen, since the legend is drawn as raster text and an HTML entity would appear literally there. relatedTracks.ra gains one-way links from mavemd to clinvar and gnomadVariants. MaveMD ships a ClinVar and gnomAD snapshot taken from the MaveDB API, and its calibrations were computed against that snapshot, so the imported values stay; the links point readers at our always-current tracks. One-way because MaveMD is too narrow to earn a line on two of the most heavily used tracks we have.
• src/hg/makeDb/trackDb/tagTypes.tab
  • lines changed 1, context: html, text, full: html, text
    98255f58460415345af88e45ad233599740585e5 Wed Sep 16 11:44:41 2026 -0700
    trackDb docs for the denseClick setting, refs #38364 Documents the setting added in the previous commit, in the five places a new trackDb statement has to appear: tagTypes.tab, trackDbLibrary.shtml, trackDbDoc.html, trackDbHub.v3.html (level-new, since it works in hubs) and changes.html. The type list is bed, bigBed, genePred, bigGenePred, psl and bigPsl, which is the same list decorator carries and matches what the code reaches: the setting hooks genericDrawItemsFullDense, so it applies to anything drawn through the linked-features path. Each type was checked by hand on hg38 in dense with the setting on, counting details links in the image map: ccdsGene, mrna, mrnaBig and knownGene all gained one link per item. The blurb says what happens when several items share a pixel, which is that the first one wins, and that a row drawn from summary data rather than from items is unaffected. That last case is real: linkedFeaturesDraw calls bigBedDrawDense when tg->items is NULL, so a zoomed-out bigBed never reaches this code and has nothing to link. Verified with "make settings", which parses the blurb cleanly, and with hubCheck -settings against the sandbox copy of trackDbHub.v3.html, which reports "denseClick new" and no errors. Regenerating trackDbSettings.yaml and .json also picks up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented on 2026-09-09 without rebuilding the generated files. Nothing is removed.
  • lines changed 2, context: html, text, full: html, text
    157d7eb9f2765db7651e12c14233c56fa7205e96 Thu Sep 17 14:46:12 2026 -0700
    VCF help: add the two settings the sweep missed, and fix a value order, refs #38010 The earlier commit left out sampleMetadataFile and showHardyWeinberg on the grounds that nothing in the tree reads them. Both are read: hgc/vcfClick.c loads the metadata file in vcfGenotypeTable, and reads showHardyWeinberg with cartOrTdbBoolean in vcfGenotypesDetails. Neither call is gated on track type, so both also apply to vcfPhasedTrio, which tagTypes.tab had not allowed; that is corrected here too. sampleColorFile and showHardyWeinberg had no entry in trackDbLibrary.shtml, so they were missing from the trackDb doc pages as well. Both get a blurb, a row in trackDbDoc.html and trackDbHub.v3.html, and a changes.html note. sampleColorFile is the last VCF setting that was recognized but undocumented. vcf.html and the three trackDb doc files listed vcfPhasedColorBy as mendelDiff|deNovo|function|noColor. vcfUi.c prints the radio buttons in the opposite order, so the lists now read noColor|function|deNovo|mendelDiff, the way hapClusterColorBy already matches its own buttons. trackDbSettings.yaml and .json are regenerated. They also pick up minAc, vcfDoMinAc and vcfPhasedColorBy, which were documented earlier in this series without a regen. Found in the v504 code review, #38354. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/makeDb/trackDb/uniprot.ra
  • lines changed 12, context: html, text, full: html, text
    9541aea4204c8079ea2e464403f2225f6aa33cfd Mon Sep 14 07:59:00 2026 -0700
    uniprot: show the splice variant track, and filter the CAT alignments Two things that were quietly missing. unipSplice has been built since the pipeline rewrite in #19351 and never had a trackDb stanza - "git log -S unipSplice" on uniprot.ra returns nothing, so it was never wired up rather than deliberately dropped. It holds UniProt's splice variant features, 28962 of them on hg38 and 29195 on hs1, and has been invisible on every assembly for years. Added to uniprot.ra and to the archive/contrib template, which is where the hub-served assemblies get their trackDb. The file exists on the same 131 assemblies as unipDomain.bb, so no stanza points at anything missing. The alignments on a CAT assembly were not being filtered with pslSelect. That filter maps a UniProt accession to the transcripts UniProt cross-references for it, and the README is blunt about how much it matters for protein families with nearly identical transcripts. It understood two kinds of id, Ensembl and RefSeq, and CAT names a transcript after its source gene, so hs1 matched neither and fell through unfiltered. Every row of the CAT bigBed carries the Ensembl transcript it was lifted from, so catSourceTransMap joins on that and UniProt's Ensembl cross-reference does the rest. The mapping had to become one-to-many for this: one Ensembl transcript can name several of ours, both because paralogs are lifted from the same source and because duplicate CAT names were given -dup suffixes earlier. It reproduces those suffixes by walking the bigBed in the same order rather than attaching every paralog to every source, which measured 234903 of 234903 hs1 transcripts mapped, no duplicates, against 330749 for the loose version. For every other gene track the lists hold one element and the behaviour is unchanged, which is checked: a single-valued entry still writes exactly one pair line and still counts a version difference, a multi-valued one writes a line per transcript, and an id we do not have is still skipped so pslSelect -qPass passes it through. refs #38300
• src/hg/makeDb/trackDb/zebrafish/danRer11/danioCode.html
  • lines changed 39, context: html, text, full: html, text
    aac682d3e89d0197a6467ae5231d3fe979f06d87 Tue Sep 15 05:30:48 2026 -0700
    danioCode: un-nest the standalone data types, split CRISPR out of DC Conservation Give RNA-seq, CAGE-seq, 3P-seq, ChIP-seq and Hi-C their own top-level track in an existing group (rna/genes/regulation) instead of hiding all eleven DANIO-CODE containers behind one superTrack that only someone already looking for DANIO-CODE would open. The Burgess lab's CRISPR/Cas9 target-site tracks, which were riding along inside the "DC Conservation" composite, get their own top-level superTrack (dcCrispr, group map) instead, mirroring where hg38 keeps its own unrelated crispr tracks, and drop the "DC" branding since they aren't DANIO-CODE's data. The rest -- a mixed bag of regulatory-element/annotation containers that don't map onto any one existing group -- stay nested under the danioCode superTrack. Implemented in danioCodeHubToRa.py (STANDALONE_GROUP/NESTED_ORDER/CRISPR_*) so the regrouping survives the next hub re-import, then regenerated danioCode.ra from the cached hub trackDb.txt. tdbQuery -check passes; still alpha only. refs #38265
  • lines changed 35, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/danioCode.ra
  • lines changed 6262, context: html, text, full: html, text
    aac682d3e89d0197a6467ae5231d3fe979f06d87 Tue Sep 15 05:30:48 2026 -0700
    danioCode: un-nest the standalone data types, split CRISPR out of DC Conservation Give RNA-seq, CAGE-seq, 3P-seq, ChIP-seq and Hi-C their own top-level track in an existing group (rna/genes/regulation) instead of hiding all eleven DANIO-CODE containers behind one superTrack that only someone already looking for DANIO-CODE would open. The Burgess lab's CRISPR/Cas9 target-site tracks, which were riding along inside the "DC Conservation" composite, get their own top-level superTrack (dcCrispr, group map) instead, mirroring where hg38 keeps its own unrelated crispr tracks, and drop the "DC" branding since they aren't DANIO-CODE's data. The rest -- a mixed bag of regulatory-element/annotation containers that don't map onto any one existing group -- stay nested under the danioCode superTrack. Implemented in danioCodeHubToRa.py (STANDALONE_GROUP/NESTED_ORDER/CRISPR_*) so the regrouping survives the next hub re-import, then regenerated danioCode.ra from the cached hub trackDb.txt. tdbQuery -check passes; still alpha only. refs #38265
  • lines changed 71, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dc3PseqComposite.html
  • lines changed 7, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcCAGEseqComposite.html
  • lines changed 6, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcChIPseqComposite.html
  • lines changed 10, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcComp.html
  • lines changed 6, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcComp_cell_type.html
  • lines changed 5, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcComparativeGenomics.html
  • lines changed 16, context: html, text, full: html, text
    aac682d3e89d0197a6467ae5231d3fe979f06d87 Tue Sep 15 05:30:48 2026 -0700
    danioCode: un-nest the standalone data types, split CRISPR out of DC Conservation Give RNA-seq, CAGE-seq, 3P-seq, ChIP-seq and Hi-C their own top-level track in an existing group (rna/genes/regulation) instead of hiding all eleven DANIO-CODE containers behind one superTrack that only someone already looking for DANIO-CODE would open. The Burgess lab's CRISPR/Cas9 target-site tracks, which were riding along inside the "DC Conservation" composite, get their own top-level superTrack (dcCrispr, group map) instead, mirroring where hg38 keeps its own unrelated crispr tracks, and drop the "DC" branding since they aren't DANIO-CODE's data. The rest -- a mixed bag of regulatory-element/annotation containers that don't map onto any one existing group -- stay nested under the danioCode superTrack. Implemented in danioCodeHubToRa.py (STANDALONE_GROUP/NESTED_ORDER/CRISPR_*) so the regrouping survives the next hub re-import, then regenerated danioCode.ra from the cached hub trackDb.txt. tdbQuery -check passes; still alpha only. refs #38265
  • lines changed 17, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcConsensus_promoters.html
  • lines changed 6, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcCopes_and_dopes.html
  • lines changed 8, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcCrispr.html
  • lines changed 87, context: html, text, full: html, text
    aac682d3e89d0197a6467ae5231d3fe979f06d87 Tue Sep 15 05:30:48 2026 -0700
    danioCode: un-nest the standalone data types, split CRISPR out of DC Conservation Give RNA-seq, CAGE-seq, 3P-seq, ChIP-seq and Hi-C their own top-level track in an existing group (rna/genes/regulation) instead of hiding all eleven DANIO-CODE containers behind one superTrack that only someone already looking for DANIO-CODE would open. The Burgess lab's CRISPR/Cas9 target-site tracks, which were riding along inside the "DC Conservation" composite, get their own top-level superTrack (dcCrispr, group map) instead, mirroring where hg38 keeps its own unrelated crispr tracks, and drop the "DC" branding since they aren't DANIO-CODE's data. The rest -- a mixed bag of regulatory-element/annotation containers that don't map onto any one existing group -- stay nested under the danioCode superTrack. Implemented in danioCodeHubToRa.py (STANDALONE_GROUP/NESTED_ORDER/CRISPR_*) so the regrouping survives the next hub re-import, then regenerated danioCode.ra from the cached hub trackDb.txt. tdbQuery -check passes; still alpha only. refs #38265
  • lines changed 13, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcEvalidation.html
  • lines changed 6, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcHiC_Composite.html
  • lines changed 7, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/makeDb/trackDb/zebrafish/danRer11/dcRNAseqComposite.html
  • lines changed 7, context: html, text, full: html, text
    73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
    danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were getting long, but the full name matters more than brevity here). Relabel the danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types, COPEs DOPEs, Enhancers, Promoters). Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics) into its own top-level track, "Burgess Fish PhastCons", group compGeno -- same treatment as the CRISPR tracks pulled out earlier, since this isn't DANIO-CODE's own data either. Rewrote every description page's intro: dropped the internal link to the superTrack in favor of a line crediting the DANIO-CODE project and linking to https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer, and dropped the general explainer paragraphs (what non-coding regulation is, what conservation is) from danioCode.html and the conservation page entirely. Turn on exactly what the hub itself already marked "on": one RNA-seq sample (Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own "visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite) stays off by default. tdbQuery -check passes; still alpha only. refs #38265
• src/hg/oneShot/methbaseDownload/methbaseDownload
  • lines changed 538, context: html, text, full: html, text
    662ca09c0ea0923d5fbcec4e75346b907a5e367d Thu Sep 17 06:00:55 2026 -0700
    methbaseDownload: mirror the MethBase2 hub and keep it up to date, refs #34246 #Preview2 week - bugs introduced now will need a build patch to fix The hub's bigDataUrls point into a tree that is not under the hub directory, so hubClone cannot mirror it: -download flattens every file into one directory per assembly, and it only skips files that already exist, so it never notices a file that changed upstream. This script keeps the remote layout, downloads in parallel, and re-fetches a file only when its size on the server differs from ours, so the same command does the first mirror and every later update. It rewrites the bigDataUrls to our copy on hgdownload, skips an assembly whose trackDb is missing rather than aborting, and appends a short per-run record to download.log.
• src/hg/pyLib/hgLib3.py
  • lines changed 318, context: html, text, full: html, text
    c62c5810fffbf329f4a2555d72de0b56fb4752a4 Thu Sep 17 06:37:59 2026 -0700
    pyLib: align the python bottleneck code with hg/lib/botDelay.c, refs #38369 #Preview2 week - bugs introduced now will need a build patch to fix hgGeneGraph is the only CGI using this library. Its bottleneck handling had drifted a long way from the C implementation, so bring it back in line: - check the hguid cookie against the userDb table before using it as the bottleneck key, and fall back to the hgsid and then the address, as getBotCheckString() in botDelay.c does. Honours newBotDelay. - port recordHguidIpAndMaybeForceCaptcha() and its hguidIpTracking.* settings, so the same hg.conf keys configure the C and the python side. - run the bottleneck before deciding whether to serve the request, matching the order in earlyBotCheck(). Skip the whole thing for bottleneck.except. - when the tracking trips, redirect to hgTracks, which runs the same check and can put up the Cloudflare challenge itself. New helpers, all ports of their C namesakes: sqlUpdate(), sqlQuickNum(), cfgOptionBooleanDefault(), cfgOptionEnvDefault(), userDbTable(), sessionDbTable(), cartDbParseId(), cartDbHasSessionKey(), isValidHguid(), isValidHgsidForEarlyBotCheck(), botException(), cgiWasSpoofed() and hConnectCentralNoCache(). pymysql does not autocommit, so sqlUpdate() commits. Also in passing: - sqlTableExists() was missing its return and so was always false. - makeRandomKey() used float division and produced a 32 character key with base64 padding where the C version produces 28 characters. - cartDbLoadFromId() called cgi.escape(), gone from python since 3.8. - cfgOptionBoolean() read the config dict without the parse guard cfgOption has. - a two statement string literal in hgBotDelay() discarded half its message.
• src/hg/utils/automation/kegAlign.json.ga
  • lines changed 161, context: html, text, full: html, text
    d362457564bc038b9ecf253a0e6ff1f86222e0fa Sat Sep 19 08:45:09 2026 -0700
    toolsets updated to fix memory leak in chainAntiRepeat refs #31811
• src/hg/utils/chainToBigChain/chainToBigChain.c
  • lines changed 48, context: html, text, full: html, text
    72327e4e2d294b9d2fcb6cbdadd4e628c006345b Fri Sep 18 10:07:40 2026 -0700
    bigChain: move the chain to bigChain conversion into the library chainRecToBigChain() and chainBlockToBigLink() sat in chainToBigChain.c next to its main(), so a CGI could not link against them. Move both into hg/lib/bigChain.c, add chainToBigChainOne() for one chain and chainToBigChainList() for a list, and leave the utility as a thin main(). No behavior change; the chainToBigChain test output is unchanged. refs #38382
• src/hg/utils/docent/README.md
  • lines changed 1, context: html, text, full: html, text
    7e89607805cac22b52143b23a45464759e66a957 Wed Sep 16 12:36:15 2026 -0700
    docent: a login: step, with the account kept per hgcentral, refs #37892 hgCollection refuses a visitor who is not signed in -- hgCollection.c doMiddle, "You must be logged in to edit collections" -- and so does the saving half of hgSession. The suite has never had a logged-in page, and until now could not: the login cookie is validated against a salted hash (login.cookieSalt, hg/lib/wikiLink.c), so there is no cookie to hand the browser. A script that needs one of those pages has to sign in the way a person does. `login:` does that, through hgLogin's own form, and it takes no credentials and cannot be given any. They come from ~/.docentLogin (DOCENT_LOGIN_FILE overrides, DOCENT_LOGIN_USER + DOCENT_LOGIN_PASSWORD override both for one run), which is refused unless it is mode 0600 -- the rule hg/lib/hgConfig.c applies to hg.conf, for the same reason. Nothing prints a password. THE FILE IS KEYED BY HGCENTRAL DATABASE, not by server. An account is a row in gbMembers in one central, the way a named session is: genome-test, hgwdev, every hgwdev-<name> sandbox and every ticket park read hgcentraltest and share one account, while hgwbeta reads hgcentralbeta and the RR reads hgcentral. So the file is one [hgcentraltest] section rather than a section per sandbox. Which central a server reads is READ from its hg.conf, following its includes the way hgConfig.c does, and not guessed from the host -- because a sandbox can point itself somewhere else and two on hgwdev do today: of the personal confs there, 45 set central.db=hgcentraltest, one sets hgcentralgsid and one hgcentralbeta. A server whose conf is on another machine falls back to a small table (the RR, the two mirrors, hgwbeta), and [default] catches the rest. A run redirected with DOCENT_TARGET looks up the server it is really driving. Two failures the step has to tell apart, and both cost a run to find: A WRONG PASSWORD IS A PERFECTLY GOOD PAGE. hgLogin answers one by drawing the same form again with a red message, so a step that just navigated on would leave every later step running logged out and the failure would surface somewhere else entirely. The step fails on #accountLoginForm still being there, and quotes what the page said. A RIGHT PASSWORD ARRIVES MID-REDIRECT. hgLogin answers one with a page that navigates ITSELF a moment later: returnToURL(150) at hgLogin.c:1160 writes setTimeout(function(){location=...}, 150). Returning while that timer is pending means the next step's goto: races it and the browser aborts one of the two, which arrives as a bare net::ERR_ABORTED on a URL that is completely fine. The step now waits for that redirect to land. The failure path is checked first, since that page never leaves hgLogin and there is no redirect to wait for. preflight reports the account as a fixture -- which central it resolved, which section it came from, and why it is unusable if it is -- so a missing password is caught before the browser starts. It attempts no login: a wrong password fails loudly at the step itself, which is the one thing preflight cannot do for it. The hg.conf reader is now in both docent.js and tests/preflight.js, beside the resolveTarget each of them already carries. Both say to keep the other in step. If that second copy is a copy too many, the two of them want a shared module.
  • lines changed 7, context: html, text, full: html, text
    9b762c146f7a44a3bdc783cdd202356fc163b8fb Wed Sep 16 12:43:32 2026 -0700
    docent: one targetConf.js for the target, the hg.conf, the central and the account, refs #37892 docent.js and tests/preflight.js had grown a second and then a third copy of the same lookups. That is exactly the pair that must not drift: preflight checks the fixtures for the server the RUN will drive, so a run that resolved its target, its hg.conf, its hgcentral or its account even slightly differently would be checked against the wrong machine, and the mismatch would show up as a green preflight in front of a red suite. targetConf.js answers the four questions in order, each from the one before: resolveTarget a `target:` (or DOCENT_TARGET) -> the .../cgi-bin URL to drive hgConfFor that URL -> the hg.conf it reads, if the server is on this box centralDbFor that conf -> which hgcentral it uses loginLookup that central -> the account to sign in with Nothing in it opens a browser or the network, which is why preflight can ask all four in the seconds before a run. loginLookup returns the facts plus, when the account cannot be used, ONE sentence saying why. That sentence is the substantive half and is now written once: the step throws it with a `login:` prefix, preflight prints it on the MISSING line. Each caller still phrases its own success line, since one logs and the other prints a fixture row. No password crosses that boundary to anything that prints. 248 lines net out of the two programs, 197 into the module. docent.js is no longer a single file, and three places now say so: its own require, the Run section of README.md, and docent.mk, whose mp4 rule gains targetConf.js as a prerequisite -- a change there changes what a tour renders, so it has to rebuild one. Nothing in the tree or in ~/docentTours copies docent.js; they all reference it where it sits, with targetConf.js beside it. Measured before and after, with no other change: preflight resolves the same account, central and conf for genome-test, hgwdev-braney, a ts park on 48087 and hgwbeta (which correctly has no section); tests/ is 18 of 18 with the derive baselines matching; tests/regress preflights 14 fixtures for 67 scripts.
  • lines changed 35, context: html, text, full: html, text
    b4e78d426dcdb47a20f1979025d5a0969ea79ac4 Thu Sep 17 17:07:49 2026 -0700
    docent: a box: check for where an element sits, and lists for text:/noText:, refs #37892 `expect:` could say what was in a page and never where it was on the screen. has:/noHas: take a CSS selector, which describes the tree; color: reads pixels but only inside a track's row. #38251 moved the narrow-window menu icon out of the blue bar with every selector still matching and every word of the page still there, and nothing in the language could ask about it. box: takes `inside:` (every edge within another element's box, with `tolerance:` px of slack), `clear:` (no overlap with anything a selector matches, `gap:` for a minimum separation) and `height:`/`width:`. Every element `sel:` matches has to satisfy every clause, so {sel: "ul.nice-menu > li", inside: "#main-menu-whole"} reads as "every menu item is in the bar". Boxes are read in document coordinates; an element with no box at all is skipped rather than treated as a zero-sized box at the origin, which would sit "inside" anything. A failure prints the measurement: #topRightLinks is not inside #main-menu-whole: 114px above it -- it is at 666,0 34x32, #main-menu-whole at 0,114 1000x32 The image height: and box's own height:/width: now share one cmpSize(), so the two cannot drift into different comparison grammars. text: and noText: take a list, the way rows:, has: and noHas: always did. They have to: a list handed to a check that stringifies its argument fails OPEN -- ["a", "b"] becomes "a,b", which no page contains, so it passes on anything, and passes silently. Six scripts in one batch were written that way and all six looked green. pagechecks and its .xfail twin cover both, the xfail with one entry aimed wrongly and one aimed rightly in each list, so it fails only if every entry is really looked at on its own.
  • lines changed 4, context: html, text, full: html, text
    4265f36510c9498df6d7b0a8ff9e5251350c133c Sun Sep 20 14:34:09 2026 -0700
    docent: read a graph's tooltip, and noTip: for a bare number, refs #37892 hgTracks has two tooltips and their ids differ only in the case of one letter. hg/js/utils.js hangs an ITEM's tooltip off `#mouseoverContainer`: the text of a map box's title, an rsID or a gene name. A WIGGLE reports itself the other way. The `mouseOver` module in hg/js/hgTracks.js writes the value under the cursor into `#mouseOverText`, read from the per-pixel spans hgTracks leaves in a trash .json. A track drawn as a coverage graph has no map boxes at all, so its numbers exist only in the second one, and `tip:` on such a track read an empty string and then waited out its timeout. Every reader of the live tooltip now takes whichever is up: `tip:`, the `mouseover:` waits, `pinShot:` and the shot clip. Two more from the same mistake. `mouseover: {value: ...}` is documented for wiggles and had never matched anything on any track: it looked in `window.mapData.spans[...]` for a member called `value`, which is hg/js/mouseOver.js, an older copy of the module the page does not load, and the member there is `v` as well. It now reads `mouseOver.items[...]` and places the cursor from `td_data_<track>`, which is what hgTracks.js itself compares the cursor against. And asking by value now skips the map boxes entirely, because a track's own center label is titled "Click to alter the display density of <track>": a probe for the value 3 hovered the center label of a track called rm38253stairs before any span was looked at. `noTip:` is new. `tip:` is a substring test, which is right for an item, whose tooltip is markup and whose tail can render differently from the title it came from. It is wrong for a graph, whose tooltip is the number and nothing else: `tip: "1"` is also satisfied by "1.5" and by "13", and a mean computed with the wrong divisor is exactly the fraction that would slip through. Naming the other digits and the decimal point leaves one number. This is the prefix trap #38279 hit with two messages sharing a head, in a place with no closing paren to carry the delimiter.
• src/hg/utils/docent/docent.js
  • lines changed 5, context: html, text, full: html, text
    cad0600bd259bae35de7b39e49c2f95af8b624ca Mon Sep 14 16:14:40 2026 -0700
    docent: point a whole test directory at another server with TARGET, refs #37892 A script says where it runs with `target:`, and until now that was the only way to say it, so trying a suite against a branch build meant editing every script in the directory. DOCENT_TARGET overrides `target:` for the run, and docentTest.mk turns a TARGET= variable into it for preflight, test, parity and derive: make test TARGET=hgwdev-demo9 make preflight TARGET=hgwdev-demo9 It takes the same values `target:` does -- a shorthand, a bare hgwdev-<name> sandbox or demo, or a full .../cgi-bin URL -- so a ts park works too. preflight.js reads the same variable, so the fixture check names the server that will actually be driven rather than the one the scripts name. The trackDb cache already keys on the resolved server, so a redirected run cannot read back a listing fetched from somewhere else. The committed scripts keep their own `target:`. Redirecting is for trying a suite elsewhere, not for moving it: the nightly reads what is in the file. Both READMEs say so, and say to read a redirected failure with the other server's trackDb in mind, since a script asserts what its own server draws. Verified against the ten methbase scripts: all ten pass with TARGET=hgwdev-demo9, and mbMouse still passes with no TARGET, against the RR.
  • lines changed 179, context: html, text, full: html, text
    7e89607805cac22b52143b23a45464759e66a957 Wed Sep 16 12:36:15 2026 -0700
    docent: a login: step, with the account kept per hgcentral, refs #37892 hgCollection refuses a visitor who is not signed in -- hgCollection.c doMiddle, "You must be logged in to edit collections" -- and so does the saving half of hgSession. The suite has never had a logged-in page, and until now could not: the login cookie is validated against a salted hash (login.cookieSalt, hg/lib/wikiLink.c), so there is no cookie to hand the browser. A script that needs one of those pages has to sign in the way a person does. `login:` does that, through hgLogin's own form, and it takes no credentials and cannot be given any. They come from ~/.docentLogin (DOCENT_LOGIN_FILE overrides, DOCENT_LOGIN_USER + DOCENT_LOGIN_PASSWORD override both for one run), which is refused unless it is mode 0600 -- the rule hg/lib/hgConfig.c applies to hg.conf, for the same reason. Nothing prints a password. THE FILE IS KEYED BY HGCENTRAL DATABASE, not by server. An account is a row in gbMembers in one central, the way a named session is: genome-test, hgwdev, every hgwdev-<name> sandbox and every ticket park read hgcentraltest and share one account, while hgwbeta reads hgcentralbeta and the RR reads hgcentral. So the file is one [hgcentraltest] section rather than a section per sandbox. Which central a server reads is READ from its hg.conf, following its includes the way hgConfig.c does, and not guessed from the host -- because a sandbox can point itself somewhere else and two on hgwdev do today: of the personal confs there, 45 set central.db=hgcentraltest, one sets hgcentralgsid and one hgcentralbeta. A server whose conf is on another machine falls back to a small table (the RR, the two mirrors, hgwbeta), and [default] catches the rest. A run redirected with DOCENT_TARGET looks up the server it is really driving. Two failures the step has to tell apart, and both cost a run to find: A WRONG PASSWORD IS A PERFECTLY GOOD PAGE. hgLogin answers one by drawing the same form again with a red message, so a step that just navigated on would leave every later step running logged out and the failure would surface somewhere else entirely. The step fails on #accountLoginForm still being there, and quotes what the page said. A RIGHT PASSWORD ARRIVES MID-REDIRECT. hgLogin answers one with a page that navigates ITSELF a moment later: returnToURL(150) at hgLogin.c:1160 writes setTimeout(function(){location=...}, 150). Returning while that timer is pending means the next step's goto: races it and the browser aborts one of the two, which arrives as a bare net::ERR_ABORTED on a URL that is completely fine. The step now waits for that redirect to land. The failure path is checked first, since that page never leaves hgLogin and there is no redirect to wait for. preflight reports the account as a fixture -- which central it resolved, which section it came from, and why it is unusable if it is -- so a missing password is caught before the browser starts. It attempts no login: a wrong password fails loudly at the step itself, which is the one thing preflight cannot do for it. The hg.conf reader is now in both docent.js and tests/preflight.js, beside the resolveTarget each of them already carries. Both say to keep the other in step. If that second copy is a copy too many, the two of them want a shared module.
  • lines changed 157, context: html, text, full: html, text
    9b762c146f7a44a3bdc783cdd202356fc163b8fb Wed Sep 16 12:43:32 2026 -0700
    docent: one targetConf.js for the target, the hg.conf, the central and the account, refs #37892 docent.js and tests/preflight.js had grown a second and then a third copy of the same lookups. That is exactly the pair that must not drift: preflight checks the fixtures for the server the RUN will drive, so a run that resolved its target, its hg.conf, its hgcentral or its account even slightly differently would be checked against the wrong machine, and the mismatch would show up as a green preflight in front of a red suite. targetConf.js answers the four questions in order, each from the one before: resolveTarget a `target:` (or DOCENT_TARGET) -> the .../cgi-bin URL to drive hgConfFor that URL -> the hg.conf it reads, if the server is on this box centralDbFor that conf -> which hgcentral it uses loginLookup that central -> the account to sign in with Nothing in it opens a browser or the network, which is why preflight can ask all four in the seconds before a run. loginLookup returns the facts plus, when the account cannot be used, ONE sentence saying why. That sentence is the substantive half and is now written once: the step throws it with a `login:` prefix, preflight prints it on the MISSING line. Each caller still phrases its own success line, since one logs and the other prints a fixture row. No password crosses that boundary to anything that prints. 248 lines net out of the two programs, 197 into the module. docent.js is no longer a single file, and three places now say so: its own require, the Run section of README.md, and docent.mk, whose mp4 rule gains targetConf.js as a prerequisite -- a change there changes what a tour renders, so it has to rebuild one. Nothing in the tree or in ~/docentTours copies docent.js; they all reference it where it sits, with targetConf.js beside it. Measured before and after, with no other change: preflight resolves the same account, central and conf for genome-test, hgwdev-braney, a ts park on 48087 and hgwbeta (which correctly has no section); tests/ is 18 of 18 with the derive baselines matching; tests/regress preflights 14 fixtures for 67 scripts.
  • lines changed 145, context: html, text, full: html, text
    b4e78d426dcdb47a20f1979025d5a0969ea79ac4 Thu Sep 17 17:07:49 2026 -0700
    docent: a box: check for where an element sits, and lists for text:/noText:, refs #37892 `expect:` could say what was in a page and never where it was on the screen. has:/noHas: take a CSS selector, which describes the tree; color: reads pixels but only inside a track's row. #38251 moved the narrow-window menu icon out of the blue bar with every selector still matching and every word of the page still there, and nothing in the language could ask about it. box: takes `inside:` (every edge within another element's box, with `tolerance:` px of slack), `clear:` (no overlap with anything a selector matches, `gap:` for a minimum separation) and `height:`/`width:`. Every element `sel:` matches has to satisfy every clause, so {sel: "ul.nice-menu > li", inside: "#main-menu-whole"} reads as "every menu item is in the bar". Boxes are read in document coordinates; an element with no box at all is skipped rather than treated as a zero-sized box at the origin, which would sit "inside" anything. A failure prints the measurement: #topRightLinks is not inside #main-menu-whole: 114px above it -- it is at 666,0 34x32, #main-menu-whole at 0,114 1000x32 The image height: and box's own height:/width: now share one cmpSize(), so the two cannot drift into different comparison grammars. text: and noText: take a list, the way rows:, has: and noHas: always did. They have to: a list handed to a check that stringifies its argument fails OPEN -- ["a", "b"] becomes "a,b", which no page contains, so it passes on anything, and passes silently. Six scripts in one batch were written that way and all six looked green. pagechecks and its .xfail twin cover both, the xfail with one entry aimed wrongly and one aimed rightly in each list, so it fails only if every entry is really looked at on its own.
  • lines changed 40, context: html, text, full: html, text
    7603e0192dd9fdf257f0a57b500d4e936cd42e93 Sat Sep 19 09:45:41 2026 -0700
    docent: drop the one-frame screenshot artifact from a recorded tour, refs #37892 #38364 A shot: taken while the video is recording can leave one bad frame in the mp4, the track image painted on a grey, unpainted page, because Playwright's screenshot captures from the same surface the recorder reads. It is a race: four renders of one script gave 2, 1, 1 and 0 of them, and a sweep of the 46 finished tours under ~/docentTours found the frame in 24, 33 frames in all. Capturing through CDP instead does keep the recorder out of it, but Chromium then ignores the clip and answers with the whole viewport, so every still would need cropping and a taller-than-viewport image could not be captured at all. Preferring page.screenshot({clip}) over an element screenshot does not help either, and shifts the still by a pixel in each dimension. Both measured. So the frame is dropped in the transcode that was going to happen anyway: the time of each shot is remembered, the webm is scanned once for a frame more than 20 luma darker than both its neighbours, and only a dip within half a second of a shot is cut. One encode, stills untouched, and a tour with no artifact runs exactly the command it ran before. The shot-time gate is what makes it safe -- an ordinary dark frame in a tour is never next to a capture. Measured: the #38364 tour twice, found and dropped at 3.68s and 3.72s, mp4 clean afterwards, and its three stills byte-identical to the stock renderer's. FAST mode is untouched, since it returns before the transcode.
  • lines changed 98, context: html, text, full: html, text
    4265f36510c9498df6d7b0a8ff9e5251350c133c Sun Sep 20 14:34:09 2026 -0700
    docent: read a graph's tooltip, and noTip: for a bare number, refs #37892 hgTracks has two tooltips and their ids differ only in the case of one letter. hg/js/utils.js hangs an ITEM's tooltip off `#mouseoverContainer`: the text of a map box's title, an rsID or a gene name. A WIGGLE reports itself the other way. The `mouseOver` module in hg/js/hgTracks.js writes the value under the cursor into `#mouseOverText`, read from the per-pixel spans hgTracks leaves in a trash .json. A track drawn as a coverage graph has no map boxes at all, so its numbers exist only in the second one, and `tip:` on such a track read an empty string and then waited out its timeout. Every reader of the live tooltip now takes whichever is up: `tip:`, the `mouseover:` waits, `pinShot:` and the shot clip. Two more from the same mistake. `mouseover: {value: ...}` is documented for wiggles and had never matched anything on any track: it looked in `window.mapData.spans[...]` for a member called `value`, which is hg/js/mouseOver.js, an older copy of the module the page does not load, and the member there is `v` as well. It now reads `mouseOver.items[...]` and places the cursor from `td_data_<track>`, which is what hgTracks.js itself compares the cursor against. And asking by value now skips the map boxes entirely, because a track's own center label is titled "Click to alter the display density of <track>": a probe for the value 3 hovered the center label of a track called rm38253stairs before any span was looked at. `noTip:` is new. `tip:` is a substring test, which is right for an item, whose tooltip is markup and whose tail can render differently from the title it came from. It is wrong for a graph, whose tooltip is the number and nothing else: `tip: "1"` is also satisfied by "1.5" and by "13", and a mean computed with the wrong divisor is exactly the fraction that would slip through. Naming the other digits and the decimal point leaves one number. This is the prefix trap #38279 hit with two messages sharing a head, in a place with no closing paren to carry the delimiter.
• src/hg/utils/docent/docent.mk
  • lines changed 4, context: html, text, full: html, text
    9b762c146f7a44a3bdc783cdd202356fc163b8fb Wed Sep 16 12:43:32 2026 -0700
    docent: one targetConf.js for the target, the hg.conf, the central and the account, refs #37892 docent.js and tests/preflight.js had grown a second and then a third copy of the same lookups. That is exactly the pair that must not drift: preflight checks the fixtures for the server the RUN will drive, so a run that resolved its target, its hg.conf, its hgcentral or its account even slightly differently would be checked against the wrong machine, and the mismatch would show up as a green preflight in front of a red suite. targetConf.js answers the four questions in order, each from the one before: resolveTarget a `target:` (or DOCENT_TARGET) -> the .../cgi-bin URL to drive hgConfFor that URL -> the hg.conf it reads, if the server is on this box centralDbFor that conf -> which hgcentral it uses loginLookup that central -> the account to sign in with Nothing in it opens a browser or the network, which is why preflight can ask all four in the seconds before a run. loginLookup returns the facts plus, when the account cannot be used, ONE sentence saying why. That sentence is the substantive half and is now written once: the step throws it with a `login:` prefix, preflight prints it on the MISSING line. Each caller still phrases its own success line, since one logs and the other prints a fixture row. No password crosses that boundary to anything that prints. 248 lines net out of the two programs, 197 into the module. docent.js is no longer a single file, and three places now say so: its own require, the Run section of README.md, and docent.mk, whose mp4 rule gains targetConf.js as a prerequisite -- a change there changes what a tour renders, so it has to rebuild one. Nothing in the tree or in ~/docentTours copies docent.js; they all reference it where it sits, with targetConf.js beside it. Measured before and after, with no other change: preflight resolves the same account, central and conf for genome-test, hgwdev-braney, a ts park on 48087 and hgwbeta (which correctly has no section); tests/ is 18 of 18 with the derive baselines matching; tests/regress preflights 14 fixtures for 67 scripts.
  • lines changed 4, context: html, text, full: html, text
    e9253ccecb8b2bbfe42f0d1e77b1db7e2a50642e Sat Sep 19 14:10:28 2026 -0700
    docent: let FIGDIR really decide where an mp4 lands, refs #37892 docent.mk documents FIGDIR as an override for where videos go, and make looks for its target there, but the recipe ran `node docent.js <script>` with no output path, so docent.js fell back to its own default: the script directory's parent. Make then never saw the file it was waiting for, and rebuilt every time. It matters now because the tours moved to the genecats repository while their output stays in ~/docentTours. Without this the video lands next to the scripts, inside the repo. docent.js has taken the path as its third argument all along; the recipe just never passed it.
• src/hg/utils/docent/targetConf.js
  • lines changed 197, context: html, text, full: html, text
    9b762c146f7a44a3bdc783cdd202356fc163b8fb Wed Sep 16 12:43:32 2026 -0700
    docent: one targetConf.js for the target, the hg.conf, the central and the account, refs #37892 docent.js and tests/preflight.js had grown a second and then a third copy of the same lookups. That is exactly the pair that must not drift: preflight checks the fixtures for the server the RUN will drive, so a run that resolved its target, its hg.conf, its hgcentral or its account even slightly differently would be checked against the wrong machine, and the mismatch would show up as a green preflight in front of a red suite. targetConf.js answers the four questions in order, each from the one before: resolveTarget a `target:` (or DOCENT_TARGET) -> the .../cgi-bin URL to drive hgConfFor that URL -> the hg.conf it reads, if the server is on this box centralDbFor that conf -> which hgcentral it uses loginLookup that central -> the account to sign in with Nothing in it opens a browser or the network, which is why preflight can ask all four in the seconds before a run. loginLookup returns the facts plus, when the account cannot be used, ONE sentence saying why. That sentence is the substantive half and is now written once: the step throws it with a `login:` prefix, preflight prints it on the MISSING line. Each caller still phrases its own success line, since one logs and the other prints a fixture row. No password crosses that boundary to anything that prints. 248 lines net out of the two programs, 197 into the module. docent.js is no longer a single file, and three places now say so: its own require, the Run section of README.md, and docent.mk, whose mp4 rule gains targetConf.js as a prerequisite -- a change there changes what a tour renders, so it has to rebuild one. Nothing in the tree or in ~/docentTours copies docent.js; they all reference it where it sits, with targetConf.js beside it. Measured before and after, with no other change: preflight resolves the same account, central and conf for genome-test, hgwdev-braney, a ts park on 48087 and hgwbeta (which correctly has no section); tests/ is 18 of 18 with the derive baselines matching; tests/regress preflights 14 fixtures for 67 scripts.
• src/hg/utils/docent/tests/README.txt
  • lines changed 17, context: html, text, full: html, text
    cad0600bd259bae35de7b39e49c2f95af8b624ca Mon Sep 14 16:14:40 2026 -0700
    docent: point a whole test directory at another server with TARGET, refs #37892 A script says where it runs with `target:`, and until now that was the only way to say it, so trying a suite against a branch build meant editing every script in the directory. DOCENT_TARGET overrides `target:` for the run, and docentTest.mk turns a TARGET= variable into it for preflight, test, parity and derive: make test TARGET=hgwdev-demo9 make preflight TARGET=hgwdev-demo9 It takes the same values `target:` does -- a shorthand, a bare hgwdev-<name> sandbox or demo, or a full .../cgi-bin URL -- so a ts park works too. preflight.js reads the same variable, so the fixture check names the server that will actually be driven rather than the one the scripts name. The trackDb cache already keys on the resolved server, so a redirected run cannot read back a listing fetched from somewhere else. The committed scripts keep their own `target:`. Redirecting is for trying a suite elsewhere, not for moving it: the nightly reads what is in the file. Both READMEs say so, and say to read a redirected failure with the other server's trackDb in mind, since a script asserts what its own server draws. Verified against the ten methbase scripts: all ten pass with TARGET=hgwdev-demo9, and mbMouse still passes with no TARGET, against the RR.
  • lines changed 24, context: html, text, full: html, text
    d08323bb2ff524e4964a0922ecfb8336ab662007 Wed Sep 16 10:25:56 2026 -0700
    docent: two tests for gaps a cart-visibility branch went past, refs #37892 The #37547 branch broke quickLift and nine scripts in tests/regress caught it without help. Two other things went past the whole suite, and these are the tests for them. heavysession is selftest on a cart with weight behind it. selftest saves a session and loads it back, which is the right shape, but the cart it round-trips holds two rows. Saving a cart is not a copy of it: outIfNotPresent() in hg/hgSession/hgSession.c writes a trackDb default for every track that is deliberately NOT in the cart, so the file says what is hidden as well as what is shown, and a two-track cart barely reaches that code. A save-and-reload path that returned a 38-row clinical session as 34 rows left selftest green. The weight comes from a Recommended Track Set, which is where a clinical user starts: View/Clinical_SNVs_hg38, out of DOCUMENT_ROOT/data/recTrackSets/recTrackSets.hg38.tab. It draws 33 tracks and the ruler, and the file the session: step writes out of that cart is 741 settings, 248 of them visibilities and 154 of those hide. Both halves of the trip assert the row set with exact: true, because rows: alone would pass on a reload that lost four of them, and a count cannot say which row went missing. It is a saved session on the server, so make preflight already checks it is still there -- a deleted one answers 200 with a page that has no track image, on which every noText: check passes. firstrequest is about a bug that lags by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and the next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. So this one asserts with no go:, open: or convert: between the track: step and the check. Which track it names is the other half, and it is not free choice. A top-level track passes on a build with the bug, because hgTracks adds every top-level track as a lightweight stub so the track controls can list it -- microsat, gtexGene and windowmaskerSdust all drew. A default-visible child passes too, because its container is in the list already: wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack and same request. So the script names wgEncodeRegMarkH3k4me1, which is visibility hide under a superTrack that is itself superTrack on hide, and the hideKids step before it is what stops the other members coming back at their own trackDb visibility. expected/firstrequest.derive is committed with it, and it is not decoration. The test only means anything while the step under test is ONE round: step 5 track {"wgEncodeRegMarkH3k4me1":"full"} round 1 (2 vars): wgEncodeRegMarkH3k4me1=full wgEncodeReg=show If Docent ever splits that the way it splits a container-plus-subtrack-hide, the browser test would go on passing and stop being able to see the bug. The baseline says so out loud. README.txt describes both, and adds hgCollection to "Still to write": no script in either directory reaches that CGI, and hg/hgCollection/hgCollection.c carries its own verbatim copy of isParentVisible() from hg/lib/trackHub.c. Nine scripts caught the trackHub.c copy on the branch; nothing caught this one. Covering it needs a collection: verb, since tracks go into a collection by dragging between two jsTrees and drag: is the genomic drag-select on the track image. Seventeen of seventeen pass in tests/, including the four xfails.
  • lines changed 27, context: html, text, full: html, text
    3d98068ead92a95ea4b714f31f01119100a3c099 Wed Sep 16 10:26:18 2026 -0700
    docent: preflight prints how the target is configured, refs #37892 make test TARGET=hgwdev-you points the whole directory at another server, and a red script there can be that machine's configuration rather than a bug. A different curatedHubPrefix genuinely changes what a quickLift hop produces, and a db.trackDb with a private table in front of the shared one changes what exact: true counts. Both failures look exactly like a bug. The README already warned about it, in prose, which arrives after someone has spent an hour. make preflight now prints the settings that decide it, so a redirected run carries its own explanation in the log: target https://hgwdev-braney.gi.ucsc.edu/cgi-bin /usr/local/apache/cgi-bin-braney/hg.conf central.db hgcentraltest db.trackDb trackDb_braney,trackDb curatedHubPrefix braney browser.quickLift on browser.quickLiftAlignments on browser.recTrackSets on central.db is on the list because preflight already checks named sessions and that setting says which hgcentral it looked in. The value printed is the EFFECTIVE one. The reader follows include and delete the way hg/lib/hgConfig.c does, with a later assignment winning over an earlier one, so a sandbox conf that sets nothing still shows what it inherits from the shared conf it includes. Reading cgi-bin-braney/hg.conf on its own says browser.quickLiftAlignments is unset; the CGI there sees it on, from the include on line 2. A ts park is the mirror case: 37547's frozen conf sets db.trackDb twice and the second one, 114 lines later, wins. Only the six keys in HG_CONF_KEYS are ever printed. The includes lead to hg.conf.private, which holds database passwords, so the reader parses whatever it is pointed at and prints nothing off that list. There is no way to ask a browser over http what its hg.conf says, so which file to read is a lookup by convention: genome-test and hgwdev read /usr/local/apache/cgi-bin/hg.conf, an hgwdev-<name> sandbox or demo reads cgi-bin-<name>, and a ts park on 127.0.0.1 is found by port in its ports.tsv. Anything off this machine -- hgwbeta, the RR -- says the conf cannot be read from here, which is true and better than a guess. Even with that in the log, a config difference and a code difference can still look alike. The reliable way to tell them apart is to swap only the binary: drop a control build's CGIs into the same sandbox, leave its hg.conf alone, and re-run. If the failures follow the binary they are the code. That experiment is what turned nine plausible failures on the #37547 branch into nine proven ones, and both READMEs now say so. Verified against all three shapes: genome-test, hgwdev-braney, hgwdev-demo9, and the #37547 park on 127.0.0.1:48087. Both test directories preflight clean.
  • lines changed 12, context: html, text, full: html, text
    7e89607805cac22b52143b23a45464759e66a957 Wed Sep 16 12:36:15 2026 -0700
    docent: a login: step, with the account kept per hgcentral, refs #37892 hgCollection refuses a visitor who is not signed in -- hgCollection.c doMiddle, "You must be logged in to edit collections" -- and so does the saving half of hgSession. The suite has never had a logged-in page, and until now could not: the login cookie is validated against a salted hash (login.cookieSalt, hg/lib/wikiLink.c), so there is no cookie to hand the browser. A script that needs one of those pages has to sign in the way a person does. `login:` does that, through hgLogin's own form, and it takes no credentials and cannot be given any. They come from ~/.docentLogin (DOCENT_LOGIN_FILE overrides, DOCENT_LOGIN_USER + DOCENT_LOGIN_PASSWORD override both for one run), which is refused unless it is mode 0600 -- the rule hg/lib/hgConfig.c applies to hg.conf, for the same reason. Nothing prints a password. THE FILE IS KEYED BY HGCENTRAL DATABASE, not by server. An account is a row in gbMembers in one central, the way a named session is: genome-test, hgwdev, every hgwdev-<name> sandbox and every ticket park read hgcentraltest and share one account, while hgwbeta reads hgcentralbeta and the RR reads hgcentral. So the file is one [hgcentraltest] section rather than a section per sandbox. Which central a server reads is READ from its hg.conf, following its includes the way hgConfig.c does, and not guessed from the host -- because a sandbox can point itself somewhere else and two on hgwdev do today: of the personal confs there, 45 set central.db=hgcentraltest, one sets hgcentralgsid and one hgcentralbeta. A server whose conf is on another machine falls back to a small table (the RR, the two mirrors, hgwbeta), and [default] catches the rest. A run redirected with DOCENT_TARGET looks up the server it is really driving. Two failures the step has to tell apart, and both cost a run to find: A WRONG PASSWORD IS A PERFECTLY GOOD PAGE. hgLogin answers one by drawing the same form again with a red message, so a step that just navigated on would leave every later step running logged out and the failure would surface somewhere else entirely. The step fails on #accountLoginForm still being there, and quotes what the page said. A RIGHT PASSWORD ARRIVES MID-REDIRECT. hgLogin answers one with a page that navigates ITSELF a moment later: returnToURL(150) at hgLogin.c:1160 writes setTimeout(function(){location=...}, 150). Returning while that timer is pending means the next step's goto: races it and the browser aborts one of the two, which arrives as a bare net::ERR_ABORTED on a URL that is completely fine. The step now waits for that redirect to land. The failure path is checked first, since that page never leaves hgLogin and there is no redirect to wait for. preflight reports the account as a fixture -- which central it resolved, which section it came from, and why it is unusable if it is -- so a missing password is caught before the browser starts. It attempts no login: a wrong password fails loudly at the step itself, which is the one thing preflight cannot do for it. The hg.conf reader is now in both docent.js and tests/preflight.js, beside the resolveTarget each of them already carries. Both say to keep the other in step. If that second copy is a copy too many, the two of them want a shared module.
  • lines changed 20, context: html, text, full: html, text
    4d991e3416f4ac769408dfd2f65d8fdb0f7fae9b Wed Sep 16 12:36:41 2026 -0700
    docent: a test for hgCollection, the one CGI nothing reached, refs #37892 hgCollection shares visibility logic with hgTracks by COPY rather than by call. hg/hgCollection/hgCollection.c carries its own isParentVisible() at line 269, a verbatim copy of the one in hg/lib/trackHub.c. Nine scripts in tests/regress caught the trackHub.c copy on the #37547 cart-visibility branch; nothing caught this one, and it was found by grep afterwards. This is the test that would have. The copy's reach is narrower than it first looks, and the script says so: its only caller is checkForVisible(), which builds the "Visible Tracks" folder at the top of the builder's source tree. It is NOT on the save path, so the bug drops a track you have on in the browser out of that shortcut folder and leaves a saved collection alone. The assertion sits on the FOLDER'S OWN jsTree CLASS, which turned out to be a cleaner statement of the bug than counting what is inside it. addVisibleTracks writes `,children:true` into the node only when checkForVisible() found something, so the folder renders jstree-open when the cart has visible tracks and jstree-leaf when it does not. The leaves inside it are then named as well. Three things the live page settled, each of which had been a guess: * The folder arrives OPEN. Clicking it CLOSES it. The first draft clicked. * A node's id is the track name, and the same id is used for that track under its group folder, so both selectors are scoped to li#visible. Unscoped, they would also match the copy under Regulation the moment anyone opened it. * wgEncodeBroadHistoneGm12878H3k4me1StdSig does not follow its siblings' naming. It is named in the assertion for that reason: a rename shows up here rather than quietly halving what this checks. The track cannot be chosen freely. It has to be a leaf whose container is hidden by default, or the test passes on a broken build (the rule in regress/README.txt: a top-level track is added as a stub whatever the cart says, and a default-visible child has its container in the list already). It also has to be one hgCollection will show at all -- trackCanBeAdded() at line 141 keeps only wig, bigWig and bedGraph leaves, which is why this names a cell-line leaf and not the multiWig container above it. wgEncodeRegMarkH3k4me1H1hesc is both. The noHas: names wgEncodeRegMarkH3k27acH1hesc, whose container IS visible by default: it says the hideKids reached hgCollection, so the has: is not passing because the whole superTrack came up on its own. The wait had to be restructured, and the reason generalises. Waiting for the folder's CHILDREN meant that a build with the bug -- where the folder is empty -- timed out after 15 seconds with a Playwright message instead of failing at the expect: with what it wanted. Waiting for either terminal state (jstree-open or jstree-leaf) fixes it: measured on genome-test the folder settles about 30ms after load whichever way it goes, and jstree-open lands about 11ms before the children, which is why the child check gets its own wait after the class has been asserted. Measured both ways. The same script with the track hidden fails in about a second: step 9 (expect) failed: nothing matches "li#visible.jstree-open"; 1 element(s) match "li#visible.jstree-leaf", wanted none README.txt describes it, drops hgCollection from "Still to write", and records what that section should now hold instead: a THIRD copy of isParentVisible(), in hg/lib/hgFind.c at line 2957, feeding isTrackVisible() at 2977. It decides whether a search result lands under "Visible Tracks" or "Hidden Tracks" on hgSearch, it is the same bug in the same shape, and it needs no login. 18 of 18 pass in tests/, and preflight resolves all four fixtures.
  • lines changed 13, context: html, text, full: html, text
    2e5054b09838bacf29b31bb64fbd234471e7cb71 Wed Sep 16 13:00:17 2026 -0700
    docent: a test for hgFind's copy of isParentVisible, on hgSearch, refs #37892 The third and last of the three. isParentVisible() exists in the tree character for character three times -- hg/lib/trackHub.c:1818, hg/hgCollection/hgCollection.c:269 and hg/lib/hgFind.c:2957 -- and each copy asks the same question, are this track's containers visible, by reading the cart itself rather than going through the accessor. Nine scripts in regress/ cover the first, collection.docent.yaml covers the second, and this covers the third. A change to how visibility is stored has to reach all three, or two of them quietly start answering from the trackDb default. What this copy decides: isTrackVisible() at hgFind.c:2977 sets category->visibility, and hgSearch groups its results by it -- "Visible Tracks" when it is set, "Currently Hidden Tracks" when it is not (hgSearch.c:174, js/hgSearch.js:44). The symptom is a search result for a track you have on in the browser filed under the hidden heading, where nobody looking for it will open it. This one needs NO LOGIN, which is why it is worth having even with the other two covered: it is the cheap one, about eight seconds. wgEncodeGencodeBasicV49 is the track because it satisfies both constraints at once. It is in hgFindSpec, so it has search results at all; and its chain is wgEncodeGencodeV49ViewGenes -> wgEncodeGencodeV49 (visibility 0) -> wgEncodeGencodeSuper, which is `superTrack on` with no `show`, so isShow is FALSE and a build that reads the cart wrongly stops the walk at the superTrack. A top-level or default-visible track would have passed on the broken build. The hideKids step is not tidiness. Showing the superTrack brings every other archived version up at its own visibility -- V50 draws alongside and lands in Visible Tracks too -- so without it the `exact:` would need rewriting every time GENCODE ships a version. It costs 31 variables, one per archived composite. ENST00000297261 rather than a gene symbol: one of SHH's transcripts, it reaches the GENCODE sets, and it searches 65 categories where SHH searches 319. No count is asserted anywhere -- every one of those numbers moves when a track is reloaded. Same wait discipline as collection.docent.yaml, and for the same reason: wait for EITHER heading, so a build that files the track on the wrong side fails at the expect: naming the selector it wanted rather than timing out 15 seconds later with a Playwright message. Measured both ways. The same script with the track left hidden fails at the assertion: step 7 (expect) failed: nothing matches "li[id="Visible Tracks"] li#wgEncodeGencodeBasicV49" No derive baseline on purpose: the hideKids expansion is one variable per archived GENCODE version, so a baseline would go red twice a year for a reason that is not a bug. views is left out of expected/ for the same reason. tests/ is 19 of 19 with four fixtures resolved and the derive baselines matching.
  • lines changed 14, context: html, text, full: html, text
    b4e78d426dcdb47a20f1979025d5a0969ea79ac4 Thu Sep 17 17:07:49 2026 -0700
    docent: a box: check for where an element sits, and lists for text:/noText:, refs #37892 `expect:` could say what was in a page and never where it was on the screen. has:/noHas: take a CSS selector, which describes the tree; color: reads pixels but only inside a track's row. #38251 moved the narrow-window menu icon out of the blue bar with every selector still matching and every word of the page still there, and nothing in the language could ask about it. box: takes `inside:` (every edge within another element's box, with `tolerance:` px of slack), `clear:` (no overlap with anything a selector matches, `gap:` for a minimum separation) and `height:`/`width:`. Every element `sel:` matches has to satisfy every clause, so {sel: "ul.nice-menu > li", inside: "#main-menu-whole"} reads as "every menu item is in the bar". Boxes are read in document coordinates; an element with no box at all is skipped rather than treated as a zero-sized box at the origin, which would sit "inside" anything. A failure prints the measurement: #topRightLinks is not inside #main-menu-whole: 114px above it -- it is at 666,0 34x32, #main-menu-whole at 0,114 1000x32 The image height: and box's own height:/width: now share one cmpSize(), so the two cannot drift into different comparison grammars. text: and noText: take a list, the way rows:, has: and noHas: always did. They have to: a list handed to a check that stringifies its argument fails OPEN -- ["a", "b"] becomes "a,b", which no page contains, so it passes on anything, and passes silently. Six scripts in one batch were written that way and all six looked green. pagechecks and its .xfail twin cover both, the xfail with one entry aimed wrongly and one aimed rightly in each list, so it fails only if every entry is really looked at on its own.
• src/hg/utils/docent/tests/collection.docent.yaml
  • lines changed 91, context: html, text, full: html, text
    4d991e3416f4ac769408dfd2f65d8fdb0f7fae9b Wed Sep 16 12:36:41 2026 -0700
    docent: a test for hgCollection, the one CGI nothing reached, refs #37892 hgCollection shares visibility logic with hgTracks by COPY rather than by call. hg/hgCollection/hgCollection.c carries its own isParentVisible() at line 269, a verbatim copy of the one in hg/lib/trackHub.c. Nine scripts in tests/regress caught the trackHub.c copy on the #37547 cart-visibility branch; nothing caught this one, and it was found by grep afterwards. This is the test that would have. The copy's reach is narrower than it first looks, and the script says so: its only caller is checkForVisible(), which builds the "Visible Tracks" folder at the top of the builder's source tree. It is NOT on the save path, so the bug drops a track you have on in the browser out of that shortcut folder and leaves a saved collection alone. The assertion sits on the FOLDER'S OWN jsTree CLASS, which turned out to be a cleaner statement of the bug than counting what is inside it. addVisibleTracks writes `,children:true` into the node only when checkForVisible() found something, so the folder renders jstree-open when the cart has visible tracks and jstree-leaf when it does not. The leaves inside it are then named as well. Three things the live page settled, each of which had been a guess: * The folder arrives OPEN. Clicking it CLOSES it. The first draft clicked. * A node's id is the track name, and the same id is used for that track under its group folder, so both selectors are scoped to li#visible. Unscoped, they would also match the copy under Regulation the moment anyone opened it. * wgEncodeBroadHistoneGm12878H3k4me1StdSig does not follow its siblings' naming. It is named in the assertion for that reason: a rename shows up here rather than quietly halving what this checks. The track cannot be chosen freely. It has to be a leaf whose container is hidden by default, or the test passes on a broken build (the rule in regress/README.txt: a top-level track is added as a stub whatever the cart says, and a default-visible child has its container in the list already). It also has to be one hgCollection will show at all -- trackCanBeAdded() at line 141 keeps only wig, bigWig and bedGraph leaves, which is why this names a cell-line leaf and not the multiWig container above it. wgEncodeRegMarkH3k4me1H1hesc is both. The noHas: names wgEncodeRegMarkH3k27acH1hesc, whose container IS visible by default: it says the hideKids reached hgCollection, so the has: is not passing because the whole superTrack came up on its own. The wait had to be restructured, and the reason generalises. Waiting for the folder's CHILDREN meant that a build with the bug -- where the folder is empty -- timed out after 15 seconds with a Playwright message instead of failing at the expect: with what it wanted. Waiting for either terminal state (jstree-open or jstree-leaf) fixes it: measured on genome-test the folder settles about 30ms after load whichever way it goes, and jstree-open lands about 11ms before the children, which is why the child check gets its own wait after the class has been asserted. Measured both ways. The same script with the track hidden fails in about a second: step 9 (expect) failed: nothing matches "li#visible.jstree-open"; 1 element(s) match "li#visible.jstree-leaf", wanted none README.txt describes it, drops hgCollection from "Still to write", and records what that section should now hold instead: a THIRD copy of isParentVisible(), in hg/lib/hgFind.c at line 2957, feeding isTrackVisible() at 2977. It decides whether a search result lands under "Visible Tracks" or "Hidden Tracks" on hgSearch, it is the same bug in the same shape, and it needs no login. 18 of 18 pass in tests/, and preflight resolves all four fixtures.
• src/hg/utils/docent/tests/docentTest.mk
  • lines changed 14, context: html, text, full: html, text
    cad0600bd259bae35de7b39e49c2f95af8b624ca Mon Sep 14 16:14:40 2026 -0700
    docent: point a whole test directory at another server with TARGET, refs #37892 A script says where it runs with `target:`, and until now that was the only way to say it, so trying a suite against a branch build meant editing every script in the directory. DOCENT_TARGET overrides `target:` for the run, and docentTest.mk turns a TARGET= variable into it for preflight, test, parity and derive: make test TARGET=hgwdev-demo9 make preflight TARGET=hgwdev-demo9 It takes the same values `target:` does -- a shorthand, a bare hgwdev-<name> sandbox or demo, or a full .../cgi-bin URL -- so a ts park works too. preflight.js reads the same variable, so the fixture check names the server that will actually be driven rather than the one the scripts name. The trackDb cache already keys on the resolved server, so a redirected run cannot read back a listing fetched from somewhere else. The committed scripts keep their own `target:`. Redirecting is for trying a suite elsewhere, not for moving it: the nightly reads what is in the file. Both READMEs say so, and say to read a redirected failure with the other server's trackDb in mind, since a script asserts what its own server draws. Verified against the ten methbase scripts: all ten pass with TARGET=hgwdev-demo9, and mbMouse still passes with no TARGET, against the RR.
• src/hg/utils/docent/tests/expected/firstrequest.derive
  • lines changed 5, context: html, text, full: html, text
    d08323bb2ff524e4964a0922ecfb8336ab662007 Wed Sep 16 10:25:56 2026 -0700
    docent: two tests for gaps a cart-visibility branch went past, refs #37892 The #37547 branch broke quickLift and nine scripts in tests/regress caught it without help. Two other things went past the whole suite, and these are the tests for them. heavysession is selftest on a cart with weight behind it. selftest saves a session and loads it back, which is the right shape, but the cart it round-trips holds two rows. Saving a cart is not a copy of it: outIfNotPresent() in hg/hgSession/hgSession.c writes a trackDb default for every track that is deliberately NOT in the cart, so the file says what is hidden as well as what is shown, and a two-track cart barely reaches that code. A save-and-reload path that returned a 38-row clinical session as 34 rows left selftest green. The weight comes from a Recommended Track Set, which is where a clinical user starts: View/Clinical_SNVs_hg38, out of DOCUMENT_ROOT/data/recTrackSets/recTrackSets.hg38.tab. It draws 33 tracks and the ruler, and the file the session: step writes out of that cart is 741 settings, 248 of them visibilities and 154 of those hide. Both halves of the trip assert the row set with exact: true, because rows: alone would pass on a reload that lost four of them, and a count cannot say which row went missing. It is a saved session on the server, so make preflight already checks it is still there -- a deleted one answers 200 with a page that has no track image, on which every noText: check passes. firstrequest is about a bug that lags by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and the next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. So this one asserts with no go:, open: or convert: between the track: step and the check. Which track it names is the other half, and it is not free choice. A top-level track passes on a build with the bug, because hgTracks adds every top-level track as a lightweight stub so the track controls can list it -- microsat, gtexGene and windowmaskerSdust all drew. A default-visible child passes too, because its container is in the list already: wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack and same request. So the script names wgEncodeRegMarkH3k4me1, which is visibility hide under a superTrack that is itself superTrack on hide, and the hideKids step before it is what stops the other members coming back at their own trackDb visibility. expected/firstrequest.derive is committed with it, and it is not decoration. The test only means anything while the step under test is ONE round: step 5 track {"wgEncodeRegMarkH3k4me1":"full"} round 1 (2 vars): wgEncodeRegMarkH3k4me1=full wgEncodeReg=show If Docent ever splits that the way it splits a container-plus-subtrack-hide, the browser test would go on passing and stop being able to see the bug. The baseline says so out loud. README.txt describes both, and adds hgCollection to "Still to write": no script in either directory reaches that CGI, and hg/hgCollection/hgCollection.c carries its own verbatim copy of isParentVisible() from hg/lib/trackHub.c. Nine scripts caught the trackHub.c copy on the branch; nothing caught this one. Covering it needs a collection: verb, since tracks go into a collection by dragging between two jsTrees and drag: is the genomic drag-select on the track image. Seventeen of seventeen pass in tests/, including the four xfails.
• src/hg/utils/docent/tests/firstrequest.docent.yaml
  • lines changed 52, context: html, text, full: html, text
    d08323bb2ff524e4964a0922ecfb8336ab662007 Wed Sep 16 10:25:56 2026 -0700
    docent: two tests for gaps a cart-visibility branch went past, refs #37892 The #37547 branch broke quickLift and nine scripts in tests/regress caught it without help. Two other things went past the whole suite, and these are the tests for them. heavysession is selftest on a cart with weight behind it. selftest saves a session and loads it back, which is the right shape, but the cart it round-trips holds two rows. Saving a cart is not a copy of it: outIfNotPresent() in hg/hgSession/hgSession.c writes a trackDb default for every track that is deliberately NOT in the cart, so the file says what is hidden as well as what is shown, and a two-track cart barely reaches that code. A save-and-reload path that returned a 38-row clinical session as 34 rows left selftest green. The weight comes from a Recommended Track Set, which is where a clinical user starts: View/Clinical_SNVs_hg38, out of DOCUMENT_ROOT/data/recTrackSets/recTrackSets.hg38.tab. It draws 33 tracks and the ruler, and the file the session: step writes out of that cart is 741 settings, 248 of them visibilities and 154 of those hide. Both halves of the trip assert the row set with exact: true, because rows: alone would pass on a reload that lost four of them, and a count cannot say which row went missing. It is a saved session on the server, so make preflight already checks it is still there -- a deleted one answers 200 with a page that has no track image, on which every noText: check passes. firstrequest is about a bug that lags by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and the next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. So this one asserts with no go:, open: or convert: between the track: step and the check. Which track it names is the other half, and it is not free choice. A top-level track passes on a build with the bug, because hgTracks adds every top-level track as a lightweight stub so the track controls can list it -- microsat, gtexGene and windowmaskerSdust all drew. A default-visible child passes too, because its container is in the list already: wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack and same request. So the script names wgEncodeRegMarkH3k4me1, which is visibility hide under a superTrack that is itself superTrack on hide, and the hideKids step before it is what stops the other members coming back at their own trackDb visibility. expected/firstrequest.derive is committed with it, and it is not decoration. The test only means anything while the step under test is ONE round: step 5 track {"wgEncodeRegMarkH3k4me1":"full"} round 1 (2 vars): wgEncodeRegMarkH3k4me1=full wgEncodeReg=show If Docent ever splits that the way it splits a container-plus-subtrack-hide, the browser test would go on passing and stop being able to see the bug. The baseline says so out loud. README.txt describes both, and adds hgCollection to "Still to write": no script in either directory reaches that CGI, and hg/hgCollection/hgCollection.c carries its own verbatim copy of isParentVisible() from hg/lib/trackHub.c. Nine scripts caught the trackHub.c copy on the branch; nothing caught this one. Covering it needs a collection: verb, since tracks go into a collection by dragging between two jsTrees and drag: is the genomic drag-select on the track image. Seventeen of seventeen pass in tests/, including the four xfails.
• src/hg/utils/docent/tests/heavysession.docent.yaml
  • lines changed 72, context: html, text, full: html, text
    d08323bb2ff524e4964a0922ecfb8336ab662007 Wed Sep 16 10:25:56 2026 -0700
    docent: two tests for gaps a cart-visibility branch went past, refs #37892 The #37547 branch broke quickLift and nine scripts in tests/regress caught it without help. Two other things went past the whole suite, and these are the tests for them. heavysession is selftest on a cart with weight behind it. selftest saves a session and loads it back, which is the right shape, but the cart it round-trips holds two rows. Saving a cart is not a copy of it: outIfNotPresent() in hg/hgSession/hgSession.c writes a trackDb default for every track that is deliberately NOT in the cart, so the file says what is hidden as well as what is shown, and a two-track cart barely reaches that code. A save-and-reload path that returned a 38-row clinical session as 34 rows left selftest green. The weight comes from a Recommended Track Set, which is where a clinical user starts: View/Clinical_SNVs_hg38, out of DOCUMENT_ROOT/data/recTrackSets/recTrackSets.hg38.tab. It draws 33 tracks and the ruler, and the file the session: step writes out of that cart is 741 settings, 248 of them visibilities and 154 of those hide. Both halves of the trip assert the row set with exact: true, because rows: alone would pass on a reload that lost four of them, and a count cannot say which row went missing. It is a saved session on the server, so make preflight already checks it is still there -- a deleted one answers 200 with a page that has no track image, on which every noText: check passes. firstrequest is about a bug that lags by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and the next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. So this one asserts with no go:, open: or convert: between the track: step and the check. Which track it names is the other half, and it is not free choice. A top-level track passes on a build with the bug, because hgTracks adds every top-level track as a lightweight stub so the track controls can list it -- microsat, gtexGene and windowmaskerSdust all drew. A default-visible child passes too, because its container is in the list already: wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack and same request. So the script names wgEncodeRegMarkH3k4me1, which is visibility hide under a superTrack that is itself superTrack on hide, and the hideKids step before it is what stops the other members coming back at their own trackDb visibility. expected/firstrequest.derive is committed with it, and it is not decoration. The test only means anything while the step under test is ONE round: step 5 track {"wgEncodeRegMarkH3k4me1":"full"} round 1 (2 vars): wgEncodeRegMarkH3k4me1=full wgEncodeReg=show If Docent ever splits that the way it splits a container-plus-subtrack-hide, the browser test would go on passing and stop being able to see the bug. The baseline says so out loud. README.txt describes both, and adds hgCollection to "Still to write": no script in either directory reaches that CGI, and hg/hgCollection/hgCollection.c carries its own verbatim copy of isParentVisible() from hg/lib/trackHub.c. Nine scripts caught the trackHub.c copy on the branch; nothing caught this one. Covering it needs a collection: verb, since tracks go into a collection by dragging between two jsTrees and drag: is the genomic drag-select on the track image. Seventeen of seventeen pass in tests/, including the four xfails.
• src/hg/utils/docent/tests/methbase/README.txt
  • lines changed 111, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
  • lines changed 10, context: html, text, full: html, text
    cad0600bd259bae35de7b39e49c2f95af8b624ca Mon Sep 14 16:14:40 2026 -0700
    docent: point a whole test directory at another server with TARGET, refs #37892 A script says where it runs with `target:`, and until now that was the only way to say it, so trying a suite against a branch build meant editing every script in the directory. DOCENT_TARGET overrides `target:` for the run, and docentTest.mk turns a TARGET= variable into it for preflight, test, parity and derive: make test TARGET=hgwdev-demo9 make preflight TARGET=hgwdev-demo9 It takes the same values `target:` does -- a shorthand, a bare hgwdev-<name> sandbox or demo, or a full .../cgi-bin URL -- so a ts park works too. preflight.js reads the same variable, so the fixture check names the server that will actually be driven rather than the one the scripts name. The trackDb cache already keys on the resolved server, so a redirected run cannot read back a listing fetched from somewhere else. The committed scripts keep their own `target:`. Redirecting is for trying a suite elsewhere, not for moving it: the nightly reads what is in the file. Both READMEs say so, and say to read a redirected failure with the other server's trackDb in mind, since a script asserts what its own server draws. Verified against the ten methbase scripts: all ten pass with TARGET=hgwdev-demo9, and mbMouse still passes with no TARGET, against the RR.
• src/hg/utils/docent/tests/methbase/makefile
  • lines changed 19, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/methbase/mbContainer.docent.yaml
  • lines changed 49, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/methbase/mbDefaults.docent.yaml
  • lines changed 61, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/methbase/mbDetails.docent.yaml
  • lines changed 53, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/methbase/mbLegacy.docent.yaml
  • lines changed 57, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/methbase/mbMatrix.docent.yaml
  • lines changed 63, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/methbase/mbMouse.docent.yaml
  • lines changed 51, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/methbase/mbPublicHub.docent.yaml
  • lines changed 41, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/methbase/mbSignal.docent.yaml
  • lines changed 48, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/methbase/mbSubtracks.docent.yaml
  • lines changed 47, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/methbase/mbZoomOut.docent.yaml
  • lines changed 48, context: html, text, full: html, text
    5f99585c343029387c5a58c7e64063eaf9b72409 Mon Sep 14 12:57:49 2026 -0700
    docent: ten tests for MethBase on the RR, refs #37892 MethBase on genome.ucsc.edu is two public hubs, not the Methbase faceted composite. That composite is on genome-test only: hgTrackUi?g=Methbase answers "Can't find Methbase in track database hg38" on both the RR and hgwbeta, and genome-mysql has no matching trackDb row for hg38 or mm39. What a user of the RR has is MethBase2 from smithlab.usc.edu, and the frozen v1 hub we serve from hgdownload. Nine of these scripts drive the first and one drives the second. The suite is a directory of its own rather than more scripts in tests/regress. Those are one per fixed bug, named for a ticket, and each carries a proof: key. These have no bug and no ticket, and they point at the RR rather than at genome-test, which tests/regress and its nightly assume throughout. So they carry no proof: key and they are not in the nightly. README.txt says what each one covers. Four of them assert a color. A remote hub whose bigDataUrl stops answering keeps its row, its name, its height and its map boxes: hgTracks catches the abort and paints the row as a 240,240,180 bar with the message inside the png, where no text check can reach it. That is the rm38310 lesson, except that here the thing that breaks is a URL on someone else's web server. Two constraints shaped the rest. A hub track's cart name carries a per-hub hub_<n>_ prefix that track: cannot write, so no script here uses track: and none names a hub number: rows match by suffix, settings pages are reached through a.trackLink[data-track$="..."], and a visibility is changed through the matrix buttons, which are named for the subgroup tag. A view's own dropdown is a select, and Docent has no verb that drives one, so no script turns on AMR, PMD or CpG reads. All ten pass against the RR, in 5m17s together.
• src/hg/utils/docent/tests/pagechecks.docent.yaml
  • lines changed 24, context: html, text, full: html, text
    b4e78d426dcdb47a20f1979025d5a0969ea79ac4 Thu Sep 17 17:07:49 2026 -0700
    docent: a box: check for where an element sits, and lists for text:/noText:, refs #37892 `expect:` could say what was in a page and never where it was on the screen. has:/noHas: take a CSS selector, which describes the tree; color: reads pixels but only inside a track's row. #38251 moved the narrow-window menu icon out of the blue bar with every selector still matching and every word of the page still there, and nothing in the language could ask about it. box: takes `inside:` (every edge within another element's box, with `tolerance:` px of slack), `clear:` (no overlap with anything a selector matches, `gap:` for a minimum separation) and `height:`/`width:`. Every element `sel:` matches has to satisfy every clause, so {sel: "ul.nice-menu > li", inside: "#main-menu-whole"} reads as "every menu item is in the bar". Boxes are read in document coordinates; an element with no box at all is skipped rather than treated as a zero-sized box at the origin, which would sit "inside" anything. A failure prints the measurement: #topRightLinks is not inside #main-menu-whole: 114px above it -- it is at 666,0 34x32, #main-menu-whole at 0,114 1000x32 The image height: and box's own height:/width: now share one cmpSize(), so the two cannot drift into different comparison grammars. text: and noText: take a list, the way rows:, has: and noHas: always did. They have to: a list handed to a check that stringifies its argument fails OPEN -- ["a", "b"] becomes "a,b", which no page contains, so it passes on anything, and passes silently. Six scripts in one batch were written that way and all six looked green. pagechecks and its .xfail twin cover both, the xfail with one entry aimed wrongly and one aimed rightly in each list, so it fails only if every entry is really looked at on its own.
• src/hg/utils/docent/tests/pagechecks.xfail.docent.yaml
  • lines changed 34, context: html, text, full: html, text
    b4e78d426dcdb47a20f1979025d5a0969ea79ac4 Thu Sep 17 17:07:49 2026 -0700
    docent: a box: check for where an element sits, and lists for text:/noText:, refs #37892 `expect:` could say what was in a page and never where it was on the screen. has:/noHas: take a CSS selector, which describes the tree; color: reads pixels but only inside a track's row. #38251 moved the narrow-window menu icon out of the blue bar with every selector still matching and every word of the page still there, and nothing in the language could ask about it. box: takes `inside:` (every edge within another element's box, with `tolerance:` px of slack), `clear:` (no overlap with anything a selector matches, `gap:` for a minimum separation) and `height:`/`width:`. Every element `sel:` matches has to satisfy every clause, so {sel: "ul.nice-menu > li", inside: "#main-menu-whole"} reads as "every menu item is in the bar". Boxes are read in document coordinates; an element with no box at all is skipped rather than treated as a zero-sized box at the origin, which would sit "inside" anything. A failure prints the measurement: #topRightLinks is not inside #main-menu-whole: 114px above it -- it is at 666,0 34x32, #main-menu-whole at 0,114 1000x32 The image height: and box's own height:/width: now share one cmpSize(), so the two cannot drift into different comparison grammars. text: and noText: take a list, the way rows:, has: and noHas: always did. They have to: a list handed to a check that stringifies its argument fails OPEN -- ["a", "b"] becomes "a,b", which no page contains, so it passes on anything, and passes silently. Six scripts in one batch were written that way and all six looked green. pagechecks and its .xfail twin cover both, the xfail with one entry aimed wrongly and one aimed rightly in each list, so it fails only if every entry is really looked at on its own.
• src/hg/utils/docent/tests/preflight.js
  • lines changed 3, context: html, text, full: html, text
    cad0600bd259bae35de7b39e49c2f95af8b624ca Mon Sep 14 16:14:40 2026 -0700
    docent: point a whole test directory at another server with TARGET, refs #37892 A script says where it runs with `target:`, and until now that was the only way to say it, so trying a suite against a branch build meant editing every script in the directory. DOCENT_TARGET overrides `target:` for the run, and docentTest.mk turns a TARGET= variable into it for preflight, test, parity and derive: make test TARGET=hgwdev-demo9 make preflight TARGET=hgwdev-demo9 It takes the same values `target:` does -- a shorthand, a bare hgwdev-<name> sandbox or demo, or a full .../cgi-bin URL -- so a ts park works too. preflight.js reads the same variable, so the fixture check names the server that will actually be driven rather than the one the scripts name. The trackDb cache already keys on the resolved server, so a redirected run cannot read back a listing fetched from somewhere else. The committed scripts keep their own `target:`. Redirecting is for trying a suite elsewhere, not for moving it: the nightly reads what is in the file. Both READMEs say so, and say to read a redirected failure with the other server's trackDb in mind, since a script asserts what its own server draws. Verified against the ten methbase scripts: all ten pass with TARGET=hgwdev-demo9, and mbMouse still passes with no TARGET, against the RR.
  • lines changed 94, context: html, text, full: html, text
    3d98068ead92a95ea4b714f31f01119100a3c099 Wed Sep 16 10:26:18 2026 -0700
    docent: preflight prints how the target is configured, refs #37892 make test TARGET=hgwdev-you points the whole directory at another server, and a red script there can be that machine's configuration rather than a bug. A different curatedHubPrefix genuinely changes what a quickLift hop produces, and a db.trackDb with a private table in front of the shared one changes what exact: true counts. Both failures look exactly like a bug. The README already warned about it, in prose, which arrives after someone has spent an hour. make preflight now prints the settings that decide it, so a redirected run carries its own explanation in the log: target https://hgwdev-braney.gi.ucsc.edu/cgi-bin /usr/local/apache/cgi-bin-braney/hg.conf central.db hgcentraltest db.trackDb trackDb_braney,trackDb curatedHubPrefix braney browser.quickLift on browser.quickLiftAlignments on browser.recTrackSets on central.db is on the list because preflight already checks named sessions and that setting says which hgcentral it looked in. The value printed is the EFFECTIVE one. The reader follows include and delete the way hg/lib/hgConfig.c does, with a later assignment winning over an earlier one, so a sandbox conf that sets nothing still shows what it inherits from the shared conf it includes. Reading cgi-bin-braney/hg.conf on its own says browser.quickLiftAlignments is unset; the CGI there sees it on, from the include on line 2. A ts park is the mirror case: 37547's frozen conf sets db.trackDb twice and the second one, 114 lines later, wins. Only the six keys in HG_CONF_KEYS are ever printed. The includes lead to hg.conf.private, which holds database passwords, so the reader parses whatever it is pointed at and prints nothing off that list. There is no way to ask a browser over http what its hg.conf says, so which file to read is a lookup by convention: genome-test and hgwdev read /usr/local/apache/cgi-bin/hg.conf, an hgwdev-<name> sandbox or demo reads cgi-bin-<name>, and a ts park on 127.0.0.1 is found by port in its ports.tsv. Anything off this machine -- hgwbeta, the RR -- says the conf cannot be read from here, which is true and better than a guess. Even with that in the log, a config difference and a code difference can still look alike. The reliable way to tell them apart is to swap only the binary: drop a control build's CGIs into the same sandbox, leave its hg.conf alone, and re-run. If the failures follow the binary they are the code. That experiment is what turned nine plausible failures on the #37547 branch into nine proven ones, and both READMEs now say so. Verified against all three shapes: genome-test, hgwdev-braney, hgwdev-demo9, and the #37547 park on 127.0.0.1:48087. Both test directories preflight clean.
  • lines changed 77, context: html, text, full: html, text
    7e89607805cac22b52143b23a45464759e66a957 Wed Sep 16 12:36:15 2026 -0700
    docent: a login: step, with the account kept per hgcentral, refs #37892 hgCollection refuses a visitor who is not signed in -- hgCollection.c doMiddle, "You must be logged in to edit collections" -- and so does the saving half of hgSession. The suite has never had a logged-in page, and until now could not: the login cookie is validated against a salted hash (login.cookieSalt, hg/lib/wikiLink.c), so there is no cookie to hand the browser. A script that needs one of those pages has to sign in the way a person does. `login:` does that, through hgLogin's own form, and it takes no credentials and cannot be given any. They come from ~/.docentLogin (DOCENT_LOGIN_FILE overrides, DOCENT_LOGIN_USER + DOCENT_LOGIN_PASSWORD override both for one run), which is refused unless it is mode 0600 -- the rule hg/lib/hgConfig.c applies to hg.conf, for the same reason. Nothing prints a password. THE FILE IS KEYED BY HGCENTRAL DATABASE, not by server. An account is a row in gbMembers in one central, the way a named session is: genome-test, hgwdev, every hgwdev-<name> sandbox and every ticket park read hgcentraltest and share one account, while hgwbeta reads hgcentralbeta and the RR reads hgcentral. So the file is one [hgcentraltest] section rather than a section per sandbox. Which central a server reads is READ from its hg.conf, following its includes the way hgConfig.c does, and not guessed from the host -- because a sandbox can point itself somewhere else and two on hgwdev do today: of the personal confs there, 45 set central.db=hgcentraltest, one sets hgcentralgsid and one hgcentralbeta. A server whose conf is on another machine falls back to a small table (the RR, the two mirrors, hgwbeta), and [default] catches the rest. A run redirected with DOCENT_TARGET looks up the server it is really driving. Two failures the step has to tell apart, and both cost a run to find: A WRONG PASSWORD IS A PERFECTLY GOOD PAGE. hgLogin answers one by drawing the same form again with a red message, so a step that just navigated on would leave every later step running logged out and the failure would surface somewhere else entirely. The step fails on #accountLoginForm still being there, and quotes what the page said. A RIGHT PASSWORD ARRIVES MID-REDIRECT. hgLogin answers one with a page that navigates ITSELF a moment later: returnToURL(150) at hgLogin.c:1160 writes setTimeout(function(){location=...}, 150). Returning while that timer is pending means the next step's goto: races it and the browser aborts one of the two, which arrives as a bare net::ERR_ABORTED on a URL that is completely fine. The step now waits for that redirect to land. The failure path is checked first, since that page never leaves hgLogin and there is no redirect to wait for. preflight reports the account as a fixture -- which central it resolved, which section it came from, and why it is unusable if it is -- so a missing password is caught before the browser starts. It attempts no login: a wrong password fails loudly at the step itself, which is the one thing preflight cannot do for it. The hg.conf reader is now in both docent.js and tests/preflight.js, beside the resolveTarget each of them already carries. Both say to keep the other in step. If that second copy is a copy too many, the two of them want a shared module.
  • lines changed 139, context: html, text, full: html, text
    9b762c146f7a44a3bdc783cdd202356fc163b8fb Wed Sep 16 12:43:32 2026 -0700
    docent: one targetConf.js for the target, the hg.conf, the central and the account, refs #37892 docent.js and tests/preflight.js had grown a second and then a third copy of the same lookups. That is exactly the pair that must not drift: preflight checks the fixtures for the server the RUN will drive, so a run that resolved its target, its hg.conf, its hgcentral or its account even slightly differently would be checked against the wrong machine, and the mismatch would show up as a green preflight in front of a red suite. targetConf.js answers the four questions in order, each from the one before: resolveTarget a `target:` (or DOCENT_TARGET) -> the .../cgi-bin URL to drive hgConfFor that URL -> the hg.conf it reads, if the server is on this box centralDbFor that conf -> which hgcentral it uses loginLookup that central -> the account to sign in with Nothing in it opens a browser or the network, which is why preflight can ask all four in the seconds before a run. loginLookup returns the facts plus, when the account cannot be used, ONE sentence saying why. That sentence is the substantive half and is now written once: the step throws it with a `login:` prefix, preflight prints it on the MISSING line. Each caller still phrases its own success line, since one logs and the other prints a fixture row. No password crosses that boundary to anything that prints. 248 lines net out of the two programs, 197 into the module. docent.js is no longer a single file, and three places now say so: its own require, the Run section of README.md, and docent.mk, whose mp4 rule gains targetConf.js as a prerequisite -- a change there changes what a tour renders, so it has to rebuild one. Nothing in the tree or in ~/docentTours copies docent.js; they all reference it where it sits, with targetConf.js beside it. Measured before and after, with no other change: preflight resolves the same account, central and conf for genome-test, hgwdev-braney, a ts park on 48087 and hgwbeta (which correctly has no section); tests/ is 18 of 18 with the derive baselines matching; tests/regress preflights 14 fixtures for 67 scripts.
  • lines changed 11, context: html, text, full: html, text
    6c2ecd053818535194c7845a899bca0ecd3c10d8 Thu Sep 17 17:08:10 2026 -0700
    docent: preflight checks a session settings file named inside a goto: URL, refs #38252 `loadSession:` is the verb for a saved state, and preflight already checks the file it names. But loadSession: builds its own URL, so a test about db= TOGETHER with a session load has to spell the whole request out in a goto: -- and db= in front of the load is the entire bug in #38184. The settings file it names rots the same way any other fixture does, silently, because hgTracks answers a missing one with a perfectly good page. Read hgS_loadUrlName out of a goto: the way hubUrl is already read out of one.
• src/hg/utils/docent/tests/proof.js
  • lines changed 3, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/README.txt
  • lines changed 51, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
  • lines changed 126, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
  • lines changed 36, context: html, text, full: html, text
    b7fe2f0542ac4a5e182b86ab6f794de34d0ff0d8 Sat Sep 19 10:17:09 2026 -0700
    docent: six regression scripts written from the tours, refs #38252 rm38335 rm38298 rm37996 rm38249 rm38268 rm38364, one per ticket that got a before-and-after tour this week. 95 scripts now, make proof 45 of 95, and the whole directory is green against genome-test. Five of the six carry release-ab and the sixth sandbox-ab on their first day, which is unusual here and is only because the before had already been rendered and measured for the ticket comment: hgwbeta returns "Mangled CGI input string bogus" (rm38335), one codon number instead of two (rm38298), the ASCII table with no banner (rm37996), "not supported by QuickLift" for all four alignment families (rm38249) and "Can't find strchive in track database hg19" (rm38268). rm38364's before is ts park 38373, master frozen two days before that feature merged, since it is newer than any release. Three things they cost, each also in README.txt: * where the text lives decides how it is read. rm38309 reads its exon text out of a map box's data-tooltip, but rm38298's codon tooltip is not in the served map at all -- the page's own JavaScript builds it on mouseenter -- so that one hovers and asserts tip:. rm38268's item label is in the drawn PNG and nowhere else, so it asserts the map box's href and tooltip rather than a text: that can never match. * a superTrack still does not pass its setting to its members. rm38268 opens strVar to reach strchive and then cannot use exact: on hg38, because the sibling repeat tracks come up at their own visibilities. * an hg.conf gate belongs in the header, loudly. rm38364 fails every check when denseClick is off, and that is a configuration answer rather than a regression, so the header carries the grep that settles it. rm37996 has a date on it. The new hgBlat results page is opt-in while it is tested and the banner names 2026-10-21 as the day it becomes the default; on that day the first two steps come out and the goto: gains blatNewPage=1.
  • lines changed 77, context: html, text, full: html, text
    c52d505f538f7636e48aa5c06b9ad208697471a9 Sun Sep 20 14:34:31 2026 -0700
    docent: three regression scripts for tickets nothing was watching, refs #38252 #38391's test registry named four tickets with neither a unit test nor a script here. These are three of them. They have little else in common, and each needed a different answer to the same question: what about this bug is steady enough to assert. 98 scripts now, `make proof` 46 of 98. rm38253, the coverage-graph rewrite. Its fixture is six features on hg38 chr1, three nested inside each other in each of two groups, so their coverage is a staircase with FLAT steps: 1 2 3 2 1 over 20 kb, and the same again over 20 bp. Flat is the point. Each pixel's value is the mean coverage of the bases it covers, and the mean of a constant is that constant, so a reading taken in the middle of a step does not depend on where a pixel boundary falls, on insideWidth, or on the window being a round number of bases. The two windows take the two sampling paths on purpose, and the 200 bp one is countsToPixelsUp, which pixelSweep.sh never reaches because every cell in it is wider than insideWidth. The script CANNOT flip on its own fix and its header says so: #38253 is a rewrite required to draw the same picture it replaced, verified with 96 pixel comparisons, so both builds answer the same and no baseline can tell them apart. What was proved instead is that the checks discriminate, by pointing each of the twelve readings at a neighbouring step's value: all twelve went red. rm38317, the knownGene page that prints an OMIM link built from uninitialised memory. The link itself is the wrong thing to test for, since it is whatever was on the stack and comes and goes between loads of the same URL; an earlier probe of this ticket asked only for its absence and concluded the bug did not reproduce. The same struct is read a second time further down, and that read is deterministic: the page stops dead right after "Links to sequence:", with no Translated Protein link and no Genomic Sequence link, on every load of all three assemblies. Lou Nassar's note of 2026-09-18 is what identified that; the description does not mention it. Measured on hgwbeta against genome-test, 2026-09-20: 17,487 to 27,026 bytes on hg19, 17,656 to 27,456 on hg38, 17,484 to 25,616 on mm39. So this one carries a release-ab line. A RefSeq page is the control, where the OMIM link is real and must still be there. rm38086, the BLAT Results track group. It runs a real search, lets the results page build its own custom track through its ajax to hgc, and then asks for the group, the Delete all button, the per-track delete icon, the row being DRAWN (5915dcd2939: a result track used to come up hidden) and both new names. Two names in full, each a different half of 4fd18ab7854: "102bp SMYD4" for a nucleotide query and "60aa TP53" for a protein one, where the aa is the point. The old name "blat YourSeq" is asserted absent. hgwbeta is NOT a baseline for it: v503_branch already carries these commits, so beta answering with the old name is almost certainly hg.conf blatResultsGroup being off, and preflight cannot read beta's hg.conf from here to settle it. Assertion-only, and the header says why rather than leaving the next reader to try beta again. The fixture hub is ~/public_html/docentFixtures/rm38253, beside the others. README.txt gains a section on all of this, including one trap worth repeating: never assert a title attribute, not even on a button. rm38086's first draft matched `button#blat_delAll[title=...]`, which is in the served HTML and matches nothing once hgTracks has copied every title into a mouseoverText attribute.
• src/hg/utils/docent/tests/regress/rm10138.docent.yaml
  • lines changed 78, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm27113.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    cd13b3b171159247f1d105f650cce2a0726cd67b Fri Sep 18 16:08:59 2026 -0700
    docent: release-ab evidence for six scripts, from the released docker images, refs #38252 Six scripts move from assertion-only to release-ab. The baselines are the released v499 to v503 docker images running on hgwdev, which give what the v503 ticket park could not: a range of releases rather than a single one. Three were watched to flip, and each flips at exactly the release its fix first ships in, which is what says the failure is the ticket's own and not version drift: rm27113 fails v499 v500, passes v501 94f9b53d3db and b551f6c3ac8 first ship in v501 rm37743 fails v499, passes v500 7b74ff4b43a first ships in v500 rm38108 fails v499 to v502, passes v503 7181c0af889 first ships in v503 Three fail on every released image and pass on genome-test, which is correct rather than a gap: their fixes are on master only and ship in v504. rm38126 the descPage- prefix is absent on every release rm38185 the position after an empty CGI pair is eaten on every release rm38309 the exon tooltip the fix adds is absent on every release Each failure was read rather than counted. rm27113 fails on a position with the page otherwise whole at 10 rows; rm37743 fails on its own error string; rm38108 draws no ruler at all, which is the crash it covers. Two candidates were rejected after looking at them, and are worth naming so they are not re-picked. rm34651 appears to flip at v500, but fa3431cd36b is already in v499 and the clinvar track simply has no density options there, so the flip is not its own. rm35333 appeared to flip and did not: it reads an hs1 hub, and hs1 was repaired in the images midway through the sweep, so the flip was the repair rather than the code. make proof: 39 of 89, up from 33.
• src/hg/utils/docent/tests/regress/rm36048.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm36059.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm36061.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm36331.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm36514.docent.yaml
  • lines changed 9, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm36888.xfail.docent.yaml
  • lines changed 15, context: html, text, full: html, text
    38f431f1d75c92d3c0518f60f141008ae72d133b Fri Sep 18 09:56:52 2026 -0700
    docent: retranslate the density-label checks after the reword, refs #38252 07f0635fd35 reworded all three density-mode labels on 2026-09-18, and the 04:10 nightly went red on rm38279 four hours later. The code is right; the scripts asserted wording that no longer exists. rm38279 covers all three causes, so all four of its label steps move. The two automatic messages are now "density graph: too many items, zoom in" for a window past maxWindowCoverage and "density graph: too many items, zoom in or use dense" for too many features, and the first is a strict PREFIX of the second. mouseoverText*= is a substring match, so a bare noHas on the shorter one matches the longer one too and fails on the track it is meant to pass. labelAddNote() wraps a note as " (%s)", so every check now carries the closing paren and each message matches itself and nothing else. Verified by pointing each has: at the wrong message in turn: all three fail, including the prefix case. Step 5 went from two noHas lines to one, since all three notes now share a prefix. rm36888.xfail asserts the same old string twice and nobody asked about it, because it is an xfail and it stayed red. It is still red for its own reason -- the run says `rows not drawn: rm36888Methyl`, with the note check second -- but the stale string was a trap on a delay: when the bigMethyl fix lands, the row draws, the note appears with the new wording, the has: still misses, and the xfail never flips. Both its selectors now match "density graph: too many items", the prefix the two automatic causes share, which is what the pre-reword string did as well. The full sentence is the one that bug should produce, but nobody can see it until the fix lands, and an exact string guessed from the source would rebuild the trap. Tighten it on the day it goes green. The release-ab line on rm38279 still stands: v503 carries no density note at all, so a reword cannot reach it.
• src/hg/utils/docent/tests/regress/rm36940.docent.yaml
  • lines changed 113, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm37520.docent.yaml
  • lines changed 9, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm37553.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm37562.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm37615.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm37743.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    cd13b3b171159247f1d105f650cce2a0726cd67b Fri Sep 18 16:08:59 2026 -0700
    docent: release-ab evidence for six scripts, from the released docker images, refs #38252 Six scripts move from assertion-only to release-ab. The baselines are the released v499 to v503 docker images running on hgwdev, which give what the v503 ticket park could not: a range of releases rather than a single one. Three were watched to flip, and each flips at exactly the release its fix first ships in, which is what says the failure is the ticket's own and not version drift: rm27113 fails v499 v500, passes v501 94f9b53d3db and b551f6c3ac8 first ship in v501 rm37743 fails v499, passes v500 7b74ff4b43a first ships in v500 rm38108 fails v499 to v502, passes v503 7181c0af889 first ships in v503 Three fail on every released image and pass on genome-test, which is correct rather than a gap: their fixes are on master only and ship in v504. rm38126 the descPage- prefix is absent on every release rm38185 the position after an empty CGI pair is eaten on every release rm38309 the exon tooltip the fix adds is absent on every release Each failure was read rather than counted. rm27113 fails on a position with the page otherwise whole at 10 rows; rm37743 fails on its own error string; rm38108 draws no ruler at all, which is the crash it covers. Two candidates were rejected after looking at them, and are worth naming so they are not re-picked. rm34651 appears to flip at v500, but fa3431cd36b is already in v499 and the clinvar track simply has no density options there, so the flip is not its own. rm35333 appeared to flip and did not: it reads an hs1 hub, and hs1 was repaired in the images midway through the sweep, so the flip was the repair rather than the code. make proof: 39 of 89, up from 33.
• src/hg/utils/docent/tests/regress/rm37815.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm37969.docent.yaml
  • lines changed 69, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm37996.docent.yaml
  • lines changed 41, context: html, text, full: html, text
    b7fe2f0542ac4a5e182b86ab6f794de34d0ff0d8 Sat Sep 19 10:17:09 2026 -0700
    docent: six regression scripts written from the tours, refs #38252 rm38335 rm38298 rm37996 rm38249 rm38268 rm38364, one per ticket that got a before-and-after tour this week. 95 scripts now, make proof 45 of 95, and the whole directory is green against genome-test. Five of the six carry release-ab and the sixth sandbox-ab on their first day, which is unusual here and is only because the before had already been rendered and measured for the ticket comment: hgwbeta returns "Mangled CGI input string bogus" (rm38335), one codon number instead of two (rm38298), the ASCII table with no banner (rm37996), "not supported by QuickLift" for all four alignment families (rm38249) and "Can't find strchive in track database hg19" (rm38268). rm38364's before is ts park 38373, master frozen two days before that feature merged, since it is newer than any release. Three things they cost, each also in README.txt: * where the text lives decides how it is read. rm38309 reads its exon text out of a map box's data-tooltip, but rm38298's codon tooltip is not in the served map at all -- the page's own JavaScript builds it on mouseenter -- so that one hovers and asserts tip:. rm38268's item label is in the drawn PNG and nowhere else, so it asserts the map box's href and tooltip rather than a text: that can never match. * a superTrack still does not pass its setting to its members. rm38268 opens strVar to reach strchive and then cannot use exact: on hg38, because the sibling repeat tracks come up at their own visibilities. * an hg.conf gate belongs in the header, loudly. rm38364 fails every check when denseClick is off, and that is a configuration answer rather than a regression, so the header carries the grep that settles it. rm37996 has a date on it. The new hgBlat results page is opt-in while it is tested and the banner names 2026-10-21 as the day it becomes the default; on that day the first two steps come out and the goto: gains blatNewPage=1.
• src/hg/utils/docent/tests/regress/rm38032.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    9d4c4b4781eb714d844a98faa90a1909af541a80 Wed Sep 16 10:26:38 2026 -0700
    docent: the sandbox-ab proof for nine scripts, and the two rules behind it, refs #38252 Nine scripts here went red on the #37547 cart-visibility branch and green on master, for their own reason, on the same machine with the same hg.conf. That is the sandbox-ab shape exactly, and it was going to be lost with the run. Each now carries the line: sandbox-ab 2026-09-16 -- failed on a #37547 cart-visibility build before the isParentVisible fix in hg/lib/trackHub.c, passed on the same sandbox with the fix and with master; the same hg.conf all three times rm36048, rm36059, rm36061, rm36331, rm37553, rm37562, rm37615, rm37815 and rm38032. make proof goes from 4 of 67 watched to fail and then pass, to 13. The cause was one line: isParentVisible() read a container's visibility from the bare cart key instead of through the accessor, so after migration it read nothing, fell back to the trackDb default, decided the container was hidden, and walkTree() left the track out of the lift hub. Two rules come out of the same session and are now a section in README.txt. Assert before you navigate. A bug in this area can lag by exactly one request: the visibility reaches the cart, the image drawn in reply does not carry the row, and any next request draws it. A script shaped track: -> go: -> expect: supplies that extra request itself and passes on the broken build. The exposure here is smaller than it looks, and worth writing down so it is not re-counted: forty of the sixty-seven scripts have no track: step at all, and of the twenty-five that do, twenty-three already assert straight after it. The two that do not are rm36514 and rm37520, and neither can. Both turn on mane at the ticket's own position, chr7:156,982,676-156,996,015, where mane has no features on hg38 -- bigBedToBed on /gbdb/hg38/mane/mane.bb returns nothing there -- so there is no row to name. The track controls read pack on a broken build too, because the cart really did take the visibility. A comment in each says so, so the gap is not read as an oversight and closed with a check that passes on anything. tests/firstrequest.docent.yaml covers the class once, on its own. Name a child of a container that is hidden by default. Two shapes of track stay green on a build whose visibility handling is broken. A top-level track is added as a lightweight stub so the track controls can list it, so it is built whatever the cart lookup returned -- microsat, gtexGene and windowmaskerSdust all drew on the broken build. A default-visible child has its container in the list already -- wgEncodeRegMarkH3k27ac drew while its sibling wgEncodeRegMarkH3k4me1 did not, same superTrack, same request, opposite verdicts. wgEncodeRegMarkH3k4me1 is the known-good example. This is the cheapest rule in the file: it changes which track a script names, not how the script is written, and without it a script reads as coverage and is not. The section on reading a redirected run points at make preflight TARGET=, which now prints the target's central.db, db.trackDb, curatedHubPrefix and quickLift settings, and says to swap only the binary when that is not enough. No script's steps changed, so nothing here alters what the nightly measures. make proof and both preflights are clean.
• src/hg/utils/docent/tests/regress/rm38086.docent.yaml
  • lines changed 96, context: html, text, full: html, text
    c52d505f538f7636e48aa5c06b9ad208697471a9 Sun Sep 20 14:34:31 2026 -0700
    docent: three regression scripts for tickets nothing was watching, refs #38252 #38391's test registry named four tickets with neither a unit test nor a script here. These are three of them. They have little else in common, and each needed a different answer to the same question: what about this bug is steady enough to assert. 98 scripts now, `make proof` 46 of 98. rm38253, the coverage-graph rewrite. Its fixture is six features on hg38 chr1, three nested inside each other in each of two groups, so their coverage is a staircase with FLAT steps: 1 2 3 2 1 over 20 kb, and the same again over 20 bp. Flat is the point. Each pixel's value is the mean coverage of the bases it covers, and the mean of a constant is that constant, so a reading taken in the middle of a step does not depend on where a pixel boundary falls, on insideWidth, or on the window being a round number of bases. The two windows take the two sampling paths on purpose, and the 200 bp one is countsToPixelsUp, which pixelSweep.sh never reaches because every cell in it is wider than insideWidth. The script CANNOT flip on its own fix and its header says so: #38253 is a rewrite required to draw the same picture it replaced, verified with 96 pixel comparisons, so both builds answer the same and no baseline can tell them apart. What was proved instead is that the checks discriminate, by pointing each of the twelve readings at a neighbouring step's value: all twelve went red. rm38317, the knownGene page that prints an OMIM link built from uninitialised memory. The link itself is the wrong thing to test for, since it is whatever was on the stack and comes and goes between loads of the same URL; an earlier probe of this ticket asked only for its absence and concluded the bug did not reproduce. The same struct is read a second time further down, and that read is deterministic: the page stops dead right after "Links to sequence:", with no Translated Protein link and no Genomic Sequence link, on every load of all three assemblies. Lou Nassar's note of 2026-09-18 is what identified that; the description does not mention it. Measured on hgwbeta against genome-test, 2026-09-20: 17,487 to 27,026 bytes on hg19, 17,656 to 27,456 on hg38, 17,484 to 25,616 on mm39. So this one carries a release-ab line. A RefSeq page is the control, where the OMIM link is real and must still be there. rm38086, the BLAT Results track group. It runs a real search, lets the results page build its own custom track through its ajax to hgc, and then asks for the group, the Delete all button, the per-track delete icon, the row being DRAWN (5915dcd2939: a result track used to come up hidden) and both new names. Two names in full, each a different half of 4fd18ab7854: "102bp SMYD4" for a nucleotide query and "60aa TP53" for a protein one, where the aa is the point. The old name "blat YourSeq" is asserted absent. hgwbeta is NOT a baseline for it: v503_branch already carries these commits, so beta answering with the old name is almost certainly hg.conf blatResultsGroup being off, and preflight cannot read beta's hg.conf from here to settle it. Assertion-only, and the header says why rather than leaving the next reader to try beta again. The fixture hub is ~/public_html/docentFixtures/rm38253, beside the others. README.txt gains a section on all of this, including one trap worth repeating: never assert a title attribute, not even on a button. rm38086's first draft matched `button#blat_delAll[title=...]`, which is in the served HTML and matches nothing once hgTracks has copied every title into a mouseoverText attribute.
• src/hg/utils/docent/tests/regress/rm38108.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    cd13b3b171159247f1d105f650cce2a0726cd67b Fri Sep 18 16:08:59 2026 -0700
    docent: release-ab evidence for six scripts, from the released docker images, refs #38252 Six scripts move from assertion-only to release-ab. The baselines are the released v499 to v503 docker images running on hgwdev, which give what the v503 ticket park could not: a range of releases rather than a single one. Three were watched to flip, and each flips at exactly the release its fix first ships in, which is what says the failure is the ticket's own and not version drift: rm27113 fails v499 v500, passes v501 94f9b53d3db and b551f6c3ac8 first ship in v501 rm37743 fails v499, passes v500 7b74ff4b43a first ships in v500 rm38108 fails v499 to v502, passes v503 7181c0af889 first ships in v503 Three fail on every released image and pass on genome-test, which is correct rather than a gap: their fixes are on master only and ship in v504. rm38126 the descPage- prefix is absent on every release rm38185 the position after an empty CGI pair is eaten on every release rm38309 the exon tooltip the fix adds is absent on every release Each failure was read rather than counted. rm27113 fails on a position with the page otherwise whole at 10 rows; rm37743 fails on its own error string; rm38108 draws no ruler at all, which is the crash it covers. Two candidates were rejected after looking at them, and are worth naming so they are not re-picked. rm34651 appears to flip at v500, but fa3431cd36b is already in v499 and the clinvar track simply has no density options there, so the flip is not its own. rm35333 appeared to flip and did not: it reads an hs1 hub, and hs1 was repaired in the images midway through the sweep, so the flip was the repair rather than the code. make proof: 39 of 89, up from 33.
• src/hg/utils/docent/tests/regress/rm38126.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    cd13b3b171159247f1d105f650cce2a0726cd67b Fri Sep 18 16:08:59 2026 -0700
    docent: release-ab evidence for six scripts, from the released docker images, refs #38252 Six scripts move from assertion-only to release-ab. The baselines are the released v499 to v503 docker images running on hgwdev, which give what the v503 ticket park could not: a range of releases rather than a single one. Three were watched to flip, and each flips at exactly the release its fix first ships in, which is what says the failure is the ticket's own and not version drift: rm27113 fails v499 v500, passes v501 94f9b53d3db and b551f6c3ac8 first ship in v501 rm37743 fails v499, passes v500 7b74ff4b43a first ships in v500 rm38108 fails v499 to v502, passes v503 7181c0af889 first ships in v503 Three fail on every released image and pass on genome-test, which is correct rather than a gap: their fixes are on master only and ship in v504. rm38126 the descPage- prefix is absent on every release rm38185 the position after an empty CGI pair is eaten on every release rm38309 the exon tooltip the fix adds is absent on every release Each failure was read rather than counted. rm27113 fails on a position with the page otherwise whole at 10 rows; rm37743 fails on its own error string; rm38108 draws no ruler at all, which is the crash it covers. Two candidates were rejected after looking at them, and are worth naming so they are not re-picked. rm34651 appears to flip at v500, but fa3431cd36b is already in v499 and the clinvar track simply has no density options there, so the flip is not its own. rm35333 appeared to flip and did not: it reads an hs1 hub, and hs1 was repaired in the images midway through the sweep, so the flip was the repair rather than the code. make proof: 39 of 89, up from 33.
• src/hg/utils/docent/tests/regress/rm38171.docent.yaml
  • lines changed 73, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38184.docent.yaml
  • lines changed 60, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38185.docent.yaml
  • lines changed 1, context: html, text, full: html, text
    cd13b3b171159247f1d105f650cce2a0726cd67b Fri Sep 18 16:08:59 2026 -0700
    docent: release-ab evidence for six scripts, from the released docker images, refs #38252 Six scripts move from assertion-only to release-ab. The baselines are the released v499 to v503 docker images running on hgwdev, which give what the v503 ticket park could not: a range of releases rather than a single one. Three were watched to flip, and each flips at exactly the release its fix first ships in, which is what says the failure is the ticket's own and not version drift: rm27113 fails v499 v500, passes v501 94f9b53d3db and b551f6c3ac8 first ship in v501 rm37743 fails v499, passes v500 7b74ff4b43a first ships in v500 rm38108 fails v499 to v502, passes v503 7181c0af889 first ships in v503 Three fail on every released image and pass on genome-test, which is correct rather than a gap: their fixes are on master only and ship in v504. rm38126 the descPage- prefix is absent on every release rm38185 the position after an empty CGI pair is eaten on every release rm38309 the exon tooltip the fix adds is absent on every release Each failure was read rather than counted. rm27113 fails on a position with the page otherwise whole at 10 rows; rm37743 fails on its own error string; rm38108 draws no ruler at all, which is the crash it covers. Two candidates were rejected after looking at them, and are worth naming so they are not re-picked. rm34651 appears to flip at v500, but fa3431cd36b is already in v499 and the clinvar track simply has no density options there, so the flip is not its own. rm35333 appeared to flip and did not: it reads an hs1 hub, and hs1 was repaired in the images midway through the sweep, so the flip was the repair rather than the code. make proof: 39 of 89, up from 33.
• src/hg/utils/docent/tests/regress/rm38198.docent.yaml
  • lines changed 70, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38200.docent.yaml
  • lines changed 72, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38205.docent.yaml
  • lines changed 71, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38206.docent.yaml
  • lines changed 77, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38223.docent.yaml
  • lines changed 51, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38236.docent.yaml
  • lines changed 74, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38248.docent.yaml
  • lines changed 80, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38249.docent.yaml
  • lines changed 42, context: html, text, full: html, text
    b7fe2f0542ac4a5e182b86ab6f794de34d0ff0d8 Sat Sep 19 10:17:09 2026 -0700
    docent: six regression scripts written from the tours, refs #38252 rm38335 rm38298 rm37996 rm38249 rm38268 rm38364, one per ticket that got a before-and-after tour this week. 95 scripts now, make proof 45 of 95, and the whole directory is green against genome-test. Five of the six carry release-ab and the sixth sandbox-ab on their first day, which is unusual here and is only because the before had already been rendered and measured for the ticket comment: hgwbeta returns "Mangled CGI input string bogus" (rm38335), one codon number instead of two (rm38298), the ASCII table with no banner (rm37996), "not supported by QuickLift" for all four alignment families (rm38249) and "Can't find strchive in track database hg19" (rm38268). rm38364's before is ts park 38373, master frozen two days before that feature merged, since it is newer than any release. Three things they cost, each also in README.txt: * where the text lives decides how it is read. rm38309 reads its exon text out of a map box's data-tooltip, but rm38298's codon tooltip is not in the served map at all -- the page's own JavaScript builds it on mouseenter -- so that one hovers and asserts tip:. rm38268's item label is in the drawn PNG and nowhere else, so it asserts the map box's href and tooltip rather than a text: that can never match. * a superTrack still does not pass its setting to its members. rm38268 opens strVar to reach strchive and then cannot use exact: on hg38, because the sibling repeat tracks come up at their own visibilities. * an hg.conf gate belongs in the header, loudly. rm38364 fails every check when denseClick is off, and that is a configuration answer rather than a regression, so the header carries the grep that settles it. rm37996 has a date on it. The new hgBlat results page is opt-in while it is tested and the banner names 2026-10-21 as the day it becomes the default; on that day the first two steps come out and the goto: gains blatNewPage=1.
• src/hg/utils/docent/tests/regress/rm38251.docent.yaml
  • lines changed 129, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38253.docent.yaml
  • lines changed 100, context: html, text, full: html, text
    c52d505f538f7636e48aa5c06b9ad208697471a9 Sun Sep 20 14:34:31 2026 -0700
    docent: three regression scripts for tickets nothing was watching, refs #38252 #38391's test registry named four tickets with neither a unit test nor a script here. These are three of them. They have little else in common, and each needed a different answer to the same question: what about this bug is steady enough to assert. 98 scripts now, `make proof` 46 of 98. rm38253, the coverage-graph rewrite. Its fixture is six features on hg38 chr1, three nested inside each other in each of two groups, so their coverage is a staircase with FLAT steps: 1 2 3 2 1 over 20 kb, and the same again over 20 bp. Flat is the point. Each pixel's value is the mean coverage of the bases it covers, and the mean of a constant is that constant, so a reading taken in the middle of a step does not depend on where a pixel boundary falls, on insideWidth, or on the window being a round number of bases. The two windows take the two sampling paths on purpose, and the 200 bp one is countsToPixelsUp, which pixelSweep.sh never reaches because every cell in it is wider than insideWidth. The script CANNOT flip on its own fix and its header says so: #38253 is a rewrite required to draw the same picture it replaced, verified with 96 pixel comparisons, so both builds answer the same and no baseline can tell them apart. What was proved instead is that the checks discriminate, by pointing each of the twelve readings at a neighbouring step's value: all twelve went red. rm38317, the knownGene page that prints an OMIM link built from uninitialised memory. The link itself is the wrong thing to test for, since it is whatever was on the stack and comes and goes between loads of the same URL; an earlier probe of this ticket asked only for its absence and concluded the bug did not reproduce. The same struct is read a second time further down, and that read is deterministic: the page stops dead right after "Links to sequence:", with no Translated Protein link and no Genomic Sequence link, on every load of all three assemblies. Lou Nassar's note of 2026-09-18 is what identified that; the description does not mention it. Measured on hgwbeta against genome-test, 2026-09-20: 17,487 to 27,026 bytes on hg19, 17,656 to 27,456 on hg38, 17,484 to 25,616 on mm39. So this one carries a release-ab line. A RefSeq page is the control, where the OMIM link is real and must still be there. rm38086, the BLAT Results track group. It runs a real search, lets the results page build its own custom track through its ajax to hgc, and then asks for the group, the Delete all button, the per-track delete icon, the row being DRAWN (5915dcd2939: a result track used to come up hidden) and both new names. Two names in full, each a different half of 4fd18ab7854: "102bp SMYD4" for a nucleotide query and "60aa TP53" for a protein one, where the aa is the point. The old name "blat YourSeq" is asserted absent. hgwbeta is NOT a baseline for it: v503_branch already carries these commits, so beta answering with the old name is almost certainly hg.conf blatResultsGroup being off, and preflight cannot read beta's hg.conf from here to settle it. Assertion-only, and the header says why rather than leaving the next reader to try beta again. The fixture hub is ~/public_html/docentFixtures/rm38253, beside the others. README.txt gains a section on all of this, including one trap worth repeating: never assert a title attribute, not even on a button. rm38086's first draft matched `button#blat_delAll[title=...]`, which is in the served HTML and matches nothing once hgTracks has copied every title into a mouseoverText attribute.
• src/hg/utils/docent/tests/regress/rm38257.docent.yaml
  • lines changed 73, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38268.docent.yaml
  • lines changed 45, context: html, text, full: html, text
    b7fe2f0542ac4a5e182b86ab6f794de34d0ff0d8 Sat Sep 19 10:17:09 2026 -0700
    docent: six regression scripts written from the tours, refs #38252 rm38335 rm38298 rm37996 rm38249 rm38268 rm38364, one per ticket that got a before-and-after tour this week. 95 scripts now, make proof 45 of 95, and the whole directory is green against genome-test. Five of the six carry release-ab and the sixth sandbox-ab on their first day, which is unusual here and is only because the before had already been rendered and measured for the ticket comment: hgwbeta returns "Mangled CGI input string bogus" (rm38335), one codon number instead of two (rm38298), the ASCII table with no banner (rm37996), "not supported by QuickLift" for all four alignment families (rm38249) and "Can't find strchive in track database hg19" (rm38268). rm38364's before is ts park 38373, master frozen two days before that feature merged, since it is newer than any release. Three things they cost, each also in README.txt: * where the text lives decides how it is read. rm38309 reads its exon text out of a map box's data-tooltip, but rm38298's codon tooltip is not in the served map at all -- the page's own JavaScript builds it on mouseenter -- so that one hovers and asserts tip:. rm38268's item label is in the drawn PNG and nowhere else, so it asserts the map box's href and tooltip rather than a text: that can never match. * a superTrack still does not pass its setting to its members. rm38268 opens strVar to reach strchive and then cannot use exact: on hg38, because the sibling repeat tracks come up at their own visibilities. * an hg.conf gate belongs in the header, loudly. rm38364 fails every check when denseClick is off, and that is a configuration answer rather than a regression, so the header carries the grep that settles it. rm37996 has a date on it. The new hgBlat results page is opt-in while it is tested and the banner names 2026-10-21 as the day it becomes the default; on that day the first two steps come out and the goto: gains blatNewPage=1.
• src/hg/utils/docent/tests/regress/rm38279.docent.yaml
  • lines changed 95, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
  • lines changed 32, context: html, text, full: html, text
    38f431f1d75c92d3c0518f60f141008ae72d133b Fri Sep 18 09:56:52 2026 -0700
    docent: retranslate the density-label checks after the reword, refs #38252 07f0635fd35 reworded all three density-mode labels on 2026-09-18, and the 04:10 nightly went red on rm38279 four hours later. The code is right; the scripts asserted wording that no longer exists. rm38279 covers all three causes, so all four of its label steps move. The two automatic messages are now "density graph: too many items, zoom in" for a window past maxWindowCoverage and "density graph: too many items, zoom in or use dense" for too many features, and the first is a strict PREFIX of the second. mouseoverText*= is a substring match, so a bare noHas on the shorter one matches the longer one too and fails on the track it is meant to pass. labelAddNote() wraps a note as " (%s)", so every check now carries the closing paren and each message matches itself and nothing else. Verified by pointing each has: at the wrong message in turn: all three fail, including the prefix case. Step 5 went from two noHas lines to one, since all three notes now share a prefix. rm36888.xfail asserts the same old string twice and nobody asked about it, because it is an xfail and it stayed red. It is still red for its own reason -- the run says `rows not drawn: rm36888Methyl`, with the note check second -- but the stale string was a trap on a delay: when the bigMethyl fix lands, the row draws, the note appears with the new wording, the has: still misses, and the xfail never flips. Both its selectors now match "density graph: too many items", the prefix the two automatic causes share, which is what the pre-reword string did as well. The full sentence is the one that bug should produce, but nobody can see it until the fix lands, and an exact string guessed from the source would rebuild the trap. Tighten it on the day it goes green. The release-ab line on rm38279 still stands: v503 carries no density note at all, so a reword cannot reach it.
• src/hg/utils/docent/tests/regress/rm38281.docent.yaml
  • lines changed 79, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38283.docent.yaml
  • lines changed 90, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38284.docent.yaml
  • lines changed 82, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38285.docent.yaml
  • lines changed 61, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38298.docent.yaml
  • lines changed 43, context: html, text, full: html, text
    b7fe2f0542ac4a5e182b86ab6f794de34d0ff0d8 Sat Sep 19 10:17:09 2026 -0700
    docent: six regression scripts written from the tours, refs #38252 rm38335 rm38298 rm37996 rm38249 rm38268 rm38364, one per ticket that got a before-and-after tour this week. 95 scripts now, make proof 45 of 95, and the whole directory is green against genome-test. Five of the six carry release-ab and the sixth sandbox-ab on their first day, which is unusual here and is only because the before had already been rendered and measured for the ticket comment: hgwbeta returns "Mangled CGI input string bogus" (rm38335), one codon number instead of two (rm38298), the ASCII table with no banner (rm37996), "not supported by QuickLift" for all four alignment families (rm38249) and "Can't find strchive in track database hg19" (rm38268). rm38364's before is ts park 38373, master frozen two days before that feature merged, since it is newer than any release. Three things they cost, each also in README.txt: * where the text lives decides how it is read. rm38309 reads its exon text out of a map box's data-tooltip, but rm38298's codon tooltip is not in the served map at all -- the page's own JavaScript builds it on mouseenter -- so that one hovers and asserts tip:. rm38268's item label is in the drawn PNG and nowhere else, so it asserts the map box's href and tooltip rather than a text: that can never match. * a superTrack still does not pass its setting to its members. rm38268 opens strVar to reach strchive and then cannot use exact: on hg38, because the sibling repeat tracks come up at their own visibilities. * an hg.conf gate belongs in the header, loudly. rm38364 fails every check when denseClick is off, and that is a configuration answer rather than a regression, so the header carries the grep that settles it. rm37996 has a date on it. The new hgBlat results page is opt-in while it is tested and the banner names 2026-10-21 as the day it becomes the default; on that day the first two steps come out and the goto: gains blatNewPage=1.
• src/hg/utils/docent/tests/regress/rm38302.docent.yaml
  • lines changed 50, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38303.docent.yaml
  • lines changed 72, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
• src/hg/utils/docent/tests/regress/rm38309.docent.yaml
  • lines changed 75, context: html, text, full: html, text
    bbe654a9dc89bc7b75d40d1fe197d5376d756580 Thu Sep 17 17:08:10 2026 -0700
    docent: 22 regression tests for a QA list, and release-ab evidence from v503, refs #38252 One script per ticket for twenty-two that had none: rm10138, rm36940, rm37969, rm38171, rm38184, rm38198, rm38200, rm38205, rm38206, rm38223, rm38236, rm38248, rm38251, rm38257, rm38279, rm38281, rm38283, rm38284, rm38285, rm38302, rm38303 and rm38309. No theme -- they run from a menu-bar color to a SIGSEGV in a quickLift view. Three fixtures came with them, under ~/public_html/docentFixtures: rm36940 (seven bigBed tracks differing only in the field count on the type line), rm38283 (one description page full of dollar variables) and rm38184 (a one-line session settings file). Every one was then run against v503, which is what the new proof level is for. v503_branch 707b184e329 went into a ticket sandbox -- CGIs, js and htdocs, since three of the fixes live in hg/js or htdocs/style -- and twenty-one of the twenty-four fix commits landed after that branch was cut, so the release has the bugs. Eighteen scripts failed there on their own check and carry a release-ab line quoting the failure. release-ab sits between sandbox-ab and server-flip. It is stronger than sandbox-ab because anyone can rebuild it from a tag, where a hand-patched tree lives in one person's home directory and does not survive a rebase. The four that v503 does not cover are each said in their own header: rm38171's fix is IN v503 so an older release is its baseline; rm36940's fixture hub draws nothing there, so the run never reaches the check; rm38257's login: dies against a park on any baseline; and rm38309 PASSES, which is exactly what its header claims, since that fix changes no byte of any page. rm38251 had to be rewritten because of what the sweep said. It PASSED on a build carrying its own bug, at every width from 390 to 1099px: the icon had slid 312px left but was still inside the bar, and the clear: check named the links, where the icon fell in a gap between two of them. Naming the menu list instead has no gaps. Without the sweep it would have sat here looking green. make proof: 89 scripts, 33 watched to fail for the reason they exist and then pass, up from 13.
  • lines changed 1, context: html, text, full: html, text
    cd13b3b171159247f1d105f650cce2a0726cd67b Fri Sep 18 16:08:59 2026 -0700
    docent: release-ab evidence for six scripts, from the released docker images, refs #38252 Six scripts move from assertion-only to release-ab. The baselines are the released v499 to v503 docker images running on hgwdev, which give what the v503 ticket park could not: a range of releases rather than a single one. Three were watched to flip, and each flips at exactly the release its fix first ships in, which is what says the failure is the ticket's own and not version drift: rm27113 fails v499 v500, passes v501 94f9b53d3db and b551f6c3ac8 first ship in v501 rm37743 fails v499, passes v500 7b74ff4b43a first ships in v500 rm38108 fails v499 to v502, passes v503 7181c0af889 first ships in v503 Three fail on every released image and pass on genome-test, which is correct rather than a gap: their fixes are on master only and ship in v504. rm38126 the descPage- prefix is absent on every release rm38185 the position after an empty CGI pair is eaten on every release rm38309 the exon tooltip the fix adds is absent on every release Each failure was read rather than counted. rm27113 fails on a position with the page otherwise whole at 10 rows; rm37743 fails on its own error string; rm38108 draws no ruler at all, which is the crash it covers. Two candidates were rejected after looking at them, and are worth naming so they are not re-picked. rm34651 appears to flip at v500, but fa3431cd36b is already in v499 and the clinvar track simply has no density options there, so the flip is not its own. rm35333 appeared to flip and did not: it reads an hs1 hub, and hs1 was repaired in the images midway through the sweep, so the flip was the repair rather than the code. make proof: 39 of 89, up from 33.
• src/hg/utils/docent/tests/regress/rm38317.docent.yaml
  • lines changed 79, context: html, text, full: html, text
    c52d505f538f7636e48aa5c06b9ad208697471a9 Sun Sep 20 14:34:31 2026 -0700
    docent: three regression scripts for tickets nothing was watching, refs #38252 #38391's test registry named four tickets with neither a unit test nor a script here. These are three of them. They have little else in common, and each needed a different answer to the same question: what about this bug is steady enough to assert. 98 scripts now, `make proof` 46 of 98. rm38253, the coverage-graph rewrite. Its fixture is six features on hg38 chr1, three nested inside each other in each of two groups, so their coverage is a staircase with FLAT steps: 1 2 3 2 1 over 20 kb, and the same again over 20 bp. Flat is the point. Each pixel's value is the mean coverage of the bases it covers, and the mean of a constant is that constant, so a reading taken in the middle of a step does not depend on where a pixel boundary falls, on insideWidth, or on the window being a round number of bases. The two windows take the two sampling paths on purpose, and the 200 bp one is countsToPixelsUp, which pixelSweep.sh never reaches because every cell in it is wider than insideWidth. The script CANNOT flip on its own fix and its header says so: #38253 is a rewrite required to draw the same picture it replaced, verified with 96 pixel comparisons, so both builds answer the same and no baseline can tell them apart. What was proved instead is that the checks discriminate, by pointing each of the twelve readings at a neighbouring step's value: all twelve went red. rm38317, the knownGene page that prints an OMIM link built from uninitialised memory. The link itself is the wrong thing to test for, since it is whatever was on the stack and comes and goes between loads of the same URL; an earlier probe of this ticket asked only for its absence and concluded the bug did not reproduce. The same struct is read a second time further down, and that read is deterministic: the page stops dead right after "Links to sequence:", with no Translated Protein link and no Genomic Sequence link, on every load of all three assemblies. Lou Nassar's note of 2026-09-18 is what identified that; the description does not mention it. Measured on hgwbeta against genome-test, 2026-09-20: 17,487 to 27,026 bytes on hg19, 17,656 to 27,456 on hg38, 17,484 to 25,616 on mm39. So this one carries a release-ab line. A RefSeq page is the control, where the OMIM link is real and must still be there. rm38086, the BLAT Results track group. It runs a real search, lets the results page build its own custom track through its ajax to hgc, and then asks for the group, the Delete all button, the per-track delete icon, the row being DRAWN (5915dcd2939: a result track used to come up hidden) and both new names. Two names in full, each a different half of 4fd18ab7854: "102bp SMYD4" for a nucleotide query and "60aa TP53" for a protein one, where the aa is the point. The old name "blat YourSeq" is asserted absent. hgwbeta is NOT a baseline for it: v503_branch already carries these commits, so beta answering with the old name is almost certainly hg.conf blatResultsGroup being off, and preflight cannot read beta's hg.conf from here to settle it. Assertion-only, and the header says why rather than leaving the next reader to try beta again. The fixture hub is ~/public_html/docentFixtures/rm38253, beside the others. README.txt gains a section on all of this, including one trap worth repeating: never assert a title attribute, not even on a button. rm38086's first draft matched `button#blat_delAll[title=...]`, which is in the served HTML and matches nothing once hgTracks has copied every title into a mouseoverText attribute.
• src/hg/utils/docent/tests/regress/rm38335.docent.yaml
  • lines changed 44, context: html, text, full: html, text
    b7fe2f0542ac4a5e182b86ab6f794de34d0ff0d8 Sat Sep 19 10:17:09 2026 -0700
    docent: six regression scripts written from the tours, refs #38252 rm38335 rm38298 rm37996 rm38249 rm38268 rm38364, one per ticket that got a before-and-after tour this week. 95 scripts now, make proof 45 of 95, and the whole directory is green against genome-test. Five of the six carry release-ab and the sixth sandbox-ab on their first day, which is unusual here and is only because the before had already been rendered and measured for the ticket comment: hgwbeta returns "Mangled CGI input string bogus" (rm38335), one codon number instead of two (rm38298), the ASCII table with no banner (rm37996), "not supported by QuickLift" for all four alignment families (rm38249) and "Can't find strchive in track database hg19" (rm38268). rm38364's before is ts park 38373, master frozen two days before that feature merged, since it is newer than any release. Three things they cost, each also in README.txt: * where the text lives decides how it is read. rm38309 reads its exon text out of a map box's data-tooltip, but rm38298's codon tooltip is not in the served map at all -- the page's own JavaScript builds it on mouseenter -- so that one hovers and asserts tip:. rm38268's item label is in the drawn PNG and nowhere else, so it asserts the map box's href and tooltip rather than a text: that can never match. * a superTrack still does not pass its setting to its members. rm38268 opens strVar to reach strchive and then cannot use exact: on hg38, because the sibling repeat tracks come up at their own visibilities. * an hg.conf gate belongs in the header, loudly. rm38364 fails every check when denseClick is off, and that is a configuration answer rather than a regression, so the header carries the grep that settles it. rm37996 has a date on it. The new hgBlat results page is opt-in while it is tested and the banner names 2026-10-21 as the day it becomes the default; on that day the first two steps come out and the goto: gains blatNewPage=1.
• src/hg/utils/docent/tests/regress/rm38364.docent.yaml
  • lines changed 45, context: html, text, full: html, text
    b7fe2f0542ac4a5e182b86ab6f794de34d0ff0d8 Sat Sep 19 10:17:09 2026 -0700
    docent: six regression scripts written from the tours, refs #38252 rm38335 rm38298 rm37996 rm38249 rm38268 rm38364, one per ticket that got a before-and-after tour this week. 95 scripts now, make proof 45 of 95, and the whole directory is green against genome-test. Five of the six carry release-ab and the sixth sandbox-ab on their first day, which is unusual here and is only because the before had already been rendered and measured for the ticket comment: hgwbeta returns "Mangled CGI input string bogus" (rm38335), one codon number instead of two (rm38298), the ASCII table with no banner (rm37996), "not supported by QuickLift" for all four alignment families (rm38249) and "Can't find strchive in track database hg19" (rm38268). rm38364's before is ts park 38373, master frozen two days before that feature merged, since it is newer than any release. Three things they cost, each also in README.txt: * where the text lives decides how it is read. rm38309 reads its exon text out of a map box's data-tooltip, but rm38298's codon tooltip is not in the served map at all -- the page's own JavaScript builds it on mouseenter -- so that one hovers and asserts tip:. rm38268's item label is in the drawn PNG and nowhere else, so it asserts the map box's href and tooltip rather than a text: that can never match. * a superTrack still does not pass its setting to its members. rm38268 opens strVar to reach strchive and then cannot use exact: on hg38, because the sibling repeat tracks come up at their own visibilities. * an hg.conf gate belongs in the header, loudly. rm38364 fails every check when denseClick is off, and that is a configuration answer rather than a regression, so the header carries the grep that settles it. rm37996 has a date on it. The new hgBlat results page is opt-in while it is tested and the banner names 2026-10-21 as the day it becomes the default; on that day the first two steps come out and the goto: gains blatNewPage=1.
• src/hg/utils/docent/tests/search.docent.yaml
  • lines changed 69, context: html, text, full: html, text
    2e5054b09838bacf29b31bb64fbd234471e7cb71 Wed Sep 16 13:00:17 2026 -0700
    docent: a test for hgFind's copy of isParentVisible, on hgSearch, refs #37892 The third and last of the three. isParentVisible() exists in the tree character for character three times -- hg/lib/trackHub.c:1818, hg/hgCollection/hgCollection.c:269 and hg/lib/hgFind.c:2957 -- and each copy asks the same question, are this track's containers visible, by reading the cart itself rather than going through the accessor. Nine scripts in regress/ cover the first, collection.docent.yaml covers the second, and this covers the third. A change to how visibility is stored has to reach all three, or two of them quietly start answering from the trackDb default. What this copy decides: isTrackVisible() at hgFind.c:2977 sets category->visibility, and hgSearch groups its results by it -- "Visible Tracks" when it is set, "Currently Hidden Tracks" when it is not (hgSearch.c:174, js/hgSearch.js:44). The symptom is a search result for a track you have on in the browser filed under the hidden heading, where nobody looking for it will open it. This one needs NO LOGIN, which is why it is worth having even with the other two covered: it is the cheap one, about eight seconds. wgEncodeGencodeBasicV49 is the track because it satisfies both constraints at once. It is in hgFindSpec, so it has search results at all; and its chain is wgEncodeGencodeV49ViewGenes -> wgEncodeGencodeV49 (visibility 0) -> wgEncodeGencodeSuper, which is `superTrack on` with no `show`, so isShow is FALSE and a build that reads the cart wrongly stops the walk at the superTrack. A top-level or default-visible track would have passed on the broken build. The hideKids step is not tidiness. Showing the superTrack brings every other archived version up at its own visibility -- V50 draws alongside and lands in Visible Tracks too -- so without it the `exact:` would need rewriting every time GENCODE ships a version. It costs 31 variables, one per archived composite. ENST00000297261 rather than a gene symbol: one of SHH's transcripts, it reaches the GENCODE sets, and it searches 65 categories where SHH searches 319. No count is asserted anywhere -- every one of those numbers moves when a track is reloaded. Same wait discipline as collection.docent.yaml, and for the same reason: wait for EITHER heading, so a build that files the track on the wrong side fails at the expect: naming the selector it wanted rather than timing out 15 seconds later with a Playwright message. Measured both ways. The same script with the track left hidden fails at the assertion: step 7 (expect) failed: nothing matches "li[id="Visible Tracks"] li#wgEncodeGencodeBasicV49" No derive baseline on purpose: the hideKids expansion is one variable per archived GENCODE version, so a baseline would go red twice a year for a reason that is not a bug. views is left out of expected/ for the same reason. tests/ is 19 of 19 with four fixtures resolved and the derive baselines matching.
• src/hg/utils/hgConfCatalog/harvestHgConf.py
  • lines changed 143, context: html, text, full: html, text
    6cf2b25c7f0c34dedf380a38e8833b58f4e078eb Wed Sep 16 09:42:24 2026 -0700
    hgConfCatalog: date a setting by the release branch that holds its commit, refs #37925 version_at() mapped a commit's timestamp onto the CGI_VERSION in effect at the time. That is off by at least one release by construction, because the version stamped at or before a commit is the release already branched, and it is off by more when a commit is written before a cut and merged after it. All 28 flips in the age cache were wrong: 25 by one release, two by two, and storeUserFiles by 22. The report built on this tells somebody to delete the hg.conf line that a flipped default made pointless, but only once the release that flipped it is installed on the machine reading the file. Acting a release early turns the feature off on hgwbeta or the RR. Date a commit by the first v*_branch that contains it instead. Checked against a containment test per branch: exact on all 30 flips and on 422 of the 429 introducing commits, the rest being 2001 settings reported as v10 rather than v9, since the oldest branch in the tree is v9. The cut points come from merge-base with master, not from the "New version number vNNN" commit, which is the branch point for the recent releases but was not for 163 of the 494 branches, the last of them v409. Also report a flip that no branch carries yet as "not released yet" rather than as "since vNNN", and refuse to write an undated cache when a checkout has no release branches at all.
• src/hg/utils/hgConfCatalog/hgConfAges.json
  • lines changed 1683, context: html, text, full: html, text
    6cf2b25c7f0c34dedf380a38e8833b58f4e078eb Wed Sep 16 09:42:24 2026 -0700
    hgConfCatalog: date a setting by the release branch that holds its commit, refs #37925 version_at() mapped a commit's timestamp onto the CGI_VERSION in effect at the time. That is off by at least one release by construction, because the version stamped at or before a commit is the release already branched, and it is off by more when a commit is written before a cut and merged after it. All 28 flips in the age cache were wrong: 25 by one release, two by two, and storeUserFiles by 22. The report built on this tells somebody to delete the hg.conf line that a flipped default made pointless, but only once the release that flipped it is installed on the machine reading the file. Acting a release early turns the feature off on hgwbeta or the RR. Date a commit by the first v*_branch that contains it instead. Checked against a containment test per branch: exact on all 30 flips and on 422 of the 429 introducing commits, the rest being 2001 settings reported as v10 rather than v9, since the oldest branch in the tree is v9. The cut points come from merge-base with master, not from the "New version number vNNN" commit, which is the branch point for the recent releases but was not for 163 of the 494 branches, the last of them v409. Also report a flip that no branch carries yet as "not released yet" rather than as "since vNNN", and refuse to write an undated cache when a checkout has no release branches at all.
• src/hg/utils/hgConfCatalog/hgConfCatalog.py
  • lines changed 12, context: html, text, full: html, text
    6cf2b25c7f0c34dedf380a38e8833b58f4e078eb Wed Sep 16 09:42:24 2026 -0700
    hgConfCatalog: date a setting by the release branch that holds its commit, refs #37925 version_at() mapped a commit's timestamp onto the CGI_VERSION in effect at the time. That is off by at least one release by construction, because the version stamped at or before a commit is the release already branched, and it is off by more when a commit is written before a cut and merged after it. All 28 flips in the age cache were wrong: 25 by one release, two by two, and storeUserFiles by 22. The report built on this tells somebody to delete the hg.conf line that a flipped default made pointless, but only once the release that flipped it is installed on the machine reading the file. Acting a release early turns the feature off on hgwbeta or the RR. Date a commit by the first v*_branch that contains it instead. Checked against a containment test per branch: exact on all 30 flips and on 422 of the 429 introducing commits, the rest being 2001 settings reported as v10 rather than v9, since the oldest branch in the tree is v9. The cut points come from merge-base with master, not from the "New version number vNNN" commit, which is the branch point for the recent releases but was not for 163 of the 494 branches, the last of them v409. Also report a flip that no branch carries yet as "not released yet" rather than as "since vNNN", and refuse to write an undated cache when a checkout has no release branches at all.
  • lines changed 15, context: html, text, full: html, text
    d24129dec53069cb4d8f0683e794e65ddd819926 Thu Sep 17 05:18:35 2026 -0700
    hgConfCatalog: register showManeInSearch gate, refs #38285
  • lines changed 42, context: html, text, full: html, text
    b45b48de3c6266567adf6784a97d71bb47f4fbd0 Fri Sep 18 08:39:32 2026 -0700
    hgConfCatalog: a run-time-built name is not a boolean flag to classify, refs #37925 hgLogin's social sign-in now reads login.oauth.<provider>.trustEmail with a boolean accessor, so the {key} row started failing the rule that every boolean flag is a gate or a knob. That row stands for a whole family, read elsewhere as a string and as a credential, so neither answer is true of it and there is no single default for the sunset report to follow. The Runtime-built names section is now exempt from that check, and the row's note lists trustEmail as the one field read as a boolean.
  • lines changed 16, context: html, text, full: html, text
    14ecff0730596f8be147b255afc8acb99e9095e2 Fri Sep 18 09:06:06 2026 -0700
    hgHubConnect: an api key made on one geo mirror now works on all of them, behind syncHubApiKeys #Preview2 week - bugs introduced now will need a build patch to fix #Preview2 week - bugs introduced now will need a build patch to fix Api keys live in the hgcentral of whichever mirror issued them, so a key made on genome.ucsc.edu was rejected on genome-euro and genome-asia, and the botDelay error told the user to go and make a second one. hgHubConnect now tells the other nodes about a key as soon as it generates or revokes one: cjGenerateApiKey and cjRevokeApiKey call syncApiKeyToOtherNodes(), which posts a hgHubSyncApiKey cartJson request to each peer, and cjSyncApiKey() on the far side writes it into that mirror's own table. The peers come from hgcentral.gbNode via the new geoMirrorNotifyOtherNodes(), and the request is signed with login.cookieSalt, which all of a site's mirrors already share - it is what makes the login cookie verifiable on each of them. So neither a list of mirror addresses nor a new shared secret needs provisioning. The notify is best effort: a peer that is down is warn()ed about and skipped, never failing the local generate or revoke, which has already committed by then. All of it is off unless hg.conf says syncHubApiKeys=on. With the gate off the sender returns at once, the receiver refuses the request outright rather than merely being unreachable, and botDelay keeps printing the old server-specific wording, which off is still the truth. Registered in hgConfCatalog.py as a gate, to be flipped once this is released. refs #38323
  • lines changed 7, context: html, text, full: html, text
    91ecf5d0d600ed4117a08a559725e80c8061a264 Sat Sep 19 02:30:57 2026 -0700
    hgConfCatalog: register sessionLoadNotice, refs #37925 Written by nightlyRegister.sh, which records the settings the tree reads that the catalog was missing. Only facts copied off the call site are filled in. No classification is guessed: a new boolean gets no role=, because calling a release gate a knob would hide it from the sunset report for good, and every row lands in the 'Awaiting review' section until somebody reads the call site. sessionLoadNotice hg/hgTracks/hgTracks.c:12219 (38157) wrote 1 row to the 'Awaiting review' section of hgConfCatalog.py; classify it and move it out
  • lines changed 28, context: html, text, full: html, text
    517996ce766d0a844b5736dc3864db2243e22577 Sat Sep 19 07:17:31 2026 -0700
    hgConfCatalog: sessionLoadNotice is a mirror knob, refs #37925 --auto-register wrote the row down on 2026-09-19; this classifies it and moves it out of Awaiting review. The flag is read with cfgOptionBooleanDefault at hg/hgTracks/hgTracks.c and decides whether hgTracks shows the note that names the session just opened and says the configuration the user had before is gone. It was born TRUE at 3bcf86a0995 and is documented in product/ex.hg.conf as an off switch, so it never held the feature back, and the off position is for a site whose users open sessions constantly and do not need telling. Filed with debatable= rather than quietly: a site-wide off switch born in the same commit as a user-visible change has the shape of a gate, and if the note turns out to be uncontroversial the switch should be deleted rather than kept.
  • lines changed 8, context: html, text, full: html, text
    e7b649e254779c18430f815e205f504c537d66c8 Sun Sep 20 02:31:07 2026 -0700
    hgConfCatalog: register denseClick, refs #37925 Written by nightlyRegister.sh, which records the settings the tree reads that the catalog was missing. Only facts copied off the call site are filled in. No classification is guessed: a new boolean gets no role=, because calling a release gate a knob would hide it from the sunset report for good, and every row lands in the 'Awaiting review' section until somebody reads the call site. denseClick hg/hgTracks/simpleTracks.c:5934 (38364) wrote 1 row to the 'Awaiting review' section of hgConfCatalog.py; classify it and move it out
• src/hg/utils/otto/genArk/loopDetect.sh
  • lines changed 51, context: html, text, full: html, text
    dda07606495066058a431b1ec5924e6d829b9562 Thu Sep 17 11:59:59 2026 -0700
    otto script to detect recursive directory loops no redmine
• src/hg/utils/otto/genArk/whatIsNew.sh
  • lines changed 7, context: html, text, full: html, text
    59737e5bedffc697d6f783c71b56ba711aac7deb Thu Sep 17 12:02:08 2026 -0700
    adding loopDetect.sh no redmine
  • lines changed 2, context: html, text, full: html, text
    2da75463c55d14a61c509128ba82733f39e0c784 Thu Sep 17 12:04:16 2026 -0700
    correctly detect count difference in dev,beta and hgw1 listings no redmine
• src/hg/utils/otto/methbase2/makefile
  • lines changed 20, context: html, text, full: html, text
    02ceec685f431d51256bdddd3cbc952ed5b37396 Fri Sep 18 07:08:40 2026 -0700
    methbase2 otto: makefile to install the scripts into the otto directory, refs #34246 #Preview2 week - bugs introduced now will need a build patch to fix "make install" syntax-checks the three scripts and copies them to /hive/data/outside/otto/methbase2/, following the other otto makefiles. methbaseDownload is installed there for the record, but it does not run there: methbaseOtto.sh ssh's to hgdownload and runs it against the mirror, so the data never crosses to hgwdev. "make installRemote" pushes it to hgdownload, kept as a separate target so copying to another machine is never a side effect of "make install".
• src/hg/utils/otto/methbase2/methbaseDownload
  • lines changed 0, context: html, text, full: html, text
    aeb4d42771eb1a6cdd300b2b5e7df3929e99566d Fri Sep 18 05:32:13 2026 -0700
    methbase2 otto: monthly mirror refresh, and move methbaseDownload into the otto dir #Preview2 week - bugs introduced now will need a build patch to fix The mirror lives on hgdownload and is refreshed there in place, so the 6 TB never passes through hgwdev. Only the trackDb text comes back: the script fetches the mirror's hg38 trackDb, drops the composite header, and commits the subtrack stanzas if they changed. Silent when nothing changed, which is the normal monthly outcome. The fetch is checked for length, for a leading track stanza and for the expected bigDataUrl prefix before anything is committed, so a truncated download or an error page cannot land in trackDb. methbaseDownload moves from oneShot to the otto directory next to processMethbaseHub, refs #34246
• src/hg/utils/otto/methbase2/methbaseOtto.sh
  • lines changed 100, context: html, text, full: html, text
    945e19cbeb388a37b2a9af831ccc395ed1d01622 Fri Sep 18 05:32:24 2026 -0700
    methbase2 otto: monthly script to refresh the mirror and the hg38 subtracks, refs #34246 #Preview2 week - bugs introduced now will need a build patch to fix The mirror lives on hgdownload and is refreshed there in place, so the 6 TB never passes through hgwdev. Only the trackDb text comes back: the script fetches the mirror's hg38 trackDb, drops the composite header, and commits the subtrack stanzas if they changed. Silent when nothing changed, which is the normal monthly outcome. The fetch is checked for length, for a leading track stanza and for the expected bigDataUrl prefix before anything is committed, so a truncated download or an error page cannot land in trackDb.
• src/hg/utils/otto/otto.crontab
  • lines changed 3, context: html, text, full: html, text
    69f4ef65bd1f1d15304c3fe0a47e701d58d6e688 Fri Sep 18 13:05:49 2026 -0700
    trackLists: take the otto assembly counts from the jobs, and quiet the weekly mail. refs #38295 Code review feedback from Lou on the mirror track lists page. The Assemblies column counted every assembly with a table matching the job's keyword, not the assemblies the job rebuilds, so NCBI RefSeq read 84 where ottoNcbiRefSeq.sh runs four and GRC Incident read 61 where runUpdate.sh works through ten. Each job now carries the list it actually builds, read out of its script; UniProt keeps the trackDb count because doUniprot chooses its assemblies at run time. A run reports either way the list and trackDb disagree. An otto job the keyword table did not recognize vanished from the page without a word, which is how STRchive went missing from it. STRchive is described now, and anything else unrecognized is named on the page and in the cron mail. The weekly mail was the whole progress log with the reachable restricted files buried in it, because every note() went to stderr and the "updated" line fired every run. Progress now goes to stdout and on to lastRun.log; stderr carries only what needs a person, so mail from this cron line means something is wrong. The "every file marked as restricted is correctly blocked" line could print on a run where no fetch got an answer, and covered only the 39 of 58 rows that name a file. It now says how many were tried and stays quiet about blocking when the check could not finish. Four rows reading "Mutation: A" through "Mutation: T" sat under M in the restricted table, nowhere near AlphaGenome. A subtrack of a listed track now sorts directly under that track, keeping its own row, since each of these is a separate file a mirror cannot have. A track named after its container is labelled with the container as well, so varFreqsBackground reads "SNV Frequencies: Population reference" rather than "Population reference". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/otto/panelApp/doPanelApp.py
  • lines changed 18, context: html, text, full: html, text
    a021efbee675889414298c97c15a6103e5f058f9 Tue Sep 15 15:58:19 2026 -0700
    Adding a flag for panelApp to ignore the threshold and update the files anyways. Previously was commenting out lines and re-adding them each time once the script finished. No redmine
• src/hg/utils/otto/trackLists/README.txt
  • lines changed 37, context: html, text, full: html, text
    69f4ef65bd1f1d15304c3fe0a47e701d58d6e688 Fri Sep 18 13:05:49 2026 -0700
    trackLists: take the otto assembly counts from the jobs, and quiet the weekly mail. refs #38295 Code review feedback from Lou on the mirror track lists page. The Assemblies column counted every assembly with a table matching the job's keyword, not the assemblies the job rebuilds, so NCBI RefSeq read 84 where ottoNcbiRefSeq.sh runs four and GRC Incident read 61 where runUpdate.sh works through ten. Each job now carries the list it actually builds, read out of its script; UniProt keeps the trackDb count because doUniprot chooses its assemblies at run time. A run reports either way the list and trackDb disagree. An otto job the keyword table did not recognize vanished from the page without a word, which is how STRchive went missing from it. STRchive is described now, and anything else unrecognized is named on the page and in the cron mail. The weekly mail was the whole progress log with the reachable restricted files buried in it, because every note() went to stderr and the "updated" line fired every run. Progress now goes to stdout and on to lastRun.log; stderr carries only what needs a person, so mail from this cron line means something is wrong. The "every file marked as restricted is correctly blocked" line could print on a run where no fetch got an answer, and covered only the 39 of 58 rows that name a file. It now says how many were tried and stays quiet about blocking when the check could not finish. Four rows reading "Mutation: A" through "Mutation: T" sat under M in the restricted table, nowhere near AlphaGenome. A subtrack of a listed track now sorts directly under that track, keeping its own row, since each of these is a separate file a mirror cannot have. A track named after its container is labelled with the container as well, so varFreqsBackground reads "SNV Frequencies: Population reference" rather than "Population reference". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/otto/trackLists/collect.py
  • lines changed 178, context: html, text, full: html, text
    69f4ef65bd1f1d15304c3fe0a47e701d58d6e688 Fri Sep 18 13:05:49 2026 -0700
    trackLists: take the otto assembly counts from the jobs, and quiet the weekly mail. refs #38295 Code review feedback from Lou on the mirror track lists page. The Assemblies column counted every assembly with a table matching the job's keyword, not the assemblies the job rebuilds, so NCBI RefSeq read 84 where ottoNcbiRefSeq.sh runs four and GRC Incident read 61 where runUpdate.sh works through ten. Each job now carries the list it actually builds, read out of its script; UniProt keeps the trackDb count because doUniprot chooses its assemblies at run time. A run reports either way the list and trackDb disagree. An otto job the keyword table did not recognize vanished from the page without a word, which is how STRchive went missing from it. STRchive is described now, and anything else unrecognized is named on the page and in the cron mail. The weekly mail was the whole progress log with the reachable restricted files buried in it, because every note() went to stderr and the "updated" line fired every run. Progress now goes to stdout and on to lastRun.log; stderr carries only what needs a person, so mail from this cron line means something is wrong. The "every file marked as restricted is correctly blocked" line could print on a run where no fetch got an answer, and covered only the 39 of 58 rows that name a file. It now says how many were tried and stays quiet about blocking when the check could not finish. Four rows reading "Mutation: A" through "Mutation: T" sat under M in the restricted table, nowhere near AlphaGenome. A subtrack of a listed track now sorts directly under that track, keeping its own row, since each of these is a separate file a mirror cannot have. A track named after its container is labelled with the container as well, so varFreqsBackground reads "SNV Frequencies: Population reference" rather than "Population reference". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/otto/trackLists/mkPage.py
  • lines changed 87, context: html, text, full: html, text
    69f4ef65bd1f1d15304c3fe0a47e701d58d6e688 Fri Sep 18 13:05:49 2026 -0700
    trackLists: take the otto assembly counts from the jobs, and quiet the weekly mail. refs #38295 Code review feedback from Lou on the mirror track lists page. The Assemblies column counted every assembly with a table matching the job's keyword, not the assemblies the job rebuilds, so NCBI RefSeq read 84 where ottoNcbiRefSeq.sh runs four and GRC Incident read 61 where runUpdate.sh works through ten. Each job now carries the list it actually builds, read out of its script; UniProt keeps the trackDb count because doUniprot chooses its assemblies at run time. A run reports either way the list and trackDb disagree. An otto job the keyword table did not recognize vanished from the page without a word, which is how STRchive went missing from it. STRchive is described now, and anything else unrecognized is named on the page and in the cron mail. The weekly mail was the whole progress log with the reachable restricted files buried in it, because every note() went to stderr and the "updated" line fired every run. Progress now goes to stdout and on to lastRun.log; stderr carries only what needs a person, so mail from this cron line means something is wrong. The "every file marked as restricted is correctly blocked" line could print on a run where no fetch got an answer, and covered only the 39 of 58 rows that name a file. It now says how many were tried and stays quiet about blocking when the check could not finish. Four rows reading "Mutation: A" through "Mutation: T" sat under M in the restricted table, nowhere near AlphaGenome. A subtrack of a listed track now sorts directly under that track, keeping its own row, since each of these is a separate file a mirror cannot have. A track named after its container is labelled with the container as well, so varFreqsBackground reads "SNV Frequencies: Population reference" rather than "Population reference". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/otto/trackLists/trackLists.sh
  • lines changed 16, context: html, text, full: html, text
    69f4ef65bd1f1d15304c3fe0a47e701d58d6e688 Fri Sep 18 13:05:49 2026 -0700
    trackLists: take the otto assembly counts from the jobs, and quiet the weekly mail. refs #38295 Code review feedback from Lou on the mirror track lists page. The Assemblies column counted every assembly with a table matching the job's keyword, not the assemblies the job rebuilds, so NCBI RefSeq read 84 where ottoNcbiRefSeq.sh runs four and GRC Incident read 61 where runUpdate.sh works through ten. Each job now carries the list it actually builds, read out of its script; UniProt keeps the trackDb count because doUniprot chooses its assemblies at run time. A run reports either way the list and trackDb disagree. An otto job the keyword table did not recognize vanished from the page without a word, which is how STRchive went missing from it. STRchive is described now, and anything else unrecognized is named on the page and in the cron mail. The weekly mail was the whole progress log with the reachable restricted files buried in it, because every note() went to stderr and the "updated" line fired every run. Progress now goes to stdout and on to lastRun.log; stderr carries only what needs a person, so mail from this cron line means something is wrong. The "every file marked as restricted is correctly blocked" line could print on a run where no fetch got an answer, and covered only the 39 of 58 rows that name a file. It now says how many were tried and stays quiet about blocking when the check could not finish. Four rows reading "Mutation: A" through "Mutation: T" sat under M in the restricted table, nowhere near AlphaGenome. A subtrack of a listed track now sorts directly under that track, keeping its own row, since each of these is a separate file a mirror cannot have. A track named after its container is labelled with the container as well, so varFreqsBackground reads "SNV Frequencies: Population reference" rather than "Population reference". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/otto/uniprot/doUniprot
  • lines changed 67, context: html, text, full: html, text
    9541aea4204c8079ea2e464403f2225f6aa33cfd Mon Sep 14 07:59:00 2026 -0700
    uniprot: show the splice variant track, and filter the CAT alignments Two things that were quietly missing. unipSplice has been built since the pipeline rewrite in #19351 and never had a trackDb stanza - "git log -S unipSplice" on uniprot.ra returns nothing, so it was never wired up rather than deliberately dropped. It holds UniProt's splice variant features, 28962 of them on hg38 and 29195 on hs1, and has been invisible on every assembly for years. Added to uniprot.ra and to the archive/contrib template, which is where the hub-served assemblies get their trackDb. The file exists on the same 131 assemblies as unipDomain.bb, so no stanza points at anything missing. The alignments on a CAT assembly were not being filtered with pslSelect. That filter maps a UniProt accession to the transcripts UniProt cross-references for it, and the README is blunt about how much it matters for protein families with nearly identical transcripts. It understood two kinds of id, Ensembl and RefSeq, and CAT names a transcript after its source gene, so hs1 matched neither and fell through unfiltered. Every row of the CAT bigBed carries the Ensembl transcript it was lifted from, so catSourceTransMap joins on that and UniProt's Ensembl cross-reference does the rest. The mapping had to become one-to-many for this: one Ensembl transcript can name several of ours, both because paralogs are lifted from the same source and because duplicate CAT names were given -dup suffixes earlier. It reproduces those suffixes by walking the bigBed in the same order rather than attaching every paralog to every source, which measured 234903 of 234903 hs1 transcripts mapped, no duplicates, against 330749 for the loose version. For every other gene track the lists hold one element and the behaviour is unchanged, which is checked: a single-valued entry still writes exactly one pair line and still counts a version difference, a multi-valued one writes a line per transcript, and an id we do not have is still skipped so pslSelect -qPass passes it through. refs #38300
  • lines changed 19, context: html, text, full: html, text
    5afbafb6dde0b144cca0fe3f377a5b284e3561d3 Mon Sep 14 10:26:17 2026 -0700
    uniprot otto: an organism whose proteins do not align should not stop the run aquChr2, the golden eagle, took the run down with a bare AssertionError: on assert(os.path.getsize(mapFname)!=0). UniProt has two proteins for taxon 216574 and neither aligned, so the mapping PSL came out empty. That is a fact about the data rather than a failure of the pipeline, and it is the same situation as an organism with no proteins at all, which is already skipped with a warning. Several more taxa in the plan are in the same position, with 1, 2, 8, 13 and 17 proteins. The assembly is now skipped with a message naming it, both when the mapping was just built and when an empty one is being reused from an earlier run. Worker failures also log a traceback now. errAbort raises a bare AssertionError, so the pool reported exactly "taxon 216574 failed: AssertionError:" and said nothing about where, which is worse than what the sequential path used to print. Checked that the errAbort message, the traceback and the file and line all appear. refs #38300
  • lines changed 19, context: html, text, full: html, text
    08221b327a1496fd1e4d3f88dc17e839b15b64f7 Tue Sep 15 05:24:18 2026 -0700
    uniprot otto: minAli must not be global, and must be part of the cache key max spotted that ci3's lower alignment threshold was assigned to the global MINALI. With --taxonThreads that leaks: ci3 sets 0.85 and every assembly whose thread reads the global afterwards uses it too. In the 2026_03 run it did exactly that - 89 of 95 alignments ran at 0.85 instead of 0.93, and which assemblies escaped depended purely on thread timing, with ce11 and cb3 getting 0.93 by luck. His fix makes it a local; this commit keeps that and closes the second half of the problem. The mapping files are cached under an md5 of the protein sequences, the transcripts, their alignment and the protein/transcript pair file. The threshold was not in it, so a mapping built at 0.85 would be silently reused on a run that asked for 0.93, and the bad alignments would have survived the rebuild that is meant to fix them. minAli is now part of the key, so changing it invalidates the mapping by itself. Checked that the key differs between 0.93 and 0.85. Also gave lastRun_noAnnotatedTranscriptIDs.txt a per-assembly name. It is written into the shared mapDir under a fixed name by every assembly, which several threads would write at once. Only a debugging artifact, nothing reads it, but it is the last shared write in the per-taxon path. Audited the rest of the module for state shared across threads: the only remaining global is flagFname, set once in main before the pool starts; dbIsHubCache can be written by two threads but only ever with the same value; and every temp directory and working file is keyed by assembly, md5 or taxon. refs #38300
  • lines changed 12, context: html, text, full: html, text
    3d9c0012a395a31127c3af3459c4e28c48dda205 Thu Sep 17 01:09:25 2026 -0700
    uniprot otto: keep updating the classic assembly when a GenArk one appears #Preview2 week - bugs introduced now will need a build patch to fix getTaxIdDbs took the first assembly per taxon by orderKey. GenArk assemblies now sort ahead of the classic ones, so the moment ARS_UCD2.0 appeared for cow, bosTau9 stopped being updated: its UniProt tracks froze at whatever release they had while the new data went into a GenArk contrib collection that is not installed. Users are still on bosTau9. The same had happened for marmoset, zebrafish, gorilla, cat, macaque, chicken, rat, horse, sheep, chimp, bonobo, orangutan, zebra finch and two more, sixteen taxa in all. Take the first of each kind instead, one GenArk and one classic, so both keep getting the current release. The plan goes from 119 to 135 assemblies; the taxa count is unchanged at 109, and no assembly appears twice, so the assertion at the end of the function still holds. Human, mouse, fly, worm and yeast are unaffected. refs #38300
  • lines changed 13, context: html, text, full: html, text
    145ba2031c9abbd4efbc5c8c8188f0b298fd5259 Thu Sep 17 05:28:05 2026 -0700
    uniprot otto: skip barely annotated taxa, and default to 20 at a time #Preview2 week - bugs introduced now will need a build patch to fix --minProteins skips a taxon with fewer than that many UniProt proteins. The default is 1, so only the empty ones go, which is what the old check did; the point of the option is running across many assemblies. UniProt annotates most species barely at all: of the 3974 taxa that have both a GenArk assembly and a SwissProt entry, the median has four proteins and only 558 have more than a hundred. Four proteins cannot make a useful track, and it is the case that took the last run down, since two proteins that do not align leave an empty mapping. Counting from the accession table the fasta was built from rather than from the file size also means the message says how many proteins there were. --taxonThreads now defaults to 20 rather than 1. The pool has a full successful run behind it at 4, the serial part of an assembly is single threaded and leaves the cluster idle, and with the smaller job count from the previous commit twenty concurrent taxa is a reasonable load. refs #38300
  • lines changed 123, context: html, text, full: html, text
    7fe928bd58b07ee240fa4b2a1cc5f7e8318f0a9a Thu Sep 17 06:11:37 2026 -0700
    uniprot otto: build the plan from the GenArk list, filtered by proteome size #Preview2 week - bugs introduced now will need a build patch to fix --genArkList takes the GenArk assembly list, the same tab separated file the genark otto job pulls from hgdownload, and builds the plan from it instead of from dbDb. dbDb holds the few thousand assemblies the browser serves directly; GenArk holds about fifty thousand. Most of those are not worth running. UniProt covers most species only through TrEMBL and often with a handful of proteins: of the 43821 distinct GenArk taxa, 32492 have some UniProt data but the median has 2214 proteins, and an organism with a few hundred cannot make a useful track. --minProteins is the filter. It wants to start high: a bacterial proteome is only a few thousand proteins, so a threshold in the low thousands pulls in the whole bacteria clade, while 20000 keeps it to the large eukaryotic proteomes. At 20000 the plan is 2224 assemblies over 664 taxa, against 52777 assemblies listed. The protein counts come from the OX= field of the UniProt fasta headers, since they have to be known before anything is parsed. Reading TrEMBL means 40 GB and about twenty minutes, so the result is cached beside the download and rebuilt only when it is older than the fasta. genArkHubDir now resolves an assembly named by its accession directly, sharding the digits three at a time, so nothing has to be in dbDb for this to work. The list is read as latin1: the organism names carry bytes that are not valid UTF-8 and python stops on the first one. refs #38300
  • lines changed 41, context: html, text, full: html, text
    62f11347129c7aa7067c03879d2b6f3d9b05426a Thu Sep 17 10:53:15 2026 -0700
    uniprot otto: a skip list of assemblies to leave alone #Preview2 week - bugs introduced now will need a build patch to fix --skipList names a file of assemblies not to build, one per line, with # comments so each entry can say why. Entries may carry the full asmId with its assembly-name suffix, which is how the GenArk orderLists write them, and only the accession part is matched, so an orderList can be handed over unchanged: --skipList=kent/src/hg/makeDb/doc/hprcAsmHub/hprc.orderList.tsv That is the case it was written for. GenArk lists 572 human assemblies, 464 of them HPRC haplotypes, and the pool works taxon by taxon, so all 572 would have run one after another in a single slot while the other 663 taxa finished in hours. Dropping the HPRC ones takes human to 108 assemblies and the whole 20000-protein plan from 2224 to 1760. Applies to the dbDb plan as well, alongside notAutoDbs, so a troublesome assembly can be kept out of the monthly run without editing the script. refs #38300
  • lines changed 21, context: html, text, full: html, text
    1769eab7a2ee4a732e0de1152e6f50c1be8dc585 Thu Sep 17 11:00:45 2026 -0700
    uniprot otto: only the newest version of an assembly, and skip whole assemblies #Preview2 week - bugs introduced now will need a build patch to fix GenArk lists every version it has ever carried, so GCA_018466835.1 sits beside GCA_018466835.2. Building the old one produces a track nobody will look at. Keep only the highest version of each accession. The order matters, and getting it wrong is silent. Doing this after the skip list leaves the old version behind wherever the new one was skipped, which is exactly the HPRC case: 94 of the 108 human assemblies that survived the HPRC orderList were .1 versions whose .2 had just been removed, so the filtering had quietly swapped the current assemblies for their predecessors. The versions are now collapsed against the whole list, before anything else is filtered. For the same reason an entry on the skip list now matches the assembly rather than one of its versions, so naming GCA_018466835.2 also drops GCA_018466835.1. Human goes from 572 GenArk assemblies to 14, and those 14 are the genuinely distinct ones: NA12878, mHomSap3, HG03492, HG01243, NA24385, RPE-1, H9 and an iPSC line. It is no longer the longest pole in the run; worm and mouse are. refs #38300
  • lines changed 5, context: html, text, full: html, text
    0ed3d3d300a94b6d8c94e98aa70ebad0294aa27e Fri Sep 18 09:44:12 2026 -0700
    uniprot otto: a missing tab/version.txt should mean parse, not crash #Preview2 week - bugs introduced now will need a build patch to fix updateUniprot opened tab/version.txt without checking it was there, so the run died with FileNotFoundError. The file is absent on a first run into an empty tab directory, and removing it is the natural way to force a reparse when the UniProt release has not changed but the set of taxa has, which is exactly the case when moving from the dbDb plan of 109 taxa to a GenArk plan of 664. Treat a missing file as "nothing parsed yet" and parse. refs #38300
  • lines changed 48, context: html, text, full: html, text
    befc3bcf5e350b84ccce24c7aba1acbb0f073c4f Sat Sep 19 09:07:54 2026 -0700
    uniprot otto: per-taxon NCBI tables, and lift info next to a reused mapping Two failures on the GenArk run, both of which would have hit about a quarter of the 664 taxa. The per-taxon NCBI gene to RefSeq tables were only built inside the download step, which --skipDownload turns off. Those tables are keyed on the plan rather than on the UniProt release, so a plan with taxa that have never been run needs them even when nothing has to be downloaded: 104 existed, for the old dbDb plan of 109 taxa, against 664 here. The download and the split are now separate, and the split runs whenever a taxon in the plan has no table yet. Separating them turned up a third thing. The download unpacked to ncbi/gene2refseq.tsv, 16 GB, while the split read ncbi/gene2refseq.gz, so the two named different files and the tables were being built from whatever .gz happened to be on disk. That was a copy from last November. It now downloads to the file the split reads, through a .tmp so an interrupted download cannot be mistaken for a complete one, and stays gzipped, which also saves the 16 GB. Separately, makeProteinGenomePsl writes the lift info only when it builds the mapping, not when it reuses one. A run interrupted between building the PSL and writing that file leaves the PSL on its own, and the next run reuses the PSL and then fails copying a file that was never written. Twenty assemblies were in that state after the aborted start earlier today. It is now written on the reuse path as well, when it is missing. refs #38300
• src/hg/utils/otto/uniprot/doUpdate.sh
  • lines changed 11, context: html, text, full: html, text
    d068f3195e28f60403917212dc139504b392e7c6 Tue Sep 15 04:56:49 2026 -0700
    uniprot otto: guard the shift in doUpdate.sh shift is a POSIX special built-in, so a shell that follows that rule exits when it fails. Under dash, which is /bin/sh on Debian and in most containers, running doUpdate.sh with no arguments ended the script at that line and doUniprot never started. Redirecting the complaint to /dev/null hid the reason, so the run log showed a START with no END, which reads exactly like a run still in progress. /bin/sh is bash on hgwdev, where it works.
• src/hg/utils/otto/uniprot/makeUniProtPsl.sh
  • lines changed 17, context: html, text, full: html, text
    17fb5d876fd59c39645d7589ed115e23208f8743 Thu Sep 17 05:27:00 2026 -0700
    uniprot otto: fewer, larger BLAST jobs #Preview2 week - bugs introduced now will need a build patch to fix The protein fasta was split into 2500 byte chunks regardless of its size, which for zebrafish meant 18894 cluster jobs averaging 13 seconds each. At that length parasol overhead and the creation of 18894 tiny result files cost more than the BLAST does. The cluster was never the problem: it finished 57 CPU hours of zebrafish alignment in 12 minutes of wall clock, a speedup of about 290. The cost is afterwards, in the single-threaded "find aligns | xargs cat | pslReps" that has to open every one of those files again over NFS, which is the slowest part of a per-assembly run. Split for a job count instead, about a thousand, keeping 2500 bytes as a floor so a small protein set still splits sensibly. Zebrafish goes from 18894 jobs to about 1000, each around five minutes, with the same total CPU and a twentieth of the files to concatenate. Also corrected the argument documentation at the top, which still described the parasol cluster argument that went away with ku. refs #38300
• src/hg/utils/otto/uniprot/trackDb.template.txt
  • lines changed 11, context: html, text, full: html, text
    9541aea4204c8079ea2e464403f2225f6aa33cfd Mon Sep 14 07:59:00 2026 -0700
    uniprot: show the splice variant track, and filter the CAT alignments Two things that were quietly missing. unipSplice has been built since the pipeline rewrite in #19351 and never had a trackDb stanza - "git log -S unipSplice" on uniprot.ra returns nothing, so it was never wired up rather than deliberately dropped. It holds UniProt's splice variant features, 28962 of them on hg38 and 29195 on hs1, and has been invisible on every assembly for years. Added to uniprot.ra and to the archive/contrib template, which is where the hub-served assemblies get their trackDb. The file exists on the same 131 assemblies as unipDomain.bb, so no stanza points at anything missing. The alignments on a CAT assembly were not being filtered with pslSelect. That filter maps a UniProt accession to the transcripts UniProt cross-references for it, and the README is blunt about how much it matters for protein families with nearly identical transcripts. It understood two kinds of id, Ensembl and RefSeq, and CAT names a transcript after its source gene, so hs1 matched neither and fell through unfiltered. Every row of the CAT bigBed carries the Ensembl transcript it was lifted from, so catSourceTransMap joins on that and UniProt's Ensembl cross-reference does the rest. The mapping had to become one-to-many for this: one Ensembl transcript can name several of ours, both because paralogs are lifted from the same source and because duplicate CAT names were given -dup suffixes earlier. It reproduces those suffixes by walking the bigBed in the same order rather than attaching every paralog to every source, which measured 234903 of 234903 hs1 transcripts mapped, no duplicates, against 330749 for the loose version. For every other gene track the lists hold one element and the behaviour is unchanged, which is checked: a single-valued entry still writes exactly one pair line and still counts a version difference, a multi-valued one writes a line per transcript, and an id we do not have is still skipped so pslSelect -qPass passes it through. refs #38300
• src/hg/utils/perfNightly/README.txt
  • lines changed 173, context: html, text, full: html, text
    cd41fd4fa0efadea0cd412d1a39e9479f3aa9832 Tue Sep 15 14:45:24 2026 -0700
    A nightly start-up and pixel check for hgTracks, refs #37547 Two builds of hgTracks, one machine, one moment: how long each takes to start up and draw, and whether they draw the same pixels. Eight cells -- native and GenArk, few tracks and many, narrow and wide. compare.sh is the engine and takes any two builds, so the same test that runs nightly against master also answers "is this branch slower than genome-test, and does it still draw the same thing?" A build is just a directory holding hgTracks and hgRenderTracks, which `make compile` in hg/hgTracks produces without installing anything. nightly.sh is the cron wrapper, in the same shape as the Docent regression nightly: its own clone, --update resets it to origin/master, a mail every night whether or not anything failed, sixty days of logs, always exit 0. Its baseline is a rolling reference binary from the last green night rather than a golden image, because two binaries reading the same data at the same moment cannot disagree about pixels for a data reason -- ClinVar, GENCODE and the GenArk hubs all move underneath a saved render. Five things here were measured rather than assumed, and each one silently produces a wrong answer if it is left out. README.txt has them with the numbers; the short version: - TRACKDB_VERSION is baked into each trackDb cache dump's filename, so a build that bumped it starts with an empty cache and pays a full parse per cell. Each side gets its own cache dir and a per-cell warmup, or a one-time cost reads as a permanent regression. - Replaying a saved session's variables as a GET does not reproduce the session: composite _sel state and the default-visibility pass live in hgSession's load path. Against a 71-track session a flat replay drew 23 of them and invented four more. The fixtures are loaded with hgS_doLoadUrl instead. - Running the binaries directly needs an htdocs symlink beside trash, or freetype cannot find its font and every render dies. - Always running side A first is worth 4.5% to side B, from the caches A just warmed. The order alternates. - trackLog names every track hgTracks loaded, not the drawn rows -- a default hg38 view logs 563 and draws 21. Both are checked. Cold and warm carts are reported separately and deliberately. A one-shot render gives every request a fresh cart, so it measures only the first load; on #37547 that reported the branch slower while a persistent cart showed it faster, on the same binaries. Nothing is wired into the tree-wide build, for the same reason the Docent tests are not: these drive a real server and need the network. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/perfNightly/compare.sh
  • lines changed 374, context: html, text, full: html, text
    cd41fd4fa0efadea0cd412d1a39e9479f3aa9832 Tue Sep 15 14:45:24 2026 -0700
    A nightly start-up and pixel check for hgTracks, refs #37547 Two builds of hgTracks, one machine, one moment: how long each takes to start up and draw, and whether they draw the same pixels. Eight cells -- native and GenArk, few tracks and many, narrow and wide. compare.sh is the engine and takes any two builds, so the same test that runs nightly against master also answers "is this branch slower than genome-test, and does it still draw the same thing?" A build is just a directory holding hgTracks and hgRenderTracks, which `make compile` in hg/hgTracks produces without installing anything. nightly.sh is the cron wrapper, in the same shape as the Docent regression nightly: its own clone, --update resets it to origin/master, a mail every night whether or not anything failed, sixty days of logs, always exit 0. Its baseline is a rolling reference binary from the last green night rather than a golden image, because two binaries reading the same data at the same moment cannot disagree about pixels for a data reason -- ClinVar, GENCODE and the GenArk hubs all move underneath a saved render. Five things here were measured rather than assumed, and each one silently produces a wrong answer if it is left out. README.txt has them with the numbers; the short version: - TRACKDB_VERSION is baked into each trackDb cache dump's filename, so a build that bumped it starts with an empty cache and pays a full parse per cell. Each side gets its own cache dir and a per-cell warmup, or a one-time cost reads as a permanent regression. - Replaying a saved session's variables as a GET does not reproduce the session: composite _sel state and the default-visibility pass live in hgSession's load path. Against a 71-track session a flat replay drew 23 of them and invented four more. The fixtures are loaded with hgS_doLoadUrl instead. - Running the binaries directly needs an htdocs symlink beside trash, or freetype cannot find its font and every render dies. - Always running side A first is worth 4.5% to side B, from the caches A just warmed. The order alternates. - trackLog names every track hgTracks loaded, not the drawn rows -- a default hg38 view logs 563 and draws 21. Both are checked. Cold and warm carts are reported separately and deliberately. A one-shot render gives every request a fresh cart, so it measures only the first load; on #37547 that reported the branch slower while a persistent cart showed it faster, on the same binaries. Nothing is wired into the tree-wide build, for the same reason the Docent tests are not: these drive a real server and need the network. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/perfNightly/nightly.sh
  • lines changed 261, context: html, text, full: html, text
    cd41fd4fa0efadea0cd412d1a39e9479f3aa9832 Tue Sep 15 14:45:24 2026 -0700
    A nightly start-up and pixel check for hgTracks, refs #37547 Two builds of hgTracks, one machine, one moment: how long each takes to start up and draw, and whether they draw the same pixels. Eight cells -- native and GenArk, few tracks and many, narrow and wide. compare.sh is the engine and takes any two builds, so the same test that runs nightly against master also answers "is this branch slower than genome-test, and does it still draw the same thing?" A build is just a directory holding hgTracks and hgRenderTracks, which `make compile` in hg/hgTracks produces without installing anything. nightly.sh is the cron wrapper, in the same shape as the Docent regression nightly: its own clone, --update resets it to origin/master, a mail every night whether or not anything failed, sixty days of logs, always exit 0. Its baseline is a rolling reference binary from the last green night rather than a golden image, because two binaries reading the same data at the same moment cannot disagree about pixels for a data reason -- ClinVar, GENCODE and the GenArk hubs all move underneath a saved render. Five things here were measured rather than assumed, and each one silently produces a wrong answer if it is left out. README.txt has them with the numbers; the short version: - TRACKDB_VERSION is baked into each trackDb cache dump's filename, so a build that bumped it starts with an empty cache and pays a full parse per cell. Each side gets its own cache dir and a per-cell warmup, or a one-time cost reads as a permanent regression. - Replaying a saved session's variables as a GET does not reproduce the session: composite _sel state and the default-visibility pass live in hgSession's load path. Against a 71-track session a flat replay drew 23 of them and invented four more. The fixtures are loaded with hgS_doLoadUrl instead. - Running the binaries directly needs an htdocs symlink beside trash, or freetype cannot find its font and every render dies. - Always running side A first is worth 4.5% to side B, from the caches A just warmed. The order alternates. - trackLog names every track hgTracks loaded, not the drawn rows -- a default hg38 view logs 563 and draws 21. Both are checked. Cold and warm carts are reported separately and deliberately. A one-shot render gives every request a fresh cart, so it measures only the first load; on #37547 that reported the branch slower while a persistent cart showed it faster, on the same binaries. Nothing is wired into the tree-wide build, for the same reason the Docent tests are not: these drive a real server and need the network. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 1, context: html, text, full: html, text
    3212c75cbe810013f1c3a0993934b0b70f6d7d2d Wed Sep 16 08:35:38 2026 -0700
    perfNightly: build hg/cgilib and optimalLeaf too, refs #37547 The build step ran topLibs plus hg/lib, which leaves jkhgapcgi.a and optimalLeaf.a alone. hgTracks links both (MYLIBS in hg/hgTracks/makefile) and has no rule to build them, so make used whatever was already on disk. A change under hg/cgilib or optimalLeaf would be pulled in by --update and then silently not relinked into today's binary, which is exactly the kind of regression this job exists to catch. 'make libs' is topLibs + hgLib + optLib and covers all four archives.
• src/hg/utils/perfNightly/scenarios.tsv
  • lines changed 30, context: html, text, full: html, text
    cd41fd4fa0efadea0cd412d1a39e9479f3aa9832 Tue Sep 15 14:45:24 2026 -0700
    A nightly start-up and pixel check for hgTracks, refs #37547 Two builds of hgTracks, one machine, one moment: how long each takes to start up and draw, and whether they draw the same pixels. Eight cells -- native and GenArk, few tracks and many, narrow and wide. compare.sh is the engine and takes any two builds, so the same test that runs nightly against master also answers "is this branch slower than genome-test, and does it still draw the same thing?" A build is just a directory holding hgTracks and hgRenderTracks, which `make compile` in hg/hgTracks produces without installing anything. nightly.sh is the cron wrapper, in the same shape as the Docent regression nightly: its own clone, --update resets it to origin/master, a mail every night whether or not anything failed, sixty days of logs, always exit 0. Its baseline is a rolling reference binary from the last green night rather than a golden image, because two binaries reading the same data at the same moment cannot disagree about pixels for a data reason -- ClinVar, GENCODE and the GenArk hubs all move underneath a saved render. Five things here were measured rather than assumed, and each one silently produces a wrong answer if it is left out. README.txt has them with the numbers; the short version: - TRACKDB_VERSION is baked into each trackDb cache dump's filename, so a build that bumped it starts with an empty cache and pays a full parse per cell. Each side gets its own cache dir and a per-cell warmup, or a one-time cost reads as a permanent regression. - Replaying a saved session's variables as a GET does not reproduce the session: composite _sel state and the default-visibility pass live in hgSession's load path. Against a 71-track session a flat replay drew 23 of them and invented four more. The fixtures are loaded with hgS_doLoadUrl instead. - Running the binaries directly needs an htdocs symlink beside trash, or freetype cannot find its font and every render dies. - Always running side A first is worth 4.5% to side B, from the caches A just warmed. The order alternates. - trackLog names every track hgTracks loaded, not the drawn rows -- a default hg38 view logs 563 and draws 21. Both are checked. Cold and warm carts are reported separately and deliberately. A one-shot render gives every request a fresh cart, so it measures only the first load; on #37547 that reported the branch slower while a persistent cart showed it faster, on the same binaries. Nothing is wired into the tree-wide build, for the same reason the Docent tests are not: these drive a real server and need the network. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/perfNightly/sessions/hg38-clinical.session
  • lines changed 754, context: html, text, full: html, text
    cd41fd4fa0efadea0cd412d1a39e9479f3aa9832 Tue Sep 15 14:45:24 2026 -0700
    A nightly start-up and pixel check for hgTracks, refs #37547 Two builds of hgTracks, one machine, one moment: how long each takes to start up and draw, and whether they draw the same pixels. Eight cells -- native and GenArk, few tracks and many, narrow and wide. compare.sh is the engine and takes any two builds, so the same test that runs nightly against master also answers "is this branch slower than genome-test, and does it still draw the same thing?" A build is just a directory holding hgTracks and hgRenderTracks, which `make compile` in hg/hgTracks produces without installing anything. nightly.sh is the cron wrapper, in the same shape as the Docent regression nightly: its own clone, --update resets it to origin/master, a mail every night whether or not anything failed, sixty days of logs, always exit 0. Its baseline is a rolling reference binary from the last green night rather than a golden image, because two binaries reading the same data at the same moment cannot disagree about pixels for a data reason -- ClinVar, GENCODE and the GenArk hubs all move underneath a saved render. Five things here were measured rather than assumed, and each one silently produces a wrong answer if it is left out. README.txt has them with the numbers; the short version: - TRACKDB_VERSION is baked into each trackDb cache dump's filename, so a build that bumped it starts with an empty cache and pays a full parse per cell. Each side gets its own cache dir and a per-cell warmup, or a one-time cost reads as a permanent regression. - Replaying a saved session's variables as a GET does not reproduce the session: composite _sel state and the default-visibility pass live in hgSession's load path. Against a 71-track session a flat replay drew 23 of them and invented four more. The fixtures are loaded with hgS_doLoadUrl instead. - Running the binaries directly needs an htdocs symlink beside trash, or freetype cannot find its font and every render dies. - Always running side A first is worth 4.5% to side B, from the caches A just warmed. The order alternates. - trackLog names every track hgTracks loaded, not the drawn rows -- a default hg38 view logs 563 and draws 21. Both are checked. Cold and warm carts are reported separately and deliberately. A one-shot render gives every request a fresh cart, so it measures only the first load; on #37547 that reported the branch slower while a persistent cart showed it faster, on the same binaries. Nothing is wired into the tree-wide build, for the same reason the Docent tests are not: these drive a real server and need the network. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/hg/utils/urlCommandCatalog/urlCommandCatalog.py
  • lines changed 18, context: html, text, full: html, text
    1eaa67ff615dc87804cf22eac3d198ca522d5f97 Fri Sep 18 08:39:39 2026 -0700
    urlCommandCatalog: baseline hgLogin's three activation-mail names, and cite a call site the same way every time, refs #37923 hgLogin_actMailProvider, hgLogin_actMailTo and hgLogin_actMailUser are written by the server and read back a few lines later. dropRequestSuppliedFlowVars() lists all three, so a copy arriving with the request is dropped before anything reads it, and none of them can be a browser URL parameter. They belong in the baseline beside oauth_pending_* and emailLogin_*, which are the same class. The citation on each baseline line was picked with setdefault over the harvest, so for a name read in several files it followed the order of the walk and moved whenever the file was regenerated from a different checkout. That rewrote about thirty unrelated lines and buried the name that changed. The pick is now the lowest (path, line), preferring a site under the CGIs the file header tells a reviewer to look at. Those thirty lines move once here and then stay put.
  • lines changed 41, context: html, text, full: html, text
    c2208fb9f91473fd3fa5d105d4a46523b374c303 Sat Sep 19 07:17:39 2026 -0700
    urlCommandCatalog: catalog the two mirror session endpoints and the session-load marker, refs #37923 All three arrived on 2026-09-18 and the nightly found them. hgS_doSessionListJson and hgS_doMirrorSessions are answered in main() before cartNew, so a node asking another node for a session list makes no cart and no userDb or sessionDb row. They carry nocart=True for that reason, and the rows name their callers: hgSession.c:2628 builds the first URL when it fans out, and hg/js/hgSession.js:769 calls the second. hgS_sessionJustLoaded is described rather than baselined. It is written by cartNew rather than typed, but it is an ordinary cart variable, so a request can supply it, and supplying it replays the note. That is the mistake that was made with hgS_shareAnon. A trackImgOnly render returns before the cartRemove, so the marker survives an image-only request and is consumed by the next full page.
• src/hg/utils/urlCommandCatalog/urlNamesNotCataloged.txt
  • lines changed 34, context: html, text, full: html, text
    1eaa67ff615dc87804cf22eac3d198ca522d5f97 Fri Sep 18 08:39:39 2026 -0700
    urlCommandCatalog: baseline hgLogin's three activation-mail names, and cite a call site the same way every time, refs #37923 hgLogin_actMailProvider, hgLogin_actMailTo and hgLogin_actMailUser are written by the server and read back a few lines later. dropRequestSuppliedFlowVars() lists all three, so a copy arriving with the request is dropped before anything reads it, and none of them can be a browser URL parameter. They belong in the baseline beside oauth_pending_* and emailLogin_*, which are the same class. The citation on each baseline line was picked with setdefault over the harvest, so for a name read in several files it followed the order of the walk and moved whenever the file was regenerated from a different checkout. That rewrote about thirty unrelated lines and buried the name that changed. The pick is now the lowest (path, line), preferring a site under the CGIs the file header tells a reviewer to look at. Those thirty lines move once here and then stay put.
  • lines changed 3, context: html, text, full: html, text
    83dd847b7449dd0fa83c22a4d60854f8d20d2847 Fri Sep 18 08:51:14 2026 -0700
    hgLogin: stop telling a CILogon user that CILogon shares no email address #Preview2 week - bugs introduced now will need a build patch to fix QA found three messages on the social sign-in pages claiming the provider does not give us an address. That was written when ORCID, which really releases none, was the only provider reaching those pages. CILogon does release the institution's address; we drop it because CILogon never marks it verified and login.oauth.cilogon.trustEmail is what says we may take it anyway. Once the address is dropped, nothing downstream could tell "released nothing" from "released something we would not take". oauthFetchIdentity now keeps the dropped address in a separate field that matching and sign-in never look at, and the three messages read it: the choose a username page, the confirmation page, and the one that turns down a typed address that already belongs to an account. The address box on the choose a username page starts from it as well, since retyping what we were just handed is the last thing a person wants to do. It still has to clear the duplicate check and be confirmed by mail, exactly as a typed address does. Three more things on the confirmation page, all from the same QA pass: The generic "a confirmation email has been sent" paragraph printed below the "use this address instead" form, so it read as if a mail had already gone to whatever was about to be typed in. It now goes above the form. Correcting the address redisplayed the same page with the new address swapped in and nothing to say the change had worked. It now says so. The rejection reason for an address that is already in use was set but never printed, so a refused change looked like nothing happening at all; that is printed now too. Reworded the unconfirmed-account explanation as QA suggested. refs #38339
• src/inc/errAbort.h
  • lines changed 3, context: html, text, full: html, text
    8c172e6c4b985af79bf1a1f47c9db92ce0578999 Tue Sep 15 05:47:36 2026 -0700
    errCatch: put errAbort's doContentType back the way it found it, refs #38353 hVaUserAbort() turns doContentType on so that a user error reaches the browser when the CGI has not pushed a warn handler of its own. Inside an errCatch that is pointless -- the abort is caught and never reaches the default handler -- and the flag stayed on afterwards, so a later genuine abort on a page that had already written its header would print a second Content-Type line into the body. Everywhere else in the tree the flag is something a caller turns back off. errCatchPushHandlers() now records the flag and errCatchEnd() restores it, which covers every caller rather than just the one that set it. errAbortGetDoContentType() is the reader.
• src/inc/errCatch.h
  • lines changed 1, context: html, text, full: html, text
    8c172e6c4b985af79bf1a1f47c9db92ce0578999 Tue Sep 15 05:47:36 2026 -0700
    errCatch: put errAbort's doContentType back the way it found it, refs #38353 hVaUserAbort() turns doContentType on so that a user error reaches the browser when the CGI has not pushed a warn handler of its own. Inside an errCatch that is pointless -- the abort is caught and never reaches the default handler -- and the flag stayed on afterwards, so a later genuine abort on a page that had already written its header would print a second Content-Type line into the body. Everywhere else in the tree the flag is something a caller turns back off. errCatchPushHandlers() now records the flag and errCatchEnd() restores it, which covers every caller rather than just the one that set it. errAbortGetDoContentType() is the reader.
• src/inc/hmac.h
  • lines changed 3, context: html, text, full: html, text
    33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
    hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323 Six things, all in the syncHubApiKeys path, which is still off everywhere. The sync never reached its handler on a real mirror. It was registered as a cartJson command, so building the cart ran forceUserIdOrCaptcha() first; a peer's request carries no hguid cookie and no apiKey= variable, so any site with cloudFlareSiteKey set -- which is every site that needs api keys in the first place -- answered it with a captcha page. Only hgwdev, where the captcha is commented out, ever ran the handler, which is why this was not caught earlier. doApiKeySyncIfRequested() now answers the request in main() before any cart exists. That also stops each sync leaving a junk userDb and sessionDb row behind: measured, 15 syncs now create none. The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key), so it does not rest on md5 resisting length extension, and it is compared in constant time. It now covers a timestamp too, and a peer refuses anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of now, so a captured sync cannot be replayed later to reinstate a key its owner has since revoked. The fields are joined with newlines and a newline in a userName or apiKey is refused, so one signature cannot cover two different splits of the same string. syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It runs after the local key is already written and in the response, so an errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's outer catch, which threw the response away -- the user saw a failure over a key that was sitting in the database. geoMirrorNotifyOtherNodes() returns each peer's response instead of discarding it, and uses netUrlMustOpenPastHeader, which errAborts on anything but a 200 and hands back the body alone. A peer answering 'bad signature' or 'not enabled on this site' is no longer indistinguishable from success; it is logged to error_log, not warn()ed at the person who clicked the button. Verified against the built CGI run as Apache runs it, with the captcha turned on: a signed sync is accepted and writes the row, a signed revoke removes it, and a replay, a swapped userName, a forged signature and malformed json are all refused. hubSpaceKeysTester covers the signature rules; it still cannot run its database half until hgcentralregress gets the UPDATE and DELETE grant.
• src/inc/htmshell.h
  • lines changed 5, context: html, text, full: html, text
    998149024f8bde04e8b76638043cfeac3d158731 Tue Sep 15 05:47:27 2026 -0700
    Queue the cart cookie and the content policy instead of printing them, refs #38353 cartWriteHeaderAndCont() guards on cgiDidContentType(), which any cgiPrintContentType() anywhere sets. An early warn() during cartNew -- "Unable to load session file" reaches the early warning handler, which calls htmlStart -- prints the header before there is a cart, and from then on the Set-Cookie and Content-Security-Policy lines were silently skipped. Nothing changed on the wire, since before the guard they landed in the page body as text and were equally inert, but skipping them silently is not the behaviour to keep. cartWriteCookie() and cspWriteResponseHeader() now hand their lines to cgiAddHttpHeader(), so whichever call prints the content type prints them too and the order of the calls no longer matters. cartAndCookieWithHtml() queues the policy before the early handlers are pushed, so even a page written by that early warn carries one. The cookie cannot be queued that early -- there is no cart yet -- and is still lost on that path; the comment says so. cgiAddHttpHeader() now does what its own comment already promised and ignores a header added after the block was closed, rather than growing a list nothing will ever print. getCspPolicyString() is declared in htmshell.h so hCommon.c can queue the value on its own.
  • lines changed 4, context: html, text, full: html, text
    3bacf54b6a23bc56f728c2e670c9b2cca96222dd Wed Sep 16 04:56:02 2026 -0700
    htmshell: drop the two CSP response-header helpers that no longer have callers, refs #38353 cspWriteResponseHeader() was their only user and it queues the policy through cgiAddHttpHeader() now, so generateCspResponseHeader() and getCspMetaResponseHeader() build a header line nothing sends. Neither had another caller at any point. The meta-tag side is untouched.
• src/inc/srcVersion.h
  • lines changed 1, context: html, text, full: html, text
    74623f843857f2254e5327c78b98f00c3f7098ff Mon Sep 21 10:00:48 2026 -0700
    New version number v504
• src/lib/cheapcgi.c
  • lines changed 2, context: html, text, full: html, text
    998149024f8bde04e8b76638043cfeac3d158731 Tue Sep 15 05:47:27 2026 -0700
    Queue the cart cookie and the content policy instead of printing them, refs #38353 cartWriteHeaderAndCont() guards on cgiDidContentType(), which any cgiPrintContentType() anywhere sets. An early warn() during cartNew -- "Unable to load session file" reaches the early warning handler, which calls htmlStart -- prints the header before there is a cart, and from then on the Set-Cookie and Content-Security-Policy lines were silently skipped. Nothing changed on the wire, since before the guard they landed in the page body as text and were equally inert, but skipping them silently is not the behaviour to keep. cartWriteCookie() and cspWriteResponseHeader() now hand their lines to cgiAddHttpHeader(), so whichever call prints the content type prints them too and the order of the calls no longer matters. cartAndCookieWithHtml() queues the policy before the early handlers are pushed, so even a page written by that early warn carries one. The cookie cannot be queued that early -- there is no cart yet -- and is still lost on that path; the comment says so. cgiAddHttpHeader() now does what its own comment already promised and ignores a header added after the block was closed, rather than growing a list nothing will ever print. getCspPolicyString() is declared in htmshell.h so hCommon.c can queue the value on its own.
  • lines changed 40, context: html, text, full: html, text
    f664e77c7e67c0342ba78b140386c4644d451a32 Thu Sep 17 01:42:56 2026 -0700
    hgHubConnect, lib: name the hub URL when a hub fails to load #Preview2 week - bugs introduced now will need a build patch to fix A hub that could not be opened showed up in the connected-hubs table as an error message and nothing else: the name, description and assembly cells were all empty, because they are only filled in from the trackHub struct that the failed load never produced. The URL was in the row only inside the javascript of the Disconnect and Retry buttons, so there was no way to tell which hub the error was about. Print the URL in the name cell instead. netParseUrl chops up its local copy of the URL as it parses, so none of its errAborts could report what they were looking at. Keep the caller's string and name it in all of them. A URL written with a single slash, https:/host/f, then reported a non-numeric port, because with no :// the protocol defaults to http and the host becomes "https" with an empty port - say what is actually wrong with it. Also tighten the encoding of the hub tables' output and of javascript string literals, and use htmlEncode, not javaScriptLiteralEncode, for the option labels in cgiMakeSelectDropList, which are HTML text.
• src/lib/errAbort.c
  • lines changed 6, context: html, text, full: html, text
    8c172e6c4b985af79bf1a1f47c9db92ce0578999 Tue Sep 15 05:47:36 2026 -0700
    errCatch: put errAbort's doContentType back the way it found it, refs #38353 hVaUserAbort() turns doContentType on so that a user error reaches the browser when the CGI has not pushed a warn handler of its own. Inside an errCatch that is pointless -- the abort is caught and never reaches the default handler -- and the flag stayed on afterwards, so a later genuine abort on a page that had already written its header would print a second Content-Type line into the body. Everywhere else in the tree the flag is something a caller turns back off. errCatchPushHandlers() now records the flag and errCatchEnd() restores it, which covers every caller rather than just the one that set it. errAbortGetDoContentType() is the reader.
• src/lib/errCatch.c
  • lines changed 6, context: html, text, full: html, text
    8c172e6c4b985af79bf1a1f47c9db92ce0578999 Tue Sep 15 05:47:36 2026 -0700
    errCatch: put errAbort's doContentType back the way it found it, refs #38353 hVaUserAbort() turns doContentType on so that a user error reaches the browser when the CGI has not pushed a warn handler of its own. Inside an errCatch that is pointless -- the abort is caught and never reaches the default handler -- and the flag stayed on afterwards, so a later genuine abort on a page that had already written its header would print a second Content-Type line into the body. Everywhere else in the tree the flag is something a caller turns back off. errCatchPushHandlers() now records the flag and errCatchEnd() restores it, which covers every caller rather than just the one that set it. errAbortGetDoContentType() is the reader.
• src/lib/hmac.c
  • lines changed 12, context: html, text, full: html, text
    33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
    hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323 Six things, all in the syncHubApiKeys path, which is still off everywhere. The sync never reached its handler on a real mirror. It was registered as a cartJson command, so building the cart ran forceUserIdOrCaptcha() first; a peer's request carries no hguid cookie and no apiKey= variable, so any site with cloudFlareSiteKey set -- which is every site that needs api keys in the first place -- answered it with a captcha page. Only hgwdev, where the captcha is commented out, ever ran the handler, which is why this was not caught earlier. doApiKeySyncIfRequested() now answers the request in main() before any cart exists. That also stops each sync leaving a junk userDb and sessionDb row behind: measured, 15 syncs now create none. The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key), so it does not rest on md5 resisting length extension, and it is compared in constant time. It now covers a timestamp too, and a peer refuses anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of now, so a captured sync cannot be replayed later to reinstate a key its owner has since revoked. The fields are joined with newlines and a newline in a userName or apiKey is refused, so one signature cannot cover two different splits of the same string. syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It runs after the local key is already written and in the response, so an errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's outer catch, which threw the response away -- the user saw a failure over a key that was sitting in the database. geoMirrorNotifyOtherNodes() returns each peer's response instead of discarding it, and uses netUrlMustOpenPastHeader, which errAborts on anything but a 200 and hands back the body alone. A peer answering 'bad signature' or 'not enabled on this site' is no longer indistinguishable from success; it is logged to error_log, not warn()ed at the person who clicked the button. Verified against the built CGI run as Apache runs it, with the captcha turned on: a signed sync is accepted and writes the row, a signed revoke removes it, and a replay, a swapped userName, a forged signature and malformed json are all refused. hubSpaceKeysTester covers the signature rules; it still cannot run its database half until hgcentralregress gets the UPDATE and DELETE grant.
• src/lib/htmshell.c
  • lines changed 19, context: html, text, full: html, text
    3bacf54b6a23bc56f728c2e670c9b2cca96222dd Wed Sep 16 04:56:02 2026 -0700
    htmshell: drop the two CSP response-header helpers that no longer have callers, refs #38353 cspWriteResponseHeader() was their only user and it queues the policy through cgiAddHttpHeader() now, so generateCspResponseHeader() and getCspMetaResponseHeader() build a header line nothing sends. Neither had another caller at any point. The meta-tag side is untouched.
• src/lib/jsonParse.c
  • lines changed 3, context: html, text, full: html, text
    0cdb95bd4f0843610653891af8400a56e3b6090b Fri Sep 18 07:21:52 2026 -0700
    jsonParse: a backslash escape no longer doubles the character it escapes #Preview2 week - bugs introduced now will need a build patch to fix The default branch of the escape switch appended the escaped character and then fell through to the shared append, so \/ came back out of the parser as //, \" as "" and \\ as \\. A session URL parsed and written back out read https:////host//s//user//name. The \u passthrough, which deliberately adds a backslash before falling through, is unchanged.
• src/lib/net.c
  • lines changed 18, context: html, text, full: html, text
    f664e77c7e67c0342ba78b140386c4644d451a32 Thu Sep 17 01:42:56 2026 -0700
    hgHubConnect, lib: name the hub URL when a hub fails to load #Preview2 week - bugs introduced now will need a build patch to fix A hub that could not be opened showed up in the connected-hubs table as an error message and nothing else: the name, description and assembly cells were all empty, because they are only filled in from the trackHub struct that the failed load never produced. The URL was in the row only inside the javascript of the Disconnect and Retry buttons, so there was no way to tell which hub the error was about. Print the URL in the name cell instead. netParseUrl chops up its local copy of the URL as it parses, so none of its errAborts could report what they were looking at. Keep the caller's string and name it in all of them. A URL written with a single slash, https:/host/f, then reported a non-numeric port, because with no :// the protocol defaults to http and the host becomes "https" with an empty port - say what is actually wrong with it. Also tighten the encoding of the hub tables' output and of javascript string literals, and use htmlEncode, not javaScriptLiteralEncode, for the option labels in cgiMakeSelectDropList, which are HTML text.
• src/lib/tests/expected/hmacTest
  • lines changed 12, context: html, text, full: html, text
    c2e25edbf8c48361a71398664ec0c279d5fc58f5 Sat Sep 19 18:11:31 2026 -0700
    hmacTest: pin the signature hgLogin puts on a pending social identity hgLogin signs a pending social identity with hmacMd5(login.cookieSalt, fields) and hands the signature to the browser in the account chooser, so a forgeable signature lets an attacker choose whose account the chooser links to. Nothing about the page looks different either way. Before #37984 that signature was a plain MD5 of the salt concatenated in front of the same fields, and a hash of a secret followed by attacker-chosen text is the wrong shape for the job. boundaryMoves() is the case that says so: with concatenation, hmacMd5("a", "bc") and hmacMd5("ab", "c") hash the same bytes and sign the same, while HMAC keeps them apart. Known answers are RFC 2202 test case 2 for MD5 and SHA1, cross-checked here with the openssl command line. Only the string-keyed vectors from the RFC can be used, since this interface takes key and data as C strings. Watched to fail and then pass: putting hmacMd5 back to the concatenation shape turns the known answer red and makes the two boundary cases identical, both of which the test catches. Recorded as sandbox-ab in utils/testRegistry. refs #37984, refs #38391
• src/lib/tests/hmacTest.c
  • lines changed 134, context: html, text, full: html, text
    c2e25edbf8c48361a71398664ec0c279d5fc58f5 Sat Sep 19 18:11:31 2026 -0700
    hmacTest: pin the signature hgLogin puts on a pending social identity hgLogin signs a pending social identity with hmacMd5(login.cookieSalt, fields) and hands the signature to the browser in the account chooser, so a forgeable signature lets an attacker choose whose account the chooser links to. Nothing about the page looks different either way. Before #37984 that signature was a plain MD5 of the salt concatenated in front of the same fields, and a hash of a secret followed by attacker-chosen text is the wrong shape for the job. boundaryMoves() is the case that says so: with concatenation, hmacMd5("a", "bc") and hmacMd5("ab", "c") hash the same bytes and sign the same, while HMAC keeps them apart. Known answers are RFC 2202 test case 2 for MD5 and SHA1, cross-checked here with the openssl command line. Only the string-keyed vectors from the RFC can be used, since this interface takes key and data as C strings. Watched to fail and then pass: putting hmacMd5 back to the concatenation shape turns the known answer red and makes the two boundary cases identical, both of which the test catches. Recorded as sandbox-ab in utils/testRegistry. refs #37984, refs #38391
• src/lib/tests/makefile
  • lines changed 9, context: html, text, full: html, text
    c2e25edbf8c48361a71398664ec0c279d5fc58f5 Sat Sep 19 18:11:31 2026 -0700
    hmacTest: pin the signature hgLogin puts on a pending social identity hgLogin signs a pending social identity with hmacMd5(login.cookieSalt, fields) and hands the signature to the browser in the account chooser, so a forgeable signature lets an attacker choose whose account the chooser links to. Nothing about the page looks different either way. Before #37984 that signature was a plain MD5 of the salt concatenated in front of the same fields, and a hash of a secret followed by attacker-chosen text is the wrong shape for the job. boundaryMoves() is the case that says so: with concatenation, hmacMd5("a", "bc") and hmacMd5("ab", "c") hash the same bytes and sign the same, while HMAC keeps them apart. Known answers are RFC 2202 test case 2 for MD5 and SHA1, cross-checked here with the openssl command line. Only the string-keyed vectors from the RFC can be used, since this interface takes key and data as C strings. Watched to fail and then pass: putting hmacMd5 back to the concatenation shape turns the known answer red and makes the two boundary cases identical, both of which the test catches. Recorded as sandbox-ab in utils/testRegistry. refs #37984, refs #38391
• src/product/ex.hg.conf
  • lines changed 5, context: html, text, full: html, text
    3bcf86a0995e5a15ad5a2a8ff5533f48d5d43653 Fri Sep 18 07:15:28 2026 -0700
    hgTracks: tell the user what a session load just did, refs #38157 #Preview2 week - bugs introduced now will need a build patch to fix When a saved session is loaded it replaces the whole cart, and the view the user had before is gone without a word. hgTracks now shows a note on that page saying which session was opened, who saved it and when, the session description if it has one, and that the previous configuration cannot be brought back. Only on that one page: cart.c leaves a marker when a full (non-merge) session load happens and hgTracks takes it out again. Recommended track sets merge into the cart, so they get no note. The note uses a new notifBoxOnce() in utils.js, which has only a Close button: a one-page message does not need 'Don't show again'. Its .notifBoxOnce style is the card grey of the new Sessions page, and it sets no width, so unlike the other notifBoxes it cannot stick out of the window. Switch off with sessionLoadNotice=off in hg.conf.
• src/product/mirrorManual.txt
  • lines changed 24, context: html, text, full: html, text
    948e76a68b4a62e4f6f90244eeb266342406ce53 Wed Sep 16 03:03:36 2026 -0700
    hgLogin: make trusting a provider's unverified email a per-provider setting Accepting an address a provider had not marked verified was applied to every provider, to keep CILogon usable: it sends the address it got from the user's institution but leaves email_verified at 0, even for a real institutional sign-in. Extending that to everyone was too much. GitHub hands over a primary address whose owner never confirmed it, and any provider a mirror adds to hg.conf was treated the same way, so an address nobody had checked was enough to be signed in to an existing account that used it. New login.oauth.<name>.trustEmail, off unless an admin sets it, says that a named provider's address may be taken without email_verified. It belongs on a provider that reads the address from somewhere the user cannot type into, which is what CILogon does and what GitHub does not. Where it is off and the provider did not verify the address, the address is dropped rather than refused, and the sign-in proceeds as one that arrived with no address at all, which is already the ORCID case: the user is asked for an address and the account does not work until they open the link mailed to it. Dropping it also keeps it out of the account matching in resolveIdentity, so it cannot reach an existing account. Documented in product/mirrorManual.txt next to the other provider settings, with the CILogon line mirrors will need, and in oauthLogin.h with the rest of the keys.
• src/utils/genark/genark
  • lines changed 24, context: html, text, full: html, text
    0b293869739f3112a74d2ee38f47785a7c245884 Wed Sep 16 06:29:25 2026 -0700
    HPRC v2 contrib hub: pcLAI defaults to dense, and genark addContrib wires the beta/public hub tiers Mark reported that the pcLAI track shows only one color on HG00408 pat (GCA_041900255.1) and that you cannot see where ancestry changes. The colors are correct: a fresh download of the HPRC source BED matches the built pclai.bb on (chrom, start, end, itemRgb) for all 25,475 windows. That haplotype is simply 99.8% one ancestry cluster, which is common in this set (71 of 460 haplotypes have a single centroid, 228 are >=99% one centroid). The display was the real problem, so hprcPclai goes from visibility pack to visibility dense plus onlyVisibility dense. At chromosome scale pack stacks the 100 kb windows into ~50 rows of 1-2 px slivers; dense draws one colored bar where the ancestry blocks are legible. While deploying that, beta.hub.txt turned out to be a week stale: it still had the pre-2026-09-09 pcaSegment field name and the double-prefixed pclaiRefPanel.json dataUrl, so the details-page scatterplot was not drawing on hgwbeta. genark addContrib only ever wired alpha.hub.txt, leaving beta and public to the next clade build. It now wires those two as well for a collection already named in betaGenArk.txt / publicGenArk.txt. This never promotes anything - the tier argument still owns the release lists - and mkGenomes.pl inlines the same per-assembly trackDb, so a later clade build converges on the same content. Ran over 462 assemblies, alpha and beta; genark checkContrib reports no contrib problems, refs #35415
• src/utils/hubtools/hubtools
  • lines changed 130, context: html, text, full: html, text
    f518e03d8f56af784ec14536a7bdc0daed9a38fd Wed Sep 16 11:11:32 2026 -0700
    Send each file's genome with a hubtools upload so hubSpace rows get a db, and stop the server rewriting a user-uploaded hub.txt, no redmine Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • lines changed 20, context: html, text, full: html, text
    547bbe0a4ad38df6b0b7a511d6a2ce24b3625fe7 Thu Sep 17 05:08:41 2026 -0700
    hubtools import session: keep the track order of the session, refs #34405 #Preview2 week - bugs introduced now will need a build patch to fix The custom track backup archive has one file per track and no ordering information: hgSession wrote the files into a directory and tar walks it in readdir order, which is not the order they were written. hubtools globbed them, and a hub track without a priority is shown in alphabetical order of its label, so an imported session came out sorted by label. hgSession now writes the priority hgTracks orders the tracks by into each track line of the archive, either the one the browser assigned when it loaded the track or the cart variable that a drag-and-drop reorder left behind. A track line that names its own priority is left alone. hubtools sorts on that priority and numbers the hub stanzas, continuing the count across assemblies so two genome stanzas cannot interleave. Archives made before this have no priority and fall back to file name order.
• src/utils/makefile
  • lines changed 4, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/qa/hubCheckPublicHubs.sh
  • lines changed 55, context: html, text, full: html, text
    738e39f7389e853f775cdee73a54087b86dfd31f Wed Sep 16 11:00:35 2026 -0700
    Add runtime limits and locking to hubCheckPublicHubs.sh so one hung hubCheck cannot wedge the monthly public hub check: timeout -k 30s 5m per hub, a six hour cap on the whole loop, flock so runs cannot stack, timeout exit codes 124 and 137 logged into the archive, and removal of a stray tail -n +2 that was silently dropping the first hub from every run, refs #38366 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
• src/utils/qa/weeklybld/README.dockerTodo
  • lines changed 19, context: html, text, full: html, text
    dbb8f6e006713f43b7dbd6100d3b5c024dc72766 Fri Sep 18 11:11:15 2026 -0700
    weeklybld: let the docker instance scripts run a past release, refs #38377 We could not reproduce a bug against the release a user is actually on. hgwdev runs tip and the current release, and nothing older. Docker Hub already has a per-release image, so the missing piece was a way to run one. The lifecycle scripts now take a release name as well as tip, beta and rel. run-instance.sh v503 starts the published v503 image as container kent-v503 on port 8503; the port is 8000 plus the release number, so the mapping needs no table and no allocator. A release instance gets no CGI overlay, for the same reason rel gets none: it has to be the code that shipped. refresh-instance.sh, remove-instance.sh and smoke-instance.sh take the same name. smoke-instance.sh checks a release instance against the version in its own name, so no --version flag is needed. start-all.sh and stop-all.sh now read the release containers that exist rather than a fixed list, so the set of releases we keep can change without editing them. tunnel-release.sh is one script for every release rather than one per release, since that set changes. v499 through v503 are running on hgwdev on ports 8499 through 8503 and pass smoke-instance.sh. Five instances cost about 1.5 GB of memory, 10 GB of docker disk, and no measurable CPU. They read tables from genome-mysql.soe.ucsc.edu and files from hgdownload.soe.ucsc.edu, so they put no load on the hgwdev MySQL or /gbdb.
• src/utils/qa/weeklybld/buildEnv.csh
  • lines changed 3, context: html, text, full: html, text
    2be17664d5eab1f5aece82ccc0e4f7afd275f78d Mon Sep 21 10:00:44 2026 -0700
    v504 final build (automated)
• src/utils/qa/weeklybld/doHgTablesTestRobot.csh
  • lines changed 83, context: html, text, full: html, text
    5a34a4caa476493be5efb117dff50073d5637e20 Wed Sep 16 12:40:05 2026 -0700
    doHgTablesTestRobot: fail when a run dies instead of reporting success, refs #38356 This is the robot that runs at the final build, against hgwbeta. Its check looked only for Total lines that carried errors. hgTablesTest writes that line after a run finishes, so a run that died partway through wrote no Total line at all, the match came back empty, and the robot mailed "done successfully". Every log back to v490 is missing the summary line, so this gate had never once fired. Give it the checks the preview2 robot already got in cf348a83d76: the exit status of each of the two runs, a count of the Total lines against the number of runs expected, the existing error-in-Total check, and a grep for crash signatures over both the report log and a separate file holding the runs' own output. A missing summary is now itself a failure, and the mail says which of those conditions tripped. Also a ROBOT_MAILTO override, so re-running the robot to diagnose something does not notify the QA list every time. Tested against a stub hgTablesTest and a stub mail, four ways. A clean run still passes. A run reporting errors in its Total line still fails. A run that dies partway, whether it exits nonzero or zero, used to pass and now fails.
• src/utils/qa/weeklybld/refresh-instance.sh
  • lines changed 9, context: html, text, full: html, text
    dbb8f6e006713f43b7dbd6100d3b5c024dc72766 Fri Sep 18 11:11:15 2026 -0700
    weeklybld: let the docker instance scripts run a past release, refs #38377 We could not reproduce a bug against the release a user is actually on. hgwdev runs tip and the current release, and nothing older. Docker Hub already has a per-release image, so the missing piece was a way to run one. The lifecycle scripts now take a release name as well as tip, beta and rel. run-instance.sh v503 starts the published v503 image as container kent-v503 on port 8503; the port is 8000 plus the release number, so the mapping needs no table and no allocator. A release instance gets no CGI overlay, for the same reason rel gets none: it has to be the code that shipped. refresh-instance.sh, remove-instance.sh and smoke-instance.sh take the same name. smoke-instance.sh checks a release instance against the version in its own name, so no --version flag is needed. start-all.sh and stop-all.sh now read the release containers that exist rather than a fixed list, so the set of releases we keep can change without editing them. tunnel-release.sh is one script for every release rather than one per release, since that set changes. v499 through v503 are running on hgwdev on ports 8499 through 8503 and pass smoke-instance.sh. Five instances cost about 1.5 GB of memory, 10 GB of docker disk, and no measurable CPU. They read tables from genome-mysql.soe.ucsc.edu and files from hgdownload.soe.ucsc.edu, so they put no load on the hgwdev MySQL or /gbdb.
• src/utils/qa/weeklybld/remove-instance.sh
  • lines changed 9, context: html, text, full: html, text
    dbb8f6e006713f43b7dbd6100d3b5c024dc72766 Fri Sep 18 11:11:15 2026 -0700
    weeklybld: let the docker instance scripts run a past release, refs #38377 We could not reproduce a bug against the release a user is actually on. hgwdev runs tip and the current release, and nothing older. Docker Hub already has a per-release image, so the missing piece was a way to run one. The lifecycle scripts now take a release name as well as tip, beta and rel. run-instance.sh v503 starts the published v503 image as container kent-v503 on port 8503; the port is 8000 plus the release number, so the mapping needs no table and no allocator. A release instance gets no CGI overlay, for the same reason rel gets none: it has to be the code that shipped. refresh-instance.sh, remove-instance.sh and smoke-instance.sh take the same name. smoke-instance.sh checks a release instance against the version in its own name, so no --version flag is needed. start-all.sh and stop-all.sh now read the release containers that exist rather than a fixed list, so the set of releases we keep can change without editing them. tunnel-release.sh is one script for every release rather than one per release, since that set changes. v499 through v503 are running on hgwdev on ports 8499 through 8503 and pass smoke-instance.sh. Five instances cost about 1.5 GB of memory, 10 GB of docker disk, and no measurable CPU. They read tables from genome-mysql.soe.ucsc.edu and files from hgdownload.soe.ucsc.edu, so they put no load on the hgwdev MySQL or /gbdb.
• src/utils/qa/weeklybld/run-instance.sh
  • lines changed 20, context: html, text, full: html, text
    dbb8f6e006713f43b7dbd6100d3b5c024dc72766 Fri Sep 18 11:11:15 2026 -0700
    weeklybld: let the docker instance scripts run a past release, refs #38377 We could not reproduce a bug against the release a user is actually on. hgwdev runs tip and the current release, and nothing older. Docker Hub already has a per-release image, so the missing piece was a way to run one. The lifecycle scripts now take a release name as well as tip, beta and rel. run-instance.sh v503 starts the published v503 image as container kent-v503 on port 8503; the port is 8000 plus the release number, so the mapping needs no table and no allocator. A release instance gets no CGI overlay, for the same reason rel gets none: it has to be the code that shipped. refresh-instance.sh, remove-instance.sh and smoke-instance.sh take the same name. smoke-instance.sh checks a release instance against the version in its own name, so no --version flag is needed. start-all.sh and stop-all.sh now read the release containers that exist rather than a fixed list, so the set of releases we keep can change without editing them. tunnel-release.sh is one script for every release rather than one per release, since that set changes. v499 through v503 are running on hgwdev on ports 8499 through 8503 and pass smoke-instance.sh. Five instances cost about 1.5 GB of memory, 10 GB of docker disk, and no measurable CPU. They read tables from genome-mysql.soe.ucsc.edu and files from hgdownload.soe.ucsc.edu, so they put no load on the hgwdev MySQL or /gbdb.
• src/utils/qa/weeklybld/smoke-instance.sh
  • lines changed 15, context: html, text, full: html, text
    dbb8f6e006713f43b7dbd6100d3b5c024dc72766 Fri Sep 18 11:11:15 2026 -0700
    weeklybld: let the docker instance scripts run a past release, refs #38377 We could not reproduce a bug against the release a user is actually on. hgwdev runs tip and the current release, and nothing older. Docker Hub already has a per-release image, so the missing piece was a way to run one. The lifecycle scripts now take a release name as well as tip, beta and rel. run-instance.sh v503 starts the published v503 image as container kent-v503 on port 8503; the port is 8000 plus the release number, so the mapping needs no table and no allocator. A release instance gets no CGI overlay, for the same reason rel gets none: it has to be the code that shipped. refresh-instance.sh, remove-instance.sh and smoke-instance.sh take the same name. smoke-instance.sh checks a release instance against the version in its own name, so no --version flag is needed. start-all.sh and stop-all.sh now read the release containers that exist rather than a fixed list, so the set of releases we keep can change without editing them. tunnel-release.sh is one script for every release rather than one per release, since that set changes. v499 through v503 are running on hgwdev on ports 8499 through 8503 and pass smoke-instance.sh. Five instances cost about 1.5 GB of memory, 10 GB of docker disk, and no measurable CPU. They read tables from genome-mysql.soe.ucsc.edu and files from hgdownload.soe.ucsc.edu, so they put no load on the hgwdev MySQL or /gbdb.
• src/utils/qa/weeklybld/start-all.sh
  • lines changed 12, context: html, text, full: html, text
    dbb8f6e006713f43b7dbd6100d3b5c024dc72766 Fri Sep 18 11:11:15 2026 -0700
    weeklybld: let the docker instance scripts run a past release, refs #38377 We could not reproduce a bug against the release a user is actually on. hgwdev runs tip and the current release, and nothing older. Docker Hub already has a per-release image, so the missing piece was a way to run one. The lifecycle scripts now take a release name as well as tip, beta and rel. run-instance.sh v503 starts the published v503 image as container kent-v503 on port 8503; the port is 8000 plus the release number, so the mapping needs no table and no allocator. A release instance gets no CGI overlay, for the same reason rel gets none: it has to be the code that shipped. refresh-instance.sh, remove-instance.sh and smoke-instance.sh take the same name. smoke-instance.sh checks a release instance against the version in its own name, so no --version flag is needed. start-all.sh and stop-all.sh now read the release containers that exist rather than a fixed list, so the set of releases we keep can change without editing them. tunnel-release.sh is one script for every release rather than one per release, since that set changes. v499 through v503 are running on hgwdev on ports 8499 through 8503 and pass smoke-instance.sh. Five instances cost about 1.5 GB of memory, 10 GB of docker disk, and no measurable CPU. They read tables from genome-mysql.soe.ucsc.edu and files from hgdownload.soe.ucsc.edu, so they put no load on the hgwdev MySQL or /gbdb.
• src/utils/qa/weeklybld/stop-all.sh
  • lines changed 10, context: html, text, full: html, text
    dbb8f6e006713f43b7dbd6100d3b5c024dc72766 Fri Sep 18 11:11:15 2026 -0700
    weeklybld: let the docker instance scripts run a past release, refs #38377 We could not reproduce a bug against the release a user is actually on. hgwdev runs tip and the current release, and nothing older. Docker Hub already has a per-release image, so the missing piece was a way to run one. The lifecycle scripts now take a release name as well as tip, beta and rel. run-instance.sh v503 starts the published v503 image as container kent-v503 on port 8503; the port is 8000 plus the release number, so the mapping needs no table and no allocator. A release instance gets no CGI overlay, for the same reason rel gets none: it has to be the code that shipped. refresh-instance.sh, remove-instance.sh and smoke-instance.sh take the same name. smoke-instance.sh checks a release instance against the version in its own name, so no --version flag is needed. start-all.sh and stop-all.sh now read the release containers that exist rather than a fixed list, so the set of releases we keep can change without editing them. tunnel-release.sh is one script for every release rather than one per release, since that set changes. v499 through v503 are running on hgwdev on ports 8499 through 8503 and pass smoke-instance.sh. Five instances cost about 1.5 GB of memory, 10 GB of docker disk, and no measurable CPU. They read tables from genome-mysql.soe.ucsc.edu and files from hgdownload.soe.ucsc.edu, so they put no load on the hgwdev MySQL or /gbdb.
• src/utils/qa/weeklybld/tunnel-release.sh
  • lines changed 43, context: html, text, full: html, text
    dbb8f6e006713f43b7dbd6100d3b5c024dc72766 Fri Sep 18 11:11:15 2026 -0700
    weeklybld: let the docker instance scripts run a past release, refs #38377 We could not reproduce a bug against the release a user is actually on. hgwdev runs tip and the current release, and nothing older. Docker Hub already has a per-release image, so the missing piece was a way to run one. The lifecycle scripts now take a release name as well as tip, beta and rel. run-instance.sh v503 starts the published v503 image as container kent-v503 on port 8503; the port is 8000 plus the release number, so the mapping needs no table and no allocator. A release instance gets no CGI overlay, for the same reason rel gets none: it has to be the code that shipped. refresh-instance.sh, remove-instance.sh and smoke-instance.sh take the same name. smoke-instance.sh checks a release instance against the version in its own name, so no --version flag is needed. start-all.sh and stop-all.sh now read the release containers that exist rather than a fixed list, so the set of releases we keep can change without editing them. tunnel-release.sh is one script for every release rather than one per release, since that set changes. v499 through v503 are running on hgwdev on ports 8499 through 8503 and pass smoke-instance.sh. Five instances cost about 1.5 GB of memory, 10 GB of docker disk, and no measurable CPU. They read tables from genome-mysql.soe.ucsc.edu and files from hgdownload.soe.ucsc.edu, so they put no load on the hgwdev MySQL or /gbdb.
• src/utils/redmineCli
  • lines changed 13, context: html, text, full: html, text
    5ef589f4bdca750ae44fabb5c648378e8264b2c9 Tue Sep 15 04:56:51 2026 -0700
    redmineCli: match the 403/404 prefix, not anywhere in the error text summarize_issue treats a ticket as unreadable when api_get reports 403 or 404, and re-raises everything else so a real failure is not hidden behind "(not readable)". The test looked for "HTTP 403 GET" anywhere in the message, but that message carries up to 500 characters of the server's response body, so a 500 or a proxy error page quoting those words in its own text was swallowed as well. Anchor it to the prefix the tool builds.
• src/utils/testRegistry/README
  • lines changed 120, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
  • lines changed 8, context: html, text, full: html, text
    eccdddfb22cf35d4e5698f72a4b12707aa43b349 Sun Sep 20 14:27:27 2026 -0700
    testRegistry: check the docent column's "-" as well as its names The docent column says which browser test watches the same ticket, and a named script was already checked: it has to exist and has to belong to that ticket. A "-" was checked by nothing, and that was the wrong half to leave out. `unwatched` -- the list of tickets where nothing anywhere would go red if the bug came back -- is built entirely out of those "-" values. They were entered by hand, by reading the docent directory once, so a script written the next day left the ticket sitting on that queue with nothing to notice. For the four tickets that are on it because their code lives inside a CGI, somebody writing the docent script is the expected outcome, which made it the likeliest kind of row to go stale. So a row claiming no docent script is now checked against the tree the same way a named one is: if rm<ticket>.*docent.yaml turns up, the check fails and asks for the row to be filled in. The failure is somebody doing the right thing and the table not having heard about it. Watched to fail and then pass: blanking the docent column of #36212, whose script is in the tree, turns the real table red with the name of the script it found. refs #38391
• src/utils/testRegistry/makefile
  • lines changed 15, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/testRegistry/registry.tsv
  • lines changed 89, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
  • lines changed 1, context: html, text, full: html, text
    c2e25edbf8c48361a71398664ec0c279d5fc58f5 Sat Sep 19 18:11:31 2026 -0700
    hmacTest: pin the signature hgLogin puts on a pending social identity hgLogin signs a pending social identity with hmacMd5(login.cookieSalt, fields) and hands the signature to the browser in the account chooser, so a forgeable signature lets an attacker choose whose account the chooser links to. Nothing about the page looks different either way. Before #37984 that signature was a plain MD5 of the salt concatenated in front of the same fields, and a hash of a secret followed by attacker-chosen text is the wrong shape for the job. boundaryMoves() is the case that says so: with concatenation, hmacMd5("a", "bc") and hmacMd5("ab", "c") hash the same bytes and sign the same, while HMAC keeps them apart. Known answers are RFC 2202 test case 2 for MD5 and SHA1, cross-checked here with the openssl command line. Only the string-keyed vectors from the RFC can be used, since this interface takes key and data as C strings. Watched to fail and then pass: putting hmacMd5 back to the concatenation shape turns the known answer red and makes the two boundary cases identical, both of which the test catches. Recorded as sandbox-ab in utils/testRegistry. refs #37984, refs #38391
  • lines changed 1, context: html, text, full: html, text
    859bc749b0593a83e1c8cd7149f956b620a24a73 Sat Sep 19 18:14:15 2026 -0700
    trashDirTester: pin which file paths the cart accepts hg/lib/cart.c screens every cart variable that names a server-made file through these functions, so they decide whether a saved session still works. Both ways of being wrong are invisible: a path wrongly refused brings the session back without its custom track or its region list and says nothing, and a path wrongly accepted says nothing either. That is what #38303 cost. A check shipped in v503 discarded the saved region list from 583 sessions on the RR and 66 on euro, and hgwdev, code review and hgwbeta were all clean, because the corpus that shows it is only on the production central. Four rules pinned, each of which cost a bug or a review round: only the configured directory is symlink-resolved and never the path from the cart; the resolved spelling of that directory is accepted as well as the configured one, since /userdata on the RR is a symlink and saved sessions hold both spellings; only an absolute directory is resolved, because a relative one would be resolved against the caller's working directory; and the acceptance runs one direction only. pathIsUnderDir's own edges are here too, including the sibling directory that merely starts the same way and the name that begins with "..". The fixture is a directory, a symlink to it and three confs naming the same place three ways, since hgConfig caches what it read and one process can only answer for one spelling. Nothing machine-specific reaches the output. Watched to fail and then pass: with the pre-#38303 code, which did not resolve at all, the resolved spelling comes back refused and the diff is that one line. Recorded as sandbox-ab in utils/testRegistry. refs #38303, refs #37623, refs #38391
  • lines changed 1, context: html, text, full: html, text
    a5d42e58d7f8ce1e985cf7422145619122ca5d25 Sat Sep 19 18:16:16 2026 -0700
    mallocTopPadTester: check the heap step setting still reaches the library cfgSetMallocTopPad() asks glibc to grow the heap in steps of mallocTopPad bytes instead of its 128 kB default, which saves a heavy hgTracks render more than a hundred thousand system calls that draw nothing. The failure to catch is backsliding: the mallopt call is dropped in a later edit, or the setting is renamed, or it stops being read before the first allocation. Every page still draws correctly and the only symptom is that renders are slower than they used to be, which nobody attributes to this. So the test measures the step, not the clock. A timing test here would be red on a busy hgwdev, green on an idle one, and deleted within a month. sbrk(0) says where the heap ends and the distance it jumps when malloc extends it is what M_TOP_PAD sets, so the answer is the same on a loaded machine as an idle one. It is read in buckets, since glibc may round and may change what it rounds to; what must not change is the order of magnitude between a configured step and the default. Allocations stay under the mmap threshold, or the break never moves and the test measures nothing while appearing to pass. Two runs, with the setting and without, because the answer is the difference between them: a single run would pass on a library whose own default happened to be large. Watched to fail and then pass: leaving the knob read but never applied gives the default step under both confs, which is the one line the diff shows. Recorded as sandbox-ab in utils/testRegistry. refs #38225, refs #38391
  • lines changed 1, context: html, text, full: html, text
    a0a3411bca2fdd6d704ce2be7c9c8fca7682913a Sat Sep 19 18:19:36 2026 -0700
    bedItemRgbTester: pin the order of the itemRgb and color tests bedItemRgb() decides whether a BED track draws its items in the colors its file carries or in the one color its stanza names. The rule has four steps, and the order of the first three is the whole of #36212: an explicit "itemRgb off" wins, then an explicit "itemRgb on" wins, and only then does the presence of a "color" setting turn item colors off by default. 88d620e6c82 folded those tests together, so a stanza saying both "itemRgb on" and "color" -- which means items from the file and labels from color -- lost its item colors. The pairs are what matter here. Every single-setting case passed while the bug was live, so a test that exercised one setting at a time would have proved nothing. A child that says "itemRgb on" under a parent that says "color" is the same case reached through trackDbSettingClosestToHome. Two runs, with and without hg.conf's alwaysItemRgb, since the last step of the rule reads it and a mirror that turns it off must still honour a stanza that asks for item colors explicitly. The test lives in hg/lib/tests because hg/cgilib has no tests directory; its link line adds jkhgapcgi.a. Watched to fail and then pass: with the color test moved back in front of the itemRgb test, three lines flip, and they are the three that pair the two settings. Recorded as sandbox-ab in utils/testRegistry. refs #36212, refs #38391
  • lines changed 1, context: html, text, full: html, text
    d898b4820087cccd79e240153bc310856bb89310 Sat Sep 19 18:22:49 2026 -0700
    hVarSubstHtmlTester: pin which variables a description page may use A native description page has been through hgTrackDb already, which resolved everything it could and deferred only ${hgsid}, so the render pass acts on that one variable and only in braces: hgTrackDb collapses an escaped $$hgsid to a literal $hgsid, and acting on the bare form here would expand the very thing the author escaped. A hub's page has never been substituted, so the whole hub list is resolved at render time, and $hgsid is deliberately not on that list. A hub page is only lightly sanitized -- an <img> with an http src survives -- so a page carrying <img src="https://example.com/px?s=${hgsid}"> would hand the reader's session id to the hub's own server, and a session id alone is enough to read and write that cart. The page renders the same either way and the request goes to somebody else's host, so nothing about this is visible here. The test builds a cart by hand rather than opening one. That is not a shortcut, it is the point: a cartless version of this test was written first, and adding hgsid back to hubHtmlVars changed not one line of its output, because the check that recognises a variable also asks whether this call can resolve it, and with no cart it cannot. cartSessionId reads two fields, so a struct built in the test is enough and no database is involved. Watched to fail and then pass: with hgsid back on the hub list, both hub cases come back with the session id substituted into the img src. Recorded as sandbox-ab in utils/testRegistry. refs #38283, refs #38391
  • lines changed 1, context: html, text, full: html, text
    471b82ec87d4b23b03ec3f3a2a193eb1f88a035c Sat Sep 19 18:24:53 2026 -0700
    dataVersionPathTester: pin which local files a hub's dataVersion may read A track's dataVersion setting is usually a version string, but for otto tracks it is the path of a local file that hgTrackUi opens and prints. On a hub track that setting is an instruction from a stranger to read a named file on our server and put it on the page. #38268 narrowed it to /gbdb, which is public data mirrored on hgdownload, and to a plain path, since a ".." component makes the name mean something outside that tree. The exception is needed: curated-hub assemblies are served as hubs, so hs1's otto tracks are hub tracks, and a quickLifted track's version file lives on the source assembly. Nothing about the failure is visible in the usual way. A refused path and an accepted one both leave a page that looks reasonable, and the difference shows only as somebody else's file appearing where a version number belongs. The test prints a classification and never a file. The three outcomes separate without looking at any contents: a refused path comes back as the path itself, an accepted path that does not exist comes back NULL, and an accepted path that exists comes back as something else. Watched to fail and then pass: with the /gbdb test dropped, /etc/passwd comes back read rather than refused, and the two climbing cases come back opened. Recorded as sandbox-ab in utils/testRegistry. refs #38268, refs #38391
  • lines changed 1, context: html, text, full: html, text
    d58bf9a182cbf4dba142a7028742abf688638829 Sat Sep 19 18:27:02 2026 -0700
    genarkLiftOverTester: pin the escaping of the GenArk accession list Before #38328 the caller pasted its accessions into one string and genarkLiftOverDbs dropped that string into a query marked NOSQLINJ, which is a promise that somebody upstream had made it safe. It now takes an slName list and builds the query itself with sqlDyStringPrintf, so the escaping happens where the values are, and a name that is not a GC accession never reaches the database. Nothing about this is visible. Correct and incorrect escaping give the same page for every input a browser sends, because the accessions come from our own chain files. The difference appears only for a value chosen to break out, which is the value that never turns up in ordinary testing. The genark table is data and it moves, so the test reads an accession out of it and asks whether the function returns that one, rather than naming an assembly that may be dropped later. Watched to fail and then pass: with the value pasted in unescaped the run dies instead of coming back with nothing, so the quoted cases turn the test red rather than quietly querying something else. Recorded as sandbox-ab in utils/testRegistry. refs #38328, refs #38391
  • lines changed 1, context: html, text, full: html, text
    89f9ce7bc7c9333c45190074088f5bee377ca1d7 Sat Sep 19 18:30:26 2026 -0700
    snapshotTypeTester: check the fast snapshotType reader against raFromString A saved session is a snapshot when its settings carry a snapshotType line, and the My Sessions listings hide those rows. #38313 replaced raFromString with a walk over the lines, because the question is asked once per row in a listing and the hash costs about 600ns and three allocations per session against about 30ns and none. A hand written parser that has to agree with a general one is the shape of bug found a year later, so this is a differential test: every case is read both ways and the two answers are printed side by side. The claim in the code is "same line semantics as raFromString", and that claim is what is checked rather than a list of answers written down once. Reading the tag wrongly is invisible either way: a snapshot is listed as an ordinary session, or an ordinary session disappears from the listing, and both look like something the user did. One case does not agree, and it is recorded rather than hidden. Given two snapshotType lines the walk returns the first and raFromString returns the last, because a hash overwrites. Real settings carry the tag once, so nothing depends on it today, but it is a difference in the semantics the code says it copies. It is marked as a known difference, and the test also fails if it ever starts agreeing, so the note cannot go stale in the other direction. Watched to fail and then pass: dropping the delimiter test after the tag makes "snapshotTypeExtra view" read as the type "Extra view" while raFromString finds none. Recorded as sandbox-ab in utils/testRegistry. refs #38313, refs #38391
  • lines changed 1, context: html, text, full: html, text
    751a1f9fff1d6d77c2da690d1369dca64ba04fee Sat Sep 19 18:31:42 2026 -0700
    sessionDirTester: pin how a saved session's data directory is named A saved session's custom tracks and region files are moved to a durable directory named from the user name and the session name. #10138 widened the session half of that name from 8 hex characters to 10: 8 characters of md5 is 4 billion values, and the birthday arithmetic over hundreds of thousands of sessions is not comfortable, since a collision puts one user's files in another user's session. Widening a name that is already on disk is the risky half. Directories written before the change carry the old width and their files are still in use, so the cleanup code has to be able to name both, which is why the width is a parameter rather than a constant in the middle of the function. The property pinned here is not the hash, it is that the short name is a PREFIX of the long one. That is what lets code holding the new name find a directory written under the old one, and it holds only because both come from the same md5 truncated to different lengths. Also pinned: the two-character spreading directory, the user name appearing as given, and the three calls that must abort rather than invent a path -- a relative sessionDataDir, and a width of 0 or 33. None of this is visible. A session whose directory is named differently does not report an error, it comes back without its custom track. Watched to fail and then pass: narrowing the width back to 8 turns it red on the width line. Recorded as sandbox-ab in utils/testRegistry. refs #10138, refs #38391
  • lines changed 16, context: html, text, full: html, text
    36977d0e699d3c853840f08cec63ab2a2dd5eaa7 Sat Sep 19 18:34:00 2026 -0700
    geoMirrorSelfTester: pin which gbNode row a server takes for itself The browser runs on several machines and each offers the others in a menu, so it has to know which gbNode row is itself. It used to answer from browser.node in hg.conf alone, so a machine serving a node other than the one browser.node names -- a sandbox, or two nodes behind one apache -- took itself for its own peer and offered the visitor a link to the site they were already on. #27988 made the host the visitor typed decide whenever that host is one of the nodes, with browser.node as the fallback. The symptom is invisible in the way that matters: every page renders, and a menu entry pointing back at itself reads as a mirror being down rather than as a bug. No domain is named here. gbNode is configuration data whose rows change, so the test reads two rows, points browser.node at the first, and asks whether the second can claim the server. What is printed is which of the two answered, not what they are called, and the port case is included because HTTP_HOST carries one whenever it is not 80 or 443. Also recorded in utils/testRegistry: what blocks a unit test on the fifteen v504 tickets still waiting for one, each with its own reason rather than a category, since the reason decides who can unblock it. Watched to fail and then pass: with the host test removed the server claims no node at all and offers all three, itself included. Recorded as sandbox-ab. refs #27988, refs #38391
  • lines changed 1, context: html, text, full: html, text
    814335cde61f999ae23626093306bdb6bf394f5e Sun Sep 20 07:00:39 2026 -0700
    hgvs: cover a deprecated protein accession, and say what is not covered A protein accession that is no longer current resolves through ncbiRefSeqLinkHistorical, which is what lets a term from an old paper still find its position. Three such terms are added to the valid-terms list: NP_000054.2, which has three deprecated transcript versions behind it, and NP_055615.8, which has versions .8, .9 and .10. The other half of #38248 is NOT pinned, and the registry row says so rather than implying otherwise. That half picks the newest deprecated transcript by sorting on length first, so that .9 does not beat .10 as a string. I could not build a term that shows it: with the order by removed entirely, and again with a plain string sort, every term gives identical coordinates. The reason is in the data -- for the accessions with both a .9 and a .10, the versions differ only in where the UTR ends, so cdsStart, cdsEnd and exonCount are the same and a coding residue lands in the same place whichever version is chosen. Recorded as assertion-only, the honest level, with what was tried written down so the next person does not repeat it. refs #38248, refs #38391
  • lines changed 2, context: html, text, full: html, text
    0f2f66270dcd4603785850ee72d7d883ff68185b Sun Sep 20 07:03:14 2026 -0700
    asForDbTester: a GenArk accession must not be looked for in MySQL asForDb() may need a database connection to fetch a track's autoSql. The name it is handed is usually an assembly, but on a quickLifted track it is the SOURCE assembly, and for a GenArk that arrives as a bare accession with no MySQL database of that name anywhere. Without #38272's isGenArk test the CGI does not return an empty field list, it dies, and the reader gets an error page instead of a details page -- only for the combination of a quickLift and a GenArk source, which is why it went unnoticed. The last case is the discriminating one: a name that is neither a GenArk accession nor a real database still aborts, which shows the GenArk case is let through deliberately rather than by connection failures having stopped mattering. The abort message carries a MySQL error number and the server's own wording, so what is pinned is whether the abort was about reaching a database of that name, not the text. Two things this test got wrong before it got them right, both recorded in its header. It has to run against a config that sets genarkHubPrefix, because isGenArk answers through that prefix and says no when it is unset, so with a developer's own hg.conf the GenArk case failed for a reason unrelated to the fix. And the accession is read from the genark table at run time, since naming an assembly makes the test go red the day that assembly is dropped. Also fixes the registry row for bedItemRgbTester, which moved to hg/cgilib and whose row still named the old path -- caught by the registry's own check, which is the first time it has failed for a real reason. refs #38272, refs #38391
  • lines changed 7, context: html, text, full: html, text
    4fa4bda98feb9c07aa1409b9bd0fb6274b388e31 Sun Sep 20 07:04:16 2026 -0700
    registry: record that four of the seven handed to docent have no docent script The decision on 2026-09-20 was that the tickets whose fix lives inside a CGI are covered by the docent suite rather than by a unit test. That is true of three of them. It is not true of #38233, #38253, #38254 and #38317, which have no docent script either, so nothing watches them at all, and the rows now say so. They belong on #38252's list. refs #38391
  • lines changed 4, context: html, text, full: html, text
    eccdddfb22cf35d4e5698f72a4b12707aa43b349 Sun Sep 20 14:27:27 2026 -0700
    testRegistry: check the docent column's "-" as well as its names The docent column says which browser test watches the same ticket, and a named script was already checked: it has to exist and has to belong to that ticket. A "-" was checked by nothing, and that was the wrong half to leave out. `unwatched` -- the list of tickets where nothing anywhere would go red if the bug came back -- is built entirely out of those "-" values. They were entered by hand, by reading the docent directory once, so a script written the next day left the ticket sitting on that queue with nothing to notice. For the four tickets that are on it because their code lives inside a CGI, somebody writing the docent script is the expected outcome, which made it the likeliest kind of row to go stale. So a row claiming no docent script is now checked against the tree the same way a named one is: if rm<ticket>.*docent.yaml turns up, the check fails and asks for the row to be filled in. The failure is somebody doing the right thing and the table not having heard about it. Watched to fail and then pass: blanking the docent column of #36212, whose script is in the tree, turns the real table red with the name of the script it found. refs #38391
• src/utils/testRegistry/testRegistry
  • lines changed 374, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
  • lines changed 22, context: html, text, full: html, text
    eccdddfb22cf35d4e5698f72a4b12707aa43b349 Sun Sep 20 14:27:27 2026 -0700
    testRegistry: check the docent column's "-" as well as its names The docent column says which browser test watches the same ticket, and a named script was already checked: it has to exist and has to belong to that ticket. A "-" was checked by nothing, and that was the wrong half to leave out. `unwatched` -- the list of tickets where nothing anywhere would go red if the bug came back -- is built entirely out of those "-" values. They were entered by hand, by reading the docent directory once, so a script written the next day left the ticket sitting on that queue with nothing to notice. For the four tickets that are on it because their code lives inside a CGI, somebody writing the docent script is the expected outcome, which made it the likeliest kind of row to go stale. So a row claiming no docent script is now checked against the tree the same way a named one is: if rm<ticket>.*docent.yaml turns up, the check fails and asks for the row to be filled in. The failure is somebody doing the right thing and the table not having heard about it. Watched to fail and then pass: blanking the docent column of #36212, whose script is in the tree, turns the real table red with the name of the script it found. refs #38391
• src/utils/testRegistry/tests/expected/check.out
  • lines changed 12, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
  • lines changed 1, context: html, text, full: html, text
    eccdddfb22cf35d4e5698f72a4b12707aa43b349 Sun Sep 20 14:27:27 2026 -0700
    testRegistry: check the docent column's "-" as well as its names The docent column says which browser test watches the same ticket, and a named script was already checked: it has to exist and has to belong to that ticket. A "-" was checked by nothing, and that was the wrong half to leave out. `unwatched` -- the list of tickets where nothing anywhere would go red if the bug came back -- is built entirely out of those "-" values. They were entered by hand, by reading the docent directory once, so a script written the next day left the ticket sitting on that queue with nothing to notice. For the four tickets that are on it because their code lives inside a CGI, somebody writing the docent script is the expected outcome, which made it the likeliest kind of row to go stale. So a row claiming no docent script is now checked against the tree the same way a named one is: if rm<ticket>.*docent.yaml turns up, the check fails and asks for the row to be filled in. The failure is somebody doing the right thing and the table not having heard about it. Watched to fail and then pass: blanking the docent column of #36212, whose script is in the tree, turns the real table red with the name of the script it found. refs #38391
• src/utils/testRegistry/tests/expected/list.out
  • lines changed 17, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/testRegistry/tests/expected/needed.out
  • lines changed 5, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/testRegistry/tests/expected/release.out
  • lines changed 15, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/testRegistry/tests/expected/test.out
  • lines changed 3, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/testRegistry/tests/expected/ticket.out
  • lines changed 6, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/testRegistry/tests/expected/tickets.out
  • lines changed 3, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/testRegistry/tests/expected/unwatched.out
  • lines changed 1, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/testRegistry/tests/expected/why.out
  • lines changed 14, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/testRegistry/tests/input/bad.tsv
  • lines changed 16, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
  • lines changed 1, context: html, text, full: html, text
    eccdddfb22cf35d4e5698f72a4b12707aa43b349 Sun Sep 20 14:27:27 2026 -0700
    testRegistry: check the docent column's "-" as well as its names The docent column says which browser test watches the same ticket, and a named script was already checked: it has to exist and has to belong to that ticket. A "-" was checked by nothing, and that was the wrong half to leave out. `unwatched` -- the list of tickets where nothing anywhere would go red if the bug came back -- is built entirely out of those "-" values. They were entered by hand, by reading the docent directory once, so a script written the next day left the ticket sitting on that queue with nothing to notice. For the four tickets that are on it because their code lives inside a CGI, somebody writing the docent script is the expected outcome, which made it the likeliest kind of row to go stale. So a row claiming no docent script is now checked against the tree the same way a named one is: if rm<ticket>.*docent.yaml turns up, the check fails and asks for the row to be filled in. The failure is somebody doing the right thing and the table not having heard about it. Watched to fail and then pass: blanking the docent column of #36212, whose script is in the tree, turns the real table red with the name of the script it found. refs #38391
• src/utils/testRegistry/tests/input/good.tsv
  • lines changed 8, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391
• src/utils/testRegistry/tests/makefile
  • lines changed 72, context: html, text, full: html, text
    5011b739391b6a1b6129d473b6c050010baf1f6f Sat Sep 19 17:57:37 2026 -0700
    testRegistry: a registry of which unit test defends which ticket A bug ticket has no way to say whether a test now defends its fix, and a test has no way to say which bug it came from. The two questions are asked from opposite ends and neither the tree nor Redmine answers either one. utils/testRegistry/registry.tsv is that table, hand written, one row per ticket per test, starting at v504. Unit tests only: the browser-page regression tests are the docent suite, refs #38252, which already names each script for its ticket, and a docent column here says which ticket has one of those too. Each row says why a unit test is the right test, because "the fix is in a library file" is wrong in both directions. A wording fix in hg/lib needs no unit test, and a performance rewrite inside hgTracks needs one badly, since nothing else can tell that the slow path has come back. The three reasons are invisible, perf and library, and a perf row says what has to be measured -- queries, passes, allocations, bytes -- never wall-clock seconds. testRegistry reads it: ticket, test, release, why, needed, unwatched, tickets, check. No network and no database, so the check can run anywhere. check is why the table is checked in rather than derived. A registry that quietly names files nobody has, saying a bug is covered when it is not, is worse than no registry, so check fails the build the day a row rots. Its own tests are in tests/, with one row of input/bad.tsv per complaint check can make, and utils/makefile runs the whole thing so it reaches make test from src. Today: 37 tickets, 12 covered by 11 tests, 25 waiting. refs #38391

• lines changed: 475861
• files changed: 627