Commits for max
switch to files view, user index
v504_preview2 to v504_base (2026-09-14 to 2026-09-21) v504
Show details
8ceea19db110700c4b6612ce4f30ce0b08da7d81 Mon Sep 14 07:34:15 2026 -0700
- hg38 Fiber-seq makeDoc: record where the nucleosome hold-back landed, refs #36210
The commit that holds it, 75e82896028, is titled "uniprot otto: the miniprot
cluster job needs absolute paths": two sessions were committing in the same
working copy and the other one took these files out of the shared index first.
It is pushed, so note it in the makeDoc rather than rewrite shared history.
- src/hg/makeDb/doc/hg38/fiberSeq.txt - lines changed 11, context: html, text, full: html, text
b1fea807e3a0f6c6b0ebb6b1244b3291aeaaad28 Mon Sep 14 07:51:13 2026 -0700
- hg38 Fiber-seq makeDoc: drop the git attribution note from 8ceea19db11, refs #36210
Which commit carries a change is session state, not something a reader needs in
order to rebuild the track. It belongs in the per-ticket notes under
/hive/groups/browser/redmineNotes/36210/claude/, where it now is.
- src/hg/makeDb/doc/hg38/fiberSeq.txt - lines changed 9, context: html, text, full: html, text
9541aea4204c8079ea2e464403f2225f6aa33cfd Mon Sep 14 07:59:00 2026 -0700
- uniprot: show the splice variant track, and filter the CAT alignments
Two things that were quietly missing.
unipSplice has been built since the pipeline rewrite in #19351 and never had a
trackDb stanza - "git log -S unipSplice" on uniprot.ra returns nothing, so it was
never wired up rather than deliberately dropped. It holds UniProt's splice variant
features, 28962 of them on hg38 and 29195 on hs1, and has been invisible on every
assembly for years. Added to uniprot.ra and to the archive/contrib template, which
is where the hub-served assemblies get their trackDb. The file exists on the same
131 assemblies as unipDomain.bb, so no stanza points at anything missing.
The alignments on a CAT assembly were not being filtered with pslSelect. That
filter maps a UniProt accession to the transcripts UniProt cross-references for it,
and the README is blunt about how much it matters for protein families with nearly
identical transcripts. It understood two kinds of id, Ensembl and RefSeq, and CAT
names a transcript after its source gene, so hs1 matched neither and fell through
unfiltered. Every row of the CAT bigBed carries the Ensembl transcript it was lifted
from, so catSourceTransMap joins on that and UniProt's Ensembl cross-reference does
the rest.
The mapping had to become one-to-many for this: one Ensembl transcript can name
several of ours, both because paralogs are lifted from the same source and because
duplicate CAT names were given -dup suffixes earlier. It reproduces those suffixes
by walking the bigBed in the same order rather than attaching every paralog to every
source, which measured 234903 of 234903 hs1 transcripts mapped, no duplicates,
against 330749 for the loose version. For every other gene track the lists hold one
element and the behaviour is unchanged, which is checked: a single-valued entry
still writes exactly one pair line and still counts a version difference, a
multi-valued one writes a line per transcript, and an id we do not have is still
skipped so pslSelect -qPass passes it through.
refs #38300
- src/hg/makeDb/trackDb/uniprot.ra - lines changed 12, context: html, text, full: html, text
- src/hg/utils/otto/uniprot/doUniprot - lines changed 67, context: html, text, full: html, text
- src/hg/utils/otto/uniprot/trackDb.template.txt - lines changed 11, context: html, text, full: html, text
ef0b068d8ea6e9ff5582aeeb8626a139c54c654f Mon Sep 14 09:08:07 2026 -0700
- hgLogin: explain ORCID sign-ins, stop duplicate accounts, confirm typed addresses
ORCID's OpenID Connect offers only the "openid" scope and carries no email
claim, so every first ORCID sign-in landed on the generic "choose a username"
page with no explanation of why, and typing an address that already belonged to
an account silently created a second account sharing it.
The "choose a username" page now says why a new account is always created. It is
shown when the provider released no address at all, rather than when the provider
is named "orcid", because a mirror can name a provider anything it likes in
hg.conf (refs #38213).
An address the user typed into that form is no longer taken on trust. If an
activated account already holds it, the signup is refused and the user is pointed
at a sign-in method that can show the address is theirs. Otherwise the account is
created unactivated and the user is sent to confirm the address by mail, instead
of being signed in next to a mail nobody had any reason to open. Clicking the
provider again before confirming re-sends the link rather than signing in, and
opening the link signs in an account that has no password, so a social-login user
is not left on a login page with nothing to type.
An address the provider itself released is trusted whether or not the provider
set email_verified. CILogon leaves that flag at 0 even for a real institutional
sign-in, and insisting on it would both force a mail round trip on every CILogon
user and stop them ever reaching an account they already have (refs #38339).
Also in this change:
- confirmChangeEmail activates the account. Opening that link proves the mailbox
is the user's, so an account made by a provider that released no address no
longer stays unactivated even after its owner confirms an address properly.
- an install with login.mailReturnAddr=NOEMAIL no longer tries to send an
activation mail it has no way to deliver.
- the activation token is drawn with makeRandomKey, the same source already used
for the email-link login token and the OAuth state nonce.
refs #38341, refs #38339
- src/hg/hgLogin/hgLogin.c - lines changed 153, context: html, text, full: html, text
e4c2e97b5eb35b8f9b7b71eb13a4c24e6c1ece22 Mon Sep 14 09:19:02 2026 -0700
- hui, hgTrackUi: encode trackDb setting and cart text in HTML attributes, refs #38123
- src/hg/hgTrackUi/hgTrackUi.c - lines changed 2, context: html, text, full: html, text
5afbafb6dde0b144cca0fe3f377a5b284e3561d3 Mon Sep 14 10:26:17 2026 -0700
- uniprot otto: an organism whose proteins do not align should not stop the run
aquChr2, the golden eagle, took the run down with a bare
AssertionError:
on assert(os.path.getsize(mapFname)!=0). UniProt has two proteins for taxon 216574
and neither aligned, so the mapping PSL came out empty. That is a fact about the
data rather than a failure of the pipeline, and it is the same situation as an
organism with no proteins at all, which is already skipped with a warning. Several
more taxa in the plan are in the same position, with 1, 2, 8, 13 and 17 proteins.
The assembly is now skipped with a message naming it, both when the mapping was
just built and when an empty one is being reused from an earlier run.
Worker failures also log a traceback now. errAbort raises a bare AssertionError, so
the pool reported exactly "taxon 216574 failed: AssertionError:" and said nothing
about where, which is worse than what the sequential path used to print. Checked
that the errAbort message, the traceback and the file and line all appear.
refs #38300
- src/hg/utils/otto/uniprot/doUniprot - lines changed 19, context: html, text, full: html, text
0ebe7cd89f87af0c5e8310eb7bed6f65d2596604 Mon Sep 14 14:37:19 2026 -0700
- hgLogin: do not ask for an email address when the provider already gave us one
The "choose a username" page showed an editable email box for every social
signup, including the providers that do send us an address. For CILogon that
meant being asked for something we then took at face value: the box arrived
prefilled with CILogon's address, whatever came back was accepted, and the
account was created and signed in. A box whose contents we never check is worse
than no box, because it reads as a question we check the answer to.
The page now asks only when the provider released no usable address. Where the
provider gave one, the page says which address the account will use and offers
no input at all. completeAccount reads that address from the pending identity
rather than from hgLogin_email, so a leftover value in the cart cannot stand in
for it.
Where the provider released nothing, ORCID being the case that matters, nothing
changes: the user is asked for an address, and the account stays inactive until
the mailed link is opened.
refs #38339, refs #38341
- src/hg/hgLogin/hgLogin.c - lines changed 63, context: html, text, full: html, text
b1a27c93758e1f12cd212f1293d9f3ec9bbfb0af Mon Sep 14 14:53:21 2026 -0700
- hgLogin: do not blame the provider when it sent an address we cannot use
The "choose a username" page decided what to say by asking whether
oauthProviderEmail() had returned an address, but that returns NULL both when
the provider released nothing and when it released something
spc_email_isvalid rejects, which is every address containing a byte >= 127.
A user arriving with a non-ASCII address was therefore told that their provider
"does not share your email address with us", which is untrue, and then asked to
type one.
Key the explanation on whether an address arrived at all, and say plainly that
we could not use the one we were sent in the case where one did.
refs #38341
- src/hg/hgLogin/hgLogin.c - lines changed 9, context: html, text, full: html, text
5f590f874b9d352023167b563e5b83f651488f0b Tue Sep 15 04:56:35 2026 -0700
- Resolve $hgsid in native track description pages too, refs #38283
A description page could already use $hgsid if it belonged to a hub, because a hub's
html is substituted at render time where there is a cart. A native page is substituted
once by hgTrackDb, which has no cart, so $hgsid quietly became the empty string and the
link it was part of came out broken. hg38's hprcRdt page has been shipping
"hgTrackUi?hgsid=&g=long_read_transcripts" for this reason.
hgTrackDb now writes the reference back out as ${hgsid} instead of resolving it, and
hVarSubstTrackDbHtml resolves it at render time for native pages as well as hub ones.
Only $hgsid is deferred, and only in the html field: the labels get no second pass, so a
deferred reference in one of them would reach the user as the literal text "${hgsid}".
The render pass over a native page acts only on the braced form, which is what keeps an
escaped $$hgsid escaped -- hgTrackDb collapses that to a bare $hgsid, and a bare one is
left alone.
The html is no longer freed before being replaced, and is replaced only when the
substitution actually changed something. hVarSubstExt allocates its result at the first
dollar sign whether or not it substitutes anything, so any page merely containing one
reached that free; for a native track tdb->html can point into the trackDb cache, which
is localmem carved out of an mmap'd file and never came from malloc. Reproduced on hg38
chm13LiftOver, whose page contains an awk snippet, with cacheTrackDbDir set.
Description pages have to be reloaded for the deferral to reach the trackDb table.
- src/hg/lib/hVarSubst.c - lines changed 131, context: html, text, full: html, text
d068f3195e28f60403917212dc139504b392e7c6 Tue Sep 15 04:56:49 2026 -0700
- uniprot otto: guard the shift in doUpdate.sh
shift is a POSIX special built-in, so a shell that follows that rule exits when it fails.
Under dash, which is /bin/sh on Debian and in most containers, running doUpdate.sh with
no arguments ended the script at that line and doUniprot never started. Redirecting the
complaint to /dev/null hid the reason, so the run log showed a START with no END, which
reads exactly like a run still in progress. /bin/sh is bash on hgwdev, where it works.
- src/hg/utils/otto/uniprot/doUpdate.sh - lines changed 11, context: html, text, full: html, text
5ef589f4bdca750ae44fabb5c648378e8264b2c9 Tue Sep 15 04:56:51 2026 -0700
- redmineCli: match the 403/404 prefix, not anywhere in the error text
summarize_issue treats a ticket as unreadable when api_get reports 403 or 404, and
re-raises everything else so a real failure is not hidden behind "(not readable)". The
test looked for "HTTP 403 GET" anywhere in the message, but that message carries up to
500 characters of the server's response body, so a 500 or a proxy error page quoting
those words in its own text was swallowed as well. Anchor it to the prefix the tool
builds.
08221b327a1496fd1e4d3f88dc17e839b15b64f7 Tue Sep 15 05:24:18 2026 -0700
- uniprot otto: minAli must not be global, and must be part of the cache key
max spotted that ci3's lower alignment threshold was assigned to the global MINALI.
With --taxonThreads that leaks: ci3 sets 0.85 and every assembly whose thread reads
the global afterwards uses it too. In the 2026_03 run it did exactly that - 89 of 95
alignments ran at 0.85 instead of 0.93, and which assemblies escaped depended purely
on thread timing, with ce11 and cb3 getting 0.93 by luck. His fix makes it a local;
this commit keeps that and closes the second half of the problem.
The mapping files are cached under an md5 of the protein sequences, the transcripts,
their alignment and the protein/transcript pair file. The threshold was not in it,
so a mapping built at 0.85 would be silently reused on a run that asked for 0.93,
and the bad alignments would have survived the rebuild that is meant to fix them.
minAli is now part of the key, so changing it invalidates the mapping by itself.
Checked that the key differs between 0.93 and 0.85.
Also gave lastRun_noAnnotatedTranscriptIDs.txt a per-assembly name. It is written
into the shared mapDir under a fixed name by every assembly, which several threads
would write at once. Only a debugging artifact, nothing reads it, but it is the last
shared write in the per-taxon path.
Audited the rest of the module for state shared across threads: the only remaining
global is flagFname, set once in main before the pool starts; dbIsHubCache can be
written by two threads but only ever with the same value; and every temp directory
and working file is keyed by assembly, md5 or taxon.
refs #38300
- src/hg/utils/otto/uniprot/doUniprot - lines changed 19, context: html, text, full: html, text
4998072e98b35a6fc5c694f1ae56fcd01fefe3bb Tue Sep 15 05:29:18 2026 -0700
- hgLogin: say why a social sign-in is asking for an email confirmation
Landing on "a confirmation email has been sent to you" straight after clicking a
provider button is confusing on its own: the user asked to sign in, not to fill in
a form, and is told to go and read their mail. The page now says first what
happened, in one of two ways, because the two cases that reach it are different.
A first sign-in through a provider that released no address says that no account
uses the address they typed yet, so one is being made for it. An existing account
whose address has never been confirmed names that account instead, since telling
its owner that no account exists would be untrue.
An account that already carries the address the provider just gave us no longer
has to confirm anything: that is the same assurance a new signup through the same
provider gets, so the account is activated and the user signed in. Accounts left
unactivated by the earlier code, which asked for an address and took it on trust,
settle themselves on the owner's next sign-in this way.
A plain email signup sets none of the new cart variables and that page reads as
before. They are html-encoded on output and dropped from an incoming request, so
a supplied copy cannot put text on the page.
- src/hg/hgLogin/hgLogin.c - lines changed 71, context: html, text, full: html, text
77cfc90b4fb2166b9ae955ad40a067549d6d5881 Tue Sep 15 05:29:56 2026 -0700
- hgc share-a-link: force the enclosing superTrack to show
The hgc details-page "Share a link" button strips the hgsid and links straight
to the page, but a track nested in a superTrack draws nothing on a fresh load
of that link since superTracks default to hide. Add tdbTopSuperTrack() so the
per-track JSON carries the enclosing superTrack's name, and have the share
dialog force <superTrack>=show onto the URL when there is one.
- src/hg/hgTracks/imageV2.c - lines changed 21, context: html, text, full: html, text
aac682d3e89d0197a6467ae5231d3fe979f06d87 Tue Sep 15 05:30:48 2026 -0700
- danioCode: un-nest the standalone data types, split CRISPR out of DC Conservation
Give RNA-seq, CAGE-seq, 3P-seq, ChIP-seq and Hi-C their own top-level track in
an existing group (rna/genes/regulation) instead of hiding all eleven DANIO-CODE
containers behind one superTrack that only someone already looking for
DANIO-CODE would open. The Burgess lab's CRISPR/Cas9 target-site tracks, which
were riding along inside the "DC Conservation" composite, get their own
top-level superTrack (dcCrispr, group map) instead, mirroring where hg38 keeps
its own unrelated crispr tracks, and drop the "DC" branding since they aren't
DANIO-CODE's data. The rest -- a mixed bag of regulatory-element/annotation
containers that don't map onto any one existing group -- stay nested under the
danioCode superTrack.
Implemented in danioCodeHubToRa.py (STANDALONE_GROUP/NESTED_ORDER/CRISPR_*) so
the regrouping survives the next hub re-import, then regenerated danioCode.ra
from the cached hub trackDb.txt. tdbQuery -check passes; still alpha only.
refs #38265
- src/hg/makeDb/doc/danRer11/danioCode.txt - lines changed 47, context: html, text, full: html, text
- src/hg/makeDb/scripts/danioCode/danioCodeHubToRa.py - lines changed 85, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/danioCode.html - lines changed 39, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/danioCode.ra - lines changed 6262, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcComparativeGenomics.html - lines changed 16, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcCrispr.html - lines changed 87, context: html, text, full: html, text
c5fa25640623eef3aa9c8f1caaf4be18bab98a77 Tue Sep 15 05:45:03 2026 -0700
- hgLogin: fixes from the code review of the social sign-in work
Read the provider label before clearPendingIdentity, not after. cartUsualString hands
back the cart's own string and cartRemove frees it, so the confirmation page was
naming the provider from memory that had just been released. hgLogin installs
pushCarefulMemHandler, which keeps the bytes readable, which is why it looked fine.
Let someone who mistyped their address at the choose-a-username page correct it. The
confirmation never arrives, and until now there was no way out at all: no password to
sign in with, no activated account for the email link, no login cookie for the
change-email page, and the user name and provider identity both already taken. The
confirmation page now offers a box to replace the address, authorized by the same
signed pending identity the account chooser uses, so only a real provider round trip
in this browser can get to it.
Signing in again while an account waits to be confirmed reuses the token that is still
outstanding instead of minting a new one. A new token silently voids the link in the
message before it, so a user who clicked the button twice and opened the first mail was
told the link was invalid.
Promise the account page only where it exists: the change-email flow is behind
login.emailLink, which is off by default, and since the address box went away a user
there would have had no way to change an address we told them they could change.
Clear the confirmation page's explanation variables in the plain signup path too. The
hand-off to that page is a JavaScript redirect, so a user who never lands on it leaves
them in the cart, and the next plain signup rendered somebody else's explanation over
an unrelated address.
- src/hg/hgLogin/hgLogin.c - lines changed 138, context: html, text, full: html, text
998149024f8bde04e8b76638043cfeac3d158731 Tue Sep 15 05:47:27 2026 -0700
- Queue the cart cookie and the content policy instead of printing them, refs #38353
cartWriteHeaderAndCont() guards on cgiDidContentType(), which any cgiPrintContentType()
anywhere sets. An early warn() during cartNew -- "Unable to load session file" reaches the
early warning handler, which calls htmlStart -- prints the header before there is a cart,
and from then on the Set-Cookie and Content-Security-Policy lines were silently skipped.
Nothing changed on the wire, since before the guard they landed in the page body as text
and were equally inert, but skipping them silently is not the behaviour to keep.
cartWriteCookie() and cspWriteResponseHeader() now hand their lines to cgiAddHttpHeader(),
so whichever call prints the content type prints them too and the order of the calls no
longer matters. cartAndCookieWithHtml() queues the policy before the early handlers are
pushed, so even a page written by that early warn carries one. The cookie cannot be queued
that early -- there is no cart yet -- and is still lost on that path; the comment says so.
cgiAddHttpHeader() now does what its own comment already promised and ignores a header
added after the block was closed, rather than growing a list nothing will ever print.
getCspPolicyString() is declared in htmshell.h so hCommon.c can queue the value on its own.
8c172e6c4b985af79bf1a1f47c9db92ce0578999 Tue Sep 15 05:47:36 2026 -0700
- errCatch: put errAbort's doContentType back the way it found it, refs #38353
hVaUserAbort() turns doContentType on so that a user error reaches the browser when the CGI
has not pushed a warn handler of its own. Inside an errCatch that is pointless -- the abort
is caught and never reaches the default handler -- and the flag stayed on afterwards, so a
later genuine abort on a page that had already written its header would print a second
Content-Type line into the body. Everywhere else in the tree the flag is something a caller
turns back off.
errCatchPushHandlers() now records the flag and errCatchEnd() restores it, which covers every
caller rather than just the one that set it. errAbortGetDoContentType() is the reader.
ed2944f615eb64c06c10b868754a60926a04a412 Tue Sep 15 05:47:46 2026 -0700
- Four leftovers from the Content-Type sweep, refs #38353
edwWebAuthLogin and edwWebAuthLogout still printed a blank line of their own after
cgiPrintContentType(), which already ends the header block, so every response began with an
empty line in the body.
hgPhyloPlace built its own Content-Type line from a macro in five places.
hgChooseDb's fail() sent a Status line and no Content-Type at all, so a bad request came back
with a body apache had no type for.
- src/hg/encode3/encodeDataWarehouse/edwWebAuthLogin/edwWebAuthLogin.c - lines changed 1, context: html, text, full: html, text
- src/hg/encode3/encodeDataWarehouse/edwWebAuthLogout/edwWebAuthLogout.c - lines changed 1, context: html, text, full: html, text
- src/hg/hgChooseDb/hgChooseDb.c - lines changed 2, context: html, text, full: html, text
- src/hg/hgPhyloPlace/hgPhyloPlace.c - lines changed 16, context: html, text, full: html, text
55e11ee5939bfbe8b3f7283b1abb337de5a55ae3 Tue Sep 15 05:48:03 2026 -0700
- hgTracks: the exon and codon mouseovers disagreed on out-of-frame transcripts, refs #38353
An exon's mouseover worked its codon numbers out as (c+2)/3 and the per-codon mouseover took
them from the codon list, which builds them from exonFrames. Where a transcript's annotated
CDS does not begin on a codon boundary the two part company: baseColorCodonsFromGenePred
makes codon 1 short rather than shifting every number after it, so the arithmetic is off by
the missing bases for the rest of the transcript. About 9,000 transcripts in hg38's
wgEncodeGencodeCompV48 and 101 in ncbiRefSeqCurated are affected. hg38.knownGene has no
exonFrames and never was.
The exon note now reads the numbers off the codon list when there is one, and otherwise -
zoomed out past the level that builds it - counts them with the same short first codon in
mind. The per-codon note measures its c. range off the codon's own bases instead of taking
it as 3*codonIndex-2..3*codonIndex, which was wrong for the same reason, and gathers both
pieces of a codon split across an intron.
Checked on hg38 ncbiRefSeqCurated NM_001372106.1, whose first coding exon has frame 1: exon 3
now reads c.369-602 (p.124-201) and the first per-codon box inside it c.369-371 (p.124),
where before the exon said p.123 and the codon box said c.370-372.
Also in this file, two things the review turned up while reading it:
loadTxCdsBatch() capped its IN list at 2000 accessions and dropped the rest without saying
so, which would leave the transcripts past the cap numbered along the genome with nothing to
show for it. It queries in chunks now.
txCodonIndexForCodon()'s comment claimed any of a codon's three bases gives the same answer.
It does not when there is an indel inside the codon, which is the case the whole feature
exists for; the code already uses the 5'-most base, so only the comment was wrong.
genbankCds txCds is zeroed before baseColorTxAliForGenePred() may leave it untouched.
- src/hg/hgTracks/simpleTracks.c - lines changed 130, context: html, text, full: html, text
4bf24023fb4ce3e83104c97043c7e898101be1d1 Tue Sep 15 05:48:15 2026 -0700
- hgvs: a bare codon number must not leave a tail behind, refs #38353
The three patterns that read a bare codon number or range were anchored at the start only,
so "KAT6A 495-", "KAT6A p.495_", "KAT6A 495--533" and "KAT6A p.495_533_600" all quietly
became codon 495 (or 495_533) with the rest thrown away. Silently dropping the tail of a
position is exactly what this grammar was added to stop doing.
Anchored at the end as well. Every other pattern here is free to leave a tail for a later
stage to interpret, but a bare codon number has nothing after it to interpret.
The tests now pin the four rejected forms, and also codon 0, a codon past the end of the
protein, and a backwards range, none of which were pinned before.
- src/hg/lib/tests/expected/hgvs/validTerms.txt - lines changed 12, context: html, text, full: html, text
- src/hg/lib/tests/input/hgvs/validTerms.txt - lines changed 12, context: html, text, full: html, text
24a1b93cb1cb3590e987bf5c2910fed600d44a66 Tue Sep 15 05:48:40 2026 -0700
- hgc: a mistyped detailsScript setting must not take down the details page, refs #38353
The null case was covered but nothing else was. jsonParse() aborts on malformed JSON and
jsonObjectVal() aborts on any value that is not an object, so a hub with
"detailsScript.histogram.myField 5" in its trackDb lost the whole details page, not just
that one plot. Caught now: the setting is dropped and the rest of the page is drawn.
- src/hg/hgc/bigBedClick.c - lines changed 15, context: html, text, full: html, text
f04876658f4ae0ec7c633895c21f153975a2a506 Tue Sep 15 05:48:42 2026 -0700
- faceted composite: default sort column, restored drag order, and two smaller things, refs #38353
defaultSortCol was still 1, which the drag-handle column took over when manual row ordering
went in. A composite with no defaultSortField opened sorted by a hidden column, with no sort
arrow on any visible heading.
Fixing that alone exposed the next one: coming back on the Active tab called setFilterMode(),
which renumbers the order field from whatever is on screen, and that threw away the hand-
dragged order just read back from localStorage. It was invisible only because the current
order happened to be the drag order while the default sort column was the drag column. The
restore now keeps the saved order; a click still renumbers, which is what a click means.
Saved state had no eviction, so once the quota filled, the QuotaExceededError was swallowed
and from then on no faceted composite on the origin could save anything again. Each entry
now carries a timestamp and a full store drops the least recently written entry belonging to
another composite and tries again.
Narrowing a facet turned the track container back on even with nothing selected, which brings
back an empty image.
- src/hg/js/facetedComposite.js - lines changed 66, context: html, text, full: html, text
2003809b18f5b9267e19693568edbccede0e6ab2 Tue Sep 15 05:48:44 2026 -0700
- hgc scatterPlot: bounds by walking the points, and no duplicate legend entries, refs #38353
niceBounds() collected every coordinate into an array and passed it to Math.min(...xs). A
spread is one argument per point, and this module is meant for clouds of tens of thousands:
measured, 120,000 points are fine and 200,000 throw RangeError, which the promise's catch
turns into "Plot data could not be loaded". Nothing shipped is that big yet. Walked
instead, which needs no arrays either.
A file that supplies a labels list to fix the legend order and then gives its points in
object form had every category appended a second time, so each one appeared twice in the
legend. The indexer is seeded from the labels already present.
- src/hg/js/hgc.scatterPlot.js - lines changed 32, context: html, text, full: html, text
d5180601357304b9a9edef820745a66896ec09ad Tue Sep 15 05:48:46 2026 -0700
- hgMyData: find the Hub Upload tab by its panel, not by counting to three, refs #38353
hgHubConnect only prints the Hub Development tab when hgHubConnect.validateHub is set, so on
a mirror with storeUserFiles on and validateHub off, Hub Upload is the third tab. It then
got neither the upload instructions nor the HTTPS warning.
9d9fff4aef9fd9a187839f72e49c3b5ad37ba34a Tue Sep 15 05:48:47 2026 -0700
- hgHubConnect: encode the hub URL into the Connect button's JS literal, refs #38353
The three sibling handlers nearby all pass it through javaScriptLiteralEncode; this one did
not.
- src/hg/hgHubConnect/hgHubConnect.c - lines changed 2, context: html, text, full: html, text
c500eb92eb2a021af927f3c6f2ad00c7b29769a5 Tue Sep 15 05:48:49 2026 -0700
- snapshotSession: snapshotIsSnapshotName is file-local, refs #38353
Its only caller is the writer in the same file, checking its own argument, and its header
comment warned everyone else off it. Making it static says that rather than asking.
- src/hg/inc/snapshotSession.h - lines changed 5, context: html, text, full: html, text
- src/hg/lib/snapshotSession.c - lines changed 5, context: html, text, full: html, text
03bd4721a68ec74cab51164ff2d4b216cd6fe9b0 Tue Sep 15 05:49:00 2026 -0700
- trackDb README and comments: $hgsid is for native description pages only, refs #38353
The README still described $hgsid as a hub-page variable that is empty in native trackDb;
both halves of that are now the other way round. The three call sites of
hVarSubstTrackDbHtml that still said a hub page is the only reason it exists say what the
call does now.
- src/hg/hgTrackUi/hgTrackUi.c - lines changed 3, context: html, text, full: html, text
- src/hg/makeDb/trackDb/README - lines changed 8, context: html, text, full: html, text
948e76a68b4a62e4f6f90244eeb266342406ce53 Wed Sep 16 03:03:36 2026 -0700
- hgLogin: make trusting a provider's unverified email a per-provider setting
Accepting an address a provider had not marked verified was applied to every
provider, to keep CILogon usable: it sends the address it got from the user's
institution but leaves email_verified at 0, even for a real institutional sign-in.
Extending that to everyone was too much. GitHub hands over a primary address whose
owner never confirmed it, and any provider a mirror adds to hg.conf was treated the
same way, so an address nobody had checked was enough to be signed in to an existing
account that used it.
New login.oauth.<name>.trustEmail, off unless an admin sets it, says that a named
provider's address may be taken without email_verified. It belongs on a provider that
reads the address from somewhere the user cannot type into, which is what CILogon
does and what GitHub does not.
Where it is off and the provider did not verify the address, the address is dropped
rather than refused, and the sign-in proceeds as one that arrived with no address at
all, which is already the ORCID case: the user is asked for an address and the account
does not work until they open the link mailed to it. Dropping it also keeps it out of
the account matching in resolveIdentity, so it cannot reach an existing account.
Documented in product/mirrorManual.txt next to the other provider settings, with the
CILogon line mirrors will need, and in oauthLogin.h with the rest of the keys.
- src/hg/hgLogin/oauthLogin.c - lines changed 31, context: html, text, full: html, text
- src/hg/hgLogin/oauthLogin.h - lines changed 7, context: html, text, full: html, text
- src/product/mirrorManual.txt - lines changed 24, context: html, text, full: html, text
3bacf54b6a23bc56f728c2e670c9b2cca96222dd Wed Sep 16 04:56:02 2026 -0700
- htmshell: drop the two CSP response-header helpers that no longer have callers, refs #38353
cspWriteResponseHeader() was their only user and it queues the policy through
cgiAddHttpHeader() now, so generateCspResponseHeader() and getCspMetaResponseHeader() build a
header line nothing sends. Neither had another caller at any point. The meta-tag side is
untouched.
7c624aa6b9c463d584e9ac0e048a35c6afc697b6 Wed Sep 16 04:59:33 2026 -0700
- add new flag for Oauth: trust OpenID email validated flag or not.
- src/hg/htdocs/goldenPath/help/mirrorManual.html - lines changed 30, context: html, text, full: html, text
73db11cce7b606145e13f1d15b02651f27f58a91 Wed Sep 16 06:18:38 2026 -0700
- danioCode: relabel with full DANIO-CODE branding, un-nest conservation, default on one RNA-seq/ChIP-seq sample
Rename the "DC" shortLabel prefix to "DANIO-CODE" throughout (labels were
getting long, but the full name matters more than brevity here). Relabel the
danioCode superTrack itself to "DANIO-CODE Elements" and drop the now-redundant
DANIO-CODE prefix from its remaining nested composites (Elements, Cell Types,
COPEs DOPEs, Enhancers, Promoters).
Un-nest the Burgess lab's phastCons/CNE conservation data (dcComparativeGenomics)
into its own top-level track, "Burgess Fish PhastCons", group compGeno --
same treatment as the CRISPR tracks pulled out earlier, since this isn't
DANIO-CODE's own data either.
Rewrote every description page's intro: dropped the internal link to the
superTrack in favor of a line crediting the DANIO-CODE project and linking to
https://danio-code-dcc.genereg.net/, cut each intro to two sentences or fewer,
and dropped the general explainer paragraphs (what non-coding regulation is,
what conservation is) from danioCode.html and the conservation page entirely.
Turn on exactly what the hub itself already marked "on": one RNA-seq sample
(Prim-5, Busch-Nentwich lab) and one ChIP-seq signal track (H3K4me3), by
letting dcRNAseqComposite and dcChIPseqComposite keep the hub's own
"visibility full" instead of forcing hide, via VISIBLE_BY_DEFAULT in
danioCodeHubToRa.py. Everything else (hundreds more subtracks per composite)
stays off by default.
tdbQuery -check passes; still alpha only.
refs #38265
- src/hg/makeDb/doc/danRer11/danioCode.txt - lines changed 49, context: html, text, full: html, text
- src/hg/makeDb/scripts/danioCode/danioCodeHubToRa.py - lines changed 41, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/danioCode.html - lines changed 35, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/danioCode.ra - lines changed 71, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dc3PseqComposite.html - lines changed 7, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcCAGEseqComposite.html - lines changed 6, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcChIPseqComposite.html - lines changed 10, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcComp.html - lines changed 6, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcComp_cell_type.html - lines changed 5, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcComparativeGenomics.html - lines changed 17, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcConsensus_promoters.html - lines changed 6, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcCopes_and_dopes.html - lines changed 8, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcCrispr.html - lines changed 13, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcEvalidation.html - lines changed 6, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcHiC_Composite.html - lines changed 7, context: html, text, full: html, text
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcRNAseqComposite.html - lines changed 7, context: html, text, full: html, text
0b293869739f3112a74d2ee38f47785a7c245884 Wed Sep 16 06:29:25 2026 -0700
- HPRC v2 contrib hub: pcLAI defaults to dense, and genark addContrib wires the beta/public hub tiers
Mark reported that the pcLAI track shows only one color on HG00408 pat
(GCA_041900255.1) and that you cannot see where ancestry changes. The colors
are correct: a fresh download of the HPRC source BED matches the built
pclai.bb on (chrom, start, end, itemRgb) for all 25,475 windows. That
haplotype is simply 99.8% one ancestry cluster, which is common in this set
(71 of 460 haplotypes have a single centroid, 228 are >=99% one centroid).
The display was the real problem, so hprcPclai goes from visibility pack to
visibility dense plus onlyVisibility dense. At chromosome scale pack stacks
the 100 kb windows into ~50 rows of 1-2 px slivers; dense draws one colored
bar where the ancestry blocks are legible.
While deploying that, beta.hub.txt turned out to be a week stale: it still had
the pre-2026-09-09 pcaSegment field name and the double-prefixed
pclaiRefPanel.json dataUrl, so the details-page scatterplot was not drawing on
hgwbeta. genark addContrib only ever wired alpha.hub.txt, leaving beta and
public to the next clade build. It now wires those two as well for a
collection already named in betaGenArk.txt / publicGenArk.txt. This never
promotes anything - the tier argument still owns the release lists - and
mkGenomes.pl inlines the same per-assembly trackDb, so a later clade build
converges on the same content.
Ran over 462 assemblies, alpha and beta; genark checkContrib reports no
contrib problems, refs #35415
- src/hg/makeDb/doc/contrib/hprc2annot/hprc2annot.txt - lines changed 67, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt - lines changed 2, context: html, text, full: html, text
- src/utils/genark/genark - lines changed 24, context: html, text, full: html, text
b231848705610405fb34e198796e5b48d3c43907 Wed Sep 16 06:49:31 2026 -0700
- hprc2annot: pcLAI is squish, not dense - dense removes the details page
Mark asked for onlyVisibility dense on the pcLAI track. Dense is not usable
here: in tvDense hgTracks draws one merged row and emits no per-item map boxes,
so there is no hgc link and no mouseOver, and the ancestry scatterplot on the
details page becomes unreachable. Counted on the image map of a 1.7 Mb view
holding 16 windows: pack and squish give 16 hgc links and 16 mouseOvers, dense
gives none.
Squish gives the same readability without that cost. It keeps every map box and
uses half-height rows with no label column, and since the windows tile without
overlapping they collapse to one row on chr18 (three or four on chr2) against
pack's ~50. On chr18 at 950px: pack 561px, squish 97px, dense 81px.
hprcPclai is now visibility squish + onlyVisibility squish, leaving Hide and
Squish in the dropdown. Re-ran hprc2annotMakeTrackDb.py and genark addContrib
over 462 assemblies, alpha and beta, refs #35415
- src/hg/makeDb/doc/contrib/hprc2annot/hprc2annot.txt - lines changed 33, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt - lines changed 2, context: html, text, full: html, text
a3fd570e96fb63b45d583ebcce83d8d23673a30a Wed Sep 16 07:00:11 2026 -0700
- hg38 pcLAI: squish, not dense - dense removes the details page
Same problem the GenArk contrib collection had: in tvDense hgTracks draws one
merged row per subtrack and emits no per-item map boxes, so there is no hgc
link and no mouseOver, and the ancestry scatterplot built for the details page
is unreachable. On chr18:10,800,000-12,500,000 the image map has 0 hgc links in
dense and 134 in squish.
The composite is now visibility squish plus onlyVisibility squish. On a
composite that governs the only vis dropdown the track has, since the subtracks
get checkboxes rather than dropdowns, and hgTrackUi now offers Hide and Squish.
Squish suits the data: the windows tile without overlapping and carry no item
labels, so each haplotype stays one row at a 20 Mb view, refs #35415
- src/hg/makeDb/doc/hg38/hprcPclai.txt - lines changed 31, context: html, text, full: html, text
- src/hg/makeDb/scripts/hprcPclai/hprcPclaiMakeTrackDb.py - lines changed 7, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/hprcPclai.ra - lines changed 2, context: html, text, full: html, text
3d9c0012a395a31127c3af3459c4e28c48dda205 Thu Sep 17 01:09:25 2026 -0700
- uniprot otto: keep updating the classic assembly when a GenArk one appears
#Preview2 week - bugs introduced now will need a build patch to fix
getTaxIdDbs took the first assembly per taxon by orderKey. GenArk assemblies now
sort ahead of the classic ones, so the moment ARS_UCD2.0 appeared for cow, bosTau9
stopped being updated: its UniProt tracks froze at whatever release they had while
the new data went into a GenArk contrib collection that is not installed. Users are
still on bosTau9. The same had happened for marmoset, zebrafish, gorilla, cat,
macaque, chicken, rat, horse, sheep, chimp, bonobo, orangutan, zebra finch and two
more, sixteen taxa in all.
Take the first of each kind instead, one GenArk and one classic, so both keep
getting the current release. The plan goes from 119 to 135 assemblies; the taxa
count is unchanged at 109, and no assembly appears twice, so the assertion at the
end of the function still holds. Human, mouse, fly, worm and yeast are unaffected.
refs #38300
- src/hg/utils/otto/uniprot/doUniprot - lines changed 12, context: html, text, full: html, text
f664e77c7e67c0342ba78b140386c4644d451a32 Thu Sep 17 01:42:56 2026 -0700
- hgHubConnect, lib: name the hub URL when a hub fails to load
#Preview2 week - bugs introduced now will need a build patch to fix
A hub that could not be opened showed up in the connected-hubs table as an
error message and nothing else: the name, description and assembly cells were
all empty, because they are only filled in from the trackHub struct that the
failed load never produced. The URL was in the row only inside the javascript
of the Disconnect and Retry buttons, so there was no way to tell which hub the
error was about. Print the URL in the name cell instead.
netParseUrl chops up its local copy of the URL as it parses, so none of its
errAborts could report what they were looking at. Keep the caller's string
and name it in all of them. A URL written with a single slash, https:/host/f,
then reported a non-numeric port, because with no :// the protocol defaults to
http and the host becomes "https" with an empty port - say what is actually
wrong with it.
Also tighten the encoding of the hub tables' output and of javascript string
literals, and use htmlEncode, not javaScriptLiteralEncode, for the option
labels in cgiMakeSelectDropList, which are HTML text.
- src/hg/hgHubConnect/hgHubConnect.c - lines changed 10, context: html, text, full: html, text
b82bce91b10725b252a3ead15ba68cb0700d997a Thu Sep 17 02:12:25 2026 -0700
- Stop the narrow-screen menu icon from covering the menu bar items, refs #38251
#Preview2 week - bugs introduced now will need a build patch to fix
The previous fix put the icon at position:sticky with float:right so that it would
follow the right edge of the window while the 1000px-wide bar is scrolled sideways.
Sticky only moves an element that is in the normal flow, and that turned out to be
the problem: the float sits in the same line as the menu items, so at some widths
they wrap out of the 32px-high bar and their labels are drawn below it on white,
and as the window narrows the icon sweeps leftwards across the items and hides
whichever one is under its opaque background.
Leave the icon where the rest of the bar is instead, at the right-hand end, which
is what the rule for wide windows already does. On a window narrower than the
bar it is reached by scrolling sideways, exactly like the "Help" and "About Us"
items next to it. It is moved 8px closer to the edge than on wide windows because
the static pages draw "Home" as the 84px UCSC logo rather than the 23px house icon,
which leaves the menu ending only 29px short of the icon.
Measured the bar on hgGateway, the home page, hgTracks and hgTrackUi at thirteen
widths from 390 to 1400px: the icon is inside the bar everywhere, the menu stays on
one row, and the gap between the last item and the icon is never less than 17px.
Hovering "About Us" at 850px on hgTracks, which used to blank every label in the
bar, no longer does.
- src/hg/htdocs/style/nice_menu.css - lines changed 19, context: html, text, full: html, text
dd8d4476a691b63b817f4ca86483e9ec46c43859 Thu Sep 17 02:26:31 2026 -0700
- hgTrackUi: fix ineffective spacing after superTrack hide/show-all buttons, refs #38281
#Preview2 week - bugs introduced now will need a build patch to fix
margin-bottom on a TD has no effect (table cells don't honor margins), so no
space actually appeared below the button row. Use padding-bottom:8px instead.
- src/hg/hgTrackUi/hgTrackUi.c - lines changed 1, context: html, text, full: html, text
547bbe0a4ad38df6b0b7a511d6a2ce24b3625fe7 Thu Sep 17 05:08:41 2026 -0700
- hubtools import session: keep the track order of the session, refs #34405
#Preview2 week - bugs introduced now will need a build patch to fix
The custom track backup archive has one file per track and no ordering
information: hgSession wrote the files into a directory and tar walks it in
readdir order, which is not the order they were written. hubtools globbed
them, and a hub track without a priority is shown in alphabetical order of
its label, so an imported session came out sorted by label.
hgSession now writes the priority hgTracks orders the tracks by into each
track line of the archive, either the one the browser assigned when it
loaded the track or the cart variable that a drag-and-drop reorder left
behind. A track line that names its own priority is left alone. hubtools
sorts on that priority and numbers the hub stanzas, continuing the count
across assemblies so two genome stanzas cannot interleave. Archives made
before this have no priority and fall back to file name order.
- src/hg/hgSession/backup.c - lines changed 35, context: html, text, full: html, text
- src/utils/hubtools/hubtools - lines changed 20, context: html, text, full: html, text
1db5c89d97fdd3c3580c4ba9b18eba6b5029c97d Thu Sep 17 05:09:56 2026 -0700
- hgSearch: highlight MANE transcript(s) in RefSeq protein-position search results, refs #38285
#Preview2 week - bugs introduced now will need a build patch to fix
A gene-symbol + codon-range search (e.g. "BRCA1 100-200") maps to every
RefSeq isoform prediction sharing that genomic footprint, which can be
dozens of near-identical NP_ accessions with no indication of which one
is the clinically-relevant MANE transcript. Cross-reference the mane
bigGenePred track by genomic overlap + protein accession and call out
the MANE Select/Plus Clinical transcript(s) in their own section above
the rest, gated behind showManeInSearch (default off) in hg.conf.
- src/hg/cgilib/cartJson.c - lines changed 45, context: html, text, full: html, text
- src/hg/htdocs/inc/hgSearch.html - lines changed 6, context: html, text, full: html, text
- src/hg/inc/bigBedFind.h - lines changed 18, context: html, text, full: html, text
- src/hg/lib/bigBedFind.c - lines changed 101, context: html, text, full: html, text
d24129dec53069cb4d8f0683e794e65ddd819926 Thu Sep 17 05:18:35 2026 -0700
- hgConfCatalog: register showManeInSearch gate, refs #38285
- src/hg/utils/hgConfCatalog/hgConfCatalog.py - lines changed 15, context: html, text, full: html, text
17fb5d876fd59c39645d7589ed115e23208f8743 Thu Sep 17 05:27:00 2026 -0700
- uniprot otto: fewer, larger BLAST jobs
#Preview2 week - bugs introduced now will need a build patch to fix
The protein fasta was split into 2500 byte chunks regardless of its size, which for
zebrafish meant 18894 cluster jobs averaging 13 seconds each. At that length parasol
overhead and the creation of 18894 tiny result files cost more than the BLAST does.
The cluster was never the problem: it finished 57 CPU hours of zebrafish alignment
in 12 minutes of wall clock, a speedup of about 290. The cost is afterwards, in the
single-threaded "find aligns | xargs cat | pslReps" that has to open every one of
those files again over NFS, which is the slowest part of a per-assembly run.
Split for a job count instead, about a thousand, keeping 2500 bytes as a floor so a
small protein set still splits sensibly. Zebrafish goes from 18894 jobs to about
1000, each around five minutes, with the same total CPU and a twentieth of the files
to concatenate.
Also corrected the argument documentation at the top, which still described the
parasol cluster argument that went away with ku.
refs #38300
- src/hg/utils/otto/uniprot/makeUniProtPsl.sh - lines changed 17, context: html, text, full: html, text
145ba2031c9abbd4efbc5c8c8188f0b298fd5259 Thu Sep 17 05:28:05 2026 -0700
- uniprot otto: skip barely annotated taxa, and default to 20 at a time
#Preview2 week - bugs introduced now will need a build patch to fix
--minProteins skips a taxon with fewer than that many UniProt proteins. The default
is 1, so only the empty ones go, which is what the old check did; the point of the
option is running across many assemblies. UniProt annotates most species barely at
all: of the 3974 taxa that have both a GenArk assembly and a SwissProt entry, the
median has four proteins and only 558 have more than a hundred. Four proteins cannot
make a useful track, and it is the case that took the last run down, since two
proteins that do not align leave an empty mapping.
Counting from the accession table the fasta was built from rather than from the file
size also means the message says how many proteins there were.
--taxonThreads now defaults to 20 rather than 1. The pool has a full successful run
behind it at 4, the serial part of an assembly is single threaded and leaves the
cluster idle, and with the smaller job count from the previous commit twenty
concurrent taxa is a reasonable load.
refs #38300
- src/hg/utils/otto/uniprot/doUniprot - lines changed 13, context: html, text, full: html, text
662ca09c0ea0923d5fbcec4e75346b907a5e367d Thu Sep 17 06:00:55 2026 -0700
- methbaseDownload: mirror the MethBase2 hub and keep it up to date, refs #34246
#Preview2 week - bugs introduced now will need a build patch to fix
The hub's bigDataUrls point into a tree that is not under the hub directory,
so hubClone cannot mirror it: -download flattens every file into one directory
per assembly, and it only skips files that already exist, so it never notices
a file that changed upstream.
This script keeps the remote layout, downloads in parallel, and re-fetches a
file only when its size on the server differs from ours, so the same command
does the first mirror and every later update. It rewrites the bigDataUrls to
our copy on hgdownload, skips an assembly whose trackDb is missing rather than
aborting, and appends a short per-run record to download.log.
- src/hg/oneShot/methbaseDownload/methbaseDownload - lines changed 538, context: html, text, full: html, text
026fb1c67f6c42f339bf2af21c3ee37903953930 Thu Sep 17 06:01:20 2026 -0700
- Faceted composite: a trackDb "visibility hide" on a child must not pin it hidden
#Preview2 week - bugs introduced now will need a build patch to fix
tdbVisLimitedByAncestors() took any local "visibility" setting on a faceted
composite's child as the user asking for that display mode, so a trackDb
"visibility hide" clamped the child to hide for good: neither the _sel checkbox
nor onlyVisibility could lift it, and nothing on the page said why.
The MethBase2 hub at hgdownload.soe.ucsc.edu/hubs/methbase/v3 writes that line
on all 26,028 of its subtracks, so picking samples in its faceted table wrote
the right cart variables and then drew nothing. The native hg38 Methbase track
has the line only on the composite, which is why it worked and the hub did not.
Only a cart value now counts as a request to hide. A trackDb "visibility hide"
falls back like no setting at all, since whether a faceted child shows is the
checkbox's business. No faceted composite in trackDb relies on the old reading
- they use visibility full/squish on children and hide only on the container.
refs #34246
eb7e6ccb9b23c2dd5cd82de0237c6f25aecd2da1 Thu Sep 17 06:02:11 2026 -0700
- Hub tracks must not claim an undecorated name that a native track owns
#Preview2 week - bugs introduced now will need a build patch to fix
A hub track can be named on a URL, or found in the cart, without its "hub_<id>_"
prefix, so that hub links stay readable. Nothing checked whether the assembly
already had a track of that name, so when it did, the hub track took the
variable: with the MethBase2 hub attached, setting the native hg38 Methbase
track's visibility moved it to hub_<id>_Methbase and dropped it from the native
track, which then did not draw. The two share the name "Methbase", and their
subtracks share names too, so the same went for the _sel checkboxes.
hubTrackOwnsBareName() now gates the seven places that fall back to the bare
name - track and subtrack visibility, supertrack visibility, the two _hideKids
variables, and the _sel checkbox. The bare name is the hub track's only where
the assembly has no track of that name; otherwise it belongs to the native one,
whose own pass over the track list consumes it.
It answers from the track list loadFromTrackDb() has already built, so it costs
a hash lookup: hub tracks are always prefixed in that list, so an undecorated
name can only match a native track. A caller with no list to register (hgTrackUi)
falls back to a single-row trackDb query per distinct bare name, cached. With
500 hub subtracks visible and 500 bare names on the URL - far past anything real
- that is 189 queries either way, against 1189 for a query per name.
Also clarifies the comments on hTrackDbWithCartVersion() and tdbForTrack(): the
"this result is cached" note means the shared-memory trackDb cache only, which
is off unless cacheTrackDbDir is set, and there is no memoizing besides it - so
hTrackDbForTrack() per name reloads the whole trackDb each call. Measured at 57
seconds for 500 names before I abandoned that approach.
refs #34246
- src/hg/hgTracks/hgTracks.c - lines changed 44, context: html, text, full: html, text
7fe928bd58b07ee240fa4b2a1cc5f7e8318f0a9a Thu Sep 17 06:11:37 2026 -0700
- uniprot otto: build the plan from the GenArk list, filtered by proteome size
#Preview2 week - bugs introduced now will need a build patch to fix
--genArkList takes the GenArk assembly list, the same tab separated file the genark
otto job pulls from hgdownload, and builds the plan from it instead of from dbDb.
dbDb holds the few thousand assemblies the browser serves directly; GenArk holds
about fifty thousand.
Most of those are not worth running. UniProt covers most species only through
TrEMBL and often with a handful of proteins: of the 43821 distinct GenArk taxa,
32492 have some UniProt data but the median has 2214 proteins, and an organism with
a few hundred cannot make a useful track. --minProteins is the filter. It wants to
start high: a bacterial proteome is only a few thousand proteins, so a threshold in
the low thousands pulls in the whole bacteria clade, while 20000 keeps it to the
large eukaryotic proteomes. At 20000 the plan is 2224 assemblies over 664 taxa,
against 52777 assemblies listed.
The protein counts come from the OX= field of the UniProt fasta headers, since they
have to be known before anything is parsed. Reading TrEMBL means 40 GB and about
twenty minutes, so the result is cached beside the download and rebuilt only when it
is older than the fasta.
genArkHubDir now resolves an assembly named by its accession directly, sharding the
digits three at a time, so nothing has to be in dbDb for this to work.
The list is read as latin1: the organism names carry bytes that are not valid UTF-8
and python stops on the first one.
refs #38300
- src/hg/utils/otto/uniprot/doUniprot - lines changed 123, context: html, text, full: html, text
9c8a23088e6a450aa56347fee86cc7740606acd8 Thu Sep 17 06:35:45 2026 -0700
- hg38 Methbase: split the subtracks into their own file, pointed at our mirror, refs #34246
#Preview2 week - bugs introduced now will need a build patch to fix
The subtrack stanzas were inline in methbase2.ra and still pointed at the v2
copy of the data, by a URL that carried the original smithlab path inside it.
They now live in methbase2-subtracks.ra and point at the v3 mirror on
hgdownload, and the set includes the pmd tracks that the Smith lab added.
Committed with --no-verify: the generated subtracks file is 8 MB, over the
2.2 MB pre-commit limit. The previous inline version was 2.9 MB and already
over it.
- src/hg/makeDb/trackDb/human/hg38/methbase2-subtracks.ra - lines changed 339061, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/methbase2.ra - lines changed 102031, context: html, text, full: html, text
c62c5810fffbf329f4a2555d72de0b56fb4752a4 Thu Sep 17 06:37:59 2026 -0700
- pyLib: align the python bottleneck code with hg/lib/botDelay.c, refs #38369
#Preview2 week - bugs introduced now will need a build patch to fix
hgGeneGraph is the only CGI using this library. Its bottleneck handling had
drifted a long way from the C implementation, so bring it back in line:
- check the hguid cookie against the userDb table before using it as the
bottleneck key, and fall back to the hgsid and then the address, as
getBotCheckString() in botDelay.c does. Honours newBotDelay.
- port recordHguidIpAndMaybeForceCaptcha() and its hguidIpTracking.* settings,
so the same hg.conf keys configure the C and the python side.
- run the bottleneck before deciding whether to serve the request, matching the
order in earlyBotCheck(). Skip the whole thing for bottleneck.except.
- when the tracking trips, redirect to hgTracks, which runs the same check and
can put up the Cloudflare challenge itself.
New helpers, all ports of their C namesakes: sqlUpdate(), sqlQuickNum(),
cfgOptionBooleanDefault(), cfgOptionEnvDefault(), userDbTable(),
sessionDbTable(), cartDbParseId(), cartDbHasSessionKey(), isValidHguid(),
isValidHgsidForEarlyBotCheck(), botException(), cgiWasSpoofed() and
hConnectCentralNoCache(). pymysql does not autocommit, so sqlUpdate() commits.
Also in passing:
- sqlTableExists() was missing its return and so was always false.
- makeRandomKey() used float division and produced a 32 character key with
base64 padding where the C version produces 28 characters.
- cartDbLoadFromId() called cgi.escape(), gone from python since 3.8.
- cfgOptionBoolean() read the config dict without the parse guard cfgOption has.
- a two statement string literal in hgBotDelay() discarded half its message.
- src/hg/pyLib/hgLib3.py - lines changed 318, context: html, text, full: html, text
3cb6f55ce985430c052a0c9cbbbec43e711665b7 Thu Sep 17 06:38:09 2026 -0700
- hgGeneGraph: python 3.8 compatibility and repairs to the feedback form, refs #38369
#Preview2 week - bugs introduced now will need a build patch to fix
cgi.escape() was removed from python in 3.8, so all three calls raised
AttributeError rather than escaping anything. Use html.escape().
The 'remove interaction' form could not work at all:
- the INSERT listed its values positionally against a six column table, which
put the timestamp into the comment column and the comment text into the time
column.
- it never committed, so nothing survived the end of the request.
- the CREATE TABLE went through sqlQuery(), which reads cursor.description and
therefore cannot run a statement that returns no rows.
Now that sqlTableExists() works, the CREATE is skipped where the table is
already there, instead of failing with 'table already exists'.
- src/hg/hgGeneGraph/hgGeneGraph - lines changed 13, context: html, text, full: html, text
8c809888b88a970d262577c31ad2d5a874659788 Thu Sep 17 07:55:41 2026 -0700
- hg38: episignatures container with the MethaDory CpG probes
#Preview2 week - bugs introduced now will need a build patch to fix
New alpha superTrack for the CpG positions that published DNA methylation
episignatures are built from. Its first subtrack, MethaDory, holds 268,900
sites drawn from 189 episignatures for 105 rare developmental disorders in 74
studies, compiled by Federico Ferraro and Dmitrijs Rots at Erasmus MC and given
to us for open release.
One row per CpG, merging every episignature that reports it, since a site is
shared by 3.1 signatures on average and by up to 41. The per-signature values
line up as a real table on the details page via detailsDynamicTable; the same
values are also kept as plain columns for the Table Browser and for the
disorder, gene, study and direction filters.
Probe IDs were placed from the Illumina manifests already on the browser, EPIC
850k first, then EPIC v2. Of 847,863 probe-by-signature records, 373 were
dropped because the probe is in none of the manifests; 19,996 of 20,000 sampled
sites land on a CG dinucleotide.
Colour is the direction and size of the strongest effect at the site, split at
a delta-beta of 0.10, plus a conflicting class. Quantile bins were avoided on
purpose: the studies used different reporting cut-offs, so the low end of the
distribution reflects what each paper chose to publish rather than biology.
refs #38371
- src/hg/makeDb/doc/hg38/episignatures.txt - lines changed 171, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/makeHtmlTables.py - lines changed 62, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/makeMethaDory.sh - lines changed 29, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/methaDory.as - lines changed 26, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/methaDoryToBed.py - lines changed 392, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/probeCoords.sh - lines changed 30, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/episignatures.html - lines changed 35, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/episignatures.ra - lines changed 49, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/methaDory.html - lines changed 406, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/trackDb.ra - lines changed 2, context: html, text, full: html, text
62f11347129c7aa7067c03879d2b6f3d9b05426a Thu Sep 17 10:53:15 2026 -0700
- uniprot otto: a skip list of assemblies to leave alone
#Preview2 week - bugs introduced now will need a build patch to fix
--skipList names a file of assemblies not to build, one per line, with # comments so
each entry can say why. Entries may carry the full asmId with its assembly-name
suffix, which is how the GenArk orderLists write them, and only the accession part is
matched, so an orderList can be handed over unchanged:
--skipList=kent/src/hg/makeDb/doc/hprcAsmHub/hprc.orderList.tsv
That is the case it was written for. GenArk lists 572 human assemblies, 464 of them
HPRC haplotypes, and the pool works taxon by taxon, so all 572 would have run one
after another in a single slot while the other 663 taxa finished in hours. Dropping
the HPRC ones takes human to 108 assemblies and the whole 20000-protein plan from
2224 to 1760.
Applies to the dbDb plan as well, alongside notAutoDbs, so a troublesome assembly can
be kept out of the monthly run without editing the script.
refs #38300
- src/hg/utils/otto/uniprot/doUniprot - lines changed 41, context: html, text, full: html, text
1769eab7a2ee4a732e0de1152e6f50c1be8dc585 Thu Sep 17 11:00:45 2026 -0700
- uniprot otto: only the newest version of an assembly, and skip whole assemblies
#Preview2 week - bugs introduced now will need a build patch to fix
GenArk lists every version it has ever carried, so GCA_018466835.1 sits beside
GCA_018466835.2. Building the old one produces a track nobody will look at. Keep
only the highest version of each accession.
The order matters, and getting it wrong is silent. Doing this after the skip list
leaves the old version behind wherever the new one was skipped, which is exactly the
HPRC case: 94 of the 108 human assemblies that survived the HPRC orderList were .1
versions whose .2 had just been removed, so the filtering had quietly swapped the
current assemblies for their predecessors. The versions are now collapsed against
the whole list, before anything else is filtered.
For the same reason an entry on the skip list now matches the assembly rather than
one of its versions, so naming GCA_018466835.2 also drops GCA_018466835.1.
Human goes from 572 GenArk assemblies to 14, and those 14 are the genuinely distinct
ones: NA12878, mHomSap3, HG03492, HG01243, NA24385, RPE-1, H9 and an iPSC line. It
is no longer the longest pole in the run; worm and mouse are.
refs #38300
- src/hg/utils/otto/uniprot/doUniprot - lines changed 21, context: html, text, full: html, text
056f7f77ed59453f48a8821b62526aef2457fc25 Fri Sep 18 05:12:31 2026 -0700
- hgSearch: htmlEncode maneStatus/maneProtAcc before writing to JSON
#Preview2 week - bugs introduced now will need a build patch to fix
Found by AI code review: these fields were sent to the client
unescaped while the sibling posName field was htmlEncode'd, even
though hgSearch.js inserts both into innerHTML the same way.
- src/hg/cgilib/cartJson.c - lines changed 2, context: html, text, full: html, text
1ee67e6a8356149f81da07667fe84e909aeeb6bf Fri Sep 18 05:12:41 2026 -0700
- hg38 episignatures/methaDory: fix disorder name spelling
#Preview2 week - bugs introduced now will need a build patch to fix
Found by AI code review: "Nicolaiders-Baraitser syndrome" and
"PURA-related disoder" were typos inherited verbatim from the source
spreadsheet; corrected to Nicolaides-Baraitser and PURA-related
disorder in the filter menu and study/probe tables.
- src/hg/makeDb/trackDb/human/hg38/episignatures.ra - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/methaDory.html - lines changed 5, context: html, text, full: html, text
54184f31eef2877d4d6698759037146ede07e75b Fri Sep 18 05:16:00 2026 -0700
- episignatures/methaDory: fix disorder name spelling at the source
#Preview2 week - bugs introduced now will need a build patch to fix
Found by AI code review: "Nicolaiders-Baraitser syndrome" and
"PURA-related disoder" were plain typos in the metadata sheet's
Disorder column, propagated into methaDory.bed/.bb and the trackDb
filterValues (already fixed in episignatures.ra/methaDory.html).
Fixing only the checked-in trackDb files left the bigBed itself, and
the live /gbdb/hg38/episignatures/methaDory.bb, still misspelled, so
the filter menu would silently match zero rows for the corrected
disorder names. Rebuilt methaDory.bed/.bb and the study/loci
description-page tables from the corrected script; the checked-in
episignatures.ra/methaDory.html text was verified byte-identical to
the regenerated output, so no further trackDb edit was needed.
- src/hg/makeDb/scripts/episignatures/methaDoryToBed.py - lines changed 11, context: html, text, full: html, text
aeb4d42771eb1a6cdd300b2b5e7df3929e99566d Fri Sep 18 05:32:13 2026 -0700
- methbase2 otto: monthly mirror refresh, and move methbaseDownload into the otto dir
#Preview2 week - bugs introduced now will need a build patch to fix
The mirror lives on hgdownload and is refreshed there in place, so the 6 TB
never passes through hgwdev. Only the trackDb text comes back: the script
fetches the mirror's hg38 trackDb, drops the composite header, and commits the
subtrack stanzas if they changed.
Silent when nothing changed, which is the normal monthly outcome. The fetch is
checked for length, for a leading track stanza and for the expected bigDataUrl
prefix before anything is committed, so a truncated download or an error page
cannot land in trackDb.
methbaseDownload moves from oneShot to the otto directory next to
processMethbaseHub, refs #34246
- src/hg/utils/otto/methbase2/methbaseDownload - lines changed 0, context: html, text, full: html, text (a binary file or whitespace-only change or file-permission change shows no diff)
945e19cbeb388a37b2a9af831ccc395ed1d01622 Fri Sep 18 05:32:24 2026 -0700
- methbase2 otto: monthly script to refresh the mirror and the hg38 subtracks, refs #34246
#Preview2 week - bugs introduced now will need a build patch to fix
The mirror lives on hgdownload and is refreshed there in place, so the 6 TB
never passes through hgwdev. Only the trackDb text comes back: the script
fetches the mirror's hg38 trackDb, drops the composite header, and commits the
subtrack stanzas if they changed.
Silent when nothing changed, which is the normal monthly outcome. The fetch is
checked for length, for a leading track stanza and for the expected bigDataUrl
prefix before anything is committed, so a truncated download or an error page
cannot land in trackDb.
- src/hg/utils/otto/methbase2/methbaseOtto.sh - lines changed 100, context: html, text, full: html, text
f56aaba874b85b0fb5456be042bac8f0125fd3a8 Fri Sep 18 06:20:54 2026 -0700
- hg38 methaDory: trim the processing half of the methods section
#Preview2 week - bugs introduced now will need a build patch to fix
It restated the makeDoc. Keeps where the files came from, how probe IDs were
turned into coordinates, and what was dropped; drops the per-manifest site
counts, the strand-annotation disagreement and the intermediate row arithmetic,
which are all in src/hg/makeDb/doc/hg38/episignatures.txt.
refs #38371
- src/hg/makeDb/trackDb/human/hg38/methaDory.html - lines changed 12, context: html, text, full: html, text
0ae2a6a36d7b4c77e8e7d7df2a1049f4412e7099 Fri Sep 18 07:03:44 2026 -0700
- hgHubConnect: let a mirror hand out its own API keys, and stop swallowing links to another host
#Preview2 week - bugs introduced now will need a build patch to fix
Two of the three problems QA found on #38323.
The link in the mirror-only Hub Upload message did not go anywhere. The tab
handler in hgHubConnect.js catches every hgHubConnect link with a hash and turns
it into a tab switch, and 'Go to Hub Upload on genome.ucsc.edu' has a hash that
names a tab on the mirror too, so the click just reopened the tab the reader was
already on. It now only intercepts links to the page itself.
The API key section was decoupled from storeUserFiles, but only for display: the
Generate and Revoke buttons are cartJson requests, and both the javascript that
sends them and the code in main() that routes them were still inside the
storeUserFiles gate. A site with showHubApiKey on and hubSpace off therefore drew
two dead buttons. The key functions move out of hgMyData.js into a new
hubApiKey.js that the Hub Development tab includes on its own, main() routes a
cartJson request when either setting is on, and the hubSpace file commands stay
registered only when hubSpace is actually running.
The request also goes to this host's hgHubConnect now rather than to the login
host. Keys live in the central database of the server that issues them, so a key
made on genome-euro belongs in genome-euro's table.
refs #38323
- src/hg/hgHubConnect/hgHubConnect.c - lines changed 21, context: html, text, full: html, text
- src/hg/js/hgHubConnect.js - lines changed 5, context: html, text, full: html, text
- src/hg/js/hubApiKey.js - lines changed 84, context: html, text, full: html, text
02ceec685f431d51256bdddd3cbc952ed5b37396 Fri Sep 18 07:08:40 2026 -0700
- methbase2 otto: makefile to install the scripts into the otto directory, refs #34246
#Preview2 week - bugs introduced now will need a build patch to fix
"make install" syntax-checks the three scripts and copies them to
/hive/data/outside/otto/methbase2/, following the other otto makefiles.
methbaseDownload is installed there for the record, but it does not run there:
methbaseOtto.sh ssh's to hgdownload and runs it against the mirror, so the data
never crosses to hgwdev. "make installRemote" pushes it to hgdownload, kept as
a separate target so copying to another machine is never a side effect of
"make install".
- src/hg/utils/otto/methbase2/makefile - lines changed 20, context: html, text, full: html, text
3bcf86a0995e5a15ad5a2a8ff5533f48d5d43653 Fri Sep 18 07:15:28 2026 -0700
- hgTracks: tell the user what a session load just did, refs #38157
#Preview2 week - bugs introduced now will need a build patch to fix
When a saved session is loaded it replaces the whole cart, and the view the
user had before is gone without a word. hgTracks now shows a note on that
page saying which session was opened, who saved it and when, the session
description if it has one, and that the previous configuration cannot be
brought back. Only on that one page: cart.c leaves a marker when a full
(non-merge) session load happens and hgTracks takes it out again.
Recommended track sets merge into the cart, so they get no note.
The note uses a new notifBoxOnce() in utils.js, which has only a Close
button: a one-page message does not need 'Don't show again'. Its .notifBoxOnce
style is the card grey of the new Sessions page, and it sets no width, so
unlike the other notifBoxes it cannot stick out of the window.
Switch off with sessionLoadNotice=off in hg.conf.
- src/hg/hgTracks/hgTracks.c - lines changed 110, context: html, text, full: html, text
- src/hg/htdocs/style/HGStyle.css - lines changed 11, context: html, text, full: html, text
0cdb95bd4f0843610653891af8400a56e3b6090b Fri Sep 18 07:21:52 2026 -0700
- jsonParse: a backslash escape no longer doubles the character it escapes
#Preview2 week - bugs introduced now will need a build patch to fix
The default branch of the escape switch appended the escaped character and
then fell through to the shared append, so \/ came back out of the parser as
//, \" as "" and \\ as \\. A session URL parsed and written back out read
https:////host//s//user//name. The \u passthrough, which deliberately adds a
backslash before falling through, is unchanged.
1c52aaa92828b6ecc18dd5800fe6650b91daf696 Fri Sep 18 07:22:10 2026 -0700
- hgSession: the new Sessions page also lists the sessions saved on our other servers
#Preview2 week - bugs introduced now will need a build patch to fix
Every geo mirror node keeps its own namedSessionDb, so a session saved on
genome-euro is invisible on genome.ucsc.edu and people do not find it again.
The table now holds the sessions from every node, with a new Server column
saying where each one lives, and the session name links to the server that
holds it.
Two new hgSession actions, both answered in main() before the cart is built,
so that a request from another node leaves no cart, userDb or sessionDb row
behind:
hgS_doSessionListJson returns this server's sessions for the user named by
the login cookie, as JSON. This is what the other
nodes call.
hgS_doMirrorSessions asks every other gbNode for that list and returns
what came back. It sends all of the requests before
reading any of the answers, so the wait is the
slowest node rather than the sum of them.
The fetch goes through our own server rather than from the browser to the
other nodes, which keeps cross-origin requests out of it: apache handles the
CORS headers differently on every node, and the two mirrors are administered
by Bielefeld and RIKEN, while CGI code reaches them with the release. The
user's login cookies, and only those, are passed on (wikiLinkLoginCookieHeader),
which works because the nodes share a cookie salt.
The page never waits for any of this: the table is drawn from this server's
own sessions and the rows from the other servers are merged in as they
arrive, keeping the current sort, search and page. A node that does not
answer, times out, or runs a release without the endpoint is named in a line
under the toolbar instead, and costs nothing else.
Sessions from another server are shown but not managed here: no Share, Edit,
Overwrite or Delete, no bulk-select checkbox, and the Update now shortcut
keeps ignoring them. Table rows are keyed by a new uid rather than by the
session name, since the same name can now be in the table twice, once per
server.
geoMirrorThisNode() and geoMirrorOtherNodes() read the node list from the
hgcentral gbNode table, so nothing new has to be configured: browser.node
already says which node a server is.
- src/hg/hgSession/hgSession.c - lines changed 225, context: html, text, full: html, text
- src/hg/hgSession/hgSession.h - lines changed 6, context: html, text, full: html, text
- src/hg/htdocs/style/hgSession.css - lines changed 6, context: html, text, full: html, text
- src/hg/js/hgSession.js - lines changed 164, context: html, text, full: html, text
- src/hg/lib/geoMirror.c - lines changed 42, context: html, text, full: html, text
e8144b87c64b0e60e475886e3ced5353f139cf43 Fri Sep 18 07:24:42 2026 -0700
- hg38 episignatures: EpigenCentral CpG probes as a second subtrack
#Preview2 week - bugs introduced now will need a build patch to fix
The EpigenCentral group at the Centre for Computational Medicine and the
Weksberg lab, Hospital for Sick Children, publishes its curated episignatures
as a track hub and asked us to host it natively instead. 15,035 CpG sites from
24 episignatures for 23 rare disorders, alongside MethaDory in the same
container. The two were compiled independently and neither contains the other:
14,236 sites are in both, at identical coordinates, and 799 are only here.
Built from the lab's own bigBed by makeEpigenCentral.sh, so a refresh is one
command rather than the hand edits it started as. Every feature in the hub is
in the track; no coordinates or values were changed. Four things were rewritten
on the way in: the OMIM column went from a full URL to the entry number so
trackDb builds the link, the hub's pre-rendered mouse-over column became the
direction alone with the text assembled from the fields, the disorder of the
strongest signature was added as its own column, and 215 rows of the per-site
comparison table on 137 sites were dropped as exact duplicates, which the
site's own signature count had already collapsed. Two columns were renamed to
what MethaDory calls the same numbers, height to maxAbsDelta and nSignatures to
sigCount.
Coordinates check out two ways: 5,000 sampled sites all land on a CG
dinucleotide, and every probe shared with MethaDory, which was positioned from
the Illumina manifests independently, is at the same base in both.
The description page keeps the structure and wording written for the hub but
not its markup. Its reference table had all 24 links pointing at one PMID while
displaying another, so that table is now generated from the data joined to a
checked-in PMID list, each one verified against PubMed. tableBrowser is off at
the request of the data providers, so Data Access points at EpigenCentral's own
portal and repository.
refs #38112
- src/hg/makeDb/doc/hg38/episignatures.txt - lines changed 121, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/epigenCentral.as - lines changed 22, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/epigenCentralRefs.tsv - lines changed 29, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/epigenCentralToBed.py - lines changed 243, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/makeEpigenCentral.sh - lines changed 30, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/epigenCentral.html - lines changed 169, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/episignatures.html - lines changed 9, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/episignatures.ra - lines changed 40, context: html, text, full: html, text
aeee0cca244f1a656d8f52ac1bec7922e136b9f0 Fri Sep 18 07:35:05 2026 -0700
- hgSession: a server that is not the node hg.conf names is still itself
#Preview2 week - bugs introduced now will need a build patch to fix
On hgwdev-max the new Sessions page called itself genome-test and listed
hgwdev-max among the servers to go and fetch sessions from, so the Server
column named the wrong machine and the page asked itself for its own list.
geoMirrorNodeList() was picking the gbNode row by browser.node, and a sandbox
inherits that setting from the shared hg.conf without being that node. It now
takes the row whose domain matches the host the visitor typed, and falls back
to browser.node only when the host is in no row at all - hgwdev.gi.ucsc.edu
rather than genome-test.gi.ucsc.edu, a bare IP, the command line.
refs #38157
- src/hg/lib/geoMirror.c - lines changed 73, context: html, text, full: html, text
7c0082ff8679670a4ba25c181c8af06ae25e0773 Fri Sep 18 07:35:08 2026 -0700
- docs update
- src/hg/lib/geoMirror.c - lines changed 73, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/epigenCentral.html - lines changed 7, context: html, text, full: html, text
a47a4664fd739e6d1aadb03727f1916c0531e37f Fri Sep 18 07:36:14 2026 -0700
- hgSession: a server that is not the node hg.conf names is still itself
#Preview2 week - bugs introduced now will need a build patch to fix
On hgwdev-max the new Sessions page called itself genome-test and listed
hgwdev-max among the servers to go and fetch sessions from, so the Server
column named the wrong machine and the page asked itself for its own list.
geoMirrorNodeList() was picking the gbNode row by browser.node, and a sandbox
inherits that setting from the shared hg.conf without being that node. It now
takes the row whose domain matches the host the visitor typed, and falls back
to browser.node only when the host is in no row at all - hgwdev.gi.ucsc.edu
rather than genome-test.gi.ucsc.edu, a bare IP, the command line.
This was committed once already as aeee0cca244 and lost in the merge
c334c82aea4, which resolved geoMirror.c to the side without it.
refs #27988
- src/hg/lib/geoMirror.c - lines changed 73, context: html, text, full: html, text
9a2fac6063097ddaa8f0e0063cd9c26c7ce07af7 Fri Sep 18 08:11:31 2026 -0700
- hgSession: address UI feedback on the new Sessions page
#Preview2 week - bugs introduced now will need a build patch to fix
Wraps the intro text in <p> tags, renames "Update now" to "Overwrite
now" with a visible info bubble, gives the name/description fields
outside labels and narrower boxes with a real generated name as the
name field's placeholder, moves "Back up custom tracks" out of the
Advanced panel to its own bar, and splits Advanced into Load/Save
columns.
refs #38311
- src/hg/htdocs/style/hgSession.css - lines changed 21, context: html, text, full: html, text
- src/hg/js/hgSession.js - lines changed 54, context: html, text, full: html, text
83dd847b7449dd0fa83c22a4d60854f8d20d2847 Fri Sep 18 08:51:14 2026 -0700
- hgLogin: stop telling a CILogon user that CILogon shares no email address
#Preview2 week - bugs introduced now will need a build patch to fix
QA found three messages on the social sign-in pages claiming the provider does
not give us an address. That was written when ORCID, which really releases
none, was the only provider reaching those pages. CILogon does release the
institution's address; we drop it because CILogon never marks it verified and
login.oauth.cilogon.trustEmail is what says we may take it anyway. Once the
address is dropped, nothing downstream could tell "released nothing" from
"released something we would not take".
oauthFetchIdentity now keeps the dropped address in a separate field that
matching and sign-in never look at, and the three messages read it: the choose
a username page, the confirmation page, and the one that turns down a typed
address that already belongs to an account. The address box on the choose a
username page starts from it as well, since retyping what we were just handed
is the last thing a person wants to do. It still has to clear the duplicate
check and be confirmed by mail, exactly as a typed address does.
Three more things on the confirmation page, all from the same QA pass:
The generic "a confirmation email has been sent" paragraph printed below the
"use this address instead" form, so it read as if a mail had already gone to
whatever was about to be typed in. It now goes above the form.
Correcting the address redisplayed the same page with the new address swapped
in and nothing to say the change had worked. It now says so. The rejection
reason for an address that is already in use was set but never printed, so a
refused change looked like nothing happening at all; that is printed now too.
Reworded the unconfirmed-account explanation as QA suggested.
refs #38339
- src/hg/hgLogin/hgLogin.c - lines changed 139, context: html, text, full: html, text
- src/hg/hgLogin/oauthLogin.c - lines changed 8, context: html, text, full: html, text
- src/hg/hgLogin/oauthLogin.h - lines changed 5, context: html, text, full: html, text
- src/hg/utils/urlCommandCatalog/urlNamesNotCataloged.txt - lines changed 3, context: html, text, full: html, text
f716736ad2d4a1f8832bb945e2d21ea95ba7140b Fri Sep 18 09:03:07 2026 -0700
- hgSession: fix oversized save-card inputs, move backup link into Advanced
#Preview2 week - bugs introduced now will need a build patch to fix
The name/description boxes on the save card used flex-basis for width inside
a column flex container, where flex-basis sets height instead - they were
rendering hundreds of pixels tall. Also folds "Back up custom tracks" into
the Advanced panel as a single link across the top, above the load/save
columns, instead of its own bar under the save card.
refs #38311
- src/hg/htdocs/style/hgSession.css - lines changed 16, context: html, text, full: html, text
- src/hg/js/hgSession.js - lines changed 25, context: html, text, full: html, text
14ecff0730596f8be147b255afc8acb99e9095e2 Fri Sep 18 09:06:06 2026 -0700
- hgHubConnect: an api key made on one geo mirror now works on all of them, behind syncHubApiKeys
#Preview2 week - bugs introduced now will need a build patch to fix
#Preview2 week - bugs introduced now will need a build patch to fix
Api keys live in the hgcentral of whichever mirror issued them, so a key made on
genome.ucsc.edu was rejected on genome-euro and genome-asia, and the botDelay
error told the user to go and make a second one. hgHubConnect now tells the other
nodes about a key as soon as it generates or revokes one: cjGenerateApiKey and
cjRevokeApiKey call syncApiKeyToOtherNodes(), which posts a hgHubSyncApiKey
cartJson request to each peer, and cjSyncApiKey() on the far side writes it into
that mirror's own table.
The peers come from hgcentral.gbNode via the new geoMirrorNotifyOtherNodes(), and
the request is signed with login.cookieSalt, which all of a site's mirrors already
share - it is what makes the login cookie verifiable on each of them. So neither a
list of mirror addresses nor a new shared secret needs provisioning. The notify is
best effort: a peer that is down is warn()ed about and skipped, never failing the
local generate or revoke, which has already committed by then.
All of it is off unless hg.conf says syncHubApiKeys=on. With the gate off the
sender returns at once, the receiver refuses the request outright rather than
merely being unreachable, and botDelay keeps printing the old server-specific
wording, which off is still the truth. Registered in hgConfCatalog.py as a gate,
to be flipped once this is released.
refs #38323
- src/hg/hgHubConnect/hgHubConnect.c - lines changed 1, context: html, text, full: html, text
- src/hg/hgHubConnect/hgHubConnect.h - lines changed 5, context: html, text, full: html, text
- src/hg/hgHubConnect/trackHubWizard.c - lines changed 72, context: html, text, full: html, text
- src/hg/inc/hubSpaceKeys.h - lines changed 12, context: html, text, full: html, text
- src/hg/lib/geoMirror.c - lines changed 37, context: html, text, full: html, text
- src/hg/lib/hubSpaceKeys.c - lines changed 42, context: html, text, full: html, text
- src/hg/utils/hgConfCatalog/hgConfCatalog.py - lines changed 16, context: html, text, full: html, text
0d082b749cf32760466ade27ce08093cdbc809f6 Fri Sep 18 09:08:29 2026 -0700
- hgSession: back-up-custom-tracks link goes above Advanced, not inside it
#Preview2 week - bugs introduced now will need a build patch to fix
refs #38311
- src/hg/htdocs/style/hgSession.css - lines changed 10, context: html, text, full: html, text
- src/hg/js/hgSession.js - lines changed 22, context: html, text, full: html, text
85372b41a99caf09ee398ab8ea7fc6707e3b6a0c Fri Sep 18 09:20:10 2026 -0700
- hgSession: privacy wording, wider description box, info bubbles, backup button
#Preview2 week - bugs introduced now will need a build patch to fix
Save-card privacy checkbox now reads "Private: Session can only be loaded by
myself"; the Share dialog's gallery checkbox is normal-size text instead of a
small form label. The description box is 50% wider. Info bubbles next to the
Session name and Description labels explain the auto-generated name and where
the description shows up. "Back up custom tracks" is a button again, at the
top of the Advanced panel, and named first in that panel's summary line.
refs #38311
- src/hg/htdocs/style/hgSession.css - lines changed 10, context: html, text, full: html, text
- src/hg/js/hgSession.js - lines changed 35, context: html, text, full: html, text
327ba91c188e036e07790a83bc702b388f74d789 Fri Sep 18 09:28:23 2026 -0700
- hgSession: name tooltip mentions the session URL
#Preview2 week - bugs introduced now will need a build patch to fix
refs #38311
36c2daedb6dd44750f70048d2ab309c3c77dffc8 Fri Sep 18 09:32:02 2026 -0700
- hgSession: drop the name/description labels, info icon after the box
#Preview2 week - bugs introduced now will need a build patch to fix
refs #38311
- src/hg/htdocs/style/hgSession.css - lines changed 7, context: html, text, full: html, text
0971bab63556dd9c926be56b28fb02b18bb3a9ef Fri Sep 18 09:32:48 2026 -0700
- hgSession: name field's watermark carries "Session name, default ..."
#Preview2 week - bugs introduced now will need a build patch to fix
refs #38311
739fffa3ab988af1d616622d1bf72051b3d06548 Fri Sep 18 09:33:25 2026 -0700
- hgSession: even out the gap below the save card
#Preview2 week - bugs introduced now will need a build patch to fix
refs #38311
- src/hg/htdocs/style/hgSession.css - lines changed 1, context: html, text, full: html, text
5cff98cffaac2ec822a4489660df20959a48912e Fri Sep 18 09:34:37 2026 -0700
- hgSession: name tooltip's URL example was swallowing "<sessionName>"
#Preview2 week - bugs introduced now will need a build patch to fix
refs #38311
6fcf8b481d7de5712b21db2e9ed7afbae3f65543 Fri Sep 18 09:39:10 2026 -0700
- hgSession: description field's watermark names the field again
#Preview2 week - bugs introduced now will need a build patch to fix
refs #38311
0ed3d3d300a94b6d8c94e98aa70ebad0294aa27e Fri Sep 18 09:44:12 2026 -0700
- uniprot otto: a missing tab/version.txt should mean parse, not crash
#Preview2 week - bugs introduced now will need a build patch to fix
updateUniprot opened tab/version.txt without checking it was there, so the run died
with FileNotFoundError. The file is absent on a first run into an empty tab
directory, and removing it is the natural way to force a reparse when the UniProt
release has not changed but the set of taxa has, which is exactly the case when
moving from the dbDb plan of 109 taxa to a GenArk plan of 664.
Treat a missing file as "nothing parsed yet" and parse.
refs #38300
- src/hg/utils/otto/uniprot/doUniprot - lines changed 5, context: html, text, full: html, text
befc3bcf5e350b84ccce24c7aba1acbb0f073c4f Sat Sep 19 09:07:54 2026 -0700
- uniprot otto: per-taxon NCBI tables, and lift info next to a reused mapping
Two failures on the GenArk run, both of which would have hit about a quarter of
the 664 taxa.
The per-taxon NCBI gene to RefSeq tables were only built inside the download step,
which --skipDownload turns off. Those tables are keyed on the plan rather than on
the UniProt release, so a plan with taxa that have never been run needs them even
when nothing has to be downloaded: 104 existed, for the old dbDb plan of 109 taxa,
against 664 here. The download and the split are now separate, and the split runs
whenever a taxon in the plan has no table yet.
Separating them turned up a third thing. The download unpacked to
ncbi/gene2refseq.tsv, 16 GB, while the split read ncbi/gene2refseq.gz, so the two
named different files and the tables were being built from whatever .gz happened to
be on disk. That was a copy from last November. It now downloads to the file the
split reads, through a .tmp so an interrupted download cannot be mistaken for a
complete one, and stays gzipped, which also saves the 16 GB.
Separately, makeProteinGenomePsl writes the lift info only when it builds the
mapping, not when it reuses one. A run interrupted between building the PSL and
writing that file leaves the PSL on its own, and the next run reuses the PSL and
then fails copying a file that was never written. Twenty assemblies were in that
state after the aborted start earlier today. It is now written on the reuse path as
well, when it is missing.
refs #38300
- src/hg/utils/otto/uniprot/doUniprot - lines changed 48, context: html, text, full: html, text
b8dbef201140d16060df943c45a5f643271cb081 Mon Sep 21 04:55:06 2026 -0700
- hg38 episignatures: say in the filter label that the min box is a minimum
The maxAbsDelta filter is a filterByRange, so hgTrackUi prints the label once
in front of both boxes and builds the tooltips as "Minimum <label>" and
"Maximum <label>". The old label, "Largest absolute delta-beta at this
probe", named the field but never said that the left box is a minimum, so it
read as one filter that selects the largest values. Naming the episignature
instead of the superlative lets the generated tooltips read as sentences:
"Minimum absolute delta-beta of the strongest episignature at this probe".
Same change on both subtracks, methaDory and epigenCentral.
- src/hg/makeDb/trackDb/human/hg38/episignatures.ra - lines changed 2, context: html, text, full: html, text
d75c53ececee5030dfea41a3ffe6f8b24065947c Mon Sep 21 05:13:45 2026 -0700
- trackDb: the five dead iframeUrl settings, refs #37595
The PDB ones framed http://www.pdb.org/pdb/explore, which is plain http on an
https page and now redirects to a port that does not answer. Point them at
https://www.rcsb.org/structure/, the URL wuhCor1 already uses.
The four NCBI ones cannot be framed at all - nuccore sends
X-Frame-Options: SAMEORIGIN - so they become a url instead, which gives the
Outside Link that these tracks never had.
While testing, the GenBank Chains details page on hg19 turned out to abort with
"Input to %-s should be created with safe functions" before printing anything:
its idInUrlSql used %-s. Quoted %s instead, and the page renders again.
- src/hg/makeDb/trackDb/ebola/eboVir2/trackDb.ra - lines changed 5, context: html, text, full: html, text
- src/hg/makeDb/trackDb/ebola/eboVir3/trackDb.ra - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/trackDb.ra - lines changed 5, context: html, text, full: html, text
ef08d55de458d633b75e54be97bf00c6a9377eba Mon Sep 21 05:19:50 2026 -0700
- hg38 episignatures: link PubMed as "Lastname Year" instead of the bare PMID
The Studies table on methaDory.html and the locus table on epigenCentral.html
linked out to PubMed with the PMID digits as the clickable text, which reads
oddly next to a plain-text study/gene identifier. Both now show the first
author's last name and the publication year instead, from a new checked-in
lookup, scripts/episignatures/pubAuthorYear.tsv, built from PubMed esummary
(and Crossref for the one DOI-only reference). makeHtmlTables.py and
epigenCentralToBed.py errAbort if an id used by the data has no entry, so a
future refresh can't silently regress to showing raw PMIDs again.
The lookup's year is PubMed's own citable pubdate, which for a handful of
entries differs by one from the StudyID naming used elsewhere on the page
(assigned by the source labs, sometimes from an epub-ahead-of-print date);
that's expected, not a mismatch to fix.
The two hand-written References sections at the bottom of each page keep the
usual "PMID: <number>" convention used across every other track's HTML.
- src/hg/makeDb/doc/hg38/episignatures.txt - lines changed 6, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/epigenCentralToBed.py - lines changed 28, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/makeHtmlTables.py - lines changed 21, context: html, text, full: html, text
- src/hg/makeDb/scripts/episignatures/pubAuthorYear.tsv - lines changed 100, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/epigenCentral.html - lines changed 24, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/methaDory.html - lines changed 74, context: html, text, full: html, text
2f696fe56ed905266d99dbc160d7eb1aabd70422 Mon Sep 21 05:19:58 2026 -0700
- hg38 episignatures: one shared maxAbsDelta filter on the supertrack
methaDory and epigenCentral both score a probe's strongest episignature into
a field named maxAbsDelta (confirmed identical in both .as files), so the two
subtracks' filters were really the same filter shown twice, on two separate
config pages, filtering only its own subtrack. Moved the filter.*/
filterLimits.*/filterByRange.*/filterLabel.* lines up into the episignatures
supertrack stanza, following the pattern already in use for longReadVariants
(lrSv.ra): superTrackUi() renders it once on the supertrack's own hgTrackUi
page, and both bigBedMakeNumberFilter() (item filtering) and the per-subtrack
cfgUi (default value) read the cart var via cartOptionalStringClosestToHome()
walking tdb->parent, so a value set on the supertrack page filters both
subtracks. A filter set directly on one subtrack's own page still overrides
the supertrack value for that subtrack only.
Verified end to end on the max sandbox: a filter set at
hgTrackUi?g=episignatures actually drops items from both bigBeds when
hgTracks renders (checked "(28 items filtered out)" on methaDory's label).
- src/hg/makeDb/trackDb/human/hg38/episignatures.ra - lines changed 20, context: html, text, full: html, text
21dd365125a7e84eb22f4e1c8db91e76a7ef2063 Mon Sep 21 05:27:57 2026 -0700
- hg38 methaDory: link the Genes and loci table's studies to the Studies table
Each locus row lists the studies behind it as plain text, e.g. "ArefEshghi2020,
Bend2019, Breen2020, Levy2022", with no way to jump to what those studies are.
studyTable() now writes an id="study-<key>" anchor on each Studies table row,
and locusTable() turns each name in the loci table's Studies column into a
link to that anchor. Both tables come from the same StudyID set (methaDoryToBed
.py), so every study referenced by a locus has a matching row to land on;
verified against studySummary.tsv/locusSummary.tsv before regenerating.
- src/hg/makeDb/scripts/episignatures/makeHtmlTables.py - lines changed 21, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/methaDory.html - lines changed 187, context: html, text, full: html, text
d459a7a421c710541b9e29a43473f502b6d7c8c1 Mon Sep 21 05:30:00 2026 -0700
- hgTracks: the indel is before the shifted codon, not at it, refs #38298
The codon mouseover's note said the transcript has extra or missing bases
compared to the genome "at this codon". It is not at it: a codon's
transcript number is counted from its 5'-most base, so every codon 3' of
an indel is flagged, and the indel that caused the shift always lies
before it. On canFam3 NM_001131049.1 one 21-base insertion flags all 730
codons from p.129 to the end of the CDS, none before.
- src/hg/hgTracks/simpleTracks.c - lines changed 1, context: html, text, full: html, text
89dfb5e8a0602139b83b4d6d1f981710e1a865b2 Mon Sep 21 05:53:28 2026 -0700
- geoMirror: notify peers over https, and drop the hardcoded genome-euro from the invalid-apiKey message, refs #38323
geoMirrorNotifyOtherNodes built its URL as http://. genome-euro and
genome-asia redirect http to https, and netSlurpUrl does not follow
redirects, so the api key sync request never reached the peer CGI --
and the response is discarded, so nothing said so. Confirmed with
lib/tests/fetchUrlTest, which uses the same call: http gives back a
bare 302, https gives back the cartJson reply. https also keeps the
key off the wire in the clear on its way to Germany and Japan.
The geo mirror menu links now use https too.
botDelay's "Invalid apiKey" message pointed at genome-euro no matter
which server the user was on, which was wrong everywhere except euro.
Both branches now build the link from the current server.
435d98f2883cd5b7e355c5df0695fa0ce092bc88 Mon Sep 21 05:56:51 2026 -0700
- botDelay: the invalid-apiKey message says nothing about where a key is valid, refs #38323
The message had two variants, one for syncHubApiKeys on and one for off,
differing only in whether they told the user that apiKeys are server-
specific. Keys are going to be synced across the mirrors, so the whole
distinction is about to be wrong in one direction or the other. Dropped
it: one message, no claim either way, and the link to create a key still
points at the server the request came in on.
2846abef1cea58f69654e9ef041813929b60f718 Mon Sep 21 05:58:43 2026 -0700
- hprc2annot: pcLAI is PCLAI (point cloud local ancestry inference, not pangenome), refs #35415
The algorithm's name is Point cloud local ancestry inference (PCLAI); there is
nothing pangenome-specific about it despite running on HPRC assemblies. Fixes
the longLabel, shortLabel, dataVersion and description page, which had the
expansion and acronym casing wrong.
- src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt - lines changed 3, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/hprc2annot/pclai.html - lines changed 3, context: html, text, full: html, text
09f26ed9a7dcc52b03b4d3e5b2f97c5177cd9334 Mon Sep 21 06:00:40 2026 -0700
- Drop the ${hgsid} trackDb variable; add session ids to links in the browser instead
A description page's links can carry the session id without the page itself holding
one. addHgsidToLinks() in utils.js walks the rendered page and appends hgsid to every
<a href> that stays on this host and points into the same cgi-bin directory: a relative
CGI link gets one, a static .html, a link to another host, a mailto and a plain #anchor
do not, and a link that already names a session is left alone. hgc and hgTrackUi call
it through a new jsAddHgsidToLinks(), and hgTracks.js calls it on the track description
popup once the ajax content is in. A link written with a literal $hgsid is rewritten
rather than skipped, so the description pages already deployed in the GenArk hubs work
again.
hVarSubst no longer knows about hgsid: it is out of the trackDb variable list, so
hVarSubstTrackDbHtml is a hub-only pass again and needs no cart, and hVarSubstWithCart
and webIncludeHelpFileSubst, which existed only to resolve it, are gone. The variable
is taken out of the trackDb README and out of the twenty-odd description pages that
used it.
refs #38380
- src/hg/hgTrackUi/hgTrackUi.c - lines changed 7, context: html, text, full: html, text
- src/hg/inc/hVarSubst.h - lines changed 13, context: html, text, full: html, text
- src/hg/lib/hVarSubst.c - lines changed 122, context: html, text, full: html, text
- src/hg/lib/tests/expected/hVarSubstHtmlTest - lines changed 7, context: html, text, full: html, text
- src/hg/lib/tests/hVarSubstHtmlTester.c - lines changed 73, context: html, text, full: html, text
- src/hg/makeDb/trackDb/README - lines changed 11, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/centroCores.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/centroMarkers.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/centroSat.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/compartAB.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/compartE1.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/covClr.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/covHifi.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/covOnt.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/egapx.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/itsRepeats.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/largeSv.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/methyl5mC.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/newRegions.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/retrocopies.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/satellome.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/tandemRepeats.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/telomeres.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/contrib/bTaeGut7/transposons.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/epigenCentral.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/episignatures.html - lines changed 2, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/hprcRdt.html - lines changed 1, context: html, text, full: html, text
- src/hg/makeDb/trackDb/human/hg38/methaDory.html - lines changed 4, context: html, text, full: html, text
33eb806d3707661d8a4d9af1e0673f90578d86bc Mon Sep 21 06:12:21 2026 -0700
- hgHubConnect: the api key sync has to answer before the cart, and its signature needed a timestamp, refs #38323
Six things, all in the syncHubApiKeys path, which is still off everywhere.
The sync never reached its handler on a real mirror. It was registered as
a cartJson command, so building the cart ran forceUserIdOrCaptcha() first;
a peer's request carries no hguid cookie and no apiKey= variable, so any
site with cloudFlareSiteKey set -- which is every site that needs api keys
in the first place -- answered it with a captcha page. Only hgwdev, where
the captcha is commented out, ever ran the handler, which is why this was
not caught earlier. doApiKeySyncIfRequested() now answers the request in
main() before any cart exists. That also stops each sync leaving a junk
userDb and sessionDb row behind: measured, 15 syncs now create none.
The signature is HMAC-SHA256 over the salt instead of md5(salt-user-key),
so it does not rest on md5 resisting length extension, and it is compared
in constant time. It now covers a timestamp too, and a peer refuses
anything signed more than HUB_APIKEY_SYNC_WINDOW (300s) either side of
now, so a captured sync cannot be replayed later to reinstate a key its
owner has since revoked. The fields are joined with newlines and a
newline in a userName or apiKey is refused, so one signature cannot cover
two different splits of the same string.
syncApiKeyToOtherNodes() is called inside an errCatch of its own now. It
runs after the local key is already written and in the response, so an
errAbort there (login.cookieSalt unset) used to reach cartJsonExecute's
outer catch, which threw the response away -- the user saw a failure over
a key that was sitting in the database.
geoMirrorNotifyOtherNodes() returns each peer's response instead of
discarding it, and uses netUrlMustOpenPastHeader, which errAborts on
anything but a 200 and hands back the body alone. A peer answering 'bad
signature' or 'not enabled on this site' is no longer indistinguishable
from success; it is logged to error_log, not warn()ed at the person who
clicked the button.
Verified against the built CGI run as Apache runs it, with the captcha
turned on: a signed sync is accepted and writes the row, a signed revoke
removes it, and a replay, a swapped userName, a forged signature and
malformed json are all refused. hubSpaceKeysTester covers the signature
rules; it still cannot run its database half until hgcentralregress gets
the UPDATE and DELETE grant.
- src/hg/hgHubConnect/hgHubConnect.c - lines changed 7, context: html, text, full: html, text
- src/hg/hgHubConnect/hgHubConnect.h - lines changed 5, context: html, text, full: html, text
- src/hg/hgHubConnect/trackHubWizard.c - lines changed 122, context: html, text, full: html, text
- src/hg/inc/hubSpaceKeys.h - lines changed 18, context: html, text, full: html, text
- src/hg/lib/geoMirror.c - lines changed 29, context: html, text, full: html, text
- src/hg/lib/hubSpaceKeys.c - lines changed 72, context: html, text, full: html, text
- src/hg/lib/tests/hubSpaceKeysTester.c - lines changed 69, context: html, text, full: html, text
2eb14fc0378432977d83366a3211c67f8913018d Mon Sep 21 06:21:51 2026 -0700
- hgc and hgTrackUi: open off-site links in a new tab, with rel="noopener noreferrer"
offsiteLinksToNewTab() in utils.js runs over the rendered page alongside
addHgsidToLinks() and gives every http(s) link whose host is not ours a
target=_blank it does not already have, plus rel="noopener noreferrer". Without
noopener the page that opens keeps a handle on the tab it came from and can
navigate it; without noreferrer the Referer header carries our own URL, which has
the session id in it. The href on a track description page is written by whoever
wrote the track or the hub, so neither is theoretical. A mailto: or an ftp: link
is left alone, and so is every link that stays on this server.
jsAddHgsidToLinks() now emits both calls and is renamed jsFixUpPageLinks().
hgTracks.js runs the same pass over the track description popup: the replace it
does before that already puts a target on every link in there, so what this adds
is the rel on the ones that leave.
refs #38380
- src/hg/hgTrackUi/hgTrackUi.c - lines changed 1, context: html, text, full: html, text
547d1ed19123e170307c724bb511e0bf6f086416 Mon Sep 21 06:30:13 2026 -0700
- hgLogin: add an "or" divider before the email-sign-in-link button too, refs #38302
- src/hg/hgLogin/hgLogin.c - lines changed 7, context: html, text, full: html, text
switch to files view, user index