File Changes for max
switch to commits view, user indexv503_base to v504_preview (2026-08-31 to 2026-09-07) v504
Show details
- src/blat/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/cgilib/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/encode/getTrackReferences/getTrackReferences
- lines changed 5, context: html, text, full: html, text
52432895f426bc722014061bc7a79eb1ca8b025d Sun Sep 6 06:55:18 2026 -0700
getTrackReferences: abort with an error on stderr when the NCBI full-text link cannot be fetched, instead of printing the error onto stdout where it ends up in the middle of the generated references. refs #38265
- lines changed 85, context: html, text, full: html, text
e3962943daee77f7327e2a49ba894f157646ec68 Sun Sep 6 07:02:29 2026 -0700
getTrackReferences: get the full text link from the eutils elink API. Scraping the full-text-links box out of the PubMed web page stopped working: NCBI now answers non-browser clients with a Javascript cookie challenge, so the page never contains any links and every citation fell back to the PubMed URL. Use elink/prlinks instead, resolve the doi.org links it returns to the publisher page, and retry on the eutils 429 rate limit. refs #38265
- src/hg/hgApi/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgBlat/hgBlat.c
- lines changed 6, context: html, text, full: html, text
42b33234bee213325c2b6eede35edbb4fb2e566c Thu Sep 3 06:40:30 2026 -0700
hgBlat: spinner while a search runs, shorter example, left-justified result columns
Four items from Lou's note-37 on #38086.
The search is a plain form POST, so nothing on screen changed between Submit and
the arrival of the results - on an assembly served by a dynamic BLAT server that
is a ten second wait with no sign that anything is happening. A busy overlay now
covers the form: a generic .gbBusyBg/.gbSpinner in gbModern.css (CSS-drawn, since
the page is already navigating away when it appears), put up by blatFormBusyWire()
350 ms after submit so a sub-second hg38 search does not flash a scrim, and
swapping its text after 8 s to say that an index is probably being loaded on
demand. Skipped for Clear and for an empty query; a pageshow handler takes it
down again after a Back navigation.
The example sequence goes from 13.8 kb of the SOD1 locus to 2.5 kb of PTP4A3,
hg38 chr8:141,424,501-141,427,000. That window is carried by both
chr8_KI270819v1_alt and chr8_MU273363v1_fix, so the example now demonstrates the
alt and fix rows and their FAQ icons instead of returning one plain hit, and it
covers two PTP4A3 exons so the result track is named after the gene.
Every column of the results table is left-justified; the numeric ones keep
tabular figures, and the score value gets a fixed width so its bars stay in a
straight line. The alt/fix explanation icon is red rather than faint grey, and
is drawn as the browser's own info-icon SVG rather than the U+24D8 glyph, which
some system fonts do not have.
refs #38086
- src/hg/hgCollection/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgFileSearch/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgFileUi/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgGateway/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgGene/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgGenome/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgIntegrator/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgLogin/hgLogin.c
- lines changed 213, context: html, text, full: html, text
1d079d218c274f09b61ec390619782e0c505d157 Tue Sep 1 06:44:56 2026 -0700
hgLogin: confirm the recovery email address by mail, refs #38197
The optional recovery address given at signup is now mailed a signed
confirmation link, and counts for sign-in and password recovery only once
that link has been opened. The link is signed with login.cookieSalt, expires
after a week and works exactly once.
Adds gbMembers.recovEmailVerified through the usual auto-upgrade. Existing
rows default to 'Y', so recovery addresses set up before this keep working
and nobody has to re-confirm one. An install that sends no mail, or that has
no login.cookieSalt to sign a link with, stores the address as it does today.
- lines changed 241, context: html, text, full: html, text
d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
hgLogin: let users set or change their recovery email address, refs #38197
Adds a page where a signed-in user can set or change the recovery address on
their account, offered in the account menu next to "Change email" -- both in
the top right blue bar popup and on the session page. It is off by default:
set login.recovEmailChange=on in hg.conf to offer it. The page also needs
login.cookieSalt, working outbound mail and the recovEmailVerified column,
and stays hidden where any of those is missing.
The new address is confirmed by mail before it takes effect, so whatever is
on the account keeps working until the link is opened and a typo costs the
user nothing. An account that has a password must supply it, since a
confirmed recovery address can sign in. Once the address does change, the
account's main address is told, the same notice that a change of the main
address already sends.
One signature now covers both the address given at signup and a later
change, so there is a single confirmation path rather than two.
- src/hg/hgMenubar/hgMenubar.c
- lines changed 5, context: html, text, full: html, text
d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
hgLogin: let users set or change their recovery email address, refs #38197
Adds a page where a signed-in user can set or change the recovery address on
their account, offered in the account menu next to "Change email" -- both in
the top right blue bar popup and on the session page. It is off by default:
set login.recovEmailChange=on in hg.conf to offer it. The page also needs
login.cookieSalt, working outbound mail and the recovEmailVerified column,
and stays hidden where any of those is missing.
The new address is confirmed by mail before it takes effect, so whatever is
on the account keeps working until the link is opened and a typo costs the
user nothing. An account that has a password must supply it, since a
confirmed recovery address can sign in. Once the address does change, the
account's main address is told, the same notice that a change of the main
address already sends.
One signature now covers both the address given at signup and a later
change, so there is a single confirmation path rather than two.
- src/hg/hgPal/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgPcr/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgPhyloPlace/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgPublicSessions/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgSession/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgSession/hgSession.c
- lines changed 95, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- lines changed 3, context: html, text, full: html, text
d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
hgLogin: let users set or change their recovery email address, refs #38197
Adds a page where a signed-in user can set or change the recovery address on
their account, offered in the account menu next to "Change email" -- both in
the top right blue bar popup and on the session page. It is off by default:
set login.recovEmailChange=on in hg.conf to offer it. The page also needs
login.cookieSalt, working outbound mail and the recovEmailVerified column,
and stays hidden where any of those is missing.
The new address is confirmed by mail before it takes effect, so whatever is
on the account keeps working until the link is opened and a typo costs the
user nothing. An account that has a password must supply it, since a
confirmed recovery address can sign in. Once the address does change, the
account's main address is told, the same notice that a change of the main
address already sends.
One signature now covers both the address given at signup and a later
change, so there is a single confirmation path rather than two.
- lines changed 93, context: html, text, full: html, text
8a648c2ffc0e2c8a671a9074e00a0a3d56575589 Wed Sep 2 04:48:57 2026 -0700
Merge remote-tracking branch 'origin/master'
# Conflicts:
# src/hg/hgSession/hgSession.c
- lines changed 2, context: html, text, full: html, text
89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
snapshotSession: rename the snapshot "reaper" to "cleaner"
Renames the snapshotReaper utility to snapshotCleaner and drops the word
"reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and
the doc/usage text now say clean/cleaned/cleaner). No behavior change.
refs #38197
- lines changed 9, context: html, text, full: html, text
ef8d1004e3d984933a79686a7369d74825886c48 Fri Sep 4 11:35:31 2026 -0700
snapshotSession: reject a snapshot missing its required var
A "blat" snapshot is a dead link without blatLastBigBed (the pinned bigPsl).
That variable is set by an async hgc buildBigPsl call, so a share clicked
before the build finished - or after it failed - would have minted a link
that reopens to nothing. A snapshotType may now name a requiredVar, and
doSaveSessionJson refuses the save (asking the caller to retry) instead of
handing out a broken link. Found in code review of the BLAT share wiring.
refs #38197
- src/hg/hgSession/hgSession.h
- lines changed 10, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- src/hg/hgSuggest/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgTables/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgTrackUi/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgTracks/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgVai/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgc/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/hgc/hgc.c
- lines changed 3, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- lines changed 27, context: html, text, full: html, text
2eecd47992bf6fcfee294854359117d81167459a Sat Sep 5 15:51:03 2026 -0700
hgc: show the trackDb iframeUrl frame with the item details, not above them
The iframe from the iframeUrl setting was written out before anything else on
the details page, which is unlike every other part of a details page. It now
appears just below the "View DNA for this feature" line, with the rest of the
details.
Rather than move each of the printIframe() calls, which all happen before the
position is known, printIframe() now only builds the html and a new
printPendingIframe() writes it out from the position-printing routines. That
covers bigBed, bigPsl and the generic click handler alike. The lorax details
page prints it immediately, since there the iframe is the page itself.
Also, a track with an iframeUrl but no url setting never showed its iframe at
all, refs #37595
- src/hg/hgc/hgc.h
- lines changed 8, context: html, text, full: html, text
2eecd47992bf6fcfee294854359117d81167459a Sat Sep 5 15:51:03 2026 -0700
hgc: show the trackDb iframeUrl frame with the item details, not above them
The iframe from the iframeUrl setting was written out before anything else on
the details page, which is unlike every other part of a details page. It now
appears just below the "View DNA for this feature" line, with the rest of the
details.
Rather than move each of the printIframe() calls, which all happen before the
position is known, printIframe() now only builds the html and a new
printPendingIframe() writes it out from the position-printing routines. That
covers bigBed, bigPsl and the generic click handler alike. The lorax details
page prints it immediately, since there the iframe is the page itself.
Also, a track with an iframeUrl but no url setting never showed its iframe at
all, refs #37595
- src/hg/hgc/loraxClick.c
- lines changed 3, context: html, text, full: html, text
2eecd47992bf6fcfee294854359117d81167459a Sat Sep 5 15:51:03 2026 -0700
hgc: show the trackDb iframeUrl frame with the item details, not above them
The iframe from the iframeUrl setting was written out before anything else on
the details page, which is unlike every other part of a details page. It now
appears just below the "View DNA for this feature" line, with the rest of the
details.
Rather than move each of the printIframe() calls, which all happen before the
position is known, printIframe() now only builds the html and a new
printPendingIframe() writes it out from the position-printing routines. That
covers bigBed, bigPsl and the generic click handler alike. The lorax details
page prints it immediately, since there the iframe is the page itself.
Also, a track with an iframeUrl but no url setting never showed its iframe at
all, refs #37595
- src/hg/htdocs/FAQ/FAQformat.html
- lines changed 9, context: html, text, full: html, text
059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245
This convention was previously only documented indirectly, as a SQL
filtering tip on the Gene tracks FAQ page. Add it next to the
cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as,
sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and
sangerGene.h, and mention it in FAQformat.html and bigGenePred.html
(via the equivalent thickStart == thickEnd check).
- src/hg/htdocs/goldenPath/help/bigGenePred.html
- lines changed 4, context: html, text, full: html, text
059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245
This convention was previously only documented indirectly, as a SQL
filtering tip on the Gene tracks FAQ page. Add it next to the
cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as,
sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and
sangerGene.h, and mention it in FAQformat.html and bigGenePred.html
(via the equivalent thickStart == thickEnd check).
- src/hg/htdocs/goldenPath/help/blatExample.fa
- lines changed 279, context: html, text, full: html, text
42b33234bee213325c2b6eede35edbb4fb2e566c Thu Sep 3 06:40:30 2026 -0700
hgBlat: spinner while a search runs, shorter example, left-justified result columns
Four items from Lou's note-37 on #38086.
The search is a plain form POST, so nothing on screen changed between Submit and
the arrival of the results - on an assembly served by a dynamic BLAT server that
is a ten second wait with no sign that anything is happening. A busy overlay now
covers the form: a generic .gbBusyBg/.gbSpinner in gbModern.css (CSS-drawn, since
the page is already navigating away when it appears), put up by blatFormBusyWire()
350 ms after submit so a sub-second hg38 search does not flash a scrim, and
swapping its text after 8 s to say that an index is probably being loaded on
demand. Skipped for Clear and for an empty query; a pageshow handler takes it
down again after a Back navigation.
The example sequence goes from 13.8 kb of the SOD1 locus to 2.5 kb of PTP4A3,
hg38 chr8:141,424,501-141,427,000. That window is carried by both
chr8_KI270819v1_alt and chr8_MU273363v1_fix, so the example now demonstrates the
alt and fix rows and their FAQ icons instead of returning one plain hit, and it
covers two PTP4A3 exons so the result track is named after the gene.
Every column of the results table is left-justified; the numeric ones keep
tabular figures, and the score value gets a fixed width so its bars stay in a
straight line. The alt/fix explanation icon is red rather than faint grey, and
is drawn as the browser's own info-icon SVG rather than the U+24D8 glyph, which
some system fonts do not have.
refs #38086
- src/hg/htdocs/inc/globalNavBar.inc
- lines changed 3, context: html, text, full: html, text
6e89bb12a35c65c0607719f2aa2c7377803d09f9 Fri Sep 4 14:03:07 2026 -0700
Keep the top-right Login/Share a link menu icon inside the blue bar on narrow windows.
On screens under 1100px the icon was positioned with position:fixed, which pins it
to the top of the window. That is the top of the blue bar only on hgTracks; on
hgGateway and the home page the bar sits below the logo banner, so the icon showed
up in the white area above it. Use position:sticky instead, which keeps the icon in
the bar on every page and still holds it at the right edge of the visible viewport
while the 1000px-wide bar is scrolled sideways. The hardcoded background colour is
replaced by the bar's own, which is not the same on all pages. The links now hang in
their own span so that opening the menu no longer pushes the page content down, and
the icon is left out entirely on mirrors that have no top-right links to show.
refs #38251
- src/hg/htdocs/license/index.html
- lines changed 13, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/htdocs/style/gbModern.css
- lines changed 19, context: html, text, full: html, text
42b33234bee213325c2b6eede35edbb4fb2e566c Thu Sep 3 06:40:30 2026 -0700
hgBlat: spinner while a search runs, shorter example, left-justified result columns
Four items from Lou's note-37 on #38086.
The search is a plain form POST, so nothing on screen changed between Submit and
the arrival of the results - on an assembly served by a dynamic BLAT server that
is a ten second wait with no sign that anything is happening. A busy overlay now
covers the form: a generic .gbBusyBg/.gbSpinner in gbModern.css (CSS-drawn, since
the page is already navigating away when it appears), put up by blatFormBusyWire()
350 ms after submit so a sub-second hg38 search does not flash a scrim, and
swapping its text after 8 s to say that an index is probably being loaded on
demand. Skipped for Clear and for an empty query; a pageshow handler takes it
down again after a Back navigation.
The example sequence goes from 13.8 kb of the SOD1 locus to 2.5 kb of PTP4A3,
hg38 chr8:141,424,501-141,427,000. That window is carried by both
chr8_KI270819v1_alt and chr8_MU273363v1_fix, so the example now demonstrates the
alt and fix rows and their FAQ icons instead of returning one plain hit, and it
covers two PTP4A3 exons so the result track is named after the gene.
Every column of the results table is left-justified; the numeric ones keep
tabular figures, and the score value gets a fixed width so its bars stay in a
straight line. The alt/fix explanation icon is red rather than faint grey, and
is drawn as the browser's own info-icon SVG rather than the U+24D8 glyph, which
some system fonts do not have.
refs #38086
- src/hg/htdocs/style/hgBlat.css
- lines changed 17, context: html, text, full: html, text
42b33234bee213325c2b6eede35edbb4fb2e566c Thu Sep 3 06:40:30 2026 -0700
hgBlat: spinner while a search runs, shorter example, left-justified result columns
Four items from Lou's note-37 on #38086.
The search is a plain form POST, so nothing on screen changed between Submit and
the arrival of the results - on an assembly served by a dynamic BLAT server that
is a ten second wait with no sign that anything is happening. A busy overlay now
covers the form: a generic .gbBusyBg/.gbSpinner in gbModern.css (CSS-drawn, since
the page is already navigating away when it appears), put up by blatFormBusyWire()
350 ms after submit so a sub-second hg38 search does not flash a scrim, and
swapping its text after 8 s to say that an index is probably being loaded on
demand. Skipped for Clear and for an empty query; a pageshow handler takes it
down again after a Back navigation.
The example sequence goes from 13.8 kb of the SOD1 locus to 2.5 kb of PTP4A3,
hg38 chr8:141,424,501-141,427,000. That window is carried by both
chr8_KI270819v1_alt and chr8_MU273363v1_fix, so the example now demonstrates the
alt and fix rows and their FAQ icons instead of returning one plain hit, and it
covers two PTP4A3 exons so the result track is named after the gene.
Every column of the results table is left-justified; the numeric ones keep
tabular figures, and the score value gets a fixed width so its bars stay in a
straight line. The alt/fix explanation icon is red rather than faint grey, and
is drawn as the browser's own info-icon SVG rather than the U+24D8 glyph, which
some system fonts do not have.
refs #38086
- src/hg/htdocs/style/nice_menu.css
- lines changed 22, context: html, text, full: html, text
6e89bb12a35c65c0607719f2aa2c7377803d09f9 Fri Sep 4 14:03:07 2026 -0700
Keep the top-right Login/Share a link menu icon inside the blue bar on narrow windows.
On screens under 1100px the icon was positioned with position:fixed, which pins it
to the top of the window. That is the top of the blue bar only on hgTracks; on
hgGateway and the home page the bar sits below the logo banner, so the icon showed
up in the white area above it. Use position:sticky instead, which keeps the icon in
the bar on every page and still holds it at the right edge of the visible viewport
while the 1000px-wide bar is scrolled sideways. The hardcoded background colour is
replaced by the bar's own, which is not the same on all pages. The links now hang in
their own span so that opening the menu no longer pushes the page content down, and
the icon is left out entirely on mirrors that have no top-right links to show.
refs #38251
- src/hg/hubApi/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/inc/genePred.h
- lines changed 4, context: html, text, full: html, text
059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245
This convention was previously only documented indirectly, as a SQL
filtering tip on the Gene tracks FAQ page. Add it next to the
cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as,
sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and
sangerGene.h, and mention it in FAQformat.html and bigGenePred.html
(via the equivalent thickStart == thickEnd check).
- src/hg/inc/hdb.h
- lines changed 4, context: html, text, full: html, text
571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260
hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands
each track to addTrackIfDataAccessible(), which frees any track whose data cannot be
reached. That is right for the browser, which should degrade gracefully rather than
draw a broken track, and wrong for hubCheck, whose main job is to say that the file
is unreachable. The track was gone before the file check ran, so hubCheck printed
nothing and exited 0:
cd hubCheck/tests/input
sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt
hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned
Split the non-pruning part of trackDbPolishAfterLinkup() into
trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller
still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour
is unchanged.
Added a tests/ case whose bigDataUrl does not exist. Before this change it produced
no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb".
Of the existing tests, only badType changes, and it gains a real error that the prune
had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was
already failing before this commit and still is, for an unrelated reason: seven tests
carry expected output with per-track "missing description page" warnings that no
longer appear, because those hubs point at remote URLs that no longer resolve and
hubCheckTrack aborts at the file check before reaching the description check. That is
stale test data, not addressed here.
- src/hg/inc/sangerGene.h
- lines changed 2, context: html, text, full: html, text
059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245
This convention was previously only documented indirectly, as a SQL
filtering tip on the Gene tracks FAQ page. Add it next to the
cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as,
sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and
sangerGene.h, and mention it in FAQformat.html and bigGenePred.html
(via the equivalent thickStart == thickEnd check).
- src/hg/inc/sessionData.h
- lines changed 4, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- src/hg/inc/snapshotSession.h
- lines changed 73, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- lines changed 7, context: html, text, full: html, text
89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
snapshotSession: rename the snapshot "reaper" to "cleaner"
Renames the snapshotReaper utility to snapshotCleaner and drops the word
"reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and
the doc/usage text now say clean/cleaned/cleaner). No behavior change.
refs #38197
- lines changed 6, context: html, text, full: html, text
ef8d1004e3d984933a79686a7369d74825886c48 Fri Sep 4 11:35:31 2026 -0700
snapshotSession: reject a snapshot missing its required var
A "blat" snapshot is a dead link without blatLastBigBed (the pinned bigPsl).
That variable is set by an async hgc buildBigPsl call, so a share clicked
before the build finished - or after it failed - would have minted a link
that reopens to nothing. A snapshotType may now name a requiredVar, and
doSaveSessionJson refuses the save (asking the caller to retry) instead of
handing out a broken link. Found in code review of the BLAT share wiring.
refs #38197
- src/hg/inc/wikiLink.h
- lines changed 7, context: html, text, full: html, text
d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
hgLogin: let users set or change their recovery email address, refs #38197
Adds a page where a signed-in user can set or change the recovery address on
their account, offered in the account menu next to "Change email" -- both in
the top right blue bar popup and on the session page. It is off by default:
set login.recovEmailChange=on in hg.conf to offer it. The page also needs
login.cookieSalt, working outbound mail and the recovEmailVerified column,
and stays hidden where any of those is missing.
The new address is confirmed by mail before it takes effect, so whatever is
on the account keeps working until the link is opened and a typo costs the
user nothing. An account that has a password must supply it, since a
confirmed recovery address can sign in. Once the address does change, the
account's main address is told, the same notice that a change of the main
address already sends.
One signature now covers both the address given at signup and a later
change, so there is a single confirmation path rather than two.
- src/hg/js/hgBlat.js
- lines changed 62, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- lines changed 103, context: html, text, full: html, text
42b33234bee213325c2b6eede35edbb4fb2e566c Thu Sep 3 06:40:30 2026 -0700
hgBlat: spinner while a search runs, shorter example, left-justified result columns
Four items from Lou's note-37 on #38086.
The search is a plain form POST, so nothing on screen changed between Submit and
the arrival of the results - on an assembly served by a dynamic BLAT server that
is a ten second wait with no sign that anything is happening. A busy overlay now
covers the form: a generic .gbBusyBg/.gbSpinner in gbModern.css (CSS-drawn, since
the page is already navigating away when it appears), put up by blatFormBusyWire()
350 ms after submit so a sub-second hg38 search does not flash a scrim, and
swapping its text after 8 s to say that an index is probably being loaded on
demand. Skipped for Clear and for an empty query; a pageshow handler takes it
down again after a Back navigation.
The example sequence goes from 13.8 kb of the SOD1 locus to 2.5 kb of PTP4A3,
hg38 chr8:141,424,501-141,427,000. That window is carried by both
chr8_KI270819v1_alt and chr8_MU273363v1_fix, so the example now demonstrates the
alt and fix rows and their FAQ icons instead of returning one plain hit, and it
covers two PTP4A3 exons so the result track is named after the gene.
Every column of the results table is left-justified; the numeric ones keep
tabular figures, and the score value gets a fixed width so its bars stay in a
straight line. The alt/fix explanation icon is red rather than faint grey, and
is drawn as the browser's own info-icon SVG rather than the U+24D8 glyph, which
some system fonts do not have.
refs #38086
- lines changed 1, context: html, text, full: html, text
89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
snapshotSession: rename the snapshot "reaper" to "cleaner"
Renames the snapshotReaper utility to snapshotCleaner and drops the word
"reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and
the doc/usage text now say clean/cleaned/cleaner). No behavior change.
refs #38197
- src/hg/js/hgSession.js
- lines changed 6, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- src/hg/js/topLinks.js
- lines changed 181, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- lines changed 3, context: html, text, full: html, text
d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
hgLogin: let users set or change their recovery email address, refs #38197
Adds a page where a signed-in user can set or change the recovery address on
their account, offered in the account menu next to "Change email" -- both in
the top right blue bar popup and on the session page. It is off by default:
set login.recovEmailChange=on in hg.conf to offer it. The page also needs
login.cookieSalt, working outbound mail and the recovEmailVerified column,
and stays hidden where any of those is missing.
The new address is confirmed by mail before it takes effect, so whatever is
on the account keeps working until the link is opened and a typo costs the
user nothing. An account that has a password must supply it, since a
confirmed recovery address can sign in. Once the address does change, the
account's main address is told, the same notice that a change of the main
address already sends.
One signature now covers both the address given at signup and a later
change, so there is a single confirmation path rather than two.
- lines changed 13, context: html, text, full: html, text
7d0ceafa7c0753b9cd7e59b5e29641508e59ec87 Fri Sep 4 12:18:57 2026 -0700
Close the Account dialog when one of its links is clicked, refs #38257
The dialog relied on navigation to remove it: every link in it leads to
another page, and loading that page took the dialog away with it. "My
Track Hubs" is the exception. On hgHubConnect it points at the page you
are already on, and hgHubConnect.js cancels the click and switches to the
Connected Hubs tab instead, so nothing ever removed the dialog.
The dialog also swallowed the click on its way up, so hgHubConnect.js
never saw it and the tab did not change either. Let clicks on links
through, and close the dialog on the next tick, after the click has been
fully dispatched.
- lines changed 4, context: html, text, full: html, text
6e89bb12a35c65c0607719f2aa2c7377803d09f9 Fri Sep 4 14:03:07 2026 -0700
Keep the top-right Login/Share a link menu icon inside the blue bar on narrow windows.
On screens under 1100px the icon was positioned with position:fixed, which pins it
to the top of the window. That is the top of the blue bar only on hgTracks; on
hgGateway and the home page the bar sits below the logo banner, so the icon showed
up in the white area above it. Use position:sticky instead, which keeps the icon in
the bar on every page and still holds it at the right edge of the visible viewport
while the 1000px-wide bar is scrolled sideways. The hardcoded background colour is
replaced by the bar's own, which is not the same on all pages. The links now hang in
their own span so that opening the menu no longer pushes the page content down, and
the icon is left out entirely on mirrors that have no top-right links to show.
refs #38251
- src/hg/lib/cart.c
- lines changed 1, context: html, text, full: html, text
5abe782bada3c851f1844c94b747cf7c769c71f9 Fri Sep 4 09:12:46 2026 -0700
cart.c: gate the captcha session-id debug print behind hg.conf captchaDebug, refs #38247
Register captchaDebug in the hg.conf catalog as a permanent debugging
knob rather than a release gate.
- src/hg/lib/ensGene.as
- lines changed 2, context: html, text, full: html, text
059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245
This convention was previously only documented indirectly, as a SQL
filtering tip on the Gene tracks FAQ page. Add it next to the
cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as,
sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and
sangerGene.h, and mention it in FAQformat.html and bigGenePred.html
(via the equivalent thickStart == thickEnd check).
- src/hg/lib/genePred.as
- lines changed 2, context: html, text, full: html, text
059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245
This convention was previously only documented indirectly, as a SQL
filtering tip on the Gene tracks FAQ page. Add it next to the
cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as,
sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and
sangerGene.h, and mention it in FAQformat.html and bigGenePred.html
(via the equivalent thickStart == thickEnd check).
- src/hg/lib/genePredExt.as
- lines changed 2, context: html, text, full: html, text
059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245
This convention was previously only documented indirectly, as a SQL
filtering tip on the Gene tracks FAQ page. Add it next to the
cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as,
sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and
sangerGene.h, and mention it in FAQformat.html and bigGenePred.html
(via the equivalent thickStart == thickEnd check).
- src/hg/lib/hdb.c
- lines changed 13, context: html, text, full: html, text
571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260
hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands
each track to addTrackIfDataAccessible(), which frees any track whose data cannot be
reached. That is right for the browser, which should degrade gracefully rather than
draw a broken track, and wrong for hubCheck, whose main job is to say that the file
is unreachable. The track was gone before the file check ran, so hubCheck printed
nothing and exited 0:
cd hubCheck/tests/input
sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt
hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned
Split the non-pruning part of trackDbPolishAfterLinkup() into
trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller
still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour
is unchanged.
Added a tests/ case whose bigDataUrl does not exist. Before this change it produced
no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb".
Of the existing tests, only badType changes, and it gains a real error that the prune
had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was
already failing before this commit and still is, for an unrelated reason: seven tests
carry expected output with per-track "missing description page" warnings that no
longer appear, because those hubs point at remote URLs that no longer resolve and
hubCheckTrack aborts at the file check before reaching the description check. That is
stale test data, not addressed here.
- src/hg/lib/hui.c
- lines changed 8, context: html, text, full: html, text
be590a4f6b10055e567a9ac717919cd99481272a Fri Sep 4 17:01:05 2026 -0700
Let a dataVersion file under /gbdb be read for hub tracks
A track may point dataVersion at a local file, and otto jobs use that to show
a live version string. Hub tracks were excluded, because a hub is supplied by
the user and could otherwise name any file on the server.
That also excludes the curated-hub assemblies. hs1 is served to the browser as
a hub, so its tracks are hub tracks, and hgTrackUi printed the raw path where
the version should be. Paths under /gbdb are safe to make an exception for:
that tree is public data, mirrored on hgdownload, so reading one discloses
nothing that is not already downloadable.
Also resolves $D against the plain database name, so it does not expand to
hub_NNNNN_hs1 on a curated hub.
refs #38268
- lines changed 28, context: html, text, full: html, text
1a85cdaa59f02695ccecc1aac8a9d40bac188064 Sun Sep 6 07:10:47 2026 -0700
hgTrackUi: tighten the dataVersion path check, refs #38268
checkDataVersion() decides whether a dataVersion setting that looks like a local
path may be opened. Do that test on the path after $D substitution rather than
on the raw setting, and accept only a plain path in the /gbdb tree. A quickLifted
track no longer needs a case of its own, its file is under /gbdb as well.
- src/hg/lib/knownGene.as
- lines changed 2, context: html, text, full: html, text
059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245
This convention was previously only documented indirectly, as a SQL
filtering tip on the Gene tracks FAQ page. Add it next to the
cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as,
sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and
sangerGene.h, and mention it in FAQformat.html and bigGenePred.html
(via the equivalent thickStart == thickEnd check).
- src/hg/lib/makefile
- lines changed 1, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- src/hg/lib/refFlat.as
- lines changed 2, context: html, text, full: html, text
059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245
This convention was previously only documented indirectly, as a SQL
filtering tip on the Gene tracks FAQ page. Add it next to the
cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as,
sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and
sangerGene.h, and mention it in FAQformat.html and bigGenePred.html
(via the equivalent thickStart == thickEnd check).
- src/hg/lib/sangerGene.as
- lines changed 2, context: html, text, full: html, text
059927383e72afe59202535b4863fc016463127a Thu Sep 3 15:04:05 2026 -0700
Document that cdsStart == cdsEnd marks a non-coding transcript in genePred format, refs #38245
This convention was previously only documented indirectly, as a SQL
filtering tip on the Gene tracks FAQ page. Add it next to the
cdsStart/cdsEnd field declarations in genePred.as, genePredExt.as,
sangerGene.as, ensGene.as, knownGene.as, refFlat.as, genePred.h and
sangerGene.h, and mention it in FAQformat.html and bigGenePred.html
(via the equivalent thickStart == thickEnd check).
- src/hg/lib/snapshotSession.c
- lines changed 279, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- lines changed 6, context: html, text, full: html, text
89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
snapshotSession: rename the snapshot "reaper" to "cleaner"
Renames the snapshotReaper utility to snapshotCleaner and drops the word
"reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and
the doc/usage text now say clean/cleaned/cleaner). No behavior change.
refs #38197
- lines changed 9, context: html, text, full: html, text
ef8d1004e3d984933a79686a7369d74825886c48 Fri Sep 4 11:35:31 2026 -0700
snapshotSession: reject a snapshot missing its required var
A "blat" snapshot is a dead link without blatLastBigBed (the pinned bigPsl).
That variable is set by an async hgc buildBigPsl call, so a share clicked
before the build finished - or after it failed - would have minted a link
that reopens to nothing. A snapshotType may now name a requiredVar, and
doSaveSessionJson refuses the save (asking the caller to retry) instead of
handing out a broken link. Found in code review of the BLAT share wiring.
refs #38197
- src/hg/lib/trackHub.c
- lines changed 14, context: html, text, full: html, text
bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
hubCheck: stop resolving bigDataUrl a second time, refs #38256
trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which
rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer
relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against
genome->trackDbFile, which prepended the hub's directory a second time. For a local
hub reached by a relative path that turned a file which is present into a file which
is reported missing:
hubCheck out/hub.txt
Couldn't open out/out/hg19/x.bb
Remote hubs and local hubs given by an absolute path were unaffected, because the
second resolution is a no-op once the path is a URL or already absolute, which is why
this went unnoticed. The tests/makefile has been working around it: the
fieldCountMismatch case has to cd into input/ before it can find its bigBed.
Added a tests/ case that does not cd, with the data file in a subdirectory below
hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately
declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report
that mismatch if it really opened the file. Before this change the test reported
"Couldn't open input/relPath/input/relPath/data/test6field.bb".
Verified against a binary built from the unmodified tree: this changes the output of
no other test in hubCheck/tests. Seven of those tests were already failing before
this commit and still are.
- src/hg/lib/web.c
- lines changed 6, context: html, text, full: html, text
d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
hgLogin: let users set or change their recovery email address, refs #38197
Adds a page where a signed-in user can set or change the recovery address on
their account, offered in the account menu next to "Change email" -- both in
the top right blue bar popup and on the session page. It is off by default:
set login.recovEmailChange=on in hg.conf to offer it. The page also needs
login.cookieSalt, working outbound mail and the recovEmailVerified column,
and stays hidden where any of those is missing.
The new address is confirmed by mail before it takes effect, so whatever is
on the account keeps working until the link is opened and a typo costs the
user nothing. An account that has a password must supply it, since a
confirmed recovery address can sign in. Once the address does change, the
account's main address is told, the same notice that a change of the main
address already sends.
One signature now covers both the address given at signup and a later
change, so there is a single confirmation path rather than two.
- src/hg/lib/wikiLink.c
- lines changed 26, context: html, text, full: html, text
d33d0e1c19117a0347dbadb99a77e9f12483c735 Tue Sep 1 06:56:09 2026 -0700
hgLogin: let users set or change their recovery email address, refs #38197
Adds a page where a signed-in user can set or change the recovery address on
their account, offered in the account menu next to "Change email" -- both in
the top right blue bar popup and on the session page. It is off by default:
set login.recovEmailChange=on in hg.conf to offer it. The page also needs
login.cookieSalt, working outbound mail and the recovEmailVerified column,
and stays hidden where any of those is missing.
The new address is confirmed by mail before it takes effect, so whatever is
on the account keeps working until the link is opened and a typo costs the
user nothing. An account that has a password must supply it, since a
confirmed recovery address can sign in. Once the address does change, the
account's main address is told, the same notice that a change of the main
address already sends.
One signature now covers both the address given at signup and a later
change, so there is a single confirmation path rather than two.
- src/hg/liftOver/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/makeDb/doc/contrib/hprc2annot.txt
- lines changed 228, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/doc/danRer11/danioCode.txt
- lines changed 140, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- src/hg/makeDb/doc/hg38/alphaGenome.txt
- lines changed 57, context: html, text, full: html, text
8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
AlphaGenome Variant Impact (AVI) score track for hg38
New track under the Deleteriousness Predictions container showing Google
DeepMind's AlphaGenome Variant Impact score for every possible single-base
substitution, as four bigWigs, one per alternate allele.
alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per
alternate allele. It copies the PHRED column through as text so no precision
is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a
long contiguous run. The input has no row where alt == ref, so each per-allele
file is missing roughly every fourth position; without the fill the wig
fragments into blocks of about 3.5 values and the bigWig doubles in size.
All 8,812,917,339 input lines were used, none skipped.
The data cannot be redistributed, so the stanzas set tableBrowser off and the
description page sends users to the AlphaGenome Atlas. Alpha only for now.
No reference yet, the score is unpublished.
refs #38261
- src/hg/makeDb/doc/hg38/imprinting.txt
- lines changed 53, context: html, text, full: html, text
684ad73a17eebb717fba52f0aa25f3d9d7adb2de Thu Sep 3 12:03:37 2026 -0700
New Imprinting track collection on hg38, first subtrack is MethBase2 ASM
Adds the "Imprinting" superTrack to the Regulation group and its first
subtrack, the allele-specific methylation summary from the MethBase2 hub
(http://smithlab.usc.edu/trackdata/methylation/hub.txt). The bigWig gives,
for each of 29,401,795 CpG sites, the fraction of MethBase2 human methylomes
in which the site fell inside an allelically methylated region called by
dnmtools amrfinder. Taken from upstream unchanged.
Alpha only for now, refs #37599
- lines changed 200, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/doc/hg38/strchive.txt
- lines changed 32, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- src/hg/makeDb/scripts/danioCode/danioCodeDownload.sh
- lines changed 38, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- src/hg/makeDb/scripts/danioCode/danioCodeHubToRa.py
- lines changed 425, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- src/hg/makeDb/scripts/hprc2annot/bigGenePred.as
- lines changed 25, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/scripts/hprc2annot/hprc2annotBuild.sh
- lines changed 108, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/scripts/hprc2annot/hprc2annotBuildOne.sh
- lines changed 247, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/scripts/hprc2annot/hprc2annotFillCdsPhase.py
- lines changed 62, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/scripts/hprc2annot/hprc2annotFixBed.sh
- lines changed 47, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/scripts/hprc2annot/hprc2annotMakeTrackDb.py
- lines changed 114, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/scripts/hprc2annot/pclai.as
- lines changed 16, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/scripts/hprc2annot/segdups.as
- lines changed 19, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/scripts/imprinting/akbariIdmr.as
- lines changed 24, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/scripts/imprinting/akbariIdmrToBed.py
- lines changed 143, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/scripts/imprinting/geneimprint.as
- lines changed 22, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/scripts/imprinting/geneimprintToBed.py
- lines changed 329, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/scripts/imprinting/omimImprint.as
- lines changed 19, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/scripts/imprinting/omimImprintToBed.py
- lines changed 201, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/trackDb/contrib/hprc2annot/catGenes.html
- lines changed 78, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/trackDb/contrib/hprc2annot/censat.html
- lines changed 71, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/trackDb/contrib/hprc2annot/censatCentromeres.html
- lines changed 63, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt
- lines changed 112, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/trackDb/contrib/hprc2annot/liftoffGenes.html
- lines changed 83, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/trackDb/contrib/hprc2annot/methylation.html
- lines changed 62, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/trackDb/contrib/hprc2annot/pclai.html
- lines changed 85, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/trackDb/contrib/hprc2annot/segdups.html
- lines changed 105, context: html, text, full: html, text
e6d1189bea4cc541396f842b65a3392c33c8e734 Wed Sep 2 02:55:03 2026 -0700
hprc2annot: put the collection in git and fix the QA findings
The HPRC Release 2 GenArk contributed track collection (7 tracks x 462
assemblies) had only its one-line betaGenArk.txt enable checked in. Add the
makeDoc, the build scripts, the seven track description pages and the trackDb
stanzas, and fix the problems QA found.
Data fixes, both rebuilt across all 462 assemblies:
- liftoff: gff3ToGenePred was naming each genePred after the gene, so every
transcript of a gene shared one name, the RefSeq accession was lost and the
transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass
-rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0
and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op
on the CAT GFF3 (byte-identical output), so both gene tracks now share one
code path and CAT needs no rebuild.
- segdups: the build read SEDEF column 6, strand1, which is "+" by construction
on every row, so every inverted duplication rendered forward. Use column 14,
strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across
the collection. Also translate the paralog partner out of PanSN through the
GenArk chromAlias, since the browser does not translate a plain text field,
and store identity as a percentage so the mouseover can read it.
hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an
already-parsed name and blanks the values. It now refuses to touch a converted
file. GCA_041900255.1 was damaged that way and is rebuilt from source.
Provenance, all from the QA report:
- stats.tsv is appended to rather than truncated on every run, and each run
regenerates log/summary.tsv, a per-track roll-up over the collection.
- dataVersion on all seven tracks.
- Rows are now dropped for exactly two reasons and both are counted: past the
end of the sequence, or a sequence name absent from the assembly, which also
warns with example names. Only GCA_018472765.3 trips the second, the known
upstream contig-version mismatch. genePredToBigGenePred failure is checked
and an empty conversion result is a failure, not a valid empty bigBed.
Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai
display conventions which still described the data before the name field was
blanked, add a color legend checked against the data, add the pcLAI preprint
(from the Crossref record, since it has no PMID), and correct the stated reason
liftoff drops transcripts.
Display: title case on the short labels, "Active centromeres" shortened to fit
the 17-character limit, pcLAI to pack since it has no readable dense state,
liftoff and segdups to dense, and a filter on the segdups original flag.
refs #35415
- src/hg/makeDb/trackDb/human/alphaGenome.html
- lines changed 77, context: html, text, full: html, text
8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
AlphaGenome Variant Impact (AVI) score track for hg38
New track under the Deleteriousness Predictions container showing Google
DeepMind's AlphaGenome Variant Impact score for every possible single-base
substitution, as four bigWigs, one per alternate allele.
alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per
alternate allele. It copies the PHRED column through as text so no precision
is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a
long contiguous run. The input has no row where alt == ref, so each per-allele
file is missing roughly every fourth position; without the fill the wig
fragments into blocks of about 3.5 values and the bigWig doubles in size.
All 8,812,917,339 input lines were used, none skipped.
The data cannot be redistributed, so the stanzas set tableBrowser off and the
description page sends users to the AlphaGenome Atlas. Alpha only for now.
No reference yet, the score is unpublished.
refs #38261
- src/hg/makeDb/trackDb/human/alphaGenome.ra
- lines changed 72, context: html, text, full: html, text
8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
AlphaGenome Variant Impact (AVI) score track for hg38
New track under the Deleteriousness Predictions container showing Google
DeepMind's AlphaGenome Variant Impact score for every possible single-base
substitution, as four bigWigs, one per alternate allele.
alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per
alternate allele. It copies the PHRED column through as text so no precision
is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a
long contiguous run. The input has no row where alt == ref, so each per-allele
file is missing roughly every fourth position; without the fill the wig
fragments into blocks of about 3.5 values and the bigWig doubles in size.
All 8,812,917,339 input lines were used, none skipped.
The data cannot be redistributed, so the stanzas set tableBrowser off and the
description page sends users to the AlphaGenome Atlas. Alpha only for now.
No reference yet, the score is unpublished.
refs #38261
- src/hg/makeDb/trackDb/human/hg38/akbariIdmr.html
- lines changed 228, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/trackDb/human/hg38/geneimprint.html
- lines changed 148, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/trackDb/human/hg38/imprinting.html
- lines changed 36, context: html, text, full: html, text
684ad73a17eebb717fba52f0aa25f3d9d7adb2de Thu Sep 3 12:03:37 2026 -0700
New Imprinting track collection on hg38, first subtrack is MethBase2 ASM
Adds the "Imprinting" superTrack to the Regulation group and its first
subtrack, the allele-specific methylation summary from the MethBase2 hub
(http://smithlab.usc.edu/trackdata/methylation/hub.txt). The bigWig gives,
for each of 29,401,795 CpG sites, the fraction of MethBase2 human methylomes
in which the site fell inside an allelically methylated region called by
dnmtools amrfinder. Taken from upstream unchanged.
Alpha only for now, refs #37599
- lines changed 48, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/trackDb/human/hg38/imprinting.ra
- lines changed 25, context: html, text, full: html, text
684ad73a17eebb717fba52f0aa25f3d9d7adb2de Thu Sep 3 12:03:37 2026 -0700
New Imprinting track collection on hg38, first subtrack is MethBase2 ASM
Adds the "Imprinting" superTrack to the Regulation group and its first
subtrack, the allele-specific methylation summary from the MethBase2 hub
(http://smithlab.usc.edu/trackdata/methylation/hub.txt). The bigWig gives,
for each of 29,401,795 CpG sites, the fraction of MethBase2 human methylomes
in which the site fell inside an allelically methylated region called by
dnmtools amrfinder. Taken from upstream unchanged.
Alpha only for now, refs #37599
- lines changed 67, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/trackDb/human/hg38/methBaseAsm.html
- lines changed 150, context: html, text, full: html, text
684ad73a17eebb717fba52f0aa25f3d9d7adb2de Thu Sep 3 12:03:37 2026 -0700
New Imprinting track collection on hg38, first subtrack is MethBase2 ASM
Adds the "Imprinting" superTrack to the Regulation group and its first
subtrack, the allele-specific methylation summary from the MethBase2 hub
(http://smithlab.usc.edu/trackdata/methylation/hub.txt). The bigWig gives,
for each of 29,401,795 CpG sites, the fraction of MethBase2 human methylomes
in which the site fell inside an allelically methylated region called by
dnmtools amrfinder. Taken from upstream unchanged.
Alpha only for now, refs #37599
- lines changed 15, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/trackDb/human/hg38/omimImprint.html
- lines changed 148, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/trackDb/human/hg38/trackDb.ra
- lines changed 2, context: html, text, full: html, text
684ad73a17eebb717fba52f0aa25f3d9d7adb2de Thu Sep 3 12:03:37 2026 -0700
New Imprinting track collection on hg38, first subtrack is MethBase2 ASM
Adds the "Imprinting" superTrack to the Regulation group and its first
subtrack, the allele-specific methylation summary from the MethBase2 hub
(http://smithlab.usc.edu/trackdata/methylation/hub.txt). The bigWig gives,
for each of 29,401,795 CpG sites, the fraction of MethBase2 human methylomes
in which the site fell inside an allelically methylated region called by
dnmtools amrfinder. Taken from upstream unchanged.
Alpha only for now, refs #37599
- lines changed 4, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- src/hg/makeDb/trackDb/human/predictionScoresSuper.html
- lines changed 25, context: html, text, full: html, text
8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
AlphaGenome Variant Impact (AVI) score track for hg38
New track under the Deleteriousness Predictions container showing Google
DeepMind's AlphaGenome Variant Impact score for every possible single-base
substitution, as four bigWigs, one per alternate allele.
alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per
alternate allele. It copies the PHRED column through as text so no precision
is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a
long contiguous run. The input has no row where alt == ref, so each per-allele
file is missing roughly every fourth position; without the fill the wig
fragments into blocks of about 3.5 values and the bigWig doubles in size.
All 8,812,917,339 input lines were used, none skipped.
The data cannot be redistributed, so the stanzas set tableBrowser off and the
description page sends users to the AlphaGenome Atlas. Alpha only for now.
No reference yet, the score is unpublished.
refs #38261
- src/hg/makeDb/trackDb/human/predictionScoresSuper.ra
- lines changed 1, context: html, text, full: html, text
8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
AlphaGenome Variant Impact (AVI) score track for hg38
New track under the Deleteriousness Predictions container showing Google
DeepMind's AlphaGenome Variant Impact score for every possible single-base
substitution, as four bigWigs, one per alternate allele.
alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per
alternate allele. It copies the PHRED column through as text so no precision
is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a
long contiguous run. The input has no row where alt == ref, so each per-allele
file is missing roughly every fourth position; without the fill the wig
fragments into blocks of about 3.5 values and the bigWig doubles in size.
All 8,812,917,339 input lines were used, none skipped.
The data cannot be redistributed, so the stanzas set tableBrowser off and the
description page sends users to the AlphaGenome Atlas. Alpha only for now.
No reference yet, the score is unpublished.
refs #38261
- src/hg/makeDb/trackDb/human/strVarNew.html
- lines changed 56, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- src/hg/makeDb/trackDb/human/strVarNew.ra
- lines changed 125, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- src/hg/makeDb/trackDb/human/strchiveNew.html
- lines changed 128, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- src/hg/makeDb/trackDb/human/trackDb.ra
- lines changed 7, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- src/hg/makeDb/trackDb/relatedTracks.ra
- lines changed 3, context: html, text, full: html, text
22fc293ddc00fce956d75fbcdb237e5d19d08fd5 Thu Sep 3 14:39:29 2026 -0700
Three more Imprinting subtracks on hg38: Geneimprint, Akbari iDMRs, OMIM
Geneimprint: the human catalog of imprinted and candidate imprinted genes
from geneimprint.com, 274 genes. The catalog gives only a cytogenetic band,
so symbols are resolved against the HGNC track, with fallbacks for small RNA
clusters and for symbols HGNC has retired.
Akbari iDMRs: the 192 imprinted differentially methylated regions that Akbari
et al. 2023 (PMID 36777186) compiled from five genome-wide studies. Their
published coordinates are 1-based inclusive, not bed, as their own PatMat
reader shows, so the starts are shifted.
OMIM: the 459 loci that OMIM curates as imprinted. That call is published only
through GeneScout and is in no OMIM download file, so the input is a GeneScout
export made by hand from a browser. OMIM phenotype entries, which are mapped
disease regions up to 90 Mb rather than gene positions, are left out.
Also unifies the color scheme across the collection, so vermillion always means
the maternal copy, blue the paternal copy and gray no parent of origin, and
relates the collection to the Human Methylation Atlas.
refs #37599
- src/hg/makeDb/trackDb/zebrafish/danRer11/danioCode.html
- lines changed 157, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 1, context: html, text, full: html, text
650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/danioCode.ra
- lines changed 10647, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dc3PseqComposite.html
- lines changed 117, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 1, context: html, text, full: html, text
650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
- lines changed 5, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcCAGEseqComposite.html
- lines changed 123, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 1, context: html, text, full: html, text
650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
- lines changed 5, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcChIPseqComposite.html
- lines changed 130, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 1, context: html, text, full: html, text
650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
- lines changed 5, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcComp.html
- lines changed 169, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 6, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcComp_cell_type.html
- lines changed 124, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 1, context: html, text, full: html, text
650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
- lines changed 5, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcComparativeGenomics.html
- lines changed 97, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 5, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcConsensus_promoters.html
- lines changed 115, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 1, context: html, text, full: html, text
650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
- lines changed 5, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcCopes_and_dopes.html
- lines changed 121, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 1, context: html, text, full: html, text
650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
- lines changed 5, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcEvalidation.html
- lines changed 114, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 1, context: html, text, full: html, text
650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
- lines changed 5, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcHiC_Composite.html
- lines changed 126, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 1, context: html, text, full: html, text
650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
- lines changed 5, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/dcRNAseqComposite.html
- lines changed 120, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- lines changed 1, context: html, text, full: html, text
650b51533a44058e6e9129448dc33eb947105a43 Sun Sep 6 06:55:18 2026 -0700
DANIO-CODE: remove leftover getTrackReferences error message from the References section of 10 track description pages. refs #38265
- lines changed 7, context: html, text, full: html, text
76eabae1c28bb07c02af6a12fa9222c348b039d9 Sun Sep 6 07:02:29 2026 -0700
DANIO-CODE: open each subtrack description with the sentence saying that the track is part of the DANIO-CODE container, linking to its hgTrackUi page, instead of burying it in a paragraph at the end of the description. refs #38265
- src/hg/makeDb/trackDb/zebrafish/danRer11/trackDb.ra
- lines changed 2, context: html, text, full: html, text
f5c96c14557e69252db6935d20ea55bdd250e519 Fri Sep 4 17:12:57 2026 -0700
DANIO-CODE as native danRer11 tracks, alpha only.
Converts the DANIO-CODE consortium's public track hub for danRer11 into a
native trackDb: 897 stanzas under one superTrack, with 11 containers for
RNA-seq, CAGE-seq, ChIP-seq, 3P-seq, Hi-C, regulatory elements, cell types,
COPEs/DOPEs, validated enhancers, conservation and consensus promoters.
The 879 data files, 69 GB, are mirrored under /gbdb/danRer11/danioCode and are
byte-identical to the consortium's copies. Four cell-type subtracks are left
out because their files 404 on the consortium's server.
refs #38265
- src/hg/near/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/hg/oneShot/alphaGenomeToWig/alphaGenomeToWig.c
- lines changed 227, context: html, text, full: html, text
8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
AlphaGenome Variant Impact (AVI) score track for hg38
New track under the Deleteriousness Predictions container showing Google
DeepMind's AlphaGenome Variant Impact score for every possible single-base
substitution, as four bigWigs, one per alternate allele.
alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per
alternate allele. It copies the PHRED column through as text so no precision
is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a
long contiguous run. The input has no row where alt == ref, so each per-allele
file is missing roughly every fourth position; without the fill the wig
fragments into blocks of about 3.5 values and the bigWig doubles in size.
All 8,812,917,339 input lines were used, none skipped.
The data cannot be redistributed, so the stanzas set tableBrowser off and the
description page sends users to the AlphaGenome Atlas. Alpha only for now.
No reference yet, the score is unpublished.
refs #38261
- src/hg/oneShot/alphaGenomeToWig/makefile
- lines changed 19, context: html, text, full: html, text
8e053a77fec3df57d9bfa44fafbb07c0cdadd1a7 Fri Sep 4 15:47:07 2026 -0700
AlphaGenome Variant Impact (AVI) score track for hg38
New track under the Deleteriousness Predictions container showing Google
DeepMind's AlphaGenome Variant Impact score for every possible single-base
substitution, as four bigWigs, one per alternate allele.
alphaGenomeToWig splits the 83 GB source table into one fixedStep wig per
alternate allele. It copies the PHRED column through as text so no precision
is lost, and zero-fills gaps up to -maxGap so a fixedStep block can span a
long contiguous run. The input has no row where alt == ref, so each per-allele
file is missing roughly every fourth position; without the fill the wig
fragments into blocks of about 3.5 values and the bigWig doubles in size.
All 8,812,917,339 input lines were used, none skipped.
The data cannot be redistributed, so the stanzas set tableBrowser off and the
description page sends users to the AlphaGenome Atlas. Alpha only for now.
No reference yet, the score is unpublished.
refs #38261
- src/hg/utils/hgConfCatalog/hgConfCatalog.py
- lines changed 20, context: html, text, full: html, text
4e816d84ac0d3079b74b96b0c827fa9d213eba2b Thu Sep 3 07:00:40 2026 -0700
hgConfCatalog: register login.recovEmailChange as a knob, refs #37925 #38197
- lines changed 6, context: html, text, full: html, text
5abe782bada3c851f1844c94b747cf7c769c71f9 Fri Sep 4 09:12:46 2026 -0700
cart.c: gate the captcha session-id debug print behind hg.conf captchaDebug, refs #38247
Register captchaDebug in the hg.conf catalog as a permanent debugging
knob rather than a release gate.
- lines changed 2, context: html, text, full: html, text
ea12fc3b55a64c2f8f4abdbd6e78f7c26db9c03d Fri Sep 4 11:53:47 2026 -0700
hgConfCatalog: fix stale snapshotReaper path in snapshot.ttlDays note, refs #38197
The auto-registered note for snapshot.ttlDays still pointed at
hg/utils/snapshotReaper/snapshotReaper.c after the snapshotReaper ->
snapshotCleaner rename. Update to hg/utils/snapshotCleaner/snapshotCleaner.c.
- src/hg/utils/hubCheck/hubCheck.c
- lines changed 4, context: html, text, full: html, text
571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260
hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands
each track to addTrackIfDataAccessible(), which frees any track whose data cannot be
reached. That is right for the browser, which should degrade gracefully rather than
draw a broken track, and wrong for hubCheck, whose main job is to say that the file
is unreachable. The track was gone before the file check ran, so hubCheck printed
nothing and exited 0:
cd hubCheck/tests/input
sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt
hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned
Split the non-pruning part of trackDbPolishAfterLinkup() into
trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller
still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour
is unchanged.
Added a tests/ case whose bigDataUrl does not exist. Before this change it produced
no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb".
Of the existing tests, only badType changes, and it gains a real error that the prune
had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was
already failing before this commit and still is, for an unrelated reason: seven tests
carry expected output with per-track "missing description page" warnings that no
longer appear, because those hubs point at remote URLs that no longer resolve and
hubCheckTrack aborts at the file check before reaching the description check. That is
stale test data, not addressed here.
- src/hg/utils/hubCheck/tests/expected/missingFile.output.txt
- lines changed 2, context: html, text, full: html, text
571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260
hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands
each track to addTrackIfDataAccessible(), which frees any track whose data cannot be
reached. That is right for the browser, which should degrade gracefully rather than
draw a broken track, and wrong for hubCheck, whose main job is to say that the file
is unreachable. The track was gone before the file check ran, so hubCheck printed
nothing and exited 0:
cd hubCheck/tests/input
sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt
hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned
Split the non-pruning part of trackDbPolishAfterLinkup() into
trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller
still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour
is unchanged.
Added a tests/ case whose bigDataUrl does not exist. Before this change it produced
no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb".
Of the existing tests, only badType changes, and it gains a real error that the prune
had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was
already failing before this commit and still is, for an unrelated reason: seven tests
carry expected output with per-track "missing description page" warnings that no
longer appear, because those hubs point at remote URLs that no longer resolve and
hubCheckTrack aborts at the file check before reaching the description check. That is
stale test data, not addressed here.
- src/hg/utils/hubCheck/tests/expected/relPath.output.txt
- lines changed 2, context: html, text, full: html, text
bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
hubCheck: stop resolving bigDataUrl a second time, refs #38256
trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which
rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer
relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against
genome->trackDbFile, which prepended the hub's directory a second time. For a local
hub reached by a relative path that turned a file which is present into a file which
is reported missing:
hubCheck out/hub.txt
Couldn't open out/out/hg19/x.bb
Remote hubs and local hubs given by an absolute path were unaffected, because the
second resolution is a no-op once the path is a URL or already absolute, which is why
this went unnoticed. The tests/makefile has been working around it: the
fieldCountMismatch case has to cd into input/ before it can find its bigBed.
Added a tests/ case that does not cd, with the data file in a subdirectory below
hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately
declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report
that mismatch if it really opened the file. Before this change the test reported
"Couldn't open input/relPath/input/relPath/data/test6field.bb".
Verified against a binary built from the unmodified tree: this changes the output of
no other test in hubCheck/tests. Seven of those tests were already failing before
this commit and still are.
- src/hg/utils/hubCheck/tests/input/relPath/data/test6field.bb
- lines changed 0, context: html, text, full: html, text
bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
hubCheck: stop resolving bigDataUrl a second time, refs #38256
trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which
rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer
relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against
genome->trackDbFile, which prepended the hub's directory a second time. For a local
hub reached by a relative path that turned a file which is present into a file which
is reported missing:
hubCheck out/hub.txt
Couldn't open out/out/hg19/x.bb
Remote hubs and local hubs given by an absolute path were unaffected, because the
second resolution is a no-op once the path is a URL or already absolute, which is why
this went unnoticed. The tests/makefile has been working around it: the
fieldCountMismatch case has to cd into input/ before it can find its bigBed.
Added a tests/ case that does not cd, with the data file in a subdirectory below
hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately
declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report
that mismatch if it really opened the file. Before this change the test reported
"Couldn't open input/relPath/input/relPath/data/test6field.bb".
Verified against a binary built from the unmodified tree: this changes the output of
no other test in hubCheck/tests. Seven of those tests were already failing before
this commit and still are.
- src/hg/utils/hubCheck/tests/input/relPath/hub.txt
- lines changed 20, context: html, text, full: html, text
bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
hubCheck: stop resolving bigDataUrl a second time, refs #38256
trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which
rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer
relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against
genome->trackDbFile, which prepended the hub's directory a second time. For a local
hub reached by a relative path that turned a file which is present into a file which
is reported missing:
hubCheck out/hub.txt
Couldn't open out/out/hg19/x.bb
Remote hubs and local hubs given by an absolute path were unaffected, because the
second resolution is a no-op once the path is a URL or already absolute, which is why
this went unnoticed. The tests/makefile has been working around it: the
fieldCountMismatch case has to cd into input/ before it can find its bigBed.
Added a tests/ case that does not cd, with the data file in a subdirectory below
hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately
declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report
that mismatch if it really opened the file. Before this change the test reported
"Couldn't open input/relPath/input/relPath/data/test6field.bb".
Verified against a binary built from the unmodified tree: this changes the output of
no other test in hubCheck/tests. Seven of those tests were already failing before
this commit and still are.
- src/hg/utils/hubCheck/tests/input/relPath/hubDescription.html
- lines changed 2, context: html, text, full: html, text
bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
hubCheck: stop resolving bigDataUrl a second time, refs #38256
trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which
rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer
relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against
genome->trackDbFile, which prepended the hub's directory a second time. For a local
hub reached by a relative path that turned a file which is present into a file which
is reported missing:
hubCheck out/hub.txt
Couldn't open out/out/hg19/x.bb
Remote hubs and local hubs given by an absolute path were unaffected, because the
second resolution is a no-op once the path is a URL or already absolute, which is why
this went unnoticed. The tests/makefile has been working around it: the
fieldCountMismatch case has to cd into input/ before it can find its bigBed.
Added a tests/ case that does not cd, with the data file in a subdirectory below
hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately
declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report
that mismatch if it really opened the file. Before this change the test reported
"Couldn't open input/relPath/input/relPath/data/test6field.bb".
Verified against a binary built from the unmodified tree: this changes the output of
no other test in hubCheck/tests. Seven of those tests were already failing before
this commit and still are.
- src/hg/utils/hubCheck/tests/input/relPath/missingFile.txt
- lines changed 18, context: html, text, full: html, text
571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260
hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands
each track to addTrackIfDataAccessible(), which frees any track whose data cannot be
reached. That is right for the browser, which should degrade gracefully rather than
draw a broken track, and wrong for hubCheck, whose main job is to say that the file
is unreachable. The track was gone before the file check ran, so hubCheck printed
nothing and exited 0:
cd hubCheck/tests/input
sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt
hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned
Split the non-pruning part of trackDbPolishAfterLinkup() into
trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller
still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour
is unchanged.
Added a tests/ case whose bigDataUrl does not exist. Before this change it produced
no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb".
Of the existing tests, only badType changes, and it gains a real error that the prune
had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was
already failing before this commit and still is, for an unrelated reason: seven tests
carry expected output with per-track "missing description page" warnings that no
longer appear, because those hubs point at remote URLs that no longer resolve and
hubCheckTrack aborts at the file check before reaching the description check. That is
stale test data, not addressed here.
- src/hg/utils/hubCheck/tests/makefile
- lines changed 10, context: html, text, full: html, text
bf60df82401b1c11eb4cc0d9f67e76a0f0749849 Fri Sep 4 12:02:30 2026 -0700
hubCheck: stop resolving bigDataUrl a second time, refs #38256
trackHubTracksForGenome() already runs expandBigDataUrl() over every stanza, which
rewrites bigDataUrl and bigDataIndex in the settings hash so they are no longer
relative to the trackDb file. hubCheckBigDataUrl() then resolved them again against
genome->trackDbFile, which prepended the hub's directory a second time. For a local
hub reached by a relative path that turned a file which is present into a file which
is reported missing:
hubCheck out/hub.txt
Couldn't open out/out/hg19/x.bb
Remote hubs and local hubs given by an absolute path were unaffected, because the
second resolution is a no-op once the path is a URL or already absolute, which is why
this went unnoticed. The tests/makefile has been working around it: the
fieldCountMismatch case has to cd into input/ before it can find its bigBed.
Added a tests/ case that does not cd, with the data file in a subdirectory below
hub.txt, the shape 'hubtools import session' now writes. Its bigBed is deliberately
declared 'type bigBed 9' when it holds 6 fields, because hubCheck can only report
that mismatch if it really opened the file. Before this change the test reported
"Couldn't open input/relPath/input/relPath/data/test6field.bb".
Verified against a binary built from the unmodified tree: this changes the output of
no other test in hubCheck/tests. Seven of those tests were already failing before
this commit and still are.
- lines changed 7, context: html, text, full: html, text
571aecdf42c832e61ee921e977589ca386467d13 Fri Sep 4 12:24:17 2026 -0700
hubCheck: report a bigDataUrl that cannot be opened instead of dropping the track, refs #38260
hubCheckGenome() called trackDbPolishAfterLinkup(), whose pruneEmpties() step hands
each track to addTrackIfDataAccessible(), which frees any track whose data cannot be
reached. That is right for the browser, which should degrade gracefully rather than
draw a broken track, and wrong for hubCheck, whose main job is to say that the file
is unreachable. The track was gone before the file check ran, so hubCheck printed
nothing and exited 0:
cd hubCheck/tests/input
sed 's|test6field.bb|noSuchFile.bb|' fieldCountMismatch.txt > tmp.txt
hubCheck tmp.txt # only the descriptionUrl warning, missing file not mentioned
Split the non-pruning part of trackDbPolishAfterLinkup() into
trackDbPolishAfterLinkupKeepAll() and call that from hubCheck. Every other caller
still goes through trackDbPolishAfterLinkup() and still prunes, so browser behaviour
is unchanged.
Added a tests/ case whose bigDataUrl does not exist. Before this change it produced
no output and exit 0; now it reports "Couldn't open input/relPath/data/noSuchFile.bb".
Of the existing tests, only badType changes, and it gains a real error that the prune
had been hiding: 'unrecognized type "wig" for track "invalidType"'. badType was
already failing before this commit and still is, for an unrelated reason: seven tests
carry expected output with per-track "missing description page" warnings that no
longer appear, because those hubs point at remote URLs that no longer resolve and
hubCheckTrack aborts at the file check before reaching the description check. That is
stale test data, not addressed here.
- src/hg/utils/makefile
- lines changed 1, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- lines changed 1, context: html, text, full: html, text
89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
snapshotSession: rename the snapshot "reaper" to "cleaner"
Renames the snapshotReaper utility to snapshotCleaner and drops the word
"reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and
the doc/usage text now say clean/cleaned/cleaner). No behavior change.
refs #38197
- src/hg/utils/otto/otto.crontab
- lines changed 5, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- src/hg/utils/otto/strchive/README.md
- lines changed 92, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- src/hg/utils/otto/strchive/makefile
- lines changed 18, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- src/hg/utils/otto/strchive/strchiveAutoPush
- lines changed 53, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- lines changed 7, context: html, text, full: html, text
4274e5376b9d13e83e67cfe1bce8a973643f0afa Sun Sep 6 07:44:02 2026 -0700
strchiveAutoPush: report and return the real rsync exit code
The failure branch read $? after two echo commands had already replaced it, so
the log said "code 0" whatever rsync did and the script exited 0 on a failed
push. Save the status right after rsync and use it in both messages and the
exit. The script is still the uninstalled draft for the sysadmins.
refs #38268
- src/hg/utils/otto/strchive/strchiveOtto.py
- lines changed 234, context: html, text, full: html, text
0871110b9e07cfb8001e6de9d156e7e7f74f8c25 Fri Sep 4 17:00:53 2026 -0700
STRchive track now updates itself, and is on hg19 and hs1 as well as hg38
STRchive builds a Genome Browser bigBed with every release now and publishes
it on their GitHub releases page, so there is nothing left for us to convert.
A new otto job checks weekly for a new release, downloads the file into its
own releases/<tag> directory, sanity-checks the locus count, and repoints the
/gbdb symlinks at it. Silent when there is no new release. See STRchive issue
333 for the correspondence that set this up.
Their releases carry hg19, hg38 and T2T-chm13 (= hs1) files and we take all
three, so the new strVar stanza lives in human/strVarNew.ra rather than under
hg38. The bigDataUrls use $D, and hgTrackDb -strict drops any member whose
file is missing, so each assembly gets only the tracks that were built for it:
STRchive on all three, the other four on hg38 only. The description page is
rewritten for the new evidence-level coloring, gains an evidence filter, and
now says that the track updates itself and that older versions are on GitHub
rather than here.
All of that is alpha only. Nothing copies /gbdb/*/strVar/ out to the RR yet,
so releasing this trackDb would have it promise data the RR does not have.
The old hg38-only version stays as human/hg38/strVar.ra, tagged beta,public,
until the sysadmins install the strchiveAutoPush cron drafted alongside the
otto job. The header of strVarNew.ra lists what to delete and rename then.
refs #38268
- src/hg/utils/snapshotCleaner/makefile
- lines changed 1, context: html, text, full: html, text
89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
snapshotSession: rename the snapshot "reaper" to "cleaner"
Renames the snapshotReaper utility to snapshotCleaner and drops the word
"reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and
the doc/usage text now say clean/cleaned/cleaner). No behavior change.
refs #38197
- src/hg/utils/snapshotCleaner/snapshotCleaner.c
- lines changed 10, context: html, text, full: html, text
89d738beff2c63765e2b6b4bc89412c621246b33 Thu Sep 3 12:25:02 2026 -0700
snapshotSession: rename the snapshot "reaper" to "cleaner"
Renames the snapshotReaper utility to snapshotCleaner and drops the word
"reap" from the API and comments (snapshotReapAnon -> snapshotCleanAnon, and
the doc/usage text now say clean/cleaned/cleaner). No behavior change.
refs #38197
- src/hg/utils/snapshotReaper/makefile
- lines changed 3, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- src/hg/utils/snapshotReaper/snapshotReaper.c
- lines changed 60, context: html, text, full: html, text
32cd2100d9dd11140705c848850e86c4f79191c4 Tue Sep 1 07:02:04 2026 -0700
Reusable "view snapshot" sessions for durable, minimal Share-a-link links
"Share a link" could only share the whole cart: it saved a session holding
every track, setting and position, which bloated the central db and leaked
the sharer's unrelated tracks to whoever opened the link. Anonymous share
sessions (under the reserved user "l") were also never reaped, so they
accumulated forever, and their 8-char names were generated client-side with
no uniqueness check, so two shares could collide and silently overwrite.
Adds a lightweight "snapshot session" facility (lib/snapshotSession.c): a
snapshot stores only the handful of cart variables a feature declares (a
registered snapshotType, e.g. "blat" -> {db, blatLastBigBed}), moves only
those variables' trash files into durable sessionData storage, and is saved
under a "__"-prefixed name. The "__" marks it machine-made: hidden from the
My Sessions list by default and eligible for reaping. For the anonymous "l"
owner the durable files fan out over two extra hash levels so one directory
never fills with millions of entries.
Every anonymous link now shares one server-side name generator
(snapshotNewName): a unique (db-checked), crypto-strong, "__"-prefixed token,
so tokens never collide. The hgSession doSaveSessionJson endpoint gained
hgS_snapshotType (save a minimal snapshot rather than the whole cart) and
hgS_doAnonName (reserve a unique anonymous name without saving, so the
top-right dialog can preview the exact link before it is created).
Wired three callers to the facility:
- hgc htcBlatAlign "Share a link": a minimal "blat" snapshot instead of a
full-cart anonymous session.
- hgBlat results "Share a link": creates a "blat" snapshot on click and
reveals its ?u=&s= reopen link (rebuilt from the durable bigPsl by the
existing doShareReopen), replacing the trash-only reveal.
- top-right "Share a link": anonymous links use the reserved server name;
logged-in named shares are unchanged.
snapshotReaper (hg/utils) garbage-collects abandoned anonymous snapshots: it
deletes user "l" "__" rows whose lastUse is older than the TTL (hg.conf
snapshot.ttlDays, default ~4 years) and removes their durable files. lastUse
is bumped on every open by the existing session load, so a link stays alive
as long as it is used. Meant to run from the trash-cleaner cron.
Also folds in the recent Share-dialog work in these files: the auto share
name is a short "_" prefix instead of "share_", the dialog previews the link
and creates it only when the button is clicked (no orphan session just from
opening the dialog), an optional name field with an overwrite warning, and
the anonymous save path forces the reap-eligible "__" name.
refs #38197
- src/hg/visiGene/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/isPcr/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/jkOwnLib/LICENSE
- lines changed 1, context: html, text, full: html, text
50ce78a22e9684817d8e5519e372ef2c313d343e Fri Sep 4 10:54:48 2026 -0700
Fix licensing page ambiguities and LICENSE tag spelling, refs #37781
The license/index.html page had several passages easy to misread: a
claim that liftOver was the "only" data exception directly contradicted
by the clinical-genetics exceptions listed two paragraphs later, a
header/body tension in the graphics-reuse section, a garbled sentence
about entity type, a missing "than", and two same-named "Software:"
headings covering opposite licensing outcomes (free vs. commercial-restricted).
Also fixes the SPDX tag referenced on that page: the LICENSE files in
restricted directories used the misspelled "LicenceRef-proprietary"
rather than the standard "LicenseRef-proprietary", so grepping the tree
for the tag as documented turned up almost nothing.
Adds a link to the (currently sandboxed) list of tracks we cannot
distribute, next to the clinical-genetics exception examples.
- src/lib/filePath.c
- lines changed 1, context: html, text, full: html, text
21123cb5a9a1542b7ff01542f29078adc04764a0 Sun Sep 6 07:31:34 2026 -0700
lib: check every component in isSafeRelativePath
The loop that walks the '/'-separated components stepped to the terminator that
replaced the slash instead of past it, so it ended after the first component and
the rest of the path went unchecked. Callers are gfServer's dynamic mode, which
takes genome and genomeDataDir from the client.
- src/utils/hubtools/hubtools
- lines changed 230, context: html, text, full: html, text
82747d60dcdd8207c9d3324fe6e8a1ba3417fd28 Thu Sep 3 12:17:58 2026 -0700
hubtools import session: one hub.txt per session, and output that passes hubCheck, refs #34405
Converting Ian Donaldson's hg38 session (30 custom tracks) turned up five
problems, all fixed here:
- Custom track lines often say autoScale=OFF, and hubCheck rejects the
uppercase spelling. The on/off settings are now lower-cased on the way out,
which was 14 of the 15 problems hubCheck reported for that session.
- hub.txt was written to outDir/<db>/hub.txt, one hub per assembly, so a
session covering two assemblies needed two hubUrls. There is now a single
outDir/hub.txt with one genome stanza per assembly and the data files in
outDir/<db>/. Two consequences of that had to be handled: track names are
not scoped per genome stanza, so the name counter runs across assemblies
now; and stanzaKey() returned ".genome" for every genome stanza, which made
'tdb add' on such a hub silently drop all but the last assembly.
- The tool said nothing about where the hub went or how to load it. It now
prints the path, the track and assembly counts, the hubUrl to open and the
hubCheck command.
- The hub was labelled "Auto-generated hub". A short session link redirects
to a URL that names the session and its owner, so the labels come from the
session itself. A local archive is named after the file. An email= line in
~/.hubtools.conf sets the contact address. The 'hub' key in tracks.json was
documented but ignored; it works now.
- Added the missing hubDescription.html, with a note saying where the hub came
from, which clears the last hubCheck warning.
Also: outDir/<db>/ is only created when a file goes into it, and a session
with no custom tracks aborts instead of writing an empty hub.
switch to commits view, user index