Commits for max

v504_preview to v504_preview2 (2026-09-07 to 2026-09-14) v504

5515006ee81e5de99ebd03581644a8a96f33599e Mon Sep 7 07:36:11 2026 -0700

• tiny text change on trackUi page, no ticket
  • src/hg/hgTrackUi/hgTrackUi.c - lines changed 18, context: html, text, full: html, text

d83026b7d8919bc8ad96679a98234aeb1308675e Mon Sep 7 10:55:13 2026 -0700

• hgTracks: say in the track label why a track is showing item density The density-mode labels existed but were only reachable through labelTrackAsFilteredNumber(), which every caller guards with if (filtered), so the note only appeared when a filter had also dropped features and the automatic cases never said anything. labelTrackAsDensityIfActive() now picks a message by cause: one for density the user asked for on the configuration page, one for a window wider than maxWindowCoverage, and one for the three paths that set limitWiggle because there are too many features to draw. It is called once from makeActiveImage() after every path into density mode has settled, over the tracks that will actually be drawn, so hidden tracks keep their plain label. refs #38279
  • src/hg/hgTracks/hgTracks.c - lines changed 38, context: html, text, full: html, text
  • src/hg/hgTracks/hgTracks.h - lines changed 9, context: html, text, full: html, text

81bd65b4c221b9288848d61f75b3c69197e9d7fd Mon Sep 7 11:28:21 2026 -0700

• removing alpha genome text temporarily as per google group refs #38261
  • src/hg/makeDb/trackDb/human/predictionScoresSuper.html - lines changed 2, context: html, text, full: html, text

08228c9a2d84f3d0a30474f1f24598dbb293dfa5 Mon Sep 7 11:29:09 2026 -0700

• cdwWebBrowse: do not build 'id IN ()' when no file matches the filters When a download filter matched no files, findDownloadableFiles() built SELECT * FROM cdwFile WHERE id IN () which MariaDB rejects with error 1064. Return an empty list instead; both callers already handle it. Found in the apache error log, email from Erich.
  • src/hg/cirm/cdw/cdwWebBrowse/cdwWebBrowse.c - lines changed 4, context: html, text, full: html, text

c94e417525c00e37443c212e0ded342ef7809e56 Mon Sep 7 12:07:18 2026 -0700

• hgTrackUi: add "Hide all tracks" and "Show all tracks" buttons to the superTrack page, and drop "Apply to all visible tracks". The two new buttons cover the two things people actually do on a container page. "Show all tracks" asks for pack and falls back to full for tracks that have no pack, i.e. signal tracks, so a container of bigWigs comes up in full. The old "Apply to all visible tracks" button needed two sentences of help text to distinguish it from "Apply to all tracks" and is gone; the dropdown plus "Apply to all tracks" still covers dense and squish. refs #38281
  • src/hg/js/hui.js - lines changed 7, context: html, text, full: html, text

c0e8fa6df3a0bd406c4188d49ee00f20aef203e5 Mon Sep 7 12:07:18 2026 -0700

• Substitute trackDb variables in hub track description pages A hub's description page comes straight off the hub's web server and has never been through variable substitution, so a $db or $parentTrack in it reached the reader as literal text. Native trackDb pages are fine, since hgTrackDb substitutes them when it loads trackDb, but there was no equivalent step for a hub. hgc's getTrackHtml and hgTrackUi's trackUi both call hVarSubstTrackDbHtml on a hub track's html. Only a short list of variables is recognized there and nothing is an error, because a hub page written before this existed can easily contain a dollar sign inside a shell example, and silently rewriting that would be worse than not substituting at all. Adds $parentTrack, the name of the container a track sits in, which is what a subtrack description page needs to link back to its superTrack or composite. Views are skipped, since a view has no page of its own, and the hub_<id>_ prefix is kept so the name works as hgTrackUi's g= parameter. Documents $track, $parentTrack and $hgsid in trackDb/README. refs #37599
  • src/hg/hgTrackUi/hgTrackUi.c - lines changed 4, context: html, text, full: html, text
  • src/hg/hgc/hgc.c - lines changed 6, context: html, text, full: html, text
  • src/hg/inc/hVarSubst.h - lines changed 9, context: html, text, full: html, text
  • src/hg/lib/hVarSubst.c - lines changed 104, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/README - lines changed 11, context: html, text, full: html, text

9c620cab64ca86a0044de2c4a7c610b13332558e Mon Sep 7 19:43:50 2026 -0700

• trackDb/README: document the braced ${name} form only The bare $name form still works and old pages use it, but braces are the form worth writing. Inside braces any character is allowed up to the closing brace, so ${name} can carry a structured name like ${hgTrackUi/caddSuper} if we ever want one, while $name stops at the first character outside [0-9A-Za-Z_] and cannot express it. Braces also settle the adjacency case: ${db}Something is unambiguous, $dbSomething reads as one long name. Every variable in the list is now written braced, and the closing paragraph says not to write or document the bare form in anything new. Also spells out why ${hgsid} is a hub page variable in practice. Native trackDb html is substituted once by hgTrackDb when it loads trackDb, and that happens without a cart, so there is no session id to put there. Only hgc and hgTrackUi substitute with a cart, at render time. One non-variable change: 'an $otherDb field' becomes 'an otherDb field'. That is the name of a .ra setting rather than a substitution, and leaving a dollar on it next to a braces-only rule would only confuse. refs #38283
  • src/hg/makeDb/trackDb/README - lines changed 38, context: html, text, full: html, text

c3322bc2d3f94a59721989762c58db1734ec7740 Mon Sep 7 20:04:43 2026 -0700

• hgTracks: show the cDNA and codon range of an exon in its mouseover instead of "Zoom in to show cDNA position" At gene-level zoom the exon popup only said "Codons: Zoom in to show cDNA position", so the only way to find a c. or p. position was to zoom into one exon after another. The popup now gives the exon's HGVS c. range and the codons it spans, e.g. "Codons: c.1364-1482 (p.455-494)", plus the c.-N / c.*N range of any UTR part of the exon. The numbers agree with the per-codon popups shown when zoomed in. Non-coding transcripts already got an n. range but only at codon-level zoom, where the number is of little use; that gate is gone, so they are labelled at every zoom too. Chain and LRG tracks have no cDNA coordinates and are left alone. refs #38278
  • src/hg/hgTracks/simpleTracks.c - lines changed 76, context: html, text, full: html, text

fa5b31d305066e1938953374d80b3782ef87e239 Mon Sep 7 23:23:40 2026 -0700

• Position box: accept a bare codon number, and a range of codon numbers "KAT6A p.495_533" used to land on codon 495 and silently drop the end of the range, and a bare codon number after a transcript accession was not understood at all, so "ENST00000265713.8 p.495" fell through the HGVS code and ended up on an unrelated locus. Nucleotide ranges already worked. The pseudo-HGVS layer now takes an optional _end on a bare codon number, and accepts a bare codon number or range after an NM_ or ENST accession as well as after a gene symbol, looking up the reference amino acids that HGVS wants and the user did not type. The accession forms require a literal "p", so "NM_006766.5 1483" keeps meaning what it meant. A hyphen is still not a range separator: c.1483-1599 is the HGVS intronic position and stays that way. Also fixes a read past the end of the protein sequence when the codon number was larger than the protein, and documents codon ranges in query.html. refs #38285
  • src/hg/htdocs/goldenPath/help/query.html - lines changed 23, context: html, text, full: html, text
  • src/hg/lib/hgHgvs.c - lines changed 102, context: html, text, full: html, text
  • src/hg/lib/tests/expected/hgvs/validTerms.txt - lines changed 10, context: html, text, full: html, text
  • src/hg/lib/tests/input/hgvs/validTerms.txt - lines changed 10, context: html, text, full: html, text

0f23d17640ca30e2c9ee456c7e15966cabd3bc57 Mon Sep 7 23:32:24 2026 -0700

• Position box: let a hyphen separate a range of codons, e.g. "BRCA1 100-200" A bare number after a gene symbol has always meant a codon, and "KAT6A 495-533" was already accepted -- it just landed on codon 495 and dropped the rest, the same silent truncation that the underscore form had. A hyphen now separates a range wherever the coordinates are protein: after a gene symbol with no prefix, and after an explicit p. with a symbol or a transcript accession. The hyphen stays out of c. and n. terms, where HGVS already uses it for an intron offset. KAT6A c.1483-1599 is still the single base 1599 nt before c.1483, not codons 1483 to 1599, and there are now regression tests pinning both readings so the two do not drift into each other. refs #38285
  • src/hg/htdocs/goldenPath/help/query.html - lines changed 20, context: html, text, full: html, text
  • src/hg/lib/hgHgvs.c - lines changed 6, context: html, text, full: html, text
  • src/hg/lib/tests/expected/hgvs/validTerms.txt - lines changed 11, context: html, text, full: html, text
  • src/hg/lib/tests/input/hgvs/validTerms.txt - lines changed 11, context: html, text, full: html, text

e955e2f314ad9ca7fddad3fddc1f3c068fddd45e Tue Sep 8 00:20:03 2026 -0700

• hgTracks: fix punctuation inconsistency in density-mode label message Use a comma instead of a dash before 'zoom in' to match the style of the neighboring too-many-items density label message.
  • src/hg/hgTracks/hgTracks.c - lines changed 1, context: html, text, full: html, text

96a903979dbe723e4f20a700dfc12881063b2c46 Tue Sep 8 00:26:18 2026 -0700

• hgTracks: support mouseOver on bigNarrowPeak tracks, and register the peak filter tags bigNarrowPeakLoadItems() had its own load loop and never looked at the mouseOver setting, so a bigNarrowPeak track silently ignored it. It now uses the mouseOverSetupForBbi() / mouseOverGetBbiText() helpers in mouseOver.c, which also gets mouseOverField support for free. As with every other bigBed-like track, the text only shows in pack or full, since dense makes no per-item map boxes. tagTypes.tab did not list bigNarrowPeak for mouseOver, scoreFilter, scoreFilterLimits, scoreMin, scoreMax, signalFilter or signalFilterLimits, so tdbQuery -strict rejected all of them even though the code reads them. pValueFilter and qValueFilter, with their Limits, were not registered for any type at all, although encodePeakCfgUi() in hui.c has always drawn them and bigNarrowPeakLoadItems() has always applied them. Added. refs #36210
  • src/hg/hgTracks/encode.c - lines changed 10, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/tagTypes.tab - lines changed 11, context: html, text, full: html, text

ce780dd2f1216ce728ab6bb69ac19a39ddc694fd Tue Sep 8 00:26:39 2026 -0700

• hg38: Fiber-seq container with accessibility, FIRE peaks and CpG methylation, 41 samples Native version of the Stergachis/Vollger lab hub at https://fiberseq.github.io/UCSC-Fiber-seq-hub/hub.txt, plus the per-sample CpG methylation Shane Neph asked to have alongside it. Both cover the same 41 samples: 14 cell lines and 27 lymphoblastoid lines from HPRC and GIAB individuals. fiberSeq container, group regulation fiberSeqAcc multiWig overlay of 7 common cell lines, on by default fiberSeqCompendium faceted composite, dataTypes acc/peaks/hap fiberSeqMeth faceted composite, dataTypes comb/hap/diffs, "Methylation" Both composites use the Methbase faceted-composite machinery. Subtracks are named <composite>_<accession>_<dataType> with the accession as the only middle component, because facetedCompositeUi() cuts the data element at the first underscore and cartDump.c reassembles the name from the pieces; the hub's <composite>_<sample>_<accession>_<type> names would have resolved to tracks that do not exist. Sample name and cell type live in the metadata TSV instead. Using dataTypes also brings onlyVisibility, which is what lets the peaks default to dense while the signal tracks default to full, the mixed-visibility default Andrew Stergachis asked for. 397 GB mirrored from the UW Kopah S3 server rather than pointed at over the network, since a native track should not depend on it. The FIRE peak bigBeds had to be rebuilt: they carry full narrowPeak data but their header records a field count of 3, which hides signalValue and qValue from the browser and would have made hgTracks errAbort in bigNarrowPeakLoadItems(). The rebuild fixes the header and rounds the two float columns to 3 decimals, 467 MB to 313 MB. It drops 421 of 9,487,043 peaks called on chrEBV, the EBV decoy of the GRCh38 analysis set, which hg38 does not have; 9,486,622 remain and every sample reconciles exactly. Reported upstream, along with GM12878's two haplotype accessibility bigWigs, which are one-base placeholders at the source. refs #36210
  • src/hg/makeDb/doc/hg38/fiberSeq.txt - lines changed 258, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqCheck.sh - lines changed 72, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqDownload.sh - lines changed 90, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqFixPeaks.sh - lines changed 108, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqSamples.tsv - lines changed 42, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqTrackDb.py - lines changed 404, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeq.html - lines changed 89, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeq.ra - lines changed 6267, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeqAcc.html - lines changed 122, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeqCompendium.html - lines changed 185, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeqMeth.html - lines changed 154, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/trackDb.ra - lines changed 2, context: html, text, full: html, text

fa8084416354441c8c817069b7f92344f833d8dc Tue Sep 8 01:13:43 2026 -0700

• hprc2annot: say in the pcLAI docs that uniform color is the expected case Two of us in a row zoomed a pcLAI track to a few megabases, saw a single flat color, and concluded the itemRgb was broken. It is not: pclai.bb for GCA_046629565.1 holds 258 distinct RGB values, but 25,416 of its 25,438 windows sit in one tight cluster in PCA space (PC1 ~0.40-0.44) and so map to one tight cluster in color space, R 203-255 G 149-164 B 255, which is a couple of perceptual steps wide. The color column is a byte-for-byte pass-through of the HPRC source BED, verified against the S3 file. Two things the page did not say. A haplotype with one ancestry throughout is uniform across every chromosome and that is the correct render. And where a haplotype does carry several ancestries, the blocks are tens of megabases long, so any view narrower than a chromosome tends to land inside one block and look uniform too. Add an example so this is checkable rather than asserted: GCA_018466835.2 (HG02257) has four segment-level ancestry calls, and its chr17 crosses four blocks with transitions near 10.0, 50.7 and 73.3 Mb. Name the three chromosomes in that same assembly, chr2 chr13 chr18, that are single-ancestry end to end, since chr2 is the one that started this. refs #35415
  • src/hg/makeDb/trackDb/contrib/hprc2annot/pclai.html - lines changed 21, context: html, text, full: html, text

b678946c7d246ae01d552242ac0296b8694d5fc3 Tue Sep 8 06:15:43 2026 -0700

• hgTrackUi: faceted composite says "Samples" when data types are on, and stops paging a short table The two filter tabs were hardcoded to "All Tracks" and "Active Tracks", which is wrong for a composite that uses dataTypes: there a row is a sample, standing for as many tracks as there are active data types. The file already had an itemLabel that resolves to "samples" or "tracks" and was feeding the DataTables strings, so the tabs now use the same variable. A composite without dataTypes reads exactly as before. Page length was a flat 25, so a table of 41 samples hid a third of itself behind a pager for no good reason. Tables under 50 rows now start out showing everything; the length menu still offers 10/25/50/100/All for bigger ones. refs #36210
  • src/hg/js/facetedComposite.js - lines changed 11, context: html, text, full: html, text

01bc05ac9a282a6862111502f13601e513d5b60b Tue Sep 8 06:16:04 2026 -0700

• hg38 Fiber-seq: merge the methylation composite into the compendium The accessibility compendium and the separate Methylation composite covered the identical 41 samples, and cartDump.c assigns priority with the data element as the outer loop and the data type as the inner one. So one composite keeps a sample's six subtracks contiguous in the image, where two composites drew an accessibility block followed by a methylation block and comparing the two assays for one sample meant reading past every other sample. Both come off the same molecules in the same experiment, so side by side is the point. fiberSeqMeth is gone and its three data types moved in as cpg, cpgHap and cpgDiff, renamed because "hap" was already taken by the accessibility overlay and a data type name cannot contain an underscore. Subtracks now carry an explicit priority, sample outer and declared data type inner. Without one they fell back to a label sort, so a first visit showed a sample's data types as Peaks, CpG, Acc rather than in the order of the checkbox row above the table. Metadata columns renamed from camelCase to Accession, Sample_class, _Cell_type and _Sample. toTitleStyle() in facetedComposite.js renders an underscore as a space but does not split camelCase, so "sampleClass" appeared verbatim as a column heading. A literal space cannot be used instead: the saved sort order is a space-separated list of column names and the submit code drops any name containing whitespace, which would have made sorting silently fail to persist. Cell type is no longer faceted. A facet value is only offered when it occurs more than once, and 12 of the 14 cell types here are a single sample, so as a facet it drew two checkboxes and left 12 samples unreachable by any cell-type filter. It is a searchable column now, and Sample_class is the only facet until the lab gives us real HPRC metadata that would facet properly. Description page intro rewritten, and it now says the assay measures the same property as DNase-seq and ATAC-seq. refs #36210
  • src/hg/makeDb/doc/hg38/fiberSeq.txt - lines changed 42, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqTrackDb.py - lines changed 197, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeq.html - lines changed 25, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeq.ra - lines changed 7205, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeqCompendium.html - lines changed 106, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeqMeth.html - lines changed 154, context: html, text, full: html, text

679b48855a32f8ea1e1ea8ca3c095a0c2c7f6c7a Tue Sep 8 10:08:41 2026 -0700

• detailsScript: add a scatterPlot plot type, and use it for pcLAI Clicking a pcLAI window now shows where that window sits in the ancestry space it was placed in: a scatterplot of the 1000 Genomes reference haplotypes with the window's own PCA coordinate and its segment's coordinate marked on it. The numbers were already on the details page and told a reader almost nothing. New plot type scatterPlot (hg/js/hgc.scatterPlot.js), driven the same way as histogram. Background points come from a JSON or TSV file named by dataUrl and may carry a category, which colors them and builds a legend, and a label, which is shown on mouseover. The cloud is drawn on a canvas, since these files hold thousands of points and that many <circle> elements make the page crawl; axes and the highlighted points stay SVG on top. Point lookup for the mouseover goes through a cell index so a large file stays smooth. Two additions serve every plot type, not just this one: - exportFields, a config key listing further bigBed fields whose values are passed to the module as a fieldValues object. Without it a plot needing two coordinates would need them packed into one field, and pcLAI keeps them in pca and pcaSegment. Only fields that exist in the bigBed are exported, at most 32, and the JSON types are checked rather than asserted because jsonListVal and jsonStringVal errAbort and this JSON is written by a hub. - a config key ending in Url is treated as a file, by the convention trackSettingIsFile() already uses, and a relative one is resolved against the track's own bigDataUrl. The module does not fetch it directly; it asks hgTrackUi for it, the route facetedComposite uses for its metadata. That checks the canonicalized path against the hubs on the cart and reads it with udc, so a hub-relative path works even for a hub loaded from a local path (the GenArk /gbdb hubs), no CORS header is needed, and a file outside a connected hub cannot be read. Verified that /etc/passwd, file://, a dot-dot escape, an unattached hub and an unrelated host are all refused with 400. When the session has file caching off, hgc now exports udcTimeout the way hgTrackUi does and the module POSTs, so the browser cannot answer from cache. Fixes a crash reachable from any hub: "detailsScript.<plotType>.<field> null" segfaulted hgc, because jsonObjectVal returns NULL for a JSON null and the hash routines dereference it. This hit the shipped histogram type too. trackDbSettingsGen.py stopped reading a setting's description at the first "Example:" paragraph and never read <ul> at all, so it dropped everything after the first example and every list item. That silently truncated 226 of the 264 descriptions, including spectrum's minGrayLevel/scoreMin/scoreMax bullets, and would have dropped this whole scatterPlot section. It now skips the Example label instead of stopping, and folds list items in. No setting loses a word and none gains or loses an example. pcLAI wiring: the background file is the authors' published reference panel (github.com/AI-sandbox/hprc-pclai reference_pca_metadata.tsv), converted by hprc2annotMakePclaiRefPanel.py -- 3122 haplotypes, 21 populations, 94 KB, one file for the collection since it is the reference space rather than per-assembly data. The four values pcaSegment takes across all 460 assemblies turn out to be the four continental cluster centres, so the highlighted segment dot always lands on one of them. genark: addContrib now rewrites a "...Url" inside a detailsScript value the same way it rewrites bigDataUrl, and symlinks the collection's shared root-level data files next to the docs, so contrib/<name>/<file> resolves in the deeper GenArk layout. It writes the alpha tier only, leaving the assembly's default hub alone, and clears any unmarked copy of the collection's stanzas that the assembly build baked in, which would otherwise leave the hub declaring each track twice. refs #35415
  • src/hg/hgc/bigBedClick.c - lines changed 74, context: html, text, full: html, text
  • src/hg/hgc/hgc.h - lines changed 8, context: html, text, full: html, text
  • src/hg/htdocs/goldenPath/help/trackDb/changes.html - lines changed 13, context: html, text, full: html, text
  • src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml - lines changed 83, context: html, text, full: html, text
  • src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.json - lines changed 2, context: html, text, full: html, text
  • src/hg/htdocs/goldenPath/help/trackDb/trackDbSettings.yaml - lines changed 1349, context: html, text, full: html, text
  • src/hg/htdocs/goldenPath/help/trackDb/trackDbSettingsGen.py - lines changed 23, context: html, text, full: html, text
  • src/hg/js/hgc.scatterPlot.js - lines changed 567, context: html, text, full: html, text
  • src/hg/js/makefile - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/scripts/hprc2annot/hprc2annotMakePclaiRefPanel.py - lines changed 81, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt - lines changed 1, context: html, text, full: html, text
  • src/utils/genark/genark - lines changed 78, context: html, text, full: html, text

905b9cb05eeaca7f2dcda42fc6abdb95a2d2da7f Wed Sep 9 05:29:28 2026 -0700

• no captcha for a command-line CGI run, and version the detailsScript module URL Two small fixes to things noticed while adding the scatterPlot plot type. A CGI run from the command line got the Cloudflare Turnstile challenge page instead of the output the caller asked for, which makes "./hgc db=hg38 g=x" - the quickest way to see what a CGI emits - useless without a hand-made hg.conf. There is no browser to solve a captcha in that situation. printCaptcha() now returns early when cgiWasSpoofed(). That flag cannot be set from an HTTP request: cgiFromCommandLine() returns early and leaves it FALSE whenever the web server has set REQUEST_METHOD. Checked that a plain argument-style run is now clean, that a run which fakes the web environment with QUERY_STRING still gets the captcha, and that an HTTP request behaves exactly as the unmodified binary does. The detailsScript module was loaded from a hardcoded import('../js/hgc.X.js'), bypassing webTimeStampedLinkToResource(), so it was the one script on the page with no ?v=<mtime>. That is the mechanism that flushes a browser's cache when the CGI version changes and that keeps a mirror from pairing an old static file with new CGIs, and without it a cached module could be handed newer bedDetails JSON than it was written for. Now built through the helper, which also fixes the already-shipped histogram type. The helper errAborts on a missing file and the plot type comes from a hub, so a plot type with no module installed falls back to the plain path: a silent failed import as before, rather than one bad hub setting taking down the whole details page. refs #35415
  • src/hg/hgc/bigBedClick.c - lines changed 26, context: html, text, full: html, text
  • src/hg/lib/cart.c - lines changed 8, context: html, text, full: html, text

444b1eb7e7ec2938a4a9a6d3ed214179073ab6f7 Wed Sep 9 05:41:23 2026 -0700

• Faceted composite: manual row reordering, group-by, saved UI state, and per-facet "only" links The Fiber-seq compendium put 41 samples times six data types into one faceted composite, which pushed on the parts of the page that were built for a flat list of tracks. Changes here, all in the shared faceted composite code rather than anything Fiber-seq specific: Row order. Track order in the image follows the table, so the table now lets you set that order by hand. Vendored DataTables RowReorder 1.5.1 adds a drag handle as the first column after the checkbox, enabled on the "shown in the browser" tab where reordering means something. The dragged order is remembered by sample name rather than by row number, so it survives a metadata file whose contents have changed. Group by. A container of six data types can be read two ways, so the page offers both: group the image by sample, keeping a sample's six tracks together, or by data type, putting all the accessibility tracks next to each other. cartDump assigns the priorities and just swaps the nesting of its two loops. trackDb sets the starting choice with defaultGroupBy. Saved state. Facets, per-column searches, sort column, page length, which tab was open and the hand-dragged order go to localStorage keyed by metadata id, so coming back to the page does not mean setting it all up again. Facet "only" links. A small "only" appears on hover behind each facet value and narrows to just that one, instead of unticking the others by hand. Column descriptions. A metadata column heading can now carry a longer explanation after a "|", shown behind an info icon on both the column header and the facet heading. Also: parseDataTypes() was returning its list reversed, since slPairAdd prepends and nothing put it back, so the data type checkboxes and the resulting subtrack order were backwards; the composite lifts itself out of hide when the user touches anything on the page, which is what they meant by touching it; the facet sidebar collapses when a table has no facetable columns; and the label wording throughout says "samples" and "in the browser" rather than "tracks" and "active". The Methbase hg38 track gets labels for its three data types, which were showing as the bare pipeline names hmr, levels and reads. refs #36210
  • src/hg/cartDump/cartDump.c - lines changed 32, context: html, text, full: html, text
  • src/hg/hgTrackUi/hgTrackUi.c - lines changed 25, context: html, text, full: html, text
  • src/hg/htdocs/style/dataTables.rowReorder-1.5.1.min.css - lines changed 1, context: html, text, full: html, text
  • src/hg/htdocs/style/facetedComposite.css - lines changed 88, context: html, text, full: html, text
  • src/hg/htdocs/style/makefile - lines changed 2, context: html, text, full: html, text
  • src/hg/js/external/dataTables.rowReorder-1.5.1.min.js - lines changed 0, context: html, text, full: html, text (a binary file or whitespace-only change or file-permission change shows no diff)
  • src/hg/js/external/makefile - lines changed 1, context: html, text, full: html, text
  • src/hg/js/facetedComposite.js - lines changed 471, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqTrackDb.py - lines changed 61, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeq.ra - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeqCompendium.html - lines changed 12, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/methbase2.ra - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/tagTypes.tab - lines changed 1, context: html, text, full: html, text

a8694a3b22d43f0536c02101e9f3b56b5339b4dc Wed Sep 9 05:47:17 2026 -0700

• hubtools: add "import igv" and "splitHap", and the bTaeGut7 zebra finch hub import igv builds a hub from an IGV session XML. Every Track element becomes a track, in session order, with the IGV display attributes translated to trackDb settings. Files the browser can read over the network are linked where they are; bed, gff, gtf, wig and bedGraph are downloaded and converted, which needs chrom.sizes and gets them from --chromSizes, from the UCSC assembly, or from a bigWig of the session itself, the only source there is for a custom assembly. The BED cleaner exists because real files are not to spec: reversed start/end, scores over 1000, "#rrggbb" colours, names past 255 characters, and columns that are not the BED field they sit in, such as trf writing the repeat motif where thickStart belongs. splitHap turns a hub built on a diploid assembly into one hub with a genome per haplotype, reading both assemblies' chrom.sizes and chromAlias from GenArk and sending each record to whichever assembly has its sequence. It writes splitHap.report.txt with the records per track per haplotype, the sequences neither assembly has, and the records reaching past a sequence end, and checks every track as it goes: records read must equal records matched plus records with no sequence, and every match must produce an output record or a drop. A track that does not add up stops the run rather than being written up as a finding. Two conversion fixes that came out of the zebra finch data. GFF3 requires unique IDs, but an annotation of a phased assembly often gives both haplotypes the same ID; gff3ToGenePred then merges the two copies into one transcript spanning two chromosomes and discards it, which was losing 31 of 182 retrocopies. IDs that occur on more than one sequence are now made unique per sequence first. And a feature name is now taken from the first non-numeric attribute, so a RepeatMasker GFF gives Motif:Tgut716A rather than the running number in ID=. genark addContrib gains --tier alpha|beta|public. It edits only betaGenArk.txt and publicGenArk.txt; beta.hub.txt and public.hub.txt are generated from those lists and shipped by quickPush.pl, so writing them by hand would push content outside the normal flow and lose it at the next clade build. The default alpha tier leaves the lists untouched, so re-running an install cannot demote a collection that is already promoted. doc/contrib/bTaeGut7 and trackDb/contrib/bTaeGut7 are the zebra finch telomere-to-telomere hub built with the above, from the IGV session the authors ship with the annotations on GenomeArk (Formenti et al, Cell 2026, PMID 42561917). 21 tracks in 6 collections plus 3 standalone, 27 description pages, and a makeDoc recording where every record went.
  • src/hg/makeDb/doc/contrib/bTaeGut7/bTaeGut7.txt - lines changed 212, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/hubHeader.txt - lines changed 7, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/makeHtml.py - lines changed 886, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/makeHub.py - lines changed 156, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/mkContribLinks.sh - lines changed 67, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/21097885.html - lines changed 11, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/26667931.html - lines changed 12, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/29109402.html - lines changed 11, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/31842948.html - lines changed 13, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/31843001.html - lines changed 13, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/35279659.html - lines changed 10, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/38709825.html - lines changed 12, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/39240653.html - lines changed 11, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/41262969.html - lines changed 11, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/42561917.html - lines changed 11, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/refs/9862982.html - lines changed 11, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/sourceTracks.tsv - lines changed 24, context: html, text, full: html, text
  • src/hg/makeDb/doc/contrib/bTaeGut7/splitHap.report.txt - lines changed 54, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/bTaeGut7.trackDb.txt - lines changed 292, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/centroCores.html - lines changed 70, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/centroMarkers.html - lines changed 81, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/centroSat.html - lines changed 79, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/centroTelo.html - lines changed 39, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/chromatin.html - lines changed 38, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/compartAB.html - lines changed 82, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/compartE1.html - lines changed 76, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/covClr.html - lines changed 63, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/covHifi.html - lines changed 68, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/covOnt.html - lines changed 67, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/egapx.html - lines changed 76, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/gcPercent.html - lines changed 59, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/genes.html - lines changed 38, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/hubDescription.html - lines changed 54, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/itsRepeats.html - lines changed 68, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/largeSv.html - lines changed 77, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/methyl5mC.html - lines changed 65, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/newRegions.html - lines changed 79, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/nonBdna.html - lines changed 107, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/readCoverage.html - lines changed 38, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/repeats.html - lines changed 38, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/retrocopies.html - lines changed 85, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/satellome.html - lines changed 70, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/seqEntropy.html - lines changed 60, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/structVar.html - lines changed 38, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/tandemRepeats.html - lines changed 79, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/telomeres.html - lines changed 69, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/bTaeGut7/transposons.html - lines changed 89, context: html, text, full: html, text
  • src/utils/genark/genark - lines changed 96, context: html, text, full: html, text
  • src/utils/hubtools/hubtools - lines changed 1701, context: html, text, full: html, text

64d40b9204a60274a49330c6b61d146e5cdc6d87 Wed Sep 9 05:54:34 2026 -0700

• Faceted composite: a container's max display mode must clamp a pinned child, not drop it Selecting only CpG methylation on the Fiber-seq compendium and hitting submit drew nothing at all, with no message to say why. Two things combined. The children of a faceted composite can be pinned to one display mode with onlyVisibility, and five of the six Fiber-seq data types are pinned to full because they are signal tracks. The container's own "Maximum display mode" is a ceiling over those children, and it was set to pack. tdbVisLimitedByAncestors() then took a pinned child that sat above the ceiling and returned hide for it, so every pinned-to-full track disappeared and only the peaks, pinned to dense, came through. With every data type but peaks unchecked, that left an empty image. A ceiling should limit a child, not delete it, so use tvMin the same way the unpinned case a line below already did. A bigWig draws the same at pack as at full, minus the horizontal grid, so nothing is lost here. The page was also asking for the wrong ceiling. On a faceted composite the dropdown is a ceiling rather than a display mode, since each child carries its own, so taking the container out of hide should ask for full, the one value that clips nothing. Pack was chosen on the reasoning that it suits a mix of signal and feature tracks, which is the right instinct for a plain composite and the wrong one here. refs #36210
  • src/hg/js/facetedComposite.js - lines changed 7, context: html, text, full: html, text
  • src/hg/lib/hui.c - lines changed 4, context: html, text, full: html, text

4624f2c72c429fd9b32532bb4d9b62f030b50182 Wed Sep 9 06:00:44 2026 -0700

• trackDb: cross-link the Deleteriousness Predictions container with the CADD 1.6, CADD 1.7, REVEL and AlphaMissense tracks on hg38 and hg19, refs #38261
  • src/hg/makeDb/trackDb/relatedTracks.ra - lines changed 18, context: html, text, full: html, text

1682366b1827b7559f8e1e41635acff6c5ea15e9 Wed Sep 9 06:05:05 2026 -0700

• hprc2annot: move the makeDoc into its own directory and repoint the links The makeDoc has grown a companion (an hg38 pcLAI doc is in progress), so it moves from doc/contrib/hprc2annot.txt into doc/contrib/hprc2annot/, matching how the scripts and trackDb copies are already laid out. The file itself gains a section on the pcLAI scatterplot on the details page: where the reference panel comes from, the four ancestry centroids the discretized field takes across the release, and why the file is read through hgTrackUi rather than fetched by the browser. All seven track description pages linked to the old flat path and would have 404'd, so they are repointed. Six of them change only that link; pclai.html has further edits still in progress and keeps its own copy of the change. refs #35415
  • src/hg/makeDb/doc/contrib/hprc2annot/hprc2annot.txt - lines changed 138, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/hprc2annot/catGenes.html - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/hprc2annot/censat.html - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/hprc2annot/censatCentromeres.html - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/hprc2annot/liftoffGenes.html - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/hprc2annot/methylation.html - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/hprc2annot/segdups.html - lines changed 1, context: html, text, full: html, text

2c0adaa48b2f9a14109c3f90713405f259d920bc Wed Sep 9 06:08:46 2026 -0700

• hg38: pcLAI local ancestry track for HPRC Release 2 haplotypes Point cloud local ancestry inference (pcLAI) for HPRC Release 2, projected onto GRCh38: a composite with one subtrack per haplotype, both haplotypes of 231 samples plus CHM13, 463 in all. 11,936,603 windows, autosomes only, no windows dropped from the source files. Each window carries the (PC1,PC2) coordinate pcLAI predicts for it, the discretized ancestry centroid, and a confidence score. The details page draws the window's position against the 1000 Genomes reference panel that defines the space, via detailsScript/scatterPlot; metaDataUrl is what lets hgTrackUi serve that panel file for a native (non-hub) track. thickStart is one base before chromStart in 54,811 of the windows (0.46%), against the format the pcLAI authors document, so bedToBigBed rejects it. Neither thick column carries information here, so both are set to the item bounds rather than dropping those windows. Testing only for now, alpha, no ticket yet; a ticket may follow if this becomes a real track. The makeDoc carries the detail in the meantime, including what was deliberately left undone.
  • src/hg/makeDb/doc/hg38/hprcPclai.txt - lines changed 137, context: html, text, full: html, text
  • src/hg/makeDb/scripts/hprcPclai/hprcPclai.as - lines changed 17, context: html, text, full: html, text
  • src/hg/makeDb/scripts/hprcPclai/hprcPclaiDefaultOn.txt - lines changed 11, context: html, text, full: html, text
  • src/hg/makeDb/scripts/hprcPclai/hprcPclaiDownload.sh - lines changed 28, context: html, text, full: html, text
  • src/hg/makeDb/scripts/hprcPclai/hprcPclaiMakeBb.sh - lines changed 46, context: html, text, full: html, text
  • src/hg/makeDb/scripts/hprcPclai/hprcPclaiMakeTrackDb.py - lines changed 130, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/hprcPclai.html - lines changed 146, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/hprcPclai.ra - lines changed 6040, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/trackDb.ra - lines changed 2, context: html, text, full: html, text

6b0035d19769346baffe193ef9419c269d46f8d8 Wed Sep 9 06:09:41 2026 -0700

• hprc2annot: pcLAI column 10 is the ancestry centroid, not a segment coordinate Reading the pcLAI authors' own format description (github.com/AI-sandbox/hprc-pclai, README "Output format (BED)") while building the same annotation as a native hg38 track showed that column 10 of the source BED had been described wrongly here. It is not the PCA coordinate of a longer ancestry segment the window belongs to; the authors call it the centroid, the discretized pcLAI ancestry of the window written as the PCA centroid of its ancestry cluster. That is why it only ever takes four values -- four clusters, not four long shared segments. The old reading also implied a segmentation step the method does not have: pcLAI predicts one coordinate per window, and the blocks visible in the display are runs of windows with similar predictions. Field renamed pcaSegment -> centroid in pclai.as with the description corrected, and the mouseOver, the detailsScript exportFields and the description page follow. The README also settles that windows are a fixed 1000 SNPs rather than a fixed number of bases, and that thickStart is specified to equal chromStart, so the occasional thickStart == chromStart-1 the converter works around is a bug in their files rather than something we misread. A field name and its description live inside each bigBed, so editing pclai.as does nothing to a built collection. hprc2annotRewriteAs.sh re-emits a built bigBed with the current .as -- no re-download, no column change, item count checked across the round trip, and safe to re-run, unlike hprc2annotFixBed.sh. All 460 pclai.bb were rewritten with it. Worth knowing: those files had been built from an older pclai.as than the tree and nothing had noticed, so this is the tool to run after any .as description edit. genark: the "...Url" inside a detailsScript value must not be rebased the way bigDataUrl is. hgc resolves a relative detailsScript Url against the track's own bigDataUrl when it builds the details page, and bigDataUrl has already been rebased, so the prefix landed twice: the pcLAI scatterplot had been asking for contrib/hprc2annot/contrib/hprc2annot/pclaiRefPanel.json and quietly getting nothing on every GenArk hub. In this layout the panel file is symlinked beside the .bb, so relative-to-the-.bb is the bare file name; rebaseBeside() does that and is idempotent, so addContrib can be re-run. refs #35415
  • src/hg/makeDb/scripts/hprc2annot/hprc2annotBuildOne.sh - lines changed 4, context: html, text, full: html, text
  • src/hg/makeDb/scripts/hprc2annot/hprc2annotFixBed.sh - lines changed 2, context: html, text, full: html, text
  • src/hg/makeDb/scripts/hprc2annot/hprc2annotRewriteAs.sh - lines changed 45, context: html, text, full: html, text
  • src/hg/makeDb/scripts/hprc2annot/pclai.as - lines changed 5, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt - lines changed 3, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/contrib/hprc2annot/pclai.html - lines changed 29, context: html, text, full: html, text
  • src/utils/genark/genark - lines changed 16, context: html, text, full: html, text

4a39786c0e5473f3e987017bcb616b63423a89e2 Wed Sep 9 06:16:03 2026 -0700

• hg38 Fiber-seq: open the compendium description by naming the collection it belongs to A subtrack description page should say which collection it is part of and link back to that page, so a reader who lands on it from a search result can get to the container. A bare hgTrackUi link rather than one carrying ${hgsid}: native trackDb html is substituted by hgTrackDb when it loads the table, where there is no cart, so $hgsid resolves to the empty string and the link would come out as 'hgsid=&g=fiberSeq'. The ${hgsid} form works on hub pages, which are substituted at render time instead. refs #36210
  • src/hg/makeDb/trackDb/human/hg38/fiberSeqCompendium.html - lines changed 2, context: html, text, full: html, text

b947161926eed5f7a41eb598465d1d5cb8e88bc1 Wed Sep 9 06:33:41 2026 -0700

• hg38 Fiber-seq: open the accessibility description by naming the collection too Matches the compendium page. This one already linked to its sibling track further down, it just did not say what container it sits in. refs #36210
  • src/hg/makeDb/trackDb/human/hg38/fiberSeqAcc.html - lines changed 2, context: html, text, full: html, text

691a2b8981d6db69e8707ea44041c4661cdac97e Wed Sep 9 06:38:29 2026 -0700

• Imprinting: add the ASM Atlas tracks, and tidy the collection's labels Adds a composite built from Rosenski et al. 2025, "Atlas of imprinted and allele-specific DNA methylation in the human body". Three subtracks: the 458 regions whose methylation follows the parent of origin, the 72 known control regions with the boundaries the paper redrew, and the pool of 385,235 regions carrying two methylation states that those came out of. A fourth set, the regions whose methylation follows a nearby SNP, is built by the scripts but its stanza is commented out, since sequence driven methylation is not imprinting. The authors released hg19 only, so all three are lifted. Their published files are close to bare BED, so the SNPs, cell types, p-values, gene links and gamete methylation on the details pages are read out of the paper's supplementary tables and joined on by position. Regions that lift but change length by more than 10%, because hg38 added sequence inside them, are kept with a note rather than dropped: one of them is TCEB3C, the only control region on chr18. Also across the collection: - long labels name their source right after "Imprinting", so that a label read on its own says where the data came from - the two gene catalogs are worded alike, and ordered OMIM, Geneimprint, MethBase2, Akbari, ASM Atlas - the OMIM curators confirmed that their (I) marker covers established and candidate imprinted genes alike, with nothing in the export to tell them apart. Labels, description page and makeDoc now say so, and the claim that the set is "more conservative" than the computational tracks is gone. The bigBed was rebuilt for the autoSql line, same 459 features. - every subtrack page opens by naming the collection, linked back to its hgTrackUi page, and no longer repeats the collection page's introduction to imprinting refs #37599
  • src/hg/makeDb/doc/hg38/imprinting.txt - lines changed 5, context: html, text, full: html, text
  • src/hg/makeDb/doc/hg38/kaplanImprint.txt - lines changed 176, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/kaplanAsm.as - lines changed 20, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/kaplanBimodal.as - lines changed 17, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/kaplanIcr.as - lines changed 18, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/kaplanIcrAddOrig.py - lines changed 40, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/kaplanImprintLift.sh - lines changed 56, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/kaplanImprintToBed.py - lines changed 357, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/kaplanImprintXlsxToTsv.py - lines changed 54, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/kaplanLiftNote.py - lines changed 41, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/kaplanParentalAsm.as - lines changed 29, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/omimImprint.as - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/scripts/imprinting/omimImprintToBed.py - lines changed 4, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/akbariIdmr.html - lines changed 8, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/geneimprint.html - lines changed 6, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/imprinting.html - lines changed 27, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/imprinting.ra - lines changed 108, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/kaplanImprint.html - lines changed 180, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/methBaseAsm.html - lines changed 7, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/omimImprint.html - lines changed 21, context: html, text, full: html, text

800ff0dc7028712146bb5ea2a691ae42d299d25b Wed Sep 9 06:40:29 2026 -0700

• hgLogin: tighten activation link handling - treat a missing or empty token as invalid, and apply the seven-day expiry that the confirmation mail already promises, refs #38302
  • src/hg/hgLogin/hgLogin.c - lines changed 10, context: html, text, full: html, text

af613a331e6839c6513c3e366abcb67af0fe8386 Wed Sep 9 06:47:14 2026 -0700

• UniProt otto: get the monthly update running again and make a stalled run visible The monthly UniProt job had produced nothing since January 2025. The tracks served release 2024_06 while the download sitting on disk was at 2026_02, on every assembly the job builds. Cause: uniprotToTab appended a personal conda site-packages directory to sys.path, and doUpdate.sh sourced a virtualenv, both built for python 3.6. A venv's python is only a symlink to the system one, so when hgwdev moved to python 3.9 the compiled lxml in there stopped loading and every run died at the parse step. Removed both. The system python3 has lxml from python3-lxml and the two are upgraded together, so there is nothing left here to go stale. Verified by parsing real 2026_02 records under python 3.9 with lxml 5.4. Why nobody noticed for nineteen months: - doUpdate.sh read $? after an intervening echo, so it captured the echo's exit code and mailed "Big Uniprot update OK" every month while the job was dying. It now reads the real exit code, says FAILED, prints the tail of the log and exits nonzero. A month with no new UniProt release stays silent, which is the normal otto behaviour, so silence again means "nothing to do". - The logs were overwritten on every run, so a failure left no trace on disk. doUpdate.sh now appends one line per run to runLog.txt, which is never truncated, and keeps a failing log as lastFail.log. - version.txt in each bigBed directory was rewritten on every run even when the release string was identical. That is the file the trackDb dataVersion setting shows, and its date is what people check to decide whether a pipeline is still alive, so a stalled track could look freshly updated. It is now written only when the release actually changes. Also, so this cannot come back: - doUniprot checks that uniprotToTab can start before the download, instead of finding out 35 minutes later. - pylint on hgwdev is itself pinned to pythons that no longer exist, so "make install" aborted on its first line and could not be used. Replaced with a syntax check that needs nothing but python3; pylint stays best-effort. - uniprotToTab, pslProtCnv, trackDb.template.txt and README.txt ran from /hive/data/outside/otto/uniprot without being in the makefile's copy list. The tree copy of uniprotToTab was still python 2 from 2021. All are now listed and in sync, and "make diff" reports drift. - Brought the two live-only fixes into the tree: mkdir -p in makeUniProtPsl.sh and the pslMap -inType/-mapType flags. refs #38300
  • src/hg/utils/otto/uniprot/README.txt - lines changed 130, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 41, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/doUpdate.sh - lines changed 64, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/makeUniProtPsl.sh - lines changed 5, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/makefile - lines changed 27, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/trackDb.template.txt - lines changed 224, context: html, text, full: html, text
  • src/utils/uniprotToTab - lines changed 183, context: html, text, full: html, text

bee41315e53da6fb5671f96302233c655906dfd5 Wed Sep 9 06:56:49 2026 -0700

• UniProt otto: rebuild the venv rather than relying on a system lxml Correction to the previous commit: hgwdev has no system-wide lxml at all. The import I tested was resolving to my own ~/.local/lib/python3.9/site-packages, which cron never sees, because it runs the pipeline as otto. So the environment is a virtualenv again, but a reproducible one. makeVenv.sh deletes venv/ and rebuilds it from /usr/bin/python3, installs lxml, opens up the permissions for otto, and then checks that lxml imports with an empty environment so we know the venv stands on its own instead of borrowing from whoever ran it. Built with --copies, so venv/bin/python is a real copy rather than a symlink that would silently follow a system python upgrade while its compiled modules stayed behind. doUpdate.sh activates venv/ again and says to run makeVenv.sh if it is missing or if the parser will not start. Verified: /usr/bin/python3 without per-user packages cannot import lxml, the venv can, and after activation the parser runs and converts real 2026_02 records. Also shortened the README to how the pipeline is started and how it works, and trimmed the history out of the code comments, leaving the ticket as the pointer. refs #38300
  • src/hg/utils/otto/uniprot/README.txt - lines changed 44, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 12, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/doUpdate.sh - lines changed 20, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/makeVenv.sh - lines changed 47, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/makefile - lines changed 8, context: html, text, full: html, text
  • src/utils/uniprotToTab - lines changed 11, context: html, text, full: html, text

28b1395f3e814f53fb8b0d75904f09469504563e Wed Sep 9 07:13:08 2026 -0700

• UniProt otto: report a held lock file as its own case, not as a failure A run that is still going, or one that crashed and left /hive/data/outside/uniProt/current/doUniprot.lock behind, made the next cron run print the full failure report. Now it says so in one line and logs LOCKED, so a long run in progress does not look like a broken pipeline, while a stale lock is still mentioned to whoever gets the mail. refs #38300
  • src/hg/utils/otto/uniprot/doUpdate.sh - lines changed 10, context: html, text, full: html, text

efe79575f0b546414196fb3b8a8871bf5f7611fc Wed Sep 9 07:16:04 2026 -0700

• UniProt otto: stop curl writing its progress bar into the run log The gene2refseq download wrote a few hundred lines of percentages into lastRun.log, which buries the lines that matter. --no-progress-meter drops the bar but keeps errors, unlike the plain -s that was removed here earlier for exactly that reason, and --fail turns an HTTP error into a curl error instead of piping NCBI's error page into zcat and failing with 'not in gzip format'. Not yet installed to /hive: a run is in progress. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 6, context: html, text, full: html, text

2096509ab88749e811913cd2f07239f4972d4f18 Wed Sep 9 07:21:43 2026 -0700

• genark: make addContrib's release tier a positional argument, not --tier
  • src/utils/genark/genark - lines changed 9, context: html, text, full: html, text

af670dcecc6911452f3e1a5c38cfea1ef0067978 Wed Sep 9 07:31:07 2026 -0700

• UniProt otto: submit the parasol batch on hgwdev, drop the ssh to ku The protein-to-transcript BLAST batch was submitted with ssh ku "cd <workdir> && para make jobList" ku has been decommissioned for about two years. It still resolves in DNS, but "ssh ku" is an immediate "No route to host", so a run would have died at the mapping stage, which comes only after days of XML parsing. hgwdev is the parasol head node now, so "para make" runs here with no ssh hop. Verified with a one-job batch using the same bare-command jobList this script writes: it lands on a compute node with its cwd set to the batch directory, the same as the old ku jobs did. Also checked from a compute node that tclsh and the blast-2.2.16 blastall/formatdb that mapUniprot_doBlast needs are still there. The cluster name is gone rather than redirected. doUniprot no longer reads /cluster/bin/scripts/cluster.txt, a file written in 2017 that still says "ku" and that nothing else in the tree read, and makeUniProtPsl.sh no longer takes a head node argument, so its positional parameters shift down by one. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 11, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/makeUniProtPsl.sh - lines changed 6, context: html, text, full: html, text

989ad01358d595f619021742182be67c01dd56fd Wed Sep 9 07:38:59 2026 -0700

• UniProt otto: log an interrupted run instead of leaving a dangling START Killing a run left "START" in runLog.txt with no line after it, which reads exactly like a run that is still going. doUpdate.sh now traps INT/TERM/HUP, logs INTERRUPTED, and removes doUniprot.lock, which doUniprot's own atexit handler does not get to run on a signal and which would otherwise block the next run. Note the shell only runs the trap once the foreground doUniprot has exited, so this fires when the whole process group is killed, which is what pkill does and what actually happens in practice. refs #38300
  • src/hg/utils/otto/uniprot/doUpdate.sh - lines changed 6, context: html, text, full: html, text

fc8de100a3437b9fc33bdeb0f459ca2a93e2f318 Wed Sep 9 08:14:32 2026 -0700

• hgSession: address the code review of the new Sessions page Rename and unshare now keep the public listing's thumbnail with the session it belongs to. The picture's file name is built from the encoded session name, so renaming a listed session left the listing pointing at nothing and the old file behind, and dropping a session from the listing to a plain shared link kept the picture. The classic page had the same problem in a subtler form: it removed the thumbnail after the row had already been renamed, so the old file survived. Saving under a name that is already in use asks before it replaces that session, using the failIfExists reply that the top-right Share a link menu already relies on. The description and "only I can load it" steps that follow a save now report a failure instead of reloading in silence, and what thumbnailAdd has to say when it cannot build a picture reaches the user instead of being freed unread. A session description no longer travels through a title attribute. The tooltip machinery in utils.js inserts its text with innerHTML and an attribute is decoded on the way, so a description containing angle brackets was interpreted as markup rather than shown as typed. It is attached, escaped, after each table draw, which also gives the rows DataTables renders later the same styled mouseovers as the rest of the page. Also: the AJAX endpoints say so when there is no session by that name, instead of reporting a no-op as a success; the new page always offers its way back to the classic page, since the cart variable that got the user there sticks; and four unused CSS rules, a dead element lookup and a dead local are gone. hgConfCatalog cited the wrong ticket for the two sessionNewPage flags. refs #38180, refs #38157
  • src/hg/hgSession/hgSession.c - lines changed 108, context: html, text, full: html, text
  • src/hg/htdocs/style/hgSession.css - lines changed 6, context: html, text, full: html, text
  • src/hg/js/hgSession.js - lines changed 100, context: html, text, full: html, text
  • src/hg/utils/hgConfCatalog/hgConfCatalog.py - lines changed 8, context: html, text, full: html, text

bf15116918c332d150006ee7b26eff5cc90cc5b9 Wed Sep 9 08:19:45 2026 -0700

• UniProt otto: a notifier for hand-started catch-up runs A run started by hand can take days, and until it ends there is nothing to tell you where it is short of going and looking. notifyRun.sh watches lastRun.log, works out which of the nine stages the pipeline has reached, and mails a note on every transition, with a "still doing X" note every twelve hours so a long TrEMBL parse does not go quiet for three days. When the process disappears it mails the outcome, read out of runLog.txt, so a killed or failed run is reported and not just an absence of mail. The cron run does not need it, doUpdate.sh already mails through otto's MAILTO. This is for watching a catch-up run in between. setsid nohup ./notifyRun.sh you@ucsc.edu > notifyRun.log 2>&1 < /dev/null & refs #38300
  • src/hg/utils/otto/uniprot/notifyRun.sh - lines changed 88, context: html, text, full: html, text

fcaa1479c3dbb46e3b3b25865432158156cf2dff Wed Sep 9 08:22:31 2026 -0700

• UniProt otto: let notifyRun.sh post to a Slack incoming webhook too Mail is the reliable channel for a run that takes days, but it does not necessarily land on a phone. With a Slack incoming webhook in ~/.hg.conf as slack.webhook=..., each stage note is also posted to that channel. Without the key, nothing changes and notifications go by mail only. The URL stays out of the command line, where ps would show it to everyone on the machine, and out of stdout and stderr: curl runs with -s and a failed post logs only that it failed. Checked that an absent webhook is a silent no-op, that an unreachable one warns without killing the watch, and that the URL appears nowhere in the output. refs #38300
  • src/hg/utils/otto/uniprot/notifyRun.sh - lines changed 16, context: html, text, full: html, text

042acb30f02d050444afd846b3f026eb33c99ab1 Wed Sep 9 08:38:02 2026 -0700

• varChat otto: don't blank the track's version file when the fetch fails /gbdb/hg38/bbi/varChatVersion.txt symlinks straight to /hive/data/outside/otto/varChat/version.txt, so the browser reads that file directly, and wget truncates its -O target before it has anything to write there. A failed or partial fetch therefore left the track showing an empty version. By that point the run has already done "mv varChat.hg38.latest.bb varChat.hg38.bb", so the new data is live and only the version string is gone, and set -e aborting afterwards does not undo it. VarChat only updates when upstream changes, and upstream has been on v1.1 - 2025-11-07 for ten months, so a blank could sit there that long. Fetch to version.new.txt, require it to be non-empty, and only then move it into place, keeping the old string with a warning otherwise. Same shape as mitoMap/checkMitoMapUpdate.sh, which already does this. Checked all three paths: the real upstream URL updates the file and leaves no temp behind, an unreachable URL and a URL returning an empty body both keep the previous string and warn. For contrast, the old one-liner against an unreachable URL truncated version.txt to zero bytes. Found while sweeping the otto updaters for the version-stamping problem behind the uniprot outage. refs #38300
  • src/hg/utils/otto/varChat/varChatOtto.sh - lines changed 12, context: html, text, full: html, text

68b9911e4c156cd1346fde5c957434b1a8780d1c Wed Sep 9 08:49:07 2026 -0700

• Show the transcript's own codon number where it differs from the genomic one The gene tracks count codons along the genome. A RefSeq transcript is a sequence in its own right, so where it has an insertion or a deletion relative to the assembly, every codon 3' of that point gets a different number here than the sequence provider gives it, one codon per three bases. DNM1 on canFam3 is the reported case: the transcript carries 21 bases canFam3 does not, so our p.249 is NCBI's p.256. Neither number is wrong, but HGVS c./p. is defined on the transcript, so the number people quote is the one we were not showing. The genomic number and amino acid are unchanged. Codons whose two numbers disagree now draw in the existing CDS_QUERY_INSERTION orange with a "!" after the codon number, and their mouseover adds the transcript number plus a link to a new FAQ entry. Both directions of indel are covered, and so is the case where the alignment does not reach the start of the CDS (801 transcripts on hg38), which needs the transcript's own CDS annotation as the anchor rather than the alignment. Numbers come from the transcript alignment, ncbiRefSeqPsl or refSeqAli, which is the same source hgvsMapToGenome already uses, so the browser now agrees with its own position search. Two queries per table per window, on the bin index, and only at zoomedToCdsColorLevel, where the mouseover carrying the numbers is drawn: on and off are within noise at every zoom. Gated by showTxCodonNumbers in hg.conf, default off, catalogued as a release gate. With it off nothing is looked up and the rendering and mouseover are byte-identical to before. refs #38298
  • src/hg/hgTracks/cds.c - lines changed 226, context: html, text, full: html, text
  • src/hg/hgTracks/cds.h - lines changed 20, context: html, text, full: html, text
  • src/hg/hgTracks/hgTracks.h - lines changed 5, context: html, text, full: html, text
  • src/hg/hgTracks/simpleTracks.c - lines changed 34, context: html, text, full: html, text
  • src/hg/htdocs/FAQ/FAQgenes.html - lines changed 59, context: html, text, full: html, text
  • src/hg/htdocs/goldenPath/help/hgCodonColoring.html - lines changed 18, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg19/refSeqComposite.html - lines changed 6, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/refSeqComposite.html - lines changed 6, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/mouse/mm10/refSeqComposite.html - lines changed 6, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/mouse/mm39/refSeqComposite.html - lines changed 6, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/refSeqComposite.html - lines changed 6, context: html, text, full: html, text
  • src/hg/utils/hgConfCatalog/hgConfCatalog.py - lines changed 14, context: html, text, full: html, text

5a249cd50f592a3b7598110eec792cfc942fab3f Wed Sep 9 08:52:06 2026 -0700

• Address the v504 code review hgSession: an anonymous share name that arrives with the request is saved only when it is not already in the table. Every anonymous link sits under the one reserved user "l", so a name already there stays as it is and the caller is told so. The top-right Share dialog is unaffected, since it passes a name it has just reserved and such a name does not exist yet. Snapshot names are now left out of both My Sessions listings, which is what their "__" prefix has claimed all along. Share dialog: "Create link & copy" reports the copy instead of promising it. copyToClipboard says whether the text reached the clipboard, the dialog passes that on when a browser refuses, and it tries the asynchronous clipboard API before giving up. The preview is built with the same encoding the server uses, so a name holding a hyphen or a slash previews as the link that really gets made. Cancelling out of the name editor no longer copies a second time, and a reply with no link in it says so rather than showing "undefined". hgBlat: a second click on the share button while the first request is still out no longer mints a second snapshot session, and a box dismissed during the wait stays closed. Also: a snapshot moves a cart value into durable storage only when it is a trash path, the way sessionData's own callers check; sqlAddressMatch keeps to its own documented precondition when handed an empty address; alphaGenomeToWig compares its output with its input rather than with itself, rejects a position that is not all digits and skips an empty score; and hgc's default iframe width reaches the browser as one percent sign. refs #38294
  • src/hg/hgLogin/hgLogin.c - lines changed 13, context: html, text, full: html, text
  • src/hg/hgSession/hgSession.c - lines changed 51, context: html, text, full: html, text
  • src/hg/hgc/hgc.c - lines changed 6, context: html, text, full: html, text
  • src/hg/js/hgBlat.js - lines changed 20, context: html, text, full: html, text
  • src/hg/js/topLinks.js - lines changed 60, context: html, text, full: html, text
  • src/hg/js/utils.js - lines changed 10, context: html, text, full: html, text
  • src/hg/lib/snapshotSession.c - lines changed 6, context: html, text, full: html, text
  • src/hg/oneShot/alphaGenomeToWig/alphaGenomeToWig.c - lines changed 19, context: html, text, full: html, text

7ee42571461ed5a384f3d1b628253a8773850cb2 Wed Sep 9 08:54:16 2026 -0700

• AlphaGenome track doc: preprint reference, threshold guidance from the authors The AlphaGenome authors reviewed the description page and asked for the peer-review status to be updated, for their threshold recommendation to be stated, and for links to the Atlas preprint and the educational guides.
  • src/hg/makeDb/trackDb/human/alphaGenome.html - lines changed 28, context: html, text, full: html, text

b761b0b0fc3cd9e830db0f11a536e0da29d7c9e5 Wed Sep 9 08:58:30 2026 -0700

• A copy-to-clipboard button goes back to its own label after three seconds Saying "Copied" and then keeping that as the label left no sign that the button could be used again. It now says "Copied" for three seconds and then puts back its own label and icon. A second copy while the message is up restarts the three seconds rather than adopting "Copied" as the label to go back to. The assembly search page carries its own borrowed copy of copyToClipboard and does not load utils.js, so it gets the same treatment there. refs #38294
  • src/hg/js/assemblySearch.js - lines changed 20, context: html, text, full: html, text
  • src/hg/js/topLinks.js - lines changed 3, context: html, text, full: html, text
  • src/hg/js/utils.js - lines changed 16, context: html, text, full: html, text

51b9776d380b32ed640c81917ec2d9a3528d4b06 Wed Sep 9 08:58:55 2026 -0700

• UniProt otto: notifyRun.sh could not see that the TrEMBL parse had started doUniprot's run() logs its "Running: <cmd>" line after os.system returns, not before, so the log says a command has finished, never that it is running. The stage ladder keyed the TrEMBL stage off that line, which meant it only noticed TrEMBL once TrEMBL was over: a three-day parse would have been reported as "parsing the SwissProt XML" throughout, including in every twelve-hour heartbeat. Ask the process table for a running uniprotToTab --trembl instead, and keep the log check as the fallback for after the process is gone. Later stages still override it, so the ladder is unchanged otherwise. Caught on the live run, which had the flag in its command line while the notifier still said SwissProt. refs #38300
  • src/hg/utils/otto/uniprot/notifyRun.sh - lines changed 5, context: html, text, full: html, text

562b1b24f9d7cf5157c799f733ba219e2e9f7da9 Wed Sep 9 09:06:30 2026 -0700

• Assembly search page uses the shared copyToClipboard instead of its own copy The page carried a copy of copyToClipboard marked "borrowed this code from utils.js", and the two had already drifted apart: the fix that stops the button claiming a copy that a browser refused went into one and not the other. The page now loads utils.js, as nine other static pages already do, and its own copy is gone. jquery is already loaded by the page header, so nothing else was needed. The page also declared a global named debug, which utils.js declares too. Both start out false and the two uses in utils.js are in functions this page never calls, so nothing was broken, but the page flag is now searchDebug. The debug URL parameter and stateObject.debug keep their names. refs #38294
  • src/hg/htdocs/assemblySearch.html - lines changed 1, context: html, text, full: html, text
  • src/hg/js/assemblySearch.js - lines changed 60, context: html, text, full: html, text

0cdd15681f10789132dc9e88fcf4be23bd47ee4b Wed Sep 9 09:14:37 2026 -0700

• New Sessions page: Replace on the save card keeps the session's sharing level Confirming Replace re-saved through the save endpoint, which always writes a session as shared by link. Replacing a session that was in the public listing took it off the list, and replacing a private one made it loadable by anyone with the link. Replace now goes through the overwrite endpoint, which reads the row's sharing level and keeps it - the same endpoint the floppy button on each table row already uses. The description and the "only I can load it" box on the save card are still applied afterwards, refs #38311
  • src/hg/js/hgSession.js - lines changed 16, context: html, text, full: html, text

cb99f0b11bdeee5dfa76064d38b6410da0f4a709 Thu Sep 10 00:55:21 2026 -0700

• Centralize CGI Content-Type printing in one cgiPrintContentType() helper Around 90 places across the tree hand-rolled the CGI response header, each with its own spelling: "Content-Type:" or "Content-type:", \n or \r\n, and the terminating blank line written as part of the same string, as a separate puts("\n") (which emits two newlines, so a stray blank line led the body) or as printf("\r\n\r\n") (two blank lines). A handful forgot the terminator entirely and relied on a following header to supply it. cgiPrintContentType() in lib/cheapcgi.c now writes the Content-Type line and the blank line that ends the header. Header lines are not ordered, so the callers that also send Status, Set-Cookie, Content-Disposition, Content-Length or X-Sendfile write those first and call this last to close the header; that keeps it to a single helper rather than a print-the-line / end-the-header pair that a caller can half-use. cart.c's existing httpHeaders list already worked this way. Only the CGI response path is touched. The dyStringPrintf("Content-type: ...") calls that build outgoing HTTP *requests* (genomeSpace, oauthLogin, eapMetaSync, edwWebAuthLogin, ga4ghToBed) are unrelated and left alone. Also fills out the apiKey error message in botDelay.c to say where to create a key and that keys are server-specific. No behavior change on the wire beyond dropping those stray blank lines and adding the missing newline after Retry-After.
  • src/ameme/ameme.c - lines changed 1, context: html, text, full: html, text
  • src/ameme/amemeFloat.c - lines changed 1, context: html, text, full: html, text
  • src/ameme/fixpAmeme.c - lines changed 1, context: html, text, full: html, text
  • src/hg/cgilib/api.c - lines changed 1, context: html, text, full: html, text
  • src/hg/cgilib/cartJson.c - lines changed 1, context: html, text, full: html, text
  • src/hg/cirm/cdw/cdwGetFile/cdwGetFile.c - lines changed 14, context: html, text, full: html, text
  • src/hg/cirm/cdw/cdwGetMetadataAsFile/cdwGetMetadataAsFile.c - lines changed 1, context: html, text, full: html, text
  • src/hg/cirm/cdw/cdwServeTagStorm/cdwServeTagStorm.c - lines changed 1, context: html, text, full: html, text
  • src/hg/cirm/cdw/cdwWebBrowse/cdwWebBrowse.c - lines changed 2, context: html, text, full: html, text
  • src/hg/cirm/cdw/lib/cdwLib.c - lines changed 2, context: html, text, full: html, text
  • src/hg/das/das.c - lines changed 5, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwScriptSubmitStatus/edwScriptSubmitStatus.c - lines changed 2, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwWebAuthLogin/edwWebAuthLogin.c - lines changed 2, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwWebAuthLogout/edwWebAuthLogout.c - lines changed 3, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/edwLib.c - lines changed 2, context: html, text, full: html, text
  • src/hg/hgChooseDb/hgChooseDb.c - lines changed 1, context: html, text, full: html, text
  • src/hg/hgCustom/hgCustom.c - lines changed 1, context: html, text, full: html, text
  • src/hg/hgGateway/hgGateway.c - lines changed 1, context: html, text, full: html, text
  • src/hg/hgGenome/hgGenome.c - lines changed 1, context: html, text, full: html, text
  • src/hg/hgMenubar/hgMenubar.c - lines changed 1, context: html, text, full: html, text
  • src/hg/hgSearch/hgSearch.c - lines changed 1, context: html, text, full: html, text
  • src/hg/hgSession/backup.c - lines changed 5, context: html, text, full: html, text
  • src/hg/hgSession/hgSession.c - lines changed 7, context: html, text, full: html, text
  • src/hg/hgSuggest/hgSuggest.c - lines changed 2, context: html, text, full: html, text
  • src/hg/hgTables/genomeSpace.c - lines changed 1, context: html, text, full: html, text
  • src/hg/hgText/hgText.c - lines changed 5, context: html, text, full: html, text
  • src/hg/hgText/hgWigText.c - lines changed 2, context: html, text, full: html, text
  • src/hg/hgTrackUi/hgTrackUi.c - lines changed 7, context: html, text, full: html, text
  • src/hg/hgTracks/hgTracks.c - lines changed 8, context: html, text, full: html, text
  • src/hg/hgTracks/mainMain.c - lines changed 1, context: html, text, full: html, text
  • src/hg/hgTracks/myVariantsTrack.c - lines changed 2, context: html, text, full: html, text
  • src/hg/hgc/hgc.c - lines changed 2, context: html, text, full: html, text
  • src/hg/hubApi/apiUtils.c - lines changed 3, context: html, text, full: html, text
  • src/hg/hubApi/blat.c - lines changed 2, context: html, text, full: html, text
  • src/hg/hubApi/hubApi.c - lines changed 4, context: html, text, full: html, text
  • src/hg/lib/botDelay.c - lines changed 13, context: html, text, full: html, text
  • src/hg/lib/cart.c - lines changed 5, context: html, text, full: html, text
  • src/hg/lib/web.c - lines changed 1, context: html, text, full: html, text
  • src/hg/oneShot/testCart/cart.c - lines changed 2, context: html, text, full: html, text
  • src/hg/phyloPng/phyloPng.c - lines changed 15, context: html, text, full: html, text
  • src/hg/sageVisCGI/sageVisCGI.c - lines changed 1, context: html, text, full: html, text
  • src/hg/useCount/useCount.c - lines changed 1, context: html, text, full: html, text
  • src/hg/visiGene/hgVisiGene/hgVisiGene.c - lines changed 4, context: html, text, full: html, text
  • src/inc/cheapcgi.h - lines changed 6, context: html, text, full: html, text
  • src/lib/cheapcgi.c - lines changed 11, context: html, text, full: html, text
  • src/lib/errAbort.c - lines changed 2, context: html, text, full: html, text
  • src/lib/htmshell.c - lines changed 6, context: html, text, full: html, text
  • src/lib/textOut.c - lines changed 9, context: html, text, full: html, text
  • src/lib/verbose.c - lines changed 2, context: html, text, full: html, text
  • src/oneShot/dateXmlCgi/dateXmlCgi.c - lines changed 3, context: html, text, full: html, text
  • src/webBlat/webBlat.c - lines changed 2, context: html, text, full: html, text
  • src/weblet/counter/counter.c - lines changed 1, context: html, text, full: html, text

cf9f4cb7f55c7beb8ad5f11118656a60770a71a5 Thu Sep 10 01:02:36 2026 -0700

• Move the extra-HTTP-header list into cheapcgi, and write the header only once Follow-on to the cgiPrintContentType() refactor. cart.c owned the mechanism for adding headers ahead of the content type: a global slPair list plus addHttpHeaders() to print it. That put it in hg/lib, out of reach of the CGIs and library code that do not use a cart, even though nothing about it is cart-specific. It now lives next to cgiPrintContentType() in lib/cheapcgi.c, behind cgiAddHttpHeader(name, value) instead of a bare global, and cgiPrintContentType() writes the queued headers itself. The one caller, hgTracks/mainMain.c, reads the same but no longer reaches into cart.h for it. cspWriteResponseHeader() stays in hg/lib where it belongs, since it needs hg.conf; cartWriteHeaderAndCont() calls it directly now, the way the other ten callers already do. cgiPrintContentType() also writes at most once per process now. A second content type cannot reach the browser as a header - it lands in the page body as text - so the later caller is always the mistaken one. cart.c had a private cartDidContentType flag for exactly this, covering only the flows that went through the cart; the guard is now in the one function every flow shares, and cartDidContentType is gone. Its public equivalent, cgiDidContentType(), is what cartWriteHeaderAndCont() checks so it does not write a second cookie. Verified: make libs, make cgi and the lib test suite are clean, hgTracks still emits Cache-Control: no-store, and hgTracks, hgc and hgTables each emit exactly one Content-Type on both their html and their text paths.
  • src/hg/hgTracks/mainMain.c - lines changed 1, context: html, text, full: html, text
  • src/hg/inc/cart.h - lines changed 3, context: html, text, full: html, text
  • src/hg/lib/cart.c - lines changed 29, context: html, text, full: html, text
  • src/inc/cheapcgi.h - lines changed 14, context: html, text, full: html, text
  • src/lib/cheapcgi.c - lines changed 33, context: html, text, full: html, text

58104a98358604975dac0a9350ca91d2d25c501d Thu Sep 10 05:02:45 2026 -0700

• hUserAbort shows its message to the user instead of turning into a 500 hUserAbort() reports an error caused by user input, so the message is written to be read by the user. It was only reaching them when a CGI had already pushed a warn handler of its own. The apiKey and bot checks call it from main() before that happens, and the default handler then writes to stderr and nothing else, unless hg.conf sets showEarlyErrors - off by default, and off on the RR. Apache turns the empty response into a 500, which is what hubApi/hubApi.c works around by pre-validating the apiKey itself. hVaUserAbort() now turns doContentType on for the rest of the process when it is running as a CGI, so the default handler emits the Content-Type line and the message. It stays off for a program that never called cgiSpoof(), and it is inert inside an errCatch, which pushes its own warn handler - so a caller that catches the abort to write its own response (hubApi's JSON) is unchanged. Fixes the va_list handling in defaultVaWarn() while in there. It read args three times but only the second and third read from a va_copy: the first vfprintf consumed args itself, so the two reads after it saw a spent va_list and the copy sent to the browser lost every %s and %d. It printed "Bad thing: [br]" where the message was "Bad thing: %s<br>". Every read now takes its own copy, and the buffer is filled with vsnprintf rather than vsprintf. No XSS: on this path defaultVaWarn replaces < and > with [ and ] across the whole formatted message, args included. The other handlers that can report an hUserAbort - earlyWarningHandler and cartEarlyWarningHandler via htmlVaEncodeErrorText, htmlVaWarn, webVaWarn - all run the arguments through vaHtmlDyStringPrintf, which html-encodes & < > / " and '. No caller passes user data as the format string.
  • src/hg/lib/hCommon.c - lines changed 14, context: html, text, full: html, text
  • src/lib/errAbort.c - lines changed 8, context: html, text, full: html, text

d033cea2063e9362949baf5b4d8b837597173a0d Thu Sep 10 05:16:05 2026 -0700

• Address the code review of the Sep 9 commits Faceted composite: text that comes from a hub - a metadata column's description, the name and title of a data type, and the values quoted back in the "could not load the metadata" row - is put on the page as text rather than as markup. The three places built their markup from template strings, so a value carrying angle brackets or a quote was read as HTML: the column description now goes through the shared htmlEncode() once where the header is parsed, and the other two build their elements as nodes. The error row reads better for it as well, since a value with brackets in it used to disappear from the message that was meant to show it. The saved UI state keys on the assembly as well as the metadata id. localStorage is per-origin, so two assemblies whose tracks share a name were sharing one entry, and a row order dragged on one came back on the other over a different set of samples. hgTrackUi passes the database down for it. State saved under the old key is dropped, which costs a facet selection or a page length. Imprinting: the five subtrack description pages link back to the container as hgTrackUi?db=$db&g=$parentTrack, without the hgsid. Native trackDb html is substituted by hgTrackDb as it loads the table, where there is no cart, so ${hgsid} came out empty and the link read 'hgsid=&g=...'. Matches what the Fiber-seq pages already do. The makeDoc note that described the old form is updated with the reason. UniProt otto: README.txt lists all eight things that reach runLog.txt. It had four, and was missing LOCKED, along with PREFLIGHT-FAIL, END and INTERRUPTED. refs #36210 refs #37599 refs #38300
  • src/hg/hgTrackUi/hgTrackUi.c - lines changed 4, context: html, text, full: html, text
  • src/hg/js/facetedComposite.js - lines changed 40, context: html, text, full: html, text
  • src/hg/makeDb/doc/hg38/kaplanImprint.txt - lines changed 6, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/akbariIdmr.html - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/geneimprint.html - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/kaplanImprint.html - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/methBaseAsm.html - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/omimImprint.html - lines changed 1, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/README.txt - lines changed 6, context: html, text, full: html, text

11d5b26d9798079ec5adedc103216ce818e2da5e Thu Sep 10 05:19:27 2026 -0700

• Widen the session data directory hash from 8 to 10 hex characters sessionDirFromNames() named a session's durable data directory with 8 hex characters of md5(sessionName). 32 bits was fine while a directory only had to be unique among one user's sessions, but every anonymous "Share a link" session belongs to the single reserved user "l", which makes it a birthday problem across all of them: two unrelated sessions land in the same directory more likely than not at around 77,000 anonymous sessions, and at 500,000 we would expect about 29 such pairs. Two sessions sharing a directory means cleaning up one takes the other's custom track files with it. sessionDirHashLen is now 10 (40 bits), which moves the even-odds point past a million sessions. The two fan-out levels snapshotSessionDir() added for user "l" do not help here, since they are a prefix of the same hash: they spread the entries over 65536 directories but leave the number of distinct leaf names unchanged. Both directory layouts change name as a result, so snapshotCleaner would have walked past anything written earlier and orphaned its files. Both dir-naming functions grew a hashLen argument, sessionDirHashLenLegacy records the old value, and snapshotCleanAnon() now tries the old spelling of both layouts as well as the new one. Existing sessions keep working either way: the cart stores the absolute path of each durable file, so nothing looks a session's directory up by name except the cleaner. refs #10138
  • src/hg/inc/sessionData.h - lines changed 16, context: html, text, full: html, text
  • src/hg/lib/sessionData.c - lines changed 17, context: html, text, full: html, text
  • src/hg/lib/snapshotSession.c - lines changed 34, context: html, text, full: html, text

543c9ee045ba1832faaa9b75c6dc1e369dffce5a Thu Sep 10 05:21:42 2026 -0700

• Login and Sign out come back to a page that was reached by POST, refs #38192 Clicking a track name in the list below the browser image submits the hgTracks form to hgTrackUi, so the request is a POST even though the track name sits in the URL. The return URL builder threw the query string away for anything that was not a GET, which left a returnto of hgTrackUi?hgsid= alone, and hgTrackUi cannot draw a page from that because the track name is deliberately not kept in the cart. Login and Sign out therefore ended in an error instead of coming back. The query string of a POST lives in the form's action URL, which is the address the browser is showing, so returning to it is no different from the visitor pressing reload. Only the form body is left behind, and the cart already holds what mattered from it. hgTracks stays the exception: its query string can hold a one-shot zoom or drag. Also, hgTrackUi now says which parameter is missing when it is reached without a track name, rather than failing on a bare hash lookup, and hgCollection's own "you must be logged in" link brings the visitor back to hgCollection instead of the sessions page.
  • src/hg/hgCollection/hgCollection.c - lines changed 9, context: html, text, full: html, text
  • src/hg/hgTrackUi/hgTrackUi.c - lines changed 8, context: html, text, full: html, text
  • src/hg/lib/wikiLink.c - lines changed 10, context: html, text, full: html, text

d2be4681e2c8493bf9dfe755037e76d2b784e9ff Thu Sep 10 05:24:00 2026 -0700

• uniprot otto: bigPslUniprot.as was missing the isMain field The committed .as declared 46 fields while doUniprot writes 47: pslToBigPsl() appends isMainStr as the last column (doUniprot line 1331, "primary sequence" or "alternative isoform"), and trackDb/uniprot.ra refers to it twice as "skipFields isMain". The field has been missing from this file since at least 2021; the copy in /hive/data/outside/otto/uniprot had been fixed by hand and never committed, which is why runs kept working and nobody noticed. I broke the running update by rsyncing this stale copy over the working one, and bedToBigBed stopped with "Expecting 46 words line 1 ... got 47" after the parse had already finished. Adding the field to the tree copy fixes both the immediate failure and the drift. Verified against the input that failed: bedToBigBed now writes 44142 records with 47 fields and both extra indexes, every record has exactly 47 columns, isMain reads back as 21362 "primary sequence" and 22780 "alternative isoform", and all eleven fields named by uniprot.ra's labelFields, mouseOver and skipFields resolve in the result. refs #38300
  • src/hg/utils/otto/uniprot/bigPslUniprot.as - lines changed 1, context: html, text, full: html, text

ac3191c7e768e8c0c8f6c376526f029ad39f4e1b Thu Sep 10 05:57:18 2026 -0700

• hgHubConnect: on mirrors, show the Hub Upload tab as instructions, not a dialog On a site that is not the login host the tab used to render the full upload UI with a warning dialog layered over it. Show only a message instead: where to upload, a direct link to the Hub Upload tab on the US site, and how to get an uploaded hub onto this site via Connected Hubs or a hub-connecting URL. refs #38323
  • src/hg/js/hgMyData.js - lines changed 28, context: html, text, full: html, text

1ea512b92ecb6ae35dae658eac7eb1dc5a03550e Thu Sep 10 05:58:37 2026 -0700

• hgHubConnect: let mirrors show the API key section without hubSpace, refs #38323 The API key section was gated on storeUserFiles && showHubApiKey, so a mirror could only offer key generation by also turning on the whole hubSpace upload stack (tusd endpoint, tusdDataDir, hubSpaceUrl). But keys are also used to bypass the download CAPTCHA and live in each machine's own hgcentral, so botDelay tells a genome-euro user that keys are server-specific and sends them to euro's hgHubConnect, where the section was not being printed. Gate on showHubApiKey alone, as before e9a3e487007. Mirrors still need showHubApiKey=on in hg.conf.
  • src/hg/hgHubConnect/hgHubConnect.c - lines changed 3, context: html, text, full: html, text

0a5f9cb363719d91237e0ec3c4b8831ad447daca Thu Sep 10 06:45:33 2026 -0700

• hgTrackUi: a missing track name is bad input, not a stack dump, refs #38192 hgTrackUi aborts when the URL carries no g= parameter, which happens with a hand-edited or truncated address and with the crawlers that trim query strings. errAbort routes that through the stack dump handler, so hg.conf browser.dumpStack turns each one into a gdb backtrace in the error log and about a third of a second of wait4. hUserAbort exists for errors that come from user input: same message on the page, one line in the log.
  • src/hg/hgTrackUi/hgTrackUi.c - lines changed 3, context: html, text, full: html, text

57490a621bd10c9c7810bdf474b7c801027d237d Thu Sep 10 06:56:54 2026 -0700

• Use one blue for the menu bar on every page, refs #38206 The bar came in three shades: #2636d1 from nice_menu.css on hgTables, hgBlat, hgc and the other cart CGIs, #00457c on hgTracks through its own set of ids, and #003a72 on the gateway and the static pages, which override nice_menu.css afterwards. They are all #003a72 now, the house deep navy already used for the gateway banner, the footer and the buttons. hgTracks no longer rewrites main-menu-whole and home-link into its own ids, so its bar and its house icon come from the same rules as everyone else's, and hgTracksHomeIconSprite.png goes away - the shared sprite is white on a transparent background and sits on whatever blue the bar has. gbStatic.css keeps its own copy of the color on purpose; the comment there says why.
  • src/hg/hgTracks/menu.c - lines changed 3, context: html, text, full: html, text
  • src/hg/htdocs/images/hgTracksHomeIconSprite.png - lines changed 0, context: html, text, full: html, text (a binary file or whitespace-only change or file-permission change shows no diff)
  • src/hg/htdocs/staticStyle/gbStatic.css - lines changed 4, context: html, text, full: html, text
  • src/hg/htdocs/style/gbAfterMenu.css - lines changed 6, context: html, text, full: html, text
  • src/hg/htdocs/style/gbStatic.css - lines changed 4, context: html, text, full: html, text
  • src/hg/htdocs/style/jWest.afterNiceMenu.css - lines changed 6, context: html, text, full: html, text
  • src/hg/htdocs/style/nice_menu.css - lines changed 35, context: html, text, full: html, text

7faad3c1104fce7a96b3aff918dcc5546035824c Fri Sep 11 05:23:29 2026 -0700

• small docs update after google request by email
  • src/hg/makeDb/trackDb/human/alphaGenome.html - lines changed 1, context: html, text, full: html, text

5783988b48ed0722ce2398b50bc1a422911c1fa7 Fri Sep 11 06:02:22 2026 -0700

• uniprot otto: teach the gene-model search about GenArk assemblies findBestGeneTable only knew how to look for MySQL tables, so on a GenArk assembly it found nothing and fell through to BLAT, which then died on a 2bit path that does not exist for a hub assembly. GenArk keeps its gene models as bigBed files in the hub instead. Search order, best first: catGenes Comparative Annotation Toolkit, shipped as a contrib collection (track hprcCatGenes, contrib/hprc2annot/catGenes.bb) ncbiRefSeq the GenArk RefSeq gene track, bbi/*.ncbiRefSeq.bb ncbiGene the annotation the submitter sent to GenBank with the assembly augustus ab initio, present nearly everywhere, so it is the last resort Matched by glob rather than by constructed name: files under bbi/ carry the full asmId including its assembly-name suffix, while the hub directory is named with the short accession. A dbDb row is recognised as GenArk by nibPath starting with "hub:", cached so the lookup happens once per db. Checked against the assemblies that are actually blocking the update: the twelve GCF ones resolve to ncbiRefSeq, GRCz12ab and calJac240_pri have no RefSeq or submitted annotation and fall to augustus, and on an HPRC assembly that carries both, catGenes wins over augustus as intended. Classic assemblies are untouched - genArkHubDir returns None for them and hg38/hg19/mm39/panPan3/rn6 still resolve exactly as before through the MySQL path. This is the search only. Building from these still needs the transcript fasta and PSL to come from the hub rather than from MySQL. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 49, context: html, text, full: html, text

ef779a5a2ed508cb00f0b0139a12d696e439ee5f Fri Sep 11 06:06:55 2026 -0700

• new hg38 track hprcRdt: reference-divergent transcripts from 206 HPRC Release 2 genomes Added as a third child of the existing long_read_transcripts superTrack, alpha only for now. Data from Max Marin (DFCI), a bigPsl of RDT cluster representative sequences aligned to GRCh38: 180,464 alignments of 120,451 distinct sequences from 412 haplotypes. Rebuilt from the submitted file only to add a name index, refs #33822
  • src/hg/makeDb/doc/hg38/hprcRdt.txt - lines changed 73, context: html, text, full: html, text
  • src/hg/makeDb/scripts/hprcRdt/hprcRdtBuild.sh - lines changed 53, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/hprcRdt.html - lines changed 134, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/hprcRdt.ra - lines changed 25, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/long_read_transcripts.ra - lines changed 1, context: html, text, full: html, text

dcf390fefe68c32f26bd32717bd858190e8875a0 Fri Sep 11 06:06:58 2026 -0700

• uniprot otto: build the transcript files for a GenArk assembly from the hub A GenArk assembly has no MySQL database, so everything makeTranscriptFiles did had to come from somewhere else. It all exists in the hub: bigGenePredToGenePred gene bigBed -> transcripts.gp genePredToFakePsl -chromSize=<hub> -> transcripts.psl getRnaPred -genomeSeqs=<2bit> -> transcripts.fa Both of those options exist precisely so the tools do not have to go through chromInfo, so no database is involved anywhere in the chain. Also made hub-aware, in the same call chain: - chrom.sizes for the bigPsl and bigBed steps, which used /hive/data/genomes/<db>/chrom.sizes and there is no such directory - getTransIds, which read the transcript IDs out of a MySQL table; the fasta we just built holds them all, so read them from there as the refGene case does - writeMapDesc, which read ncbiRefSeqVersion.txt; date the models by the gene bigBed we actually read instead Helpers genArkTwoBit and genArkChromSizes glob for the single matching file and abort if there is not exactly one, since the hub names those with the short accession while the files under bbi/ carry the full asmId. Verified on bonobo mPanPan1_v2.0: 95418 transcripts, identical ID sets across the fasta, genePred and PSL; getTransIds returns those 95418 without touching MySQL; writeMapDesc dates the models 2025-06-02, which matches the date embedded in the build's own ncbiRefSeq GTF filename. hg38 still reads its real ncbiRefSeqVersion.txt and is otherwise untouched. Still to do: the outputs. /gbdb/<db>/uniprot and a trackDb .ra stanza are not how a GenArk assembly is served. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 54, context: html, text, full: html, text

953e29496b012cc36ee7933ce94cefea2f66ec38 Fri Sep 11 09:21:22 2026 -0700

• uniprot otto: align proteins with miniprot instead of BLAT or Augustus Where an assembly has no gene models worth mapping through, the pipeline used "blat -q=prot -t=dnax", which is slow enough that hs1 was excluded from the whole job over it. On GenArk assemblies the alternative was Augustus, which is an ab initio prediction, so mapping UniProt through it stacks its errors on top of ours. Aligning the proteins straight to the genome avoids both. miniprot 0.18 built from github into /hive/data/outside/otto/uniprot/bin/miniprot, linked against system libraries only and runnable by otto. miniprotProteins() replaces blatProteinsKeepBest(). miniprot reads fasta rather than 2bit, so the genome is unpacked to a temp file and removed again once the alignment is done. Its GFF3 needs some care before gff3ToGenePred will take it: the ##PAF meta lines and the capitalised Rank/Identity attributes have to go, and alignments are named MP000001 with the UniProt accession hidden in Target=, so the ids are rewritten to the accession. The uniquifying suffix that GFF3 requires is stripped again afterwards, leaving the bare accession as qName, which is what the rest of the pipeline keys on. Augustus is dropped from the GenArk gene source list entirely, so the order is now catGenes, ncbiRefSeq, ncbiGene, then miniprot. The classic "blat" fallback becomes miniprot too. Checked on real data: 300 bonobo UniProt proteins against one 227 Mb chromosome give 138 alignments over 102 distinct proteins, pslCheck reports 138 checked and 0 failed, the temp genome fasta is cleaned up, and the aligner version is recorded in the mapping stats. GRCz12ab and calJac240_pri, the two assemblies that had nothing but Augustus, now resolve to miniprot. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 82, context: html, text, full: html, text

d886333094fd66bac7a872f685c3464a6d667244 Fri Sep 11 09:52:30 2026 -0700

• uniprot otto: run miniprot on the cluster, with the RAM and CPUs it actually needs para's default RAM per job is the node's RAM divided by its CPU count, which for a 16 CPU job is far less than miniprot wants, and without -cpu parasol would pack more of these onto a node than it has cores for. Both are now passed. The RAM figure is measured rather than guessed. Peak RSS is linear in genome size at about 11 GB per Gb of sequence (2.60 GB for 227 Mb, 4.69 GB for 424 Mb, 8.93 GB for 811 Mb) and is flat in the thread count, because the index is built once and shared: 2.49 GB at -t 1 against 2.60 GB at -t 16 on the same sequence. So the reservation is sized off the genome alone, with headroom, and floored at 8g. Zebrafish comes out at 21g and marmoset at 42g. Note that -cpu is a scheduling reservation, not a limit: the job still sees every core on the node. It stops parasol oversubscribing the machine, and pairing it with miniprot's own -t is what makes the two agree. Checked with a real one-job batch at -cpu=16 -ram=42g: accepted, ran on a compute node, and the binary is reachable from there. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 38, context: html, text, full: html, text

edf45d7b82651acd7bb9cf00cc492e9b19936f1f Fri Sep 11 09:58:18 2026 -0700

• uniprot otto: write a GenArk contrib collection instead of /gbdb symlinks A GenArk assembly is not served out of /gbdb/<db>/, so the symlinks and the trackDb .ra stanza that classic assemblies get do not apply to it. Its files belong in a contrib collection. For a GenArk assembly the pipeline now writes contrib/uniprot/<accession>/ with the bigBeds and a trackDb.txt, and skips three things that only make sense for a classic assembly: the /gbdb/<db>/uniprot symlinks, the version.txt symlink, and the goldenPath archive copy, whose download path does not exist for a hub assembly. The trackDb comes from the same template the archive hub uses, with three changes: track names must not carry the release or they would change every month and break saved sessions; the data files sit beside trackDb.txt rather than in a per-release subdirectory; and dataVersion has to be the literal release string because a contrib trackDb cannot read a /gbdb path. It deliberately stops there. Nothing is symlinked into the GenArk build directories and no hub.txt is touched: installing the collection is a separate, deliberate step with "genark addContrib uniprot", not something a monthly data update should do on its own. Checked by writing a collection for one assembly and running the genark tool's --dry-run over it: all 16 bigDataUrls resolve to files that are present and non-empty, the html path resolves to the shared docs page, and the tool reports it would wire the block into alpha.hub.txt only. The shared docs page is the trackDb description with the archive-hub paragraph and the /gbdb download links removed, since neither exists for a GenArk assembly. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 80, context: html, text, full: html, text

89e1e4c6ebefe29b7219fc98417b73b6a1755f05 Fri Sep 11 10:07:02 2026 -0700

• uniprot otto: handle hub assemblies that are not GenArk, and bring hs1 back in Not every assembly with a "hub:" nibPath is a GenArk assembly. hs1 is served as a hub from /gbdb/hs1/hubs but keeps its track files in /gbdb/hs1/ exactly like a classic assembly, so treating every hub as GenArk put its gene model search in the wrong place and would have sent its output to a contrib collection it does not belong in. Split the two questions. genArkHubDir still answers "is this served as a hub, and from where", and a new isGenArk asks the narrower question the output routing actually cares about: is this under /gbdb/genark with the sharded layout. The /gbdb symlinks, the version.txt symlink, the goldenPath archive and the contrib collection now all key on isGenArk, so hs1 gets the classic treatment. The gene model search now takes several globs per source and looks through both the hub directory and /gbdb/<db>/, which covers both layouts: a GenArk assembly keeps its models under bbi/ with CAT in a contrib collection, hs1 keeps them in /gbdb/hs1/<trackName>/. That makes hs1 work, so it comes out of notAutoDbs. It was excluded for having "no good gene model" and because the BLAT protein search took forever; it actually has CAT genes, 234903 of them in catLiftOffGenesV1, which is the best source we look for, and BLAT is gone anyway. Resolutions checked: hs1 -> catGenes from /gbdb/hs1/catLiftOffGenesV1, not GenArk; mPanPan1_v2.0 -> ncbiRefSeq from its hub bbi/, GenArk; GRCz12ab -> miniprot, GenArk; hg38 and panPan3 unchanged on the MySQL path. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 42, context: html, text, full: html, text

55a768d2e0ced94dc3ba7ab322b24daa67ea3570 Fri Sep 11 10:09:16 2026 -0700

• uniprot otto: resolve the 2bit and chrom.sizes for every kind of assembly The first GenArk run stopped on hs1 with "expected exactly one chrom.sizes file in /gbdb/hs1/hubs, found 0". hs1 is served as a hub but keeps its 2bit at /gbdb/hs1/hs1.2bit and its chrom.sizes at /hive/data/genomes/hs1/chrom.sizes, exactly where a classic assembly keeps them; only a real GenArk assembly keeps them in the hub directory. twoBitFname now keys on isGenArk rather than on being a hub, and a matching chromSizesFile does the same, so both are resolved the same way everywhere. The transcript building still uses the bigBed path for any hub assembly, because hs1 has no MySQL tables either, but it asks these two helpers for the supporting files. Checked that all four resolve to files that exist: hs1 and hg38 to the classic locations, mPanPan1_v2.0 and GRCz12ab to their hub directories. Also fixed the README, which documented the option to limit the run as --onlyDbs. It is --dbs; --onlyDbs is the internal dest name and the run fails with "no such option". refs #38300
  • src/hg/utils/otto/uniprot/README.txt - lines changed 1, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 19, context: html, text, full: html, text

155514c1985d0fdc25f4a33f4e1786f35b47b09d Fri Sep 11 11:11:49 2026 -0700

• uniprot otto: make CAT transcript names unique before deriving the fasta and PSL The hs1 run got through a 52 minute BLAST batch with no crashes and then stopped in pslMap with Error: inPsl RBMY1F-1 tSize (1887) != mapPsl RBMY1F-1 qSize (1718) CAT names its transcripts after the source gene, so paralogs share a name. The hs1 CAT set has 777 duplicated names among 234903 transcripts, and RBMY1F-1 is three Y chromosome paralogs of 1810, 1718 and 1887 bases. The protein-to-transcript alignment and the transcript-to-genome alignment then disagree about which transcript the name refers to and pslMap refuses to map. This is a property of CAT, not of hub assemblies: the GenArk ncbiRefSeq sets use accessions and have no duplicates at all. Uniquify the names on the genePred before the fasta and the PSL are derived from it, so the two can never disagree. Verified against the real hs1 CAT data: 234903 rows and 234903 distinct names, no duplicates in the genePred, fasta or PSL, the three RBMY1F-1 paralogs now separate, and every one of the 234903 PSL rows has a qSize equal to the length of its own fasta sequence, which is the invariant pslMap enforces. On bonobo ncbiRefSeq it is a no-op: 95418 rows, no name changed. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 16, context: html, text, full: html, text

f7a079f0d9b56429eb44109f1d05be2ab759294a Fri Sep 11 12:13:22 2026 -0700

• uniprot otto: the duplicate-name count reported zero every time The line that was meant to say how many CAT transcript names had been made unique subtracted the distinct name count of the file it had just uniquified, which is equal to its row count by construction, so the answer was always zero and the message never appeared. The rename itself was working: the hs1 run has 3769 rows carrying a -dup suffix and all 234903 names distinct. Count the names in the input instead, and say which of the two numbers is which. Verified against the real hs1 files: 234903 rows, 231134 distinct names before, so it now reports 3769 renamed. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 4, context: html, text, full: html, text

947dcb525b76d579bd6ab425cf71a221d9bab347 Fri Sep 11 12:50:09 2026 -0700

• uniprot otto: give the unfiltered gene-track case a map source hs1 cleared pslMap after the duplicate-name fix, ran the full 91 minute alignment, and then died writing the bigPsl with KeyError: 'default' at mapSource = accToMapSource["default"] buildSelectFile returns an empty dict when no UniProt cross-reference matches the ids in the gene track, and pslToBigPsl looks up accToMapSource["default"] for every protein it writes. That branch was unreachable before: every gene table findBestGeneTable could return was handled somewhere else, with augustus and the direct protein alignment each setting their own default. catGenes and ncbiGene are the first to reach it. It now returns {"default":"best"}, the same answer the augustus case gives, which is the honest description: the alignment is the best match we found, we just cannot name the transcript evidence behind it. Hardened the lookup as well. protMapSource on the supported path only ever gets per-accession entries and never a "default" key, so any accession missing from it would have raised the same KeyError on hg38. It now falls back to "best" rather than throwing away hours of cluster work over one protein. Also reworded the log line, which said "No supported gene track found" while a perfectly good gene track was in use; the missing thing is a cross-reference that matches its ids. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 12, context: html, text, full: html, text

5bf1bd7150fc659d3a1bd9f34e4daad593728cb3 Fri Sep 11 17:55:11 2026 -0700

• refGene never got the transcript codon number: the genbank CDS tables are in hgFixed, so their names are database-qualified and hTableExists could not see them. Use sqlTableExists, which can, refs #38298
  • src/hg/hgTracks/cds.c - lines changed 9, context: html, text, full: html, text

d07356d76401059fbe6c2d0890b1490546a62a4e Mon Sep 14 02:54:50 2026 -0700

• hg38 Fiber-seq: reissued GM12878 data and a nucleosome density track, refs #36210 The lab reprocessed GM12878 (PM00001) and replaced the files in place under the same hash directory. Ten of its twelve files changed; a size sweep over all 41 samples confirmed no other sample is affected. This fixes the two placeholder haplotype accessibility bigWigs that covered a single base, so that overlay now draws real data for the sample that comes up by default. Its peak calls changed substantially as well, 429,883 source peaks before and 196,742 now, which is noted on the description page since figures made from the first version of the track will not reproduce for GM12878. The downloader now fetches into <file>.part and moves it into place when complete. It used curl -C - straight onto the final file, which is right for an interrupted transfer and silently corrupting when the server has replaced the file: it would have appended the tail of the new 5.2 GB hap1 file to the 512-byte stub, and the size check afterwards would have passed. It also takes an optional list of accessions now, to refresh one sample without walking all 41. Nucleosome density (all.nucleosome.coverage.bw) was sent separately and is not in the lab's own hub. It is on the server for all 41 samples and is added as a seventh data type in the compendium. Unlike every other wiggle here it is a read depth rather than a percentage, so it cannot take fixed viewLimits: the genome-wide mean runs from 25 to 142 across samples with sequencing depth and single loci reach 1.7e5. It is drawn with autoScale, which the description page explains, and reads as the complement of the accessibility signal.
  • src/hg/makeDb/doc/hg38/fiberSeq.txt - lines changed 75, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqCheck.sh - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqDownload.sh - lines changed 30, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqTrackDb.py - lines changed 33, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeq.ra - lines changed 699, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeqCompendium.html - lines changed 30, context: html, text, full: html, text

75e828960283291546d2c1a27845e2cf3823adcd Mon Sep 14 05:29:02 2026 -0700

• uniprot otto: the miniprot cluster job needs absolute paths GRCz12ab failed with the parasol job crashing four times, return 1, no output. The wrapper I wrote ran miniprot -t 16 --gff protToGenome/GRCz12ab/.../genome.fa fasta/7955.fa > $1 and a parasol job runs with its working directory set to the batch directory, not to the directory the pipeline runs in, so neither input existed from the job's point of view. The BLAST batch next door gets away with relative paths because it cds into its own workdir and its jobList is written relative to that; this batch directory sits a level deeper and its paths were relative to the otto root. Every path in the wrapper, the jobList command and the output check is now absolute. Verified on the cluster against the real 1.48 Gb zebrafish genome: successful batch, a 195 MB GFF with 93518 mRNA records. refs #38300
  • src/hg/makeDb/doc/hg38/fiberSeq.txt - lines changed 22, context: html, text, full: html, text
  • src/hg/makeDb/scripts/fiberSeq/fiberSeqTrackDb.py - lines changed 48, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeq.ra - lines changed 699, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/fiberSeqCompendium.html - lines changed 23, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 7, context: html, text, full: html, text

66db3fb64ca38726355efb68c340c93fd83282bb Mon Sep 14 05:33:23 2026 -0700

• uniprot otto: the run watcher could never see a run end notifyRun.sh decided whether a run was still going with pgrep -f "doUniprot run" which matches far more than the pipeline: the shell that launched the watcher carries that string in its own command line and stays alive for as long as the watcher does, and so does every status command anyone types. The test was therefore true forever. The watcher started on 9 September sat in its loop through a failure on 11 September and three days of nothing, never reached the code that reports the outcome, and never sent the mail it exists to send. Its log was empty the whole time, which looked exactly like healthy silence. Both process searches are now restricted to the user the pipeline runs as, otto by default and overridable as the fourth argument. Measured against the live process list: the old pattern matched 5 processes where there is 1 run, the new one matches exactly 1. refs #38300
  • src/hg/utils/otto/uniprot/notifyRun.sh - lines changed 9, context: html, text, full: html, text

b50a995e326f37cb2248e9a04628c15fb1b58215 Mon Sep 14 05:40:04 2026 -0700

• My Sessions no longer hides a user's own "__" sessions, refs #38313 Both My Sessions listings decided a row was a share token by testing the session name for the "__" prefix, so they also hid sessions a user had named that way themselves - eleven of them on the RR, across four accounts, and their owners could no longer rename, describe, reshare or delete them. The prefix is a naming convention we follow, not a namespace we own. The authoritative mark is the "snapshotType <type>" line saveSnapshotSession() already writes into the settings column, so test that instead. Both queries already select settings, so no query changes. snapshotTypeFromSettings() walks the settings lines rather than calling raFromString(): it runs once per listed row, and a hash there costs ~600ns and three allocations for every session that has a description, against ~30ns and none for the walk. Rows with empty settings, the common case, short-circuit in both.
  • src/hg/hgSession/hgSession.c - lines changed 9, context: html, text, full: html, text
  • src/hg/inc/snapshotSession.h - lines changed 32, context: html, text, full: html, text
  • src/hg/lib/snapshotSession.c - lines changed 46, context: html, text, full: html, text

b8320ab5d2dd4d520742faedc2def3144595c450 Mon Sep 14 05:49:34 2026 -0700

• uniprot otto: writeMapDesc crashed on a hub assembly with no gene models GRCz12ab got through miniprot on the cluster and then died writing its lift info: os.path.getmtime(geneBb) -> stat on None The hub branch dates the gene models by the bigBed that was read, but an assembly that has no gene models is precisely the one that reaches miniprot, and there is no bigBed to date. The miniprot case was already handled further down the chain, so the fix is only to test it first. Checked every combination that can occur: GRCz12ab and calJac240_pri, hub assemblies with no models, report "direct"; hs1 dates its CAT models 2022-03-15; mPanPan1_v2.0 dates its GenArk RefSeq 2025-06-02; hg38 still reads its real ncbiRefSeqVersion.txt and rn6 still queries trackVersion. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 12, context: html, text, full: html, text

da869954bf88c67e25a8b68cfd49ad50c5cec7d0 Mon Sep 14 05:58:57 2026 -0700

• uniprot otto: optionally work on several taxa at once The assemblies were processed strictly one after another, and that leaves the cluster mostly idle. Each assembly submits its BLAST batch, waits for it to drain, and then spends the best part of an hour in the single-threaded pslReps that follows before the next assembly submits anything. Measured on the hs1 batch: 332 hours of CPU finished in 30 minutes of wall clock, a speedup of about 660, and then 58 minutes of one core concatenating and filtering 34335 PSL files. With 119 assemblies in the plan and a 1568 CPU cluster, that ordering costs about a day. --taxonThreads=N runs N taxa at a time. It defaults to 1, so nothing changes unless it is asked for; 4 to 6 is a reasonable range. Taxa, not assemblies. The assemblies of one taxon share fasta/<taxId>.fa, which is rebuilt at the start of each taxon, so two threads on one taxon would race on it. Below that everything is per-assembly - protToGenome/<db>, bigBed/<db>, the cluster batch directory - so separate taxa do not share files. os.makedirs calls are now exist_ok, since two taxa starting together can both find a directory missing. A failing worker is escalated, not swallowed. run() reports a failed command with sys.exit(), which raises SystemExit; in a worker thread that would kill only that thread and leave the run looking successful, which is the failure mode this pipeline has a long history of. runTaxa catches BaseException per taxon, names each one that failed, and aborts the run at the end. Checked: a worker calling sys.exit aborts the run, a worker raising an ordinary exception aborts the run, both after the other taxa have still been attempted, an all-good set returns normally, --taxonThreads=1 keeps the old sequential order, and --dbs still selects which taxa run. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 66, context: html, text, full: html, text

99a26061f6cf8d3ee709f3468dda88af74e4049d Mon Sep 14 06:17:21 2026 -0700

• uniprot otto: convert miniprot alignments to PSL properly GRCz12ab cleared miniprot and then died in the annotation lift: Error: inPsl Q98TT6 tSize (336) != mapPsl Q98TT6 qSize (339) The lift maps annotations that are given in protein coordinates, so the mapping PSL has to have the protein as its query, with qSize three times the protein length. Routing miniprot's output through gff3ToGenePred and genePredToFakePsl does not give that: it makes the query the transcript implied by the alignment, so qSize comes out as the aligned CDS length. Measured over 93518 zebrafish alignments, that was wrong for 95% of them - 79% out by exactly one codon, the trailing stop, and 16% out by other amounts where miniprot aligned only part of the protein. pafToPsl cannot do it either: it rejects miniprot's CIGAR, which is splice aware and uses operators it does not know. But every CDS line of a miniprot GFF carries its own "Target=<acc> <start> <end>" giving the protein range that block covers, so the alignment can be reconstructed exactly. New miniprotToPsl does that. Two things it has to get right. On the minus strand miniprot lists blocks in protein order, which is descending genomic order, while a PSL lists them ascending and puts the block starts on the reverse complemented query, with qStart and qEnd still in forward coordinates. And a block whose genomic span is shorter than its protein range implies, where miniprot placed a frameshift, is clamped to the genome, with the overall ranges then derived from the blocks rather than from the protein ranges. Verified against real bonobo alignments: pslCheck reports 65 checked, 0 failed; qSize is three times the protein length for every row; qName, strand and tStart match miniprot exactly for all 65; and tEnd matches for 43, is short by exactly 3 on 12, and those 12 are the ones carrying a stop_codon feature, which is correct since the protein has no stop codon. 26 of the rows are on the minus strand. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 31, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/makefile - lines changed 1, context: html, text, full: html, text
  • src/hg/utils/otto/uniprot/miniprotToPsl - lines changed 180, context: html, text, full: html, text

a77bab60dc03de789c241d97692ba1f672dfddab Mon Sep 14 07:30:12 2026 -0700

• uniprot otto: use the miniprot in /cluster/bin/x86_64 max built the current miniprot into /cluster/bin/x86_64/miniprot, which is the right home for it: shared, on the PATH the cluster nodes already use, and maintained with the rest of the kent binaries rather than by this pipeline. Dropped the copy I had built under the otto directory, so there is only one. Both were 0.18-r281; checked that otto can run the system one. refs #38300
  • src/hg/utils/otto/uniprot/doUniprot - lines changed 1, context: html, text, full: html, text