Commits for mspeir

v503_base to v504_preview (2026-08-31 to 2026-09-07) v504

4a635b51567e14c426be09c8f027c2ff43179b57 Wed Sep 2 14:05:32 2026 -0700

• store: add chain file and Docker image sections, update the EULA to the Nov 2022 revision Adds a UCSC chain files section covering the per-user commercial license, and a Genome Browser Docker image section. The two LiftOver entries now point at the chain file section, since the over.chain files are licensed separately from the program. Replaces store/UCSC_GB_EULA.pdf with the November 22, 2022 revision. That is the revision whose Appendix B lists UCSC Chain Files and CRISPOR, so every product on the page can link to a single agreement. It also broadens "Commercial Purposes" to any use by a for-profit organization, which now applies to the source code, LiftOver, GBiC and Docker entries as well. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/htdocs/store.html - lines changed 151, context: html, text, full: html, text
  • src/hg/htdocs/store/UCSC_GB_EULA.pdf - lines changed 0, context: html, text, full: html, text (a binary file or whitespace-only change or file-permission change shows no diff)

3334b0dc02b7d88cd21a987b3765b30d581339cd Wed Sep 2 14:09:21 2026 -0700

• store, docker help: document the prebuilt Docker Hub image and correct the image size Braney pointed out on the ticket that neither page mentions genomebrowser/server, the image the weekly build pushes to Docker Hub (buildReleaseDocker.sh). It is rebuilt every release, latest currently points at v502, and one tag covers amd64 and arm64, so pulling it is the better path for most people than building the Dockerfile. Both pages now say so, and docker.html gains a section with the pull and run commands ahead of the build instructions. The store page said the built image is around 3GB, which was true a few releases ago. Building the current Dockerfile gives about 5.6GB on disk, and the published image is a 2.6GB download that unpacks to about 7.4GB, so the page now tells people to plan on 8GB for the image alone, separate from track data. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/htdocs/goldenPath/help/docker.html - lines changed 32, context: html, text, full: html, text
  • src/hg/htdocs/store.html - lines changed 4, context: html, text, full: html, text

d8a3c66525910314ea3cdcc00b694138e769e699 Wed Sep 2 14:22:52 2026 -0700

• store: make the internal links relative Every internal link on the page was written against https://genome.ucsc.edu, so on a sandbox or a mirror the page sent people to production instead of to the copy they were looking at. That bit us reviewing this ticket: the "Terms and conditions" links fetched the production EULA rather than the new one sitting next to the page. Eight links: the five EULA links, hgLiftOver, gbic.html and docker.html. The gbic.html link also had goldenpath in lowercase, which production tolerates but the tree does not, so it is now goldenPath like its neighbor. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/htdocs/store.html - lines changed 8, context: html, text, full: html, text

ecb6ac9da06193c59e5a59dee24d6a02872eb407 Thu Sep 3 09:39:08 2026 -0700

• store: fix the last two absolute links on the page Code review caught two links d8a3c66 missed. The gbic.html one in the GBiC firewall list is the same problem that commit fixed and gets the same fix: it is now relative, so it resolves against whichever copy of the page you are reading. The other is not the same problem. It was a bare URL in parentheses rather than a link, and genome.ucsc.edu/downloads.html is not a page in htdocs at all, it is a redirect to hgdownload. Making it relative would have given a 404 on every host including production. It now links to hgdownload.gi.ucsc.edu/downloads.html, matching the two other download links already on this page, and the sentence loses the stray period it had before the parenthetical. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/htdocs/store.html - lines changed 2, context: html, text, full: html, text

5b54eba6619b55ebe18c183dbc65c26e7da82e18 Thu Sep 3 09:39:15 2026 -0700

• docker help: promote the prebuilt image section to h2 and split out the build path Code review pointed out that the section 3334b0d added was an h3 nested under "Using Docker Desktop for UCSC Genome Browser" while every other entry in the page's table of contents is an h2. It is now an h2. That alone left the four build-your-own h3s that follow it (Obtaining a Dockerfile, Creating a Image, Creating a Container, Using Docker Desktop to Create a Container) nested under the prebuilt image section, which is the wrong parent since they describe building rather than pulling. They now sit under a new "Building the Image Yourself" h2, which also gets a table of contents entry. Every h2 on the page is now listed in the contents and every h3 is under the h2 it belongs to. refs #37764 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/htdocs/goldenPath/help/docker.html - lines changed 5, context: html, text, full: html, text

dcb1fa309fa0157eb7076263b1a0a36b0c7babec Thu Sep 3 18:28:29 2026 -0700

• Add trackLists otto generator: tracks we cannot redistribute, tracks that update themselves, and contributed tracks, refs #37781 Builds one page answering the three questions mirror sites keep asking. List 1 unions several tests rather than relying on one, because no single trackDb setting marks every restricted track: tableBrowser off is the usual marker but OMIM instead sets tableBrowser noGenome with a noGenomeReason naming its distribution terms, so a query for "off" alone misses it. Not all noGenome is about licensing either -- CRISPR and JASPAR set it because a genome-wide query times out -- so the reason text is what separates them. The convention of putting restricted files under an underscore directory is real but partial: decipher, mexbb, spliceAI, cosmicRegions and hgmd are restricted and are not under one. It also checks the download server both directions. A trackDb track whose MySQL table exists here but is missing from hgdownload is almost certainly restricted, and a file we call restricted that hgdownload still serves is a bug the script prints so cron mails it. mkPage.py omits that second cross-check unless --internal is given, since naming reachable restricted files on a world-readable page would defeat the point. The generated page is not committed, matching allTips.html and thumbNailLinks.html, which live only in htdocs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/utils/otto/trackLists/README.txt - lines changed 64, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/collect.py - lines changed 357, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/mkPage.py - lines changed 178, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/trackLists.sh - lines changed 39, context: html, text, full: html, text

fbca0ee885a9c343e160f7ab7d584d44fa1cf8a5 Thu Sep 3 18:36:22 2026 -0700

• trackLists: emit the page in the standard static-page style, refs #37781 Drops the hand-rolled stylesheet and uses the house shape instead: the gbPageStart/gbPageEnd SSI includes for the menu bar and footer, h2 per section with an h6 contents list, and bare tables, which gbStatic.css already styles through the .gbsPage wrapper. Source is ASCII-only, lowercase tags, and no line over 100 characters, per the QA conventions; long cells wrap rather than run off. The deploy step now chmods the page to 775. Apache runs the SSI includes on a .html file only when its execute bit is set (XBitHack), so without it the page is served verbatim and the reader gets bare content with no menu bar and no stylesheets. allTips.html is in that state on the RR today, which is how the behaviour came to light. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/utils/otto/trackLists/README.txt - lines changed 6, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/mkPage.py - lines changed 281, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/trackLists.sh - lines changed 6, context: html, text, full: html, text

497f2788d49fda531c0836618688f954486eb87c Fri Sep 4 09:44:13 2026 -0700

• trackLists: do not let the public page claim the download cross-check is clean, refs #37781 The static-page rewrite collapsed mkPage.py's three-way branch on the hgdownload cross-check into two, so the public run fell into the all-clear wording whether or not collect.py had found a restricted file that is still reachable. collect.py's current output has three such files, so the page was asserting the opposite of what the check found. Restored as three cases: the public page says only that the check runs and that anything found is reported privately, the --internal page keeps the table of paths, and the all-clear wording is used only when the list really is empty. The comment explaining why the public branch must not be merged back into the all-clear one is back too. collect.py counted any HTTP code that was not literally 404 as reachable, which made a curl timeout or a failed connection (empty output, or 000) look like an exposed file. That mails false alarms and, now that the branch above depends on it, would drop the all-clear line from the public page on a network blip. Only 2xx and 3xx count as served, 4xx as blocked, and anything else is reported separately as not checked, in collected.json and on stderr. Also: the remaining unquoted interpolations into shell=True commands go through the existing q() helper, rows_by_track() no longer builds the "why" set that the new table does not use, and US spelling throughout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/utils/otto/trackLists/collect.py - lines changed 28, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/mkPage.py - lines changed 22, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/trackLists.sh - lines changed 1, context: html, text, full: html, text

d92252cc474cc9fc079cbad1b020446d97d50428 Fri Sep 4 10:04:48 2026 -0700

• relatedTracks.ra: 275 new track relationships from the #38016 audit Audits all six assemblies that appear in relatedTracks.ra (hg38, hg19, mm39, mm10, hs1, danRer11) for track pairs that a reader would want linked and that the file did not already connect. Adds 172 relationships, which come to 275 lines, taking the file from 139 relationship lines to 414. Version successions run one way, from the older track to the newer one, on the grounds that someone reading the current track does not need a pointer back to the retired one, while someone who lands on the retired track does need sending forward. Archives are the exception and stay reciprocal, since browsing an archive to reach a specific release is the point of it. Direction was checked against each track's longLabel rather than inferred from the table name, which mattered for the COVID GWAS pair: covidHgiGwas is release 3 and covidHgiGwasR4Pval is release 4, so the table name reads older than the track is. The variant impact predictors are capped at eight related tracks rather than five, because comparing one score against another is the main reason anyone opens them. That cap raise is what let spliceImpactSuper and its four children, the three constraintSuper children, mutScore and caddSuper1_7 get any related tracks at all; every one of them had none. Also relates three pairs one way where the older half was frozen years ago: Platinum Genomes to Genome In A Bottle, the Coriell cell line CNVs to ClinGen, and CCDS to MANE. refs #38016 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/makeDb/trackDb/relatedTracks.ra - lines changed 338, context: html, text, full: html, text

a98dfd4707e69c1617982e26ddf3310a8b7a3277 Fri Sep 4 10:11:06 2026 -0700

• relatedTracks.ra: relate OMIM to GenCC, Orphanet and G2P The gene-disease cluster added in the previous commit was built around GenCC as a hub, with every spoke sharing the reason "one of the sources GenCC aggregates". OMIM is not a GenCC submitter, so it never matched that phrasing and ended up connected only to GeneReviews, despite being the reference the other resources cite. The related-track cap was not the constraint; GenCC was at four of five and OMIM at one of five. Adds three reciprocal pairs on hg38 and hg19: OMIM with GenCC, with Orphanet and with G2P. GenCC now sits at its cap of five and OMIM at four. PanelApp is left out on purpose. Its panels do cite OMIM, but including it would put five gene-disease resources on OMIM's page mostly describing one another, and PanelApp is the most specialized of them. refs #38016 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/makeDb/trackDb/relatedTracks.ra - lines changed 12, context: html, text, full: html, text

da5860e01a546d10d1276880758dcfaa8e573467 Fri Sep 4 10:24:18 2026 -0700

• relatedTracks.ra: connect RepeatMasker, GWAS Catalog, HGNC and CRISPR Follow-up to the #38016 audit, which organized its work by writing one reason template per cluster of tracks. That approach only linked a track if it matched some cluster's phrasing, so tracks that are conceptually adjacent but described differently were skipped. Four such cases, all of which had no related tracks at all before this: RepeatMasker. The audit's repeat cluster was written around tandem repeats, so rmsk, which annotates interspersed repeats, fell outside the wording. It is one of the most widely used tracks in the browser and had nothing. Now relates to Interrupted Repeats and RepeatMasker Visualization, which are derived from it, to Simple Repeats as the tandem counterpart, and to WindowMasker + SDust as the library-free alternative. GWAS Catalog, SNPedia and Variants in Papers. The audit built a cluster for variant impact prediction and another for gene-disease curation, but none for variants reported in the literature, so all three were left isolated. GWAS Catalog also now relates to the GTEx cis-eQTLs, since looking for the gene behind an association signal is a common reason to open either one. HGNC. The gene symbol authority, unlinked from the gene tracks that use its nomenclature. Now relates to GENCODE, NCBI RefSeq and MANE. CRISPR. The 200 bp window track is superseded by the 10 kb one, computed by the same pipeline over a wider region, so it links forward to it and not back, the same rule the audit applies to other version successions. Counts: 32 relationships, 70 lines, taking the file to 487 relationship lines. refs #38016 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/makeDb/trackDb/relatedTracks.ra - lines changed 70, context: html, text, full: html, text

3181982d60858062c80fd6db95294172f41525be Fri Sep 4 16:21:50 2026 -0700

• Add the otto cron line for cbPingHidden, the monthly reminder to submitters of hidden Cell Browser datasets. It goes in the block whose MAILTO is cells@ucsc.edu, next to the other Cell Browser jobs. The run only queues one draft email per dataset that is due and mails the report; nothing reaches an author until someone reads the queue and runs "cbPingHidden send". Three departures from the line suggested in the script's README. The report address is spelled out, because a bare --mail-report falls back to $USER@ucsc.edu and $USER here is otto. The umask keeps the drafts otto writes into the queue group-writable, so they can still be edited by hand before they go out. And the schedule is the 1st of the month rather than the first Monday: "1-7 * 1" would have been read as the 1st through the 7th OR every Monday, which is roughly ten runs a month, not one. refs #38246 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/utils/otto/otto.crontab - lines changed 6, context: html, text, full: html, text

70ac948e82b41ef316523635c04e5e2c4a89c417 Fri Sep 4 16:57:54 2026 -0700

• trackLists: give the variant frequency projects their own table, move the page to goldenPath/help/mirrorTracks.html, add the otto cron line, refs #37781 Half the restricted list was national sequencing cohorts sitting under varFreqs and phasedVars, and in one alphabetical table they buried the tracks people actually write in about, OMIM, HGMD and DECIPHER. Those cohorts now get a table of their own below the rest. The split is read off the trackDb parent chain, so the next cohort added under varFreqs lands in the right table with no edit here. The page moves off the htdocs root to goldenPath/help/mirrorTracks.html, beside mirror.html, which now links to it. That link goes in src/product/README.txt, the pandoc source mirror.html is generated from. The licensing page link follows the move, and the page title changes with the file name. Also adds the weekly otto line, placed above the HGDB_CONF that would otherwise apply to it, and keeps the job from listing itself as a self-updating track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/htdocs/goldenPath/help/mirror.html - lines changed 11, context: html, text, full: html, text
  • src/hg/htdocs/license/index.html - lines changed 1, context: html, text, full: html, text
  • src/hg/utils/otto/otto.crontab - lines changed 7, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/README.txt - lines changed 15, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/collect.py - lines changed 28, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/mkPage.py - lines changed 54, context: html, text, full: html, text
  • src/hg/utils/otto/trackLists/trackLists.sh - lines changed 6, context: html, text, full: html, text
  • src/product/README.txt - lines changed 9, context: html, text, full: html, text

d953601aa1ab7e04ad424f10ec36d7bbd2c8c34b Sat Sep 5 17:13:58 2026 -0700

• trackLists: say what the malacards table is actually for, refs #37781 The row read "which no track points at", which is true and misleading in the same breath. hgGene's malaCardsSection joins the table against kgXref and builds the MalaCards links on the gene details page, so a mirror that skips the table loses that section. Nothing about the schedule or the hg38-only load changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/utils/otto/trackLists/collect.py - lines changed 2, context: html, text, full: html, text

3522a9acc8256a35230b07f2e7ab3fc08c827f33 Sat Sep 5 17:19:20 2026 -0700

• singleCellSignalsPeaks: correct the hg38 description page counts and cite BrainVar, refs #38219 From the v503 code review. Commit 3aac3982aa2 dropped 5 hg38 subtracks and updated the makeDoc, but the dataset list on the description page was missed, so the page disagreed with both the .ra and the facet menu: - Risk Loci in Alzheimer's and Parkinson's: 1 peak subtrack, not 2 (neuro-degen-atac/peaks.bb was the one dropped) - BrainVar: 9 signal subtracks, not 13, and the text still described tracks "for all nuclei together" -- the 4 combined-stage tracks are exactly the ones dropped, so every remaining BrainVar track carries a life stage Counts re-derived from the .ra by type (bigWig = signal, else peak): the other 7 hg38 datasets and all 9 mm10 datasets were already right, 929 and 587 total. BrainVar was also the only hg38 dataset with no citation, while its methods text had grown quite specific. It now cites Werling et al. 2020, which described the cohort, with the caveat that the single-nucleus data shown here were not part of that paper -- the same distinction the Cell Browser desc.conf makes. Citing it bare would credit a bulk RNA-seq and WGS study for 10x Multiome data. References are alphabetical, so hg38 is now 10 and mm10 still 7. copySingleCellSignalsPeaksFiles.py: colors_json() reads the palette in a with block, and a malformed R,G,B row now raises the SystemExit the rest of the file uses, naming the file, line number and offending field, instead of a bare ValueError or TypeError from int() or the %02X format. Output is unchanged. makeDoc: the mm10 cell-class note now records that the bare "Progenitor" row in celltype-class.tsv is live rather than leftover -- build_stanzas' class_key() collapses plurals, so BrainVar's "Progenitors" looks up under the singular key and takes its class and color from that one row. Retiring or qualifying the row would grey out that track. A non-neural label needs a specific cell type added instead, which is how "Nephron progenitors" comes out Stromal. Also fixed a contradiction there: the file claimed celltype-class.tsv is built by build_celltype_crosswalks.py a hundred lines above the note saying, correctly, that it is hand-curated and not generated. The submitters confirmed BrainVar is 100 bp tiles, so the 1 kb in their methods text is an error and the page is right; recorded in the hg38 makeDoc. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
  • src/hg/makeDb/doc/hg38/singleCellSignalsPeaks.txt - lines changed 12, context: html, text, full: html, text
  • src/hg/makeDb/doc/mm10/singleCellSignalsPeaks.txt - lines changed 14, context: html, text, full: html, text
  • src/hg/makeDb/scripts/singleCellSignalsPeaks/copySingleCellSignalsPeaksFiles.py - lines changed 7, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/hg38/singleCellSignalsPeaks.html - lines changed 18, context: html, text, full: html, text