Commits for max

v498_preview2 to v498_base (2026-05-04 to 2026-05-11) v498

986c4ede954e44904eb314772fb2cf83a48d307c Wed May 6 06:24:47 2026 -0700

• varFreqs: lift GenomeAsia (gasp + gaspIndel) GRCh37 -> hg38 Both subtracks were served at /gbdb/hg38/ but the upstream callset is GRCh37 (caught in QA, see #36642 note 2026-05-04). Lifted with CrossMap using hg19ToHg38.over.chain.gz; recipe matches tishkoff180 / mxbFreq. gasp (SNVs): 66,236,516 -> 66,222,771 (99.98%; 6,240 unmapped + 7,505 alt/random) gaspIndel: 4,415,156 -> 4,410,871 (99.90%; 3,332 unmapped + 953 alt/random) New driver script: scripts/varFreqs/gaspLift.sh. gaspIndel bigDataUrl renamed from All.indels.annot.cont_withmaf.vcf.gz to ga100k.indels.vcf.gz (old name was a verbatim copy of the upstream download name). varFreqsAll combined bigBed regenerated to fold in the corrected coordinates (36.5 GB, 1,166,451,644 items, 125 fields). refs #36642 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  • src/hg/makeDb/doc/hg38/varFreqs.txt - lines changed 88, context: html, text, full: html, text
  • src/hg/makeDb/scripts/varFreqs/databases.tsv - lines changed 3, context: html, text, full: html, text
  • src/hg/makeDb/scripts/varFreqs/gaspLift.sh - lines changed 74, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/gasp.html - lines changed 6, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/varFreqs.ra - lines changed 8, context: html, text, full: html, text

65da29c9d74d4dd832ab7f16899ad3b209b92da4 Wed May 6 08:43:57 2026 -0700

• varFreqs: 5 vcfToBigBed.py fixes + add NPM Singapore to combined track vcfToBigBed.py and mergeAndAnnotate.sh moved into kent (they were hive-only); the build is now reproducible from a fresh kent checkout. Five vcfToBigBed.py fixes (all caught by Lou's QA pass on #36642): - normalize_consequence(): bcftools csq emits "&"-joined compound terms like "stop_gained&frameshift" which exact-match-failed the old 8-bucket consequence filter and orphaned ~8.5M records. Rewrites "&" to "," so a single record can match multiple buckets, and appends ",others" to any token list with no named-filter token. Trackdb gains 4 buckets (3' UTR, 5' UTR, Non-coding, Other) and switches to filterType.consequence multipleListOr. - Source-attribution bug: the old check only inspected the unified AC/AF slot. AllOfUs ships only per-population fields ("." in the unified slot), so all 67M+ AllOfUs variants got no source attribution -- ~43M rows in the previous bigBed had an empty "sources" column. Fix scans per-population slots before declaring "no data". - parse_bcsq() returns "" instead of "." for aaChange/dnaChange on non-coding variants, so the mouseOver and detail page render a clean blank line. - maxAF format: "{:.6g}" -> "{:.6f}" so very small AFs render as "0.000003" instead of "3.31347e-06". - autoSql `table varFreqs` -> `table varFreqsAll` (matches the bigBed filename; required for hgIntegrator wiring). NPM Singapore (SG10K_Health, 9.7k WGS) added to databases.tsv, files.txt, populations.tsv (SgChinese / SgMalay / SgIndian) and the trackDb filter UI. NPM individual subtrack stays tableBrowser off (license); folded into varFreqsAll same as finngen / kova / mgrb / swefreq / tishkoff180. varFreqsAll bigBed rebuild is in progress at /hive/data/genomes/hg38/ bed/varFreqs/all/; will land in /gbdb when the bedToBigBed step completes. refs #36642 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
  • src/hg/makeDb/doc/hg38/varFreqs.txt - lines changed 51, context: html, text, full: html, text
  • src/hg/makeDb/scripts/varFreqs/databases.tsv - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/scripts/varFreqs/mergeAndAnnotate.sh - lines changed 256, context: html, text, full: html, text
  • src/hg/makeDb/scripts/varFreqs/populations.tsv - lines changed 4, context: html, text, full: html, text
  • src/hg/makeDb/scripts/varFreqs/vcfToBigBed.py - lines changed 652, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/varFreqs.ra - lines changed 20, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/varFreqsAll.html - lines changed 20, context: html, text, full: html, text

00e086afbcb8296c6aef946c13eeb777f34bac02 Mon May 11 07:03:10 2026 -0700

• mei: new track for HGSVC3 mobile element insertions on hg38 and hs1 12,642 MEIs on hg38, 12,919 on hs1, from the HGSVC3 callset (Logsdon et al. 2025). Single shared trackDb stanza in human/mei.ra with $D substitution. Okabe-Ito colors, INS-svLen:carrierCount item names, inserted DNA in an insertSeq field. Cross-link to lrSv via relatedTracks.ra. refs #37524
  • src/hg/makeDb/doc/hg38/mei.txt - lines changed 67, context: html, text, full: html, text
  • src/hg/makeDb/doc/hs1/mei.txt - lines changed 6, context: html, text, full: html, text
  • src/hg/makeDb/scripts/mei/meiHgsvc3.as - lines changed 29, context: html, text, full: html, text
  • src/hg/makeDb/scripts/mei/meiHgsvc3CsvToBed.py - lines changed 197, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/mei.html - lines changed 74, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/mei.ra - lines changed 46, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/meiHgsvc3.html - lines changed 158, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/human/trackDb.ra - lines changed 1, context: html, text, full: html, text
  • src/hg/makeDb/trackDb/relatedTracks.ra - lines changed 4, context: html, text, full: html, text

ac18a42f0dafb4febaaeaebcd53fe75df9b83234 Mon May 11 08:29:14 2026 -0700

• lrSv: add Coverage column, drop redundant Sequencing column, rename SVs to SV count Coverage values pulled from the per-subtrack methods sections and the underlying papers (Han 17x, deCODE 17x, GA4K 27x, HPRC 60x HiFi + 30x ONT, etc.). Sequencing technology is now folded into the Coverage cells. Also cross-links to the new HGSVC3 Mobile Insertions tracks. refs #36642
  • src/hg/makeDb/trackDb/human/lrSv.html - lines changed 29, context: html, text, full: html, text

8b1ebf72b693afd8705c36af91d85e9596d1654f Mon May 11 08:31:13 2026 -0700

• adding kozak strengths to ncOrfs, refs #35101, this is only the tdb part
  • src/hg/makeDb/trackDb/human/hg38/ncOrfs.ra - lines changed 96, context: html, text, full: html, text