4f8f8773bec66a9e993e9897e0b032c6e97dead8 max Fri May 15 10:12:29 2026 -0700 mei: add HMEID, SweGen, and euL1db subtracks Three new MEI catalogues under the existing mei superTrack: meiHmeid (hg38) 36,699 MELT MEIs from HMEID v1.1 (NyuWa+1KGP, 5,675 individuals, Niu et al. 2022, PMID 35212372). Site-level VCF; per-cohort and per-1KGP super- population AC/AN/AF; SVTYPE Alu/L1/SVA/HERVK. meiSwegen (hg38 lifted) 18,090 MELT MEIs from the SweGen 1,000-sample Swedish cohort (Ameur 2017, PMID 28832569; Gardner 2017, PMID 28855259). Built on hg19, liftOver to hg38 (10 unmapped). tableBrowser off per SweGen distribution terms. meiEul1db (hg19+hg38) 8,988 curated L1-HS insertion polymorphisms (MRIPs) from euL1db v1.00 (Mir 2015, PMID 25352549), aggregating 142,495 sample-level SRIPs across 32 published studies. Coloured by lineage (germline/somatic/mixed). Built on hg19, liftOver to hg38 (3 unmapped). Helman2014 used numeric chrom names (23=X, 24=Y) which are renamed during the build. meiEul1dbRef (hg19+hg38) 1,540 reference-genome L1-HS copies catalogued by euL1db (companion to meiEul1db). Single shared mei.ra (in human/) uses $D substitution so each stanza serves both assemblies where applicable. refs #37524 diff --git src/hg/makeDb/doc/hg38/varFreqs.txt src/hg/makeDb/doc/hg38/varFreqs.txt index 39cd22e14e6..c2fc59ed0c7 100644 --- src/hg/makeDb/doc/hg38/varFreqs.txt +++ src/hg/makeDb/doc/hg38/varFreqs.txt @@ -590,15 +590,75 @@ | python3 ~/kent/src/hg/makeDb/scripts/varFreqs/wbbcFix.py \ | bgzip -@ 8 -c > wbbc.vcf.gz tabix -p vcf wbbc.vcf.gz rm -f wbbc.allChrom.vcf.gz # Final: 2.2 GB bgzip + 2.2 MB tabix index, 78,594,274 variants. # 1,495,967 rows (~1.86%) with AC=0 were dropped. # Symlinks placed under /gbdb/hg38/varFreqs/wbbc/ for the wbbc stanza # in trackDb/human/varFreqs.ra. # # databases.tsv and populations.tsv now list WBBC + the four Han # regional groups, so the next varFreqsAll rebuild will pick the cohort # up. The trackDb filter UI (filterByRange.WBBCAF etc.) was deliberately # NOT added yet - those will be added together with the next rebuild # of varFreqsAll, otherwise the filters would appear in the UI before # the underlying bigBed has the columns. + +# Taiwan Precision Medicine Initiative (TPMI) - 2026-05-15 Claude max +# TPMI is a Han-Chinese cohort of 565,390 participants (Yang et al. 2025, +# Nature, PMID 41092961). Genotyping was done with two custom Axiom SNP +# arrays (TPMv1 on 165,596 individuals; TPMv2 on 321,360 individuals). +# Thermo Fisher publishes a chip annotation CSV for each array; the TPM1 +# annotation embeds the TPMI cohort allele frequency in a column named +# "Allele Frequency". The TPM2 annotation does not (it only carries the +# Affymetrix-bundled CEU/CHB/JPT/YRI HapMap freqs), so this track is +# TPM1-only. The annotation CSVs declare hg38 coordinates, so no lift. +cd /hive/data/genomes/hg38/bed/varFreqs/tpmi +# TPM1_Array_Annotation.csv (~926 MB) and TPM2_Array_Annotation.csv +# (~1.0 GB) were already in this directory at the start. Only the TPM1 +# file is used. +# CSV -> VCF: filters out rows with no AF / blacklist / alt-or-random +# chrom, anchors indels (Affymetrix uses "-" for the empty allele) by +# fetching the anchor base from hg38.2bit via twoBitToFa, and derives +# AC = round(AF * 100000). AN=100000 is the implied precision of the +# AF values (every reported AF rounds to an exact integer at 1/100000; +# AF=0 is also kept, AC=0). The true TPMv1 cohort AN is ~330,000 for +# autosomes, so AC values are proportional to but smaller than the +# real counts; documented in the VCF header and tpmi.html caveat. +python3 ~/kent/src/hg/makeDb/scripts/tpmi/tpmiToVcf.py \ + TPM1_Array_Annotation.csv /hive/data/genomes/hg38/hg38.2bit tpmi.vcf +bcftools sort -Oz -o tpmi.vcf.gz tpmi.vcf +tabix -f -p vcf tpmi.vcf.gz +rm -f tpmi.vcf +# 752,921 source rows -> 672,843 VCF records. Skipped: 80,034 with no +# reported AF (includes the entire chrY content); 36 on alt/random +# contigs; 8 with no defined REF. Blacklist rows (~61k) all have no AF +# so they are filtered by the no-AF rule. Distribution: 623,203 SNVs, +# 34,954 deletions, 13,537 insertions, 1,149 MNVs. +# Symlinks placed under /gbdb/hg38/varFreqs/tpmi/ for the tpmi stanza +# in trackDb/human/varFreqs.ra (priority 5.6, next to wbbc/tommo). + +########## +# 2026-05-15 Claude max +# Strip three unused FILTER definitions from the tishkoff180 VCF header. +# Matt Hansen (mhansen@upenn.edu, 2026-04-29 email): the source header +# declared LowQual, VQSRTrancheSNP99.90to100.00, and +# VQSRTrancheSNP99.90to100.00+ but he had already removed the +# corresponding INFO values and only kept PASS variants. The hgTracks +# VCF code auto-generates filter checkboxes from ##FILTER lines, so +# these spurious entries showed up on hgTrackUi. Confirmed every record +# has FILTER=PASS, then rewrote only the header (body untouched) with +# bcftools reheader and re-indexed. +cd /hive/data/genomes/hg38/bed/varFreqs/tishkoff/ +bcftools view -h tishkoff180.vcf.gz \ + | grep -vE '^##FILTER= header.new.txt +bcftools reheader -h header.new.txt -o tishkoff180.new.vcf.gz tishkoff180.vcf.gz +tabix -p vcf tishkoff180.new.vcf.gz +# Sanity check: same 33,600,472 records, only PASS remains. +mv tishkoff180.vcf.gz tishkoff180.vcf.gz.bak +mv tishkoff180.vcf.gz.tbi tishkoff180.vcf.gz.tbi.bak +mv tishkoff180.new.vcf.gz tishkoff180.vcf.gz +mv tishkoff180.new.vcf.gz.tbi tishkoff180.vcf.gz.tbi +rm header.new.txt +# /gbdb/hg38/varFreqs/tishkoff/ symlinks unchanged. The .bak originals +# can be removed once the change has been verified on hgwdev.