4f8f8773bec66a9e993e9897e0b032c6e97dead8
max
  Fri May 15 10:12:29 2026 -0700
mei: add HMEID, SweGen, and euL1db subtracks

Three new MEI catalogues under the existing mei superTrack:

meiHmeid     (hg38)        36,699 MELT MEIs from HMEID v1.1 (NyuWa+1KGP,
5,675 individuals, Niu et al. 2022, PMID 35212372).
Site-level VCF; per-cohort and per-1KGP super-
population AC/AN/AF; SVTYPE Alu/L1/SVA/HERVK.

meiSwegen    (hg38 lifted) 18,090 MELT MEIs from the SweGen 1,000-sample
Swedish cohort (Ameur 2017, PMID 28832569;
Gardner 2017, PMID 28855259). Built on hg19,
liftOver to hg38 (10 unmapped). tableBrowser off
per SweGen distribution terms.

meiEul1db   (hg19+hg38)    8,988 curated L1-HS insertion polymorphisms
(MRIPs) from euL1db v1.00 (Mir 2015, PMID
25352549), aggregating 142,495 sample-level
SRIPs across 32 published studies. Coloured by
lineage (germline/somatic/mixed). Built on hg19,
liftOver to hg38 (3 unmapped). Helman2014 used
numeric chrom names (23=X, 24=Y) which are
renamed during the build.

meiEul1dbRef (hg19+hg38)   1,540 reference-genome L1-HS copies catalogued
by euL1db (companion to meiEul1db).

Single shared mei.ra (in human/) uses $D substitution so each stanza
serves both assemblies where applicable.

refs #37524

diff --git src/hg/makeDb/doc/hg38/varFreqs.txt src/hg/makeDb/doc/hg38/varFreqs.txt
index 39cd22e14e6..c2fc59ed0c7 100644
--- src/hg/makeDb/doc/hg38/varFreqs.txt
+++ src/hg/makeDb/doc/hg38/varFreqs.txt
@@ -590,15 +590,75 @@
     | python3 ~/kent/src/hg/makeDb/scripts/varFreqs/wbbcFix.py \
     | bgzip -@ 8 -c > wbbc.vcf.gz
 tabix -p vcf wbbc.vcf.gz
 rm -f wbbc.allChrom.vcf.gz
 # Final: 2.2 GB bgzip + 2.2 MB tabix index, 78,594,274 variants.
 # 1,495,967 rows (~1.86%) with AC=0 were dropped.
 # Symlinks placed under /gbdb/hg38/varFreqs/wbbc/ for the wbbc stanza
 # in trackDb/human/varFreqs.ra.
 #
 # databases.tsv and populations.tsv now list WBBC + the four Han
 # regional groups, so the next varFreqsAll rebuild will pick the cohort
 # up. The trackDb filter UI (filterByRange.WBBCAF etc.) was deliberately
 # NOT added yet - those will be added together with the next rebuild
 # of varFreqsAll, otherwise the filters would appear in the UI before
 # the underlying bigBed has the columns.
+
+# Taiwan Precision Medicine Initiative (TPMI) - 2026-05-15 Claude max
+# TPMI is a Han-Chinese cohort of 565,390 participants (Yang et al. 2025,
+# Nature, PMID 41092961). Genotyping was done with two custom Axiom SNP
+# arrays (TPMv1 on 165,596 individuals; TPMv2 on 321,360 individuals).
+# Thermo Fisher publishes a chip annotation CSV for each array; the TPM1
+# annotation embeds the TPMI cohort allele frequency in a column named
+# "Allele Frequency". The TPM2 annotation does not (it only carries the
+# Affymetrix-bundled CEU/CHB/JPT/YRI HapMap freqs), so this track is
+# TPM1-only. The annotation CSVs declare hg38 coordinates, so no lift.
+cd /hive/data/genomes/hg38/bed/varFreqs/tpmi
+# TPM1_Array_Annotation.csv (~926 MB) and TPM2_Array_Annotation.csv
+# (~1.0 GB) were already in this directory at the start. Only the TPM1
+# file is used.
+# CSV -> VCF: filters out rows with no AF / blacklist / alt-or-random
+# chrom, anchors indels (Affymetrix uses "-" for the empty allele) by
+# fetching the anchor base from hg38.2bit via twoBitToFa, and derives
+# AC = round(AF * 100000). AN=100000 is the implied precision of the
+# AF values (every reported AF rounds to an exact integer at 1/100000;
+# AF=0 is also kept, AC=0). The true TPMv1 cohort AN is ~330,000 for
+# autosomes, so AC values are proportional to but smaller than the
+# real counts; documented in the VCF header and tpmi.html caveat.
+python3 ~/kent/src/hg/makeDb/scripts/tpmi/tpmiToVcf.py \
+    TPM1_Array_Annotation.csv /hive/data/genomes/hg38/hg38.2bit tpmi.vcf
+bcftools sort -Oz -o tpmi.vcf.gz tpmi.vcf
+tabix -f -p vcf tpmi.vcf.gz
+rm -f tpmi.vcf
+# 752,921 source rows -> 672,843 VCF records. Skipped: 80,034 with no
+# reported AF (includes the entire chrY content); 36 on alt/random
+# contigs; 8 with no defined REF. Blacklist rows (~61k) all have no AF
+# so they are filtered by the no-AF rule. Distribution: 623,203 SNVs,
+# 34,954 deletions, 13,537 insertions, 1,149 MNVs.
+# Symlinks placed under /gbdb/hg38/varFreqs/tpmi/ for the tpmi stanza
+# in trackDb/human/varFreqs.ra (priority 5.6, next to wbbc/tommo).
+
+##########
+# 2026-05-15 Claude max
+# Strip three unused FILTER definitions from the tishkoff180 VCF header.
+# Matt Hansen (mhansen@upenn.edu, 2026-04-29 email): the source header
+# declared LowQual, VQSRTrancheSNP99.90to100.00, and
+# VQSRTrancheSNP99.90to100.00+ but he had already removed the
+# corresponding INFO values and only kept PASS variants. The hgTracks
+# VCF code auto-generates filter checkboxes from ##FILTER lines, so
+# these spurious entries showed up on hgTrackUi. Confirmed every record
+# has FILTER=PASS, then rewrote only the header (body untouched) with
+# bcftools reheader and re-indexed.
+cd /hive/data/genomes/hg38/bed/varFreqs/tishkoff/
+bcftools view -h tishkoff180.vcf.gz \
+    | grep -vE '^##FILTER=<ID=(LowQual|VQSRTrancheSNP99\.90to100\.00\+?),' \
+    > header.new.txt
+bcftools reheader -h header.new.txt -o tishkoff180.new.vcf.gz tishkoff180.vcf.gz
+tabix -p vcf tishkoff180.new.vcf.gz
+# Sanity check: same 33,600,472 records, only PASS remains.
+mv tishkoff180.vcf.gz     tishkoff180.vcf.gz.bak
+mv tishkoff180.vcf.gz.tbi tishkoff180.vcf.gz.tbi.bak
+mv tishkoff180.new.vcf.gz     tishkoff180.vcf.gz
+mv tishkoff180.new.vcf.gz.tbi tishkoff180.vcf.gz.tbi
+rm header.new.txt
+# /gbdb/hg38/varFreqs/tishkoff/ symlinks unchanged. The .bak originals
+# can be removed once the change has been verified on hgwdev.