ef61e73fc416622d8557ec2439df2344a1cc80c3 max Tue Jun 9 15:10:01 2026 -0700 lrSv: replace HPRC v2.0 pangenome SV track with v2.1 (hprc2v21Sv) Drop the v2.0 wave-decomposed hprc2Sv track and add hprc2v21Sv built from the HPRC v2.1 minigraph-cactus raw vg deconstruct VCFs (gref95.ro), on both hg38 (GRCh38 path, 596,063 SVs) and hs1 (T2T-CHM13 path, 608,435 SVs). The v2.1 files lack per-allele TYPE/LEN, so the new converter classifies INS/DEL by parsimony-trimming REF/ALT and the net length change. The v2.0 build recipe, converter and schema are kept but commented out in the makeDocs in case wave-decomposed VCFs are released again, refs #36258 diff --git src/hg/makeDb/doc/hg38/lrSv.txt src/hg/makeDb/doc/hg38/lrSv.txt index 3d100f95523..6633bcdc2ba 100644 --- src/hg/makeDb/doc/hg38/lrSv.txt +++ src/hg/makeDb/doc/hg38/lrSv.txt @@ -303,59 +303,97 @@ # an RGN_REF_INNER column. The converter merges them into a single bigBed. python3 ~/kent/src/hg/makeDb/scripts/lrSv/lrSvHgsvc2TsvToBed.py \ variants_freeze4_sv_insdel.tsv.gz \ variants_freeze4_sv_inv.tsv.gz \ hgsvc2.bed bedSort hgsvc2.bed hgsvc2.sorted.bed bedToBigBed -type=bed9+ -as=$HOME/kent/src/hg/makeDb/scripts/lrSv/lrSvHgsvc2.as \ -tab hgsvc2.sorted.bed /hive/data/genomes/hg38/chrom.sizes hgsvc2.bb # 1KG 3202 short-read comparator moved to the srSv supertrack. # See doc/hg38/srSv.txt for that build. ########## # 2026-04-20 Claude max +# +# OUTDATED, DO NOT RUN. The HPRC v2.0 wave-decomposed track (hprc2Sv) was +# replaced by the v2.1 hprc2v21Sv track below on 2026-06-08 and removed +# from trackDb. This block, the converter lrSvHprc2VcfToBed.py and the +# schema lrSvHprc2.as are kept on purpose: HPRC may publish wave-decomposed +# VCFs again, and the converter handles the per-allele TYPE/LEN/INV fields +# that the v2.1 raw deconstruct file does not have. The whole recipe is +# commented out so it is not run by accident. +# +# # Thirteenth subtrack: HPRC release-2 pangenome SVs (233 samples). +# # No peer-reviewed publication yet; see HPRC release page: +# # https://humanpangenome.org/hprc-data-release-2/ +# # Sample list (alignments v2.0): +# # https://github.com/human-pangenomics/hprc_intermediate_assembly/blob/main/data_tables/pangenomes/alignments_v2.0.csv +# +# mkdir -p /hive/data/genomes/hg38/bed/lrSv/hprc2 +# cd /hive/data/genomes/hg38/bed/lrSv/hprc2 +# +# # Pangenome graph (referenced in the doc html): +# wget https://s3-us-west-2.amazonaws.com/human-pangenomics/pangenomes/freeze/release2/minigraph-cactus/hprc-v2.0-mc-grch38.sv.gfa.gz +# # wave-decomposed VCF (what we actually convert): +# wget https://s3-us-west-2.amazonaws.com/human-pangenomics/pangenomes/freeze/release2/minigraph-cactus/hprc-v2.0-mc-grch38.wave.vcf.gz +# +# # The wave VCF contains ~20M atomic alleles including SNVs. The converter +# # streams the multi-allelic rows, explodes one BED row per ALT, and keeps +# # only SV-sized alleles (|LEN| >= 50 bp) plus all records carrying the +# # INV flag. 1,483,114 SVs kept (1,106,190 INS, 192,597 DEL, 178,178 +# # COMPLEX, 6,149 INV). +# +# python3 ~/kent/src/hg/makeDb/scripts/lrSv/lrSvHprc2VcfToBed.py \ +# hprc-v2.0-mc-grch38.wave.vcf.gz hprc2.bed +# bedSort hprc2.bed hprc2.sorted.bed +# bedToBigBed -type=bed9+ -as=$HOME/kent/src/hg/makeDb/scripts/lrSv/lrSvHprc2.as \ +# -tab hprc2.sorted.bed /hive/data/genomes/hg38/chrom.sizes hprc2.bb +# +# # HPRC also releases a wave VCF against T2T-CHM13; the hs1 version of this +# # subtrack is built in ~/kent/src/hg/makeDb/doc/hs1/lrSv.txt. -# Thirteenth subtrack: HPRC release-2 pangenome SVs (233 samples). -# No peer-reviewed publication yet; see HPRC release page: -# https://humanpangenome.org/hprc-data-release-2/ -# Sample list (alignments v2.0): -# https://github.com/human-pangenomics/hprc_intermediate_assembly/blob/main/data_tables/pangenomes/alignments_v2.0.csv - -mkdir -p /hive/data/genomes/hg38/bed/lrSv/hprc2 -cd /hive/data/genomes/hg38/bed/lrSv/hprc2 - -# Pangenome graph (referenced in the doc html): -wget https://s3-us-west-2.amazonaws.com/human-pangenomics/pangenomes/freeze/release2/minigraph-cactus/hprc-v2.0-mc-grch38.sv.gfa.gz -# wave-decomposed VCF (what we actually convert): -wget https://s3-us-west-2.amazonaws.com/human-pangenomics/pangenomes/freeze/release2/minigraph-cactus/hprc-v2.0-mc-grch38.wave.vcf.gz - -# The wave VCF contains ~20M atomic alleles including SNVs. The converter -# streams the multi-allelic rows, explodes one BED row per ALT, and keeps -# only SV-sized alleles (|LEN| >= 50 bp) plus all records carrying the -# INV flag. 1,483,114 SVs kept (1,106,190 INS, 192,597 DEL, 178,178 -# COMPLEX, 6,149 INV). - -python3 ~/kent/src/hg/makeDb/scripts/lrSv/lrSvHprc2VcfToBed.py \ - hprc-v2.0-mc-grch38.wave.vcf.gz hprc2.bed -bedSort hprc2.bed hprc2.sorted.bed -bedToBigBed -type=bed9+ -as=$HOME/kent/src/hg/makeDb/scripts/lrSv/lrSvHprc2.as \ - -tab hprc2.sorted.bed /hive/data/genomes/hg38/chrom.sizes hprc2.bb - -# HPRC also releases a wave VCF against T2T-CHM13; the hs1 version of this -# subtrack is built in ~/kent/src/hg/makeDb/doc/hs1/lrSv.txt. +########## +# 2026-06-08 Claude max + +# hprc2v21Sv: updated HPRC release-2.1 minigraph-cactus pangenome SVs +# (233 samples). This is a second copy of the HPRC pangenome SV track, +# loaded next to hprc2Sv so the two callsets can be compared. +# +# The v2.1 file is the raw `vg deconstruct` output (it carries graph +# traversals AT, nested snarls LV and a parent-snarl pointer PS) and, unlike +# the v2.0 "wave" VCF, has NO per-allele TYPE or LEN fields. The converter +# therefore classifies each ALT by parsimony-trimming the shared prefix and +# suffix of REF/ALT and looking at the net length change: alt longer -> INS, +# ref longer -> DEL. Alleles with |net length| >= 50 bp are kept. No +# inversion annotation is present, so no INV rows are emitted; balanced +# equal-length substitutions do not occur at SV size. + +mkdir -p /hive/data/genomes/hg38/bed/lrSv/hprc2v21 +cd /hive/data/genomes/hg38/bed/lrSv/hprc2v21 + +# VCF provided by Glenn Hickey (HPRC graph team): +wget https://public.gi.ucsc.edu/~ghickey/debug/hprc-v2.1-mc-grch38.gref95.ro.vcf.gz + +python3 ~/kent/src/hg/makeDb/scripts/lrSv/lrSvHprc2RoVcfToBed.py \ + hprc-v2.1-mc-grch38.gref95.ro.vcf.gz hprc2v21.bed +# kept 596063 SV-sized alleles: 447578 INS, 148485 DEL, 0 CPX +# (43580 at nested snarl levels LV>0) +bedSort hprc2v21.bed hprc2v21.sorted.bed +bedToBigBed -type=bed9+ -as=$HOME/kent/src/hg/makeDb/scripts/lrSv/lrSvHprc2Ro.as \ + -tab hprc2v21.sorted.bed /hive/data/genomes/hg38/chrom.sizes hprc2v21.bb ########## # 2026-04-20 Claude max # CPC + HPRC Phase 1 pangenome SVs (105 samples). # Paper: Gao et al. 2023, Nature, PMID 37316654 # Data : https://github.com/Shuhua-Group/Chinese-Pangenome-Consortium-Phase-I # The VCF is on T2T-CHM13v2 (hs1) contigs renamed "CHM13v2.chrN". # Source VCF (CPC.HPRC.Phase1.processed.SVs.normed.vcf.gz, 3.7 GB) was # produced with pggb + vcfwave + bcftools norm; each graph snarl appears # as one VCF row per alternative allele, with genotypes for 105 samples. mkdir -p /hive/data/genomes/hg38/bed/lrSv/cpc1 cd /hive/data/genomes/hg38/bed/lrSv/cpc1