b85c12cf9af0ee1a954b8cced961bcbd909b7979 lrnassar Wed Apr 29 12:04:36 2026 -0700 Expand dbVar tracks to expose all six nstd186 source studies and add new Somatic and Other composites. refs #37406 Restructure the dbVar supertrack: - Renamed from "dbVar Common Struct Var" to "dbVar Struct Var". - dbVar Common SV: added subtracks for Lee, Abel, and Byrska-Bishop (the three nstd186 source studies that were missing from our Curated Common track), and for the American/East Asian/South Asian/Other populations. - dbVar Conflict SV: description page refreshed; subtrack longLabel clarified. - dbVar Somatic SV (new): single subtrack pulling somatic_sv.bb from the dbVar hub. Default hidden. - dbVar Other SV (new): residual bucket for dbVar SVs not classified as common, somatic, or clinical, split into Healthy and Phenotype subtracks. Default hidden. NCBI sometimes calls this "presumed normal"; the description page notes the equivalence. mergeSpannedItems on for the dense subtracks (normal_healthy ~5.6M items, normal_phenotype ~410K, somatic_sv ~67K). - ClinVar SVs are not duplicated; description pages cross-link to the existing ClinVar track instead. Description pages: rewrite dbVarCommon.html and dbVarCurated.html, refresh dbVarConflict.html, add dbVarSomatic.html and dbVarOther.html. Retire the unused dbVar_common.html. Methods links now point at NCBI's dbVar Overview rather than the FTP directory listing. searchTable termRegex widened to ^[den]ssv[0-9]+ so dssv* accessions in normal_healthy resolve. Otto: stage downloads to release/\${db}.new/, validate per file (size floor and 10% itemCount delta vs the current live copy), then atomically swap via directory rename with a one-cycle .prev rollback. On validation failure, leave .new/ in place for human inspection and exit non-zero so the wrapper emails. On no-op runs the wrapper now stays silent. checkNstd175.sh's "update done" message moved inside the update branch so silence is honoured. New-file detection (via a knownFiles.txt manifest) emails when NCBI adds a file we don't yet expose. knownFiles.txt itself lives only at the deployment path under /hive/data/outside/otto/dbVar/, not in the tree. diff --git src/hg/makeDb/trackDb/human/trackDb.dbVar.ra src/hg/makeDb/trackDb/human/trackDb.dbVar.ra index c423fbe9595..884843e5b83 100644 --- src/hg/makeDb/trackDb/human/trackDb.dbVar.ra +++ src/hg/makeDb/trackDb/human/trackDb.dbVar.ra @@ -1,139 +1,343 @@ track dbVarSv superTrack on group varRep html dbVarCurated -shortLabel dbVar Common Struct Var -longLabel NCBI Curated Common Structural Variants from dbVar +shortLabel dbVar Struct Var +longLabel NCBI dbVar Structural Variants track dbVar_common superTrack dbVarSv pack - group varRep compositeTrack on visibility pack itemRgb on type bigBed 9 + . html dbVarCommon shortLabel dbVar Common SV longLabel NCBI dbVar Curated Common Structural Variants mouseOverField label searchIndex name + hideEmptySubtracks on filterLabel.type Variant Type - filterValues.type alu deletion,copy number gain,copy number loss,copy number variation,deletion,duplication,herv deletion,line1 deletion,sva deletion + filterValues.type alu deletion,alu insertion,copy number gain,copy number loss,copy number variation,deletion,duplication,herv deletion,insertion,line1 deletion,line1 insertion,mobile element deletion,mobile element insertion,sva deletion,sva insertion filterLabel.length Variant Size filterValues.length Under 10KB,10KB to 100KB,100KB to 1MB,Over 1MB + filterLabel.freq_range Frequency Range + filterValues.freq_range Under 0.02,0.02 to 0.05,0.05 to 0.1,0.1 to 0.2,0.2 to 0.5,Over 0.5 track dbVar_common_gnomad parent dbVar_common on shortLabel dbVar Curated gnomAD SVs longLabel NCBI dbVar Curated Common SVs: all populations from gnomAD bigDataUrl /gbdb/$D/bbi/dbVar/common_gnomad.bb type bigBed 9 + . url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ urlLabel NCBI Variant Page: + priority 1 track dbVar_common_1000g parent dbVar_common on shortLabel dbVar Curated 1000 Genomes SVs longLabel NCBI dbVar Curated Common SVs: all populations from 1000 Genomes bigDataUrl /gbdb/$D/bbi/dbVar/common_1000g.bb type bigBed 9 + . url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ urlLabel NCBI Variant Page: + priority 2 track dbVar_common_decipher parent dbVar_common on shortLabel dbVar Curated DECIPHER SVs longLabel NCBI dbVar Curated Common SVs: all populations from DECIPHER bigDataUrl /gbdb/$D/bbi/dbVar/common_decipher.bb type bigBed 9 + . url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ urlLabel NCBI Variant Page: + priority 3 + + track dbVar_common_lee + parent dbVar_common off + shortLabel dbVar Curated Lee SVs + longLabel NCBI dbVar Curated Common SVs: all populations from Lee + bigDataUrl /gbdb/$D/bbi/dbVar/common_lee.bb + type bigBed 9 + . + url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ + urlLabel NCBI Variant Page: + priority 4 + + track dbVar_common_abel + parent dbVar_common off + shortLabel dbVar Curated Abel SVs + longLabel NCBI dbVar Curated Common SVs: all populations from Abel + bigDataUrl /gbdb/$D/bbi/dbVar/common_abel.bb + type bigBed 9 + . + url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ + urlLabel NCBI Variant Page: + priority 5 + + track dbVar_common_byrska_bishop + parent dbVar_common off + shortLabel dbVar Curated Byrska-Bishop SVs + longLabel NCBI dbVar Curated Common SVs: all populations from Byrska-Bishop + bigDataUrl /gbdb/$D/bbi/dbVar/common_byrska_bishop.bb + type bigBed 9 + . + url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ + urlLabel NCBI Variant Page: + priority 6 + + track dbVar_common_global + parent dbVar_common on + shortLabel dbVar Curated All Populations + longLabel NCBI dbVar Curated Common SVs: all populations + bigDataUrl /gbdb/$D/bbi/dbVar/common_global.bb + type bigBed 9 + . + url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ + urlLabel NCBI Variant Page: + priority 10 track dbVar_common_african parent dbVar_common on shortLabel dbVar Curated African SVs longLabel NCBI dbVar Curated Common SVs: African bigDataUrl /gbdb/$D/bbi/dbVar/common_african.bb type bigBed 9 + . url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ urlLabel NCBI Variant Page: + priority 11 + + track dbVar_common_american + parent dbVar_common off + shortLabel dbVar Curated American SVs + longLabel NCBI dbVar Curated Common SVs: American + bigDataUrl /gbdb/$D/bbi/dbVar/common_american.bb + type bigBed 9 + . + url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ + urlLabel NCBI Variant Page: + priority 12 + + track dbVar_common_east_asian + parent dbVar_common off + shortLabel dbVar Curated East Asian SVs + longLabel NCBI dbVar Curated Common SVs: East Asian + bigDataUrl /gbdb/$D/bbi/dbVar/common_east_asian.bb + type bigBed 9 + . + url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ + urlLabel NCBI Variant Page: + priority 13 track dbVar_common_european parent dbVar_common on shortLabel dbVar Curated European SVs longLabel NCBI dbVar Curated Common SVs: European bigDataUrl /gbdb/$D/bbi/dbVar/common_european.bb type bigBed 9 + . url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ urlLabel NCBI Variant Page: + priority 14 - track dbVar_common_global - parent dbVar_common on - shortLabel dbVar Curated All Populations - longLabel NCBI dbVar Curated Common SVs: all populations - bigDataUrl /gbdb/$D/bbi/dbVar/common_global.bb + track dbVar_common_south_asian + parent dbVar_common off + shortLabel dbVar Curated South Asian SVs + longLabel NCBI dbVar Curated Common SVs: South Asian + bigDataUrl /gbdb/$D/bbi/dbVar/common_south_asian.bb type bigBed 9 + . url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ urlLabel NCBI Variant Page: + priority 15 + + track dbVar_common_other + parent dbVar_common off + shortLabel dbVar Curated Other Pop SVs + longLabel NCBI dbVar Curated Common SVs: Other + bigDataUrl /gbdb/$D/bbi/dbVar/common_other.bb + type bigBed 9 + . + url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ + urlLabel NCBI Variant Page: + priority 16 track dbVar_conflict superTrack dbVarSv pack compositeTrack on visibility pack itemRgb on type bigBed 9 + . html dbVarConflict shortLabel dbVar Conflict SV longLabel NCBI dbVar Curated Conflict Variants mouseOverField label searchIndex name filterLabel.overlap Variant Overlap filterValues.overlap 10 to 25,25 to 50,50 to 75,75 to 90,90 to 100 filterLabel.type Variant Type - filterValues.type alu deletion,copy number gain,copy number loss,copy number variation,deletion,duplication,herv deletion,line1 deletion,sva deletion - filterLabel.length Variant Length + filterValues.type copy number gain,copy number loss,copy number variation,deletion,duplication + filterLabel.length Variant Size filterValues.length Under 10KB,10KB to 100KB,100KB to 1MB,Over 1MB track dbVar_conflict_pathogenic parent dbVar_conflict on shortLabel dbVar Curated Conflict SVs - longLabel NCBI dbVar Curated Common SVs: Conflicts with Pathogenic + longLabel NCBI dbVar Common SVs in Conflict with Pathogenic Variants bigDataUrl /gbdb/$D/bbi/dbVar/conflict_pathogenic.bb type bigBed 9 + . url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ urlLabel NCBI Variant Page: + track dbVar_somatic + superTrack dbVarSv + compositeTrack on + visibility hide + itemRgb on + type bigBed 9 + . + html dbVarSomatic + shortLabel dbVar Somatic SV + longLabel NCBI dbVar Somatic Structural Variants + mouseOverField label + searchIndex name + filterLabel.type Variant Type + filterValues.type complex substitution,copy number gain,copy number loss,copy-neutral loss of heterozygosity,deletion,duplication,insertion,inversion,mobile element insertion,tandem duplication + filterLabel.length Variant Size + filterValues.length Under 10KB,10KB to 100KB,100KB to 1MB,Over 1MB + filterLabel.method Discovery Method + filterValues.method Curated,Multiple,SNP array,Sequencing + filterLabel.overlap Pathogenic Reciprocal Overlap + filterValues.overlap none,10 to 25,25 to 50,50 to 75,75 to 90,90 to 100 + + track dbVar_somatic_sv + parent dbVar_somatic off + shortLabel dbVar Somatic SVs + visibility pack + longLabel NCBI dbVar Somatic Structural Variants + bigDataUrl /gbdb/$D/bbi/dbVar/somatic_sv.bb + type bigBed 9 + . + url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ + urlLabel NCBI Variant Page: + mergeSpannedItems on + + track dbVar_other + superTrack dbVarSv + compositeTrack on + visibility hide + itemRgb on + type bigBed 9 + . + html dbVarOther + shortLabel dbVar Other SV + longLabel NCBI dbVar Other Structural Variants + mouseOverField label + searchIndex name + hideEmptySubtracks on + filterLabel.type Variant Type + filterValues.type alu deletion,alu insertion,complex substitution,copy-neutral loss of heterozygosity,copy number gain,copy number loss,copy number variation,deletion,delins,duplication,herv deletion,herv insertion,insertion,inversion,line1 deletion,line1 insertion,mobile element deletion,mobile element insertion,novel sequence insertion,sequence alteration,sva deletion,sva insertion,tandem duplication + filterLabel.length Variant Size + filterValues.length Under 10KB,10KB to 100KB,100KB to 1MB,Over 1MB + filterLabel.method Discovery Method + filterValues.method Curated,Merging,Multiple,Oligo aCGH,Optical mapping,SNP array,Sequencing,other + filterLabel.overlap Pathogenic Reciprocal Overlap + filterValues.overlap none,10 to 25,25 to 50,50 to 75,75 to 90,90 to 100 + filterLabel.population Population Code + filterValues.population AFR,AMR,EAS,EUR,OTH,SAS,mixed,multiple,none,unknown + + track dbVar_other_healthy + parent dbVar_other on + visibility pack + shortLabel dbVar Healthy SVs + longLabel NCBI dbVar SVs with no reported phenotype + bigDataUrl /gbdb/$D/bbi/dbVar/normal_healthy.bb + type bigBed 9 + . + url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ + urlLabel NCBI Variant Page: + mergeSpannedItems on + + track dbVar_other_phenotype + parent dbVar_other on + visibility pack + shortLabel dbVar Phenotype SVs + longLabel NCBI dbVar SVs with Phenotype (excluding clinical and somatic) + bigDataUrl /gbdb/$D/bbi/dbVar/normal_phenotype.bb + type bigBed 9 + . + url https://www.ncbi.nlm.nih.gov/dbvar/variants/$$ + urlLabel NCBI Variant Page: + mergeSpannedItems on + searchTable dbVar_common_gnomad searchMethod exact searchType bigBed -termRegex ^[ne]ssv[0-9]+ +termRegex ^[den]ssv[0-9]+ searchTable dbVar_common_1000g searchMethod exact searchType bigBed -termRegex ^[ne]ssv[0-9]+ +termRegex ^[den]ssv[0-9]+ searchTable dbVar_common_decipher searchMethod exact searchType bigBed -termRegex ^[ne]ssv[0-9]+ +termRegex ^[den]ssv[0-9]+ -searchTable dbVar_common_european +searchTable dbVar_common_lee searchMethod exact searchType bigBed -termRegex ^[ne]ssv[0-9]+ +termRegex ^[den]ssv[0-9]+ -searchTable dbVar_common_african +searchTable dbVar_common_abel searchMethod exact searchType bigBed -termRegex ^[ne]ssv[0-9]+ +termRegex ^[den]ssv[0-9]+ + +searchTable dbVar_common_byrska_bishop +searchMethod exact +searchType bigBed +termRegex ^[den]ssv[0-9]+ searchTable dbVar_common_global searchMethod exact searchType bigBed -termRegex ^[ne]ssv[0-9]+ +termRegex ^[den]ssv[0-9]+ + +searchTable dbVar_common_african +searchMethod exact +searchType bigBed +termRegex ^[den]ssv[0-9]+ + +searchTable dbVar_common_american +searchMethod exact +searchType bigBed +termRegex ^[den]ssv[0-9]+ + +searchTable dbVar_common_east_asian +searchMethod exact +searchType bigBed +termRegex ^[den]ssv[0-9]+ + +searchTable dbVar_common_european +searchMethod exact +searchType bigBed +termRegex ^[den]ssv[0-9]+ + +searchTable dbVar_common_south_asian +searchMethod exact +searchType bigBed +termRegex ^[den]ssv[0-9]+ + +searchTable dbVar_common_other +searchMethod exact +searchType bigBed +termRegex ^[den]ssv[0-9]+ searchTable dbVar_conflict_pathogenic searchMethod exact searchType bigBed -termRegex ^[ne]ssv[0-9]+ +termRegex ^[den]ssv[0-9]+ + +searchTable dbVar_somatic_sv +searchMethod exact +searchType bigBed +termRegex ^[den]ssv[0-9]+ + +searchTable dbVar_other_healthy +searchMethod exact +searchType bigBed +termRegex ^[den]ssv[0-9]+ + +searchTable dbVar_other_phenotype +searchMethod exact +searchType bigBed +termRegex ^[den]ssv[0-9]+