af9a5b388259e680dd34bc47b2cad4ff6e3d162f lrnassar Sat Jun 13 03:00:51 2026 -0700 varFreqs: pre-release polish from comprehensive sanity check. * Sync the new combined-track shortLabels into the four description pages: "Affected/Case Individuals" -> "Disease cohorts" and "Population + Unaffected" -> "Population reference" (matches the trackdb shortLabels users now see). * Add a paragraph in the supertrack Methods section describing the pooled affectedAF / backgroundAF formulation (sum AC / sum AN) and the default_an configuration that handles AF-only cohorts. * Update the in-track Methods paragraphs on varFreqsAffected.html and varFreqsBackground.html: replace "summed/maximized" with "pooled". * Fix supertrack table downloadability column to match the underscore-prefix convention: allofus "Yes" -> "No" (description page already says license restricted); gregor "No" -> "Yes" (description page says VCF is on our download server, and the gbdb path is not underscore-prefixed). * Add a 2026-06-12 makedoc section documenting the pooled-AF rebuild, the default_an mechanism, the new affectedAN/backgroundAN columns, the before/after spot-check at APOE rs429358, and the build commands. refs #36642 diff --git src/hg/makeDb/trackDb/human/varFreqs.html src/hg/makeDb/trackDb/human/varFreqs.html index 4bfa6e2345d..3c715c1b35f 100644 --- src/hg/makeDb/trackDb/human/varFreqs.html +++ src/hg/makeDb/trackDb/human/varFreqs.html @@ -20,47 +20,55 @@ Their VCF versions below show only the isolate frequency per variant.

Please contact us (genome@soe.ucsc.edu) if you know of a project that we should add. So far, Regeneron's Million Exomes and Mexico City Studies (request rejected) and Taiwan Biobank (pending).

Combined Tracks

Three combined tracks merge variants from the individual subtracks into single bigBed files with predicted protein consequences and cross-database filtering. All three use the same filter conventions (variant type, consequence, source database, allele frequency, allele count, and per-database AF/AC).

+

+On the Disease and Population reference tracks, Affected AF and Background AF +are pooled across contributing cohort arms (sum of allele counts divided by sum of allele +numbers), not the maximum across arms, so the displayed frequency matches the carrier-count +scale and a small cohort with a high local frequency does not dominate the value. See the +"Pooled allele frequency" section on each combined track's description page for +which cohorts contribute to the pool numerator and denominator. +

+

Consequence filter — the "Other" bucket

All three combined tracks share the same Consequence filter (Missense, Synonymous, Stop Gained, Frameshift, Splice Donor, Splice Acceptor, Intron, 3' UTR, 5' UTR, Non-coding, Intergenic, Other). The filter uses OR logic across the comma-separated consequence tokens on each variant: a variant tagged stop_gained,frameshift is selected by either the "Stop Gained" or the "Frameshift" filter. The "Other" bucket catches the less common Sequence Ontology consequence terms emitted by bcftools csq that don't fit the named buckets above. Examples include splice_region (variant near a splice site but outside the canonical donor/acceptor), start_lost / stop_lost (variant disrupts the start codon or replaces the stop codon with a coding amino acid), stop_retained (variant changes the stop codon but keeps it a stop), inframe_insertion / inframe_deletion (in-frame indel that adds or @@ -70,65 +78,65 @@

Available Datasets

- + - + - + @@ -330,31 +338,31 @@ - + @@ -442,42 +450,48 @@ letters. All VCF files are normalized, with one allele per annotation (no multi-allele lines).

Methods

Each subtrack includes the upstream project's VCF largely as-released; per-subtrack pipelines (coordinate liftover, format conversion, header normalization) are documented on each subtrack's own description page and recorded in the build documentation. The conversion scripts (e.g. finngen_to_vcf.py, kovaToVcf.py, schema_addAcAnAf.py, svatalogFreqToVcf.py) live alongside the makedoc in the scripts directory.

-The combined Affected and Background tracks are built by a separate pipeline: -each per-subtrack VCF is normalized (bcftools norm), all sites are merged into a single -callset, consequence annotations are recomputed against Ensembl with bcftools csq, -and the merged callset is split by phenotype into the two bigBed files via -vcfToBigBed.py + bedToBigBed. The mapping from upstream INFO fields to -bigBed columns is driven by two configuration files in the scripts directory: -databases.tsv (one row per source dataset, flagging which cohorts study a disease) -and populations.tsv (per-population AC/AF columns within each source, including the -affected and unaffected arm of each disease cohort). Editing those two files and rerunning -mergeAndAnnotate.sh followed by vcfToBigBed.py --split-affected rebuilds -the two tracks. The Genotyping Array Databases Combined track is built the same way from the -array cohorts only. +The combined Disease cohorts and Population reference tracks are built by a separate +pipeline: each per-subtrack VCF is normalized (bcftools norm), all sites are +merged into a single callset, consequence annotations are recomputed against Ensembl with +bcftools csq, and the merged callset is split by phenotype into the two bigBed +files via vcfToBigBed.py + bedToBigBed. Within each combined +track, the Affected AF and Background AF columns are +pooled across contributing cohort arms (sum of allele counts divided by sum of +allele numbers, with the per-arm AN derived from each cohort's AC and AF), so the displayed +frequency matches the carrier-count scale and a small cohort with a high local AF cannot +dominate the value. The mapping from upstream INFO fields to bigBed columns is driven by +two configuration files in the scripts directory: databases.tsv (one row per +source dataset, flagging which cohorts study a disease, and optionally a +default_an for cohorts that publish only AF) and populations.tsv +(per-population AC/AF columns within each source, including the affected and unaffected arm +of each disease cohort). Editing those two files and rerunning +mergeAndAnnotate.sh followed by vcfToBigBed.py --split-affected +rebuilds the two tracks. The Genotyping Array Databases Combined track is built the same +way from the array cohorts only.

Data Access

All the data is publicly available. The table above indicates if we are allowed to distribute it in VCF format. Most of the databases do not allow us to redistribute the data files directly from our website, but it can always be downloaded from the original websites in some form. Click the database link in the table above and see the "Data Access" section of the respective track for a description of where to download the data. When the data is freely available from our website, the Data Access section will also indicate the VCF file location on our download server. Because it contains some licensed data, the combined track is not available for download, but can be recreated using the conversion scripts in our GitHub repository and the accompanying documentation file.

Credits

This track is only possible thanks to the data from millions of volunteers around the world, who donated blood, signed consent forms and provided health information about themselves and sometimes their families. Click any of the tracks in the list above to see the specific credits for each project. Thanks to Alex Ioannidis, UCSC, for the inspiration for this track and to Andreas Lahner, MGZ, for feedback.

References

All of Us Research Program Genomics Investigators.

Database Region N Data Type Cohort Sub-populations Downloadable from UCSC
Affected/Case IndividualsDisease cohorts Sequencing-based disease cohorts ~130k WGS/WES/long-read Affected/case arms of SFARI SPARK WES/WGS, SCHEMA, GREGoR, GA4K Affected/case AF and AC; background AF for contrast No
Population + UnaffectedPopulation reference Sequencing-based, population + unaffected ~1.5mil WGS/WES/long-read Population cohorts + unaffected/control arms Background AF and AC; per-cohort and ancestry breakdowns No
Genotyping Array Databases Combined TPMI, MexBB, UKBB ~530k Array / imputed 14.7M variants No
AllOfUs v7 USA 245k WGS General population, diverse African, Indigenous American, East Asian, European, Oceanian, South Asian (local ancestry; see Notes below)YesNo
TOPMED Freeze 10 USA 151k WGS Heart, lung, blood, sleep disorder cohorts No
SFARI SPARK WES USA 140k WESSGDP Global 279 WGS 142 diverse populations worldwide By population Yes
GREGoR R4 USA 3.6k WGS Rare disease families (10.7k participants, 4.4k families) NoYes
gnomAD HGDP+1kG Global 4k WGS 80 populations (HGDP + 1000 Genomes reprocessed) 4k-cohort total AF only; per-population AF columns are full gnomAD v3.1.2 release values (~76k genomes), see Notes below Yes
GA4K USA 552