This track collection gathers variant allele frequencies from population-scale sequencing and genotyping projects worldwide, from a total of ~1.7 million genomes/exomes/arrays. -The data was not reprocessed in a harmonized way; the variant VCFs were collected from the +Unlike gnomAD, the data was not reprocessed in a harmonized way; the variant VCFs were collected from the projects as-is. The goal is a single place to compare how common a variant is across -different populations, ancestries, and cohorts, for projects that cannot be recomputed by -gnomAD soon. Three combined tracks aggregate the source data along different lines, and +different populations, ancestries, and cohorts, for projects that gnomAD is unlikely to +reprocess soon. Three combined tracks aggregate the source data along different lines, and there is also one subtrack per project with the original VCF data and all the annotations that the project provides. The different projects use different pipelines and sequencing technologies. Click any of the projects above or below for a summary of their sample selection, sequencing assay and software pipeline. Many projects do not allow us to -distribute the data, but we document how to request it and provide all converters. +distribute the data, but we document how to request it and provide all converters, see Data Download below.
-Data from projects that provide haplotype-phased genotypes can also be found -elsewhere: 1000 Genomes is also a separate track, and the phased genotypes HGDP, SGDP, -HGDP+1000 Genomes and Mexico Biobank can also be found in the "Phased Variants" track. -Their VCF versions below show only the isolate frequency per variant. +The browser has other tracks with variant frequencies. We have of course the data +from gnomAD in separate tracks. Two projects that +provide haplotype-phased genotypes can also be found in their own tracks: +1000 Genomes is a separate track, and the phased +genotypes HGDP, SGDP, HGDP+1000 Genomes and Mexico Biobank are in the +Phased Variants track. Their VCF versions below show +only the allele frequency per variant, not the phased genotypes.
Please contact us (genome@soe.ucsc.edu) if you know of a project that we should add. So far, -Regeneron's Million Exomes and Mexico City Studies (request rejected) and Taiwan Biobank (pending). -
+we have requested data from Regeneron's Million Exomes and the Mexico City studies (both requests rejected); +Taiwan Biobank and the full UK Biobank WGS data requests are pending.Three combined tracks merge variants from the individual subtracks into single bigBed files with predicted protein consequences and cross-database filtering. All three use the same filter conventions (variant type, consequence, source database, allele frequency, allele count, and per-database AF/AC).
All three combined tracks share the same Consequence filter (Missense, Synonymous, Stop
Gained, Frameshift, Splice Donor, Splice Acceptor, Intron, 3' UTR, 5' UTR, Non-coding,
Intergenic, Other). The filter uses OR logic across the comma-separated consequence tokens
on each variant: a variant tagged stop_gained,frameshift is selected by either
the "Stop Gained" or the "Frameshift" filter. The "Other"
bucket catches the less common
Sequence Ontology consequence
-terms emitted by bcftools csq that don't fit the named buckets above. Examples
+ that don't fit the named buckets above. Examples
include splice_region (variant near a splice site but outside the canonical
donor/acceptor), start_lost / stop_lost (variant disrupts the
start codon or replaces the stop codon with a coding amino acid),
stop_retained (variant changes the stop codon but keeps it a stop),
inframe_insertion / inframe_deletion (in-frame indel that adds or
removes whole codons), and coding_sequence (CDS variant where the precise
impact is undetermined). If you include "Other" in the filter selection, no
records will be hidden by the consequence filter.
| Combined tracks | |||||||||
| Database | Region | N | Data Type | Cohort | Sub-populations | Downloadable from UCSC | |||
|---|---|---|---|---|---|---|---|---|---|
| Disease cohorts | Sequencing-based disease cohorts | ~130k | WGS/WES/long-read | Affected/case arms of SFARI SPARK WES/WGS, SCHEMA, GREGoR, GA4K | @@ -104,30 +123,40 @@~1.5mil | WGS/WES/long-read | Population cohorts + unaffected/control arms | Background AF and AC; per-cohort and ancestry breakdowns | No |
| Genotyping Array Databases Combined | TPMI, MexBB, UKBB | ~530k | Array / imputed | 14.7M variants | — | No | |||
| Individual project datasets | |||||||||
| Database | +Region | +N | +Data Type | +Cohort | +Sub-populations | +Downloadable from UCSC | +|||
| AllOfUs v7 | USA | 245k | WGS | General population, diverse | African, Indigenous American, East Asian, European, Oceanian, South Asian (local ancestry; see Notes below) | No | |||
| TOPMED Freeze 10 | USA | 151k | WGS | @@ -388,115 +417,76 @@GWAS SVatalog cohort: 101 samples with matched long-read SVs (see chirmade101Sv) | — | Yes | |||
| Indigenous Africans 180 | Africa (Ethiopia, Tanzania, Cameroon, Botswana) | 180 | WGS (>30x) | 12 indigenous populations across all four African language phyla (Khoesan, Niger-Congo, Nilo-Saharan, Afroasiatic) | — | No | |||
-The AllOfUs subtrack provides local-ancestry-stratified allele frequencies, not the -global ancestry categories used in the All of Us Research Program 2024 Nature paper -(see References). Each variant's per-ancestry AF/AC counts only the haplotypes whose -inferred local ancestry at that exact genomic position belongs to the named group -(strict-both-haps mode). The six ancestry classes -(African, Indigenous American, East Asian, European, Oceanian, South Asian) match HGDP-derived -local-ancestry reference panels and so include Oceanian, which is not one of the -paper's six global Rye categories (those are AFR, AMR, EAS, EUR, Middle Eastern, SAS). -For an admixed individual, the local-ancestry AF at a position can therefore differ -substantially from the AF among self-reported members of the same ancestry group. -The Ioannidis lab (Phoenix, UCSC) developed the pipeline that produced this VCF -and applied it to the AllOfUs v7 release; only variants with cohort allele count ≥ 20 -were retained. -
- --This subtrack derives from the gnomAD v3.1.2 release, which embeds the -4,094-genome jointly-called HGDP+1kG cohort (Koenig et al. 2024) inside the larger -gnomAD aggregation. To save space, we kept only INFO fields useful for clinical and -population-genetic interpretation. Two allele-frequency -sets are exposed: -
--The filter labels on the track configuration page, and the field descriptions in the -combined-track bigBed, reflect this distinction. Per-population -HGDP+1kG-cohort frequencies are not exposed because the cohort is too small for -stable per-population estimates in many populations. -
-Most tracks only show the variant and allele frequencies on mouseover or clicks. When zoomed in, tracks display alleles with base-specific coloring. Homozygote data are shown as one letter; heterozygotes are shown with both letters. All VCF files are normalized, with one allele per annotation (no multi-allele lines).
-Each subtrack includes the upstream project's VCF largely as-released; per-subtrack pipelines
-(coordinate liftover, format conversion, header normalization) are documented on each
+Each subtrack includes the upstream project's VCF largely as-released,
+sometimes converted from other file formats; per-subtrack pipelines (coordinate
+liftover, format conversion, header normalization) are documented on each
subtrack's own description page and recorded in the
build documentation.
-The conversion scripts (e.g. finngen_to_vcf.py, kovaToVcf.py,
-schema_addAcAnAf.py, svatalogFreqToVcf.py) live alongside the makedoc
+The conversion scripts
+live alongside the makedoc
in the scripts directory.
The combined Disease cohorts and Population reference tracks are built by a separate
pipeline: each per-subtrack VCF is normalized (bcftools norm), all sites are
merged into a single callset, consequence annotations are recomputed against Ensembl with
-bcftools csq, and the merged callset is split by phenotype into the two bigBed
-files via vcfToBigBed.py + bedToBigBed. Within each combined
+bcftools csq, and the merged callset is split by phenotype. Within each combined
track, the Affected AF and Background AF columns are
pooled across contributing cohort arms (sum of allele counts divided by sum of
allele numbers, with the per-arm AN derived from each cohort's AC and AF), so the displayed
-frequency matches the carrier-count scale and a small cohort with a high local AF cannot
-dominate the value. The mapping from upstream INFO fields to bigBed columns is driven by
-two configuration files in the scripts directory: databases.tsv (one row per
-source dataset, flagging which cohorts study a disease, and optionally a
-default_an for cohorts that publish only AF) and populations.tsv
-(per-population AC/AF columns within each source, including the affected and unaffected arm
-of each disease cohort). Editing those two files and rerunning
-mergeAndAnnotate.sh followed by vcfToBigBed.py --split-affected
-rebuilds the two tracks. The Genotyping Array Databases Combined track is built the same
+frequency matches the carrier-count.
+The Genotyping Array Databases Combined track is built the same
way from the array cohorts only.
All the data is publicly available. The table above indicates if we are allowed to distribute it in VCF format. Most of the databases do not allow us to redistribute the data files directly from our website, but it can always be downloaded from the original websites in some form. Click the database link in the table above and see the "Data Access" section of the respective track for a description of where to download the data. When the data is freely available from our website, the Data Access section will also indicate the VCF file location on our download server. Because it contains some licensed data, the combined track is not available for download, but can be recreated using the conversion scripts in our GitHub repository and the accompanying documentation file. -
+Many of these databases have restrictions on redistribution and download. +The table above indicates if we are allowed to distribute it in VCF format. +Click the database link in the table above and see the "Data Access" +section of the respective track for a description of where to download the +data. When the data is freely available from our website, the Data Access +section will also indicate the VCF file location on our download server. +Because it contains some licensed data, the combined track is not available for +download, but can be recreated using the conversion scripts in our GitHub repository and the accompanying documentation file.
This track is only possible thanks to the data from millions of volunteers around the world, who donated blood, signed consent forms and provided health information about themselves and sometimes their families. Click any of the tracks in the list above to see the specific credits for each project. Thanks to Alex Ioannidis, UCSC, for the inspiration for this track and to Andreas Lahner, MGZ, for feedback.
All of Us Research Program Genomics Investigators. Genomic data in the All of Us Research Program. Nature. 2024 Mar;627(8003):340-346. PMID: 38374255; PMC: PMC10937371