e8144b87c64b0e60e475886e3ced5353f139cf43 max Fri Sep 18 07:24:42 2026 -0700 hg38 episignatures: EpigenCentral CpG probes as a second subtrack #Preview2 week - bugs introduced now will need a build patch to fix The EpigenCentral group at the Centre for Computational Medicine and the Weksberg lab, Hospital for Sick Children, publishes its curated episignatures as a track hub and asked us to host it natively instead. 15,035 CpG sites from 24 episignatures for 23 rare disorders, alongside MethaDory in the same container. The two were compiled independently and neither contains the other: 14,236 sites are in both, at identical coordinates, and 799 are only here. Built from the lab's own bigBed by makeEpigenCentral.sh, so a refresh is one command rather than the hand edits it started as. Every feature in the hub is in the track; no coordinates or values were changed. Four things were rewritten on the way in: the OMIM column went from a full URL to the entry number so trackDb builds the link, the hub's pre-rendered mouse-over column became the direction alone with the text assembled from the fields, the disorder of the strongest signature was added as its own column, and 215 rows of the per-site comparison table on 137 sites were dropped as exact duplicates, which the site's own signature count had already collapsed. Two columns were renamed to what MethaDory calls the same numbers, height to maxAbsDelta and nSignatures to sigCount. Coordinates check out two ways: 5,000 sampled sites all land on a CG dinucleotide, and every probe shared with MethaDory, which was positioned from the Illumina manifests independently, is at the same base in both. The description page keeps the structure and wording written for the hub but not its markup. Its reference table had all 24 links pointing at one PMID while displaying another, so that table is now generated from the data joined to a checked-in PMID list, each one verified against PubMed. tableBrowser is off at the request of the data providers, so Data Access points at EpigenCentral's own portal and repository. refs #38112 diff --git src/hg/makeDb/doc/hg38/episignatures.txt src/hg/makeDb/doc/hg38/episignatures.txt index 0d3b7a55368..7ec1c97b09a 100644 --- src/hg/makeDb/doc/hg38/episignatures.txt +++ src/hg/makeDb/doc/hg38/episignatures.txt @@ -157,15 +157,136 @@ # integer-like keys numerically, other string keys keep insertion order. # The field is built by jsonTable() in methaDoryToBed.py, ASCII-escaped, since # kent's jsonParse passes \uXXXX through untouched. # Track search: the position box has to find a probe by its cg number. The # bigBed carries -extraIndex=name, and episignatures.ra has a matching # searchTable stanza. That stanza needs termRegex and semiShortCircuit, not just # searchPriority: hgFind splits specs into a short-circuiting list and a long # list (hgPositionsFind in hg/lib/hgFind.c), only specs with a termRegex land on # the short list, and as soon as one short spec matches the long list is never # run at all. Without the termRegex the spec loaded fine and searching a probe # silently returned only the Illumina array hits. semiShortCircuit then stops # our spec from suppressing those array hits in turn. hgsql hg38 -Ne "select searchName, shortCircuit, searchPriority from hgFindSpec_max where searchTable='methaDory'" # methaDory 1 50 + +############################################################################## +# EpigenCentral episignature loci (DONE 2026-09-18) +############################################################################## + +# 2026-09-18 Claude max: refs #38112 +# +# The second subtrack of the episignatures container. The EpigenCentral group at the +# Centre for Computational Medicine / Weksberg lab (Hospital for Sick Children, +# Toronto) publishes its episignatures as a track hub: +# https://github.com/ccmbioinfo/EpigenCentral-UCSC-Genome-Browser +# and asked us to host it as a native track instead. The portal itself is at +# https://epigen.ccm.sickkids.ca/ and is described in PMID 32623772. +# +# Barali Kitiyakara and Eliza Alde reviewed the hub, cloned it and worked out the +# trackDb settings; that work is on the ticket and at +# ~bkitiyak/public_html/epigenCentralHub/ +# What is here is the same set of changes redone as a repeatable build from the +# upstream file, so a refresh is one command. +# +# The hub's episignatures.bb already has the probe coordinates on hg38, so there is +# nothing to lift or map. The build downloads it and rewrites four things: +# - the OMIM column holds a full https://omim.org/entry/NNNNNN URL; only the number +# is kept so trackDb's "urls displayOmim=" can build the link +# - the hub renders its own mouse-over into a column ("NSD1|Loss|-0.346") and points +# mouseOverField at it. That column becomes the direction alone and trackDb builds +# the mouse-over from the fields, which is what the rest of our tracks do +# - 215 rows of the per-site comparison table, on 137 sites, are exact duplicates of +# another row for the same signature. The site's own nSignatures/signatureList +# columns already count each signature once, so the duplicates are dropped +# - the disorder of the strongest signature is added as its own column, so the +# mouse-over can name the disorder and not only the gene +# Two columns are also renamed to what the MethaDory subtrack calls the same numbers: +# height -> maxAbsDelta and nSignatures -> sigCount. + +mkdir -p /hive/data/genomes/hg38/bed/episignatures/epigenCentral +cd /hive/data/genomes/hg38/bed/episignatures/epigenCentral +bash ~/kent/src/hg/makeDb/scripts/episignatures/makeEpigenCentral.sh + +# Numbers it reports, for the record: +# upstream md5: 1f2cd0ef372fefa3b42c796fb89f6f40 +# features written: 15035 (same as the upstream itemCount) +# episignatures: 24, over 23 disorders +# displayed direction: Gain 5972, Loss 9063 +# duplicate table rows dropped: 215, on 137 probes +# table rows with no delta-beta: 1 +# widest comparison table: 478 bytes (cg26004771), hgc limit 4096 +# +# No feature is dropped: every probe in the hub is in the track. The one row with no +# delta-beta is the Dup7 signature at cg19457237, which the source has as NA in both +# the direction and the delta-beta column while giving it an adjusted p-value; it is +# shown as NA. The script errAborts rather than guessing if the displayed signature +# itself were the one missing a value. +# +# The widest comparison table matters because that field uses the ';' rows / '|' cells +# encoding of detailsDynamicTable, which printEmbeddedTable() in hg/hgc/hgc.c expands +# inside a fixed char[4096]. At 478 bytes there is a wide margin, so this track does +# not need the _json encoding that methaDory has to use. The script checks the limit +# on every build and stops if a refresh ever crosses it. + +# Sanity check on the coordinates, the same one the MethaDory build does: every cg +# probe should sit on a CG dinucleotide. The features are 1 bp, so take start..start+2. +awk -F'\t' 'BEGIN{OFS="\t"} {print $1,$2,$2+2,$4}' epigenCentral.bed \ + | shuf -n 5000 > /tmp/samp.bed +twoBitToFa /gbdb/hg38/hg38.2bit stdout -bed=/tmp/samp.bed | grep -v '^>' \ + | tr 'a-z' 'A-Z' | sort | uniq -c +# 5000 CG, no exceptions. + +# Cross-check against the MethaDory subtrack, which got its positions from the Illumina +# manifests rather than from EpigenCentral. 14,236 of the 15,035 probes are in both and +# every one of them is at the same position; 799 are only here. +bigBedToBed /gbdb/hg38/episignatures/methaDory.bb stdout \ + | awk -F'\t' '{print $4"\t"$1":"$2}' | sort -u > /tmp/md.pos +awk -F'\t' '{print $4"\t"$1":"$2}' epigenCentral.bed | sort -u > /tmp/ec.pos +join -t$'\t' /tmp/md.pos /tmp/ec.pos | awk -F'\t' '$2!=$3' | wc -l +# 0 + +# The signature filter menu and the "Included episignatures" table of the description +# page are generated by the build, into epigenCentralFilters.ra and +# epigenCentralTable.html. On a refresh, paste the two .ra lines back into +# human/hg38/episignatures.ra, and the table back into human/hg38/epigenCentral.html +# between the "<!-- BEGIN generated ... -->" and "<!-- END generated -->" markers, +# rather than editing either by hand. The counts quoted in the Description and Methods +# paragraphs of that page have to be updated at the same time. +# +# The publication behind each signature comes from the hub's README and is kept in +# scripts/episignatures/epigenCentralRefs.tsv, since the README is not machine +# readable. All 18 distinct PMIDs were checked against PubMed esummary before being +# written down; EHMT1 is the one signature with no PubMed record and is cited by DOI. +# The hub's own HTML page has every one of these links pointing at PMID 31311581 while +# displaying a different number, which is why they are generated here instead. + +# A comma in a filterValues entry is the entry separator and cannot be escaped when the +# filterType is one of the *List* kinds, so the commas inside four of the disorder names +# ("Dystonia 28, childhood-onset" and friends) are dropped in the menu labels by +# writeRa() in epigenCentralToBed.py. The bigBed keeps the names as the source has them. + +# Track search: the position box finds a probe by its cg number. The bigBed carries +# -extraIndex=name and episignatures.ra has a matching searchTable stanza. Both this +# spec and the methaDory one carry termRegex and semiShortCircuit, so a probe ID that +# is in both tracks returns a hit in both, plus the Illumina array tracks. +hgsql hg38 -Ne "select searchName, shortCircuit, searchPriority from hgFindSpec_max where searchTable='epigenCentral'" +# epigenCentral 1 51 + +# Lou asked on the ticket for downloads to be off, so the stanza has "tableBrowser off" +# and the Data Access section points at EpigenCentral's own portal and repository. That +# switch covers the Table Browser, the Data Integrator and the REST API; the bigBed +# still sits in /gbdb and is reachable on hgdownload, so it is not a hard block, and QA +# should confirm with the lab that this is what they wanted. + +# Visibility: the track is sparse almost everywhere, 15,035 sites over the genome, but it +# has one hot spot at the HOXA cluster. Measured heights at 1100px wide, pack, with no +# other track on: +# chr7:27,140,000-27,250,000 (110 kb) 961 px +# chr7:27,000,000-28,000,000 (1 Mb) 1937 px +# chr7 (whole chromosome) 3345 px +# There is no automatic fallback at any of those, so the stanza has the same +# maxWindowCoverage 200000 as methaDory. Above 200 kb the track draws as a coverage +# graph, 81 px, and below it stays in pack with the probe IDs, which is the point of +# zooming in. The 110 kb view is still tall; a reader who does not want the labels can +# switch that one to squish, 121 px.