01bc05ac9a282a6862111502f13601e513d5b60b max Tue Sep 8 06:16:04 2026 -0700 hg38 Fiber-seq: merge the methylation composite into the compendium The accessibility compendium and the separate Methylation composite covered the identical 41 samples, and cartDump.c assigns priority with the data element as the outer loop and the data type as the inner one. So one composite keeps a sample's six subtracks contiguous in the image, where two composites drew an accessibility block followed by a methylation block and comparing the two assays for one sample meant reading past every other sample. Both come off the same molecules in the same experiment, so side by side is the point. fiberSeqMeth is gone and its three data types moved in as cpg, cpgHap and cpgDiff, renamed because "hap" was already taken by the accessibility overlay and a data type name cannot contain an underscore. Subtracks now carry an explicit priority, sample outer and declared data type inner. Without one they fell back to a label sort, so a first visit showed a sample's data types as Peaks, CpG, Acc rather than in the order of the checkbox row above the table. Metadata columns renamed from camelCase to Accession, Sample_class, _Cell_type and _Sample. toTitleStyle() in facetedComposite.js renders an underscore as a space but does not split camelCase, so "sampleClass" appeared verbatim as a column heading. A literal space cannot be used instead: the saved sort order is a space-separated list of column names and the submit code drops any name containing whitespace, which would have made sorting silently fail to persist. Cell type is no longer faceted. A facet value is only offered when it occurs more than once, and 12 of the 14 cell types here are a single sample, so as a facet it drew two checkboxes and left 12 samples unreachable by any cell-type filter. It is a searchable column now, and Sample_class is the only facet until the lab gives us real HPRC metadata that would facet properly. Description page intro rewritten, and it now says the assay measures the same property as DNase-seq and ATAC-seq. refs #36210 diff --git src/hg/makeDb/doc/hg38/fiberSeq.txt src/hg/makeDb/doc/hg38/fiberSeq.txt index e74c04daf74..c22ae9d3be3 100644 --- src/hg/makeDb/doc/hg38/fiberSeq.txt +++ src/hg/makeDb/doc/hg38/fiberSeq.txt @@ -151,40 +151,47 @@ # chromCount 195, primary plus alts and patches # --------------------------------------------------------------------------- # trackDb, metadata and colors # --------------------------------------------------------------------------- # One script writes the whole track stanza plus the two faceted-composite # metadata and color files, so the 585 stanzas stay consistent: ~/kent/src/hg/makeDb/scripts/fiberSeq/fiberSeqTrackDb.py # It writes # ~/kent/src/hg/makeDb/trackDb/human/hg38/fiberSeq.ra # /hive/data/genomes/hg38/bed/fiberSeq/fiberSeqCompendium_metadata.tsv # /hive/data/genomes/hg38/bed/fiberSeq/fiberSeqCompendium_colors.json -# /hive/data/genomes/hg38/bed/fiberSeq/fiberSeqMeth_metadata.tsv -# /hive/data/genomes/hg38/bed/fiberSeq/fiberSeqMeth_colors.json # Structure: # fiberSeq container, group regulation # fiberSeqAcc multiWig overlay of 7 common cell lines, shown by default -# fiberSeqCompendium faceted composite, dataTypes acc/peaks/hap, 41 samples -# fiberSeqMeth faceted composite, dataTypes comb/hap/diffs, 41 samples +# fiberSeqCompendium faceted composite, 41 samples, six data types: +# acc, peaks, hap, cpg, cpgHap, cpgDiff # -# The two composites use the same faceted-composite machinery as Methbase +# Accessibility and methylation started as two composites, fiberSeqCompendium and +# fiberSeqMeth, and were merged into one. They cover the identical 41 samples, +# and cartDump.c assigns priority with the data element as the OUTER loop and the +# data type as the inner one, so a single composite keeps a sample's six subtracks +# contiguous in the image. As two composites the display was an accessibility +# block followed by a methylation block, so comparing the two assays for one +# sample meant reading across every other sample - which is the whole point, +# since both come off the same molecules in the same experiment. +# +# The composite uses the same faceted-composite machinery as Methbase # (methbase2.ra): metaDataUrl for the sample table, colorSettingsUrl for the # facet swatches, primaryKey for the row id, and dataTypes for the per-sample # data kinds. Two things about that machinery drove the naming: # # 1. Subtracks must be named <composite>_<accession>_<dataType>, with the # accession as the ONLY middle component. facetedCompositeUi() in # hg/hgTrackUi/hgTrackUi.c cuts the data element at the first underscore # after the composite name, and cartDump.c rebuilds subtrack names as # <composite>_<element>_<type>. The lab's hub used # fireCompendium_GM12878_PM00001_all, which would resolve to element # "GM12878" and then look for a track named fireCompendium_GM12878_all that # does not exist. Sample name and cell type live in the metadata TSV. # # 2. dataTypes brings onlyVisibility with it, which is what lets the bigWig # subtracks default to full while the peak subtracks default to dense. A @@ -209,50 +216,75 @@ # registered for the first time. encodePeakCfgUi() # in hg/lib/hui.c and bigNarrowPeakLoadItems() both # already handled the q/pValue filters; they had # just never been declared here, so tdbQuery # -strict rejected them. # # 4. The peak filters are the ENCODE peak settings, signalFilter, qValueFilter # and scoreFilter with their *Limits, which encodePeakCfgUi() draws with its # own labels ("Minimum Signal value", "Minimum Q-Value (-log10)"). The # bigBed-generic filter.<field> settings are NOT read by this type. All # three defaults are the full range, so nothing is hidden until the user # narrows one. Verified in the rendered UI: # hgTrackUi?db=hg38&g=fiberSeqCompendium_PM00004_peaks # draws min/max boxes for all three with the right limit hints. # -# 5. mouseOver text only appears in pack or full. Dense makes no per-item map +# 5. Facet columns: only "sampleClass" is faceted. facetedComposite.js offers +# a facet value only when it occurs more than once (a checkbox matching a +# single row is just a slow search box), so: +# accession 41 distinct, all count 1, and excluded anyway as primaryKey +# _sample 41 distinct, all count 1 - can never be a facet +# _cellType 14 distinct but only 2 with count > 1 (Lymphoblastoid 27, +# Embryonic stem cell 2), so as a facet it drew two checkboxes +# and left 12 samples unreachable. Underscored, so it is a +# searchable and sortable column instead. +# sampleClass 3 values, all count > 1. Derived from the lab's free-text +# cell type, and worth replacing when they give us real HPRC +# metadata (donor sex, population) that would facet properly. +# +# 6. Subtracks need an explicit priority. Without one they fall back to a label +# sort, which showed a sample's data types as Peaks, CpG, Acc on a first +# visit. The script now numbers them sample-outer, declared-data-type-inner +# (i*10 + j + 1), which matches the row of data type checkboxes across the +# top of the table. cartDump.c clears "<mdid>_*.priority" and writes its own +# on every submit, so this only sets the starting order. +# +# 7. mouseOver text only appears in pack or full. Dense makes no per-item map # boxes at all, so no bigBed-like track has a per-item hover there. Since # Andrew asked for peaks in dense by default, the mouseover is there for # whoever switches a peak track to pack. Checked by temporarily dropping # onlyVisibility and reading the image map: in pack the AREA tags carry # data-tooltip='<b>K562 FIRE peak</b><br>FIRE score: 17.76015 # <br>-log10 FDR: 22.807437495448326<br>Score: 177' # and in dense there are no per-peak AREA tags. Note the tooltip is HTML # entity encoded in the page, so grep for it with the entities in mind. # --------------------------------------------------------------------------- # Rendering checks # --------------------------------------------------------------------------- # ACTB promoter, the accessibility overlay, all seven cell lines in color: # hgRenderTracks?db=hg38&position=chr7:5,527,000-5,533,000&hideTracks=1 # &fiberSeq=show&fiberSeqAcc=full # Note that a child of a container needs the container shown as well; without # fiberSeq=show only the ruler comes back. # # The compendium at the same locus, showing that onlyVisibility works: the # three default samples come up with peaks in dense and signal in full, in one # composite. # ...&fiberSeqCompendium=full # # GNAS, an imprinted locus, for the methylation composite. GM12878's # haplotype overlay switches from orange (haplotype 2 methylated) to blue # (haplotype 1 methylated) across the locus, and the difference track follows: # hgRenderTracks?db=hg38&position=chr20:58,838,000-58,860,000&hideTracks=1 -# &fiberSeq=show&fiberSeqMeth=full -# &fiberSeqMeth_PM00001_hap_sel=1&fiberSeqMeth_PM00001_diffs_sel=1 +# &fiberSeq=show&fiberSeqCompendium=full +# &fiberSeqCompendium_PM00001_cpgHap_sel=1 +# &fiberSeqCompendium_PM00001_cpgDiff_sel=1 +# +# The merged composite at the same locus, with the three default data types on, +# renders each sample's tracks together and in the declared order: +# GM12878 Acc / GM12878 Peaks / GM12878 CpG / K562 Acc / K562 Peaks / ... # # The faceted table's embedded JSON on the composite UI page confirms the # naming works: dataElements comes back as bare accessions ("PM00001", # "PM00004", "PM00005") and dataTypes as acc/peaks active, hap inactive.