bdf448295c10bee74b6baa773f6733ce33171996 markd Thu Oct 1 20:56:37 2026 -0700 Store the ProCapNet minus strand negated, and fix group autoscale. refs #35528 Two of the three problems Jairo found on the composite. The negate control did not work on ProCapNet. Its minus-strand files held positive values and were flipped at display time with trackDb negateValues. The composite's control sets one shared value, which replaced the per-track setting and sent both strands the same way, with no route back to the default short of a cart reset. encode4ProCap was fine because ENCODE publishes its minus strand negative already. So put the sign in the data: proCapNetPredToFixedStep gains --negate, proCapNetPredBuild passes it for the minus strand, and negateValues is gone from the stanzas. The twelve existing files were rewritten rather than rebuilt, since the downloads had been deleted; each was checked against its original for identical nBasesCovered, a mirrored value range and exactly negated values. The originals are kept in previousPred, about 197 GB, until QA is done. Group autoscale on the composite did nothing, which was my error: the multiWig containers and the contribution leaves each carried autoScale on, and the child setting wins. The wiggle settings now live only on the composite. The third, an empty row when both strands of a cell line are deselected, is #38441 and is not fixed here. Note in both makedocs and in the generator that this arrangement only partly works and is expected to become a superTrack of multiWig overlays: under a composite, hgTrackUi lists leaves rather than containers, so an overlay gets no inline config block and leaves an empty row when deselected. diff --git src/hg/makeDb/doc/hs1/transcriptionStart.txt src/hg/makeDb/doc/hs1/transcriptionStart.txt index 476bfdbe0be..9e123f340bf 100644 --- src/hg/makeDb/doc/hs1/transcriptionStart.txt +++ src/hg/makeDb/doc/hs1/transcriptionStart.txt @@ -1,42 +1,86 @@ # Transcription start sites: ProCapNet predictions, #35528, Claude Thu Sep 17 2026 (Claude/markd) +# STATUS: the composite only partly works, and the plan is to replace it with a +# superTrack holding the multiWig overlays directly, the way fantom5 does. A +# multiWig under a composite draws correctly and the composite's wiggle controls +# apply, but hgTrackUi lists descendant leaves rather than containers, so the +# overlays get no inline config block and turning off both strands of a cell line +# leaves an empty row where the overlay was (#38441). Under a superTrack each +# overlay is a track in its own right, with its own full configuration page and +# no flattening. The cost is losing the subtrack matrix and the sample class +# filter. + # hs1 has the ProCapNet predictions only. There is no PRO-cap experiment and no # sequence-contribution score set on this assembly. See # makeDb/doc/hg38/transcriptionStart.txt for the hg38 build, which carries all # three. Scripts are in ~/kent/src/hg/makeDb/outside/proCapNet. mkdir -p /hive/data/outside/proCapNet/hs1 /hive/data/genomes/hs1/bed/proCapNet cd /hive/data/genomes/hs1/bed/proCapNet ~/kent/src/hg/makeDb/outside/proCapNet/proCapNetDownload hs1 \ ~/kent/src/hg/makeDb/outside/proCapNet/proCapNetExperiments.tsv \ /hive/data/outside/proCapNet/hs1/pred # 300 GB, about 30 minutes at six parallel streams # Re-encode from one bedGraph interval per base into fixedStep sections. Unlike # hg38 these files hold no NaN, so every base is kept. ~/kent/src/hg/makeDb/outside/proCapNet/proCapNetPredBuild \ /hive/data/genomes/hs1/chrom.sizes /hive/data/outside/proCapNet/hs1/pred pred 6 # all 12 files report: # 24 chroms, 3117275501 bases, 3117275501 with data, 0 dropped # about 25 GB in, 16.7 to 17.0 GB out per file +# proCapNetPredBuild writes the minus strand negated, so it draws below the +# baseline from its own values. Doing it in the data rather than with the +# trackDb negateValues setting is what makes the composite's own negate control +# work: that control sets one shared value for the composite, which replaced the +# per-track negateValues and sent both strands the same way, with no way back to +# the default short of a cart reset. encode4ProCap never had the problem, +# because ENCODE already publishes its minus strand negative. + # the mean matches the published file exactly, so nothing was altered bigWigInfo pred/K562.proCapNet.pos.bw | egrep 'basesCovered|mean' # basesCovered: 3,117,275,501 # mean: 0.019773 +############################################################################## +# negating the minus strand, after the fact +############################################################################## + +# The prediction files were first built with the minus strand positive and +# flipped at display time with the trackDb negateValues setting. That broke the +# composite's negate control, as described above, so the twelve minus-strand +# files were rewritten with negated values rather than rebuilt from the +# downloads, which had already been deleted: +# +# for db in hg38 hs1 ; do +# base=/hive/data/genomes/$db/bed/proCapNet +# mkdir -p $base/previousPred +# ls $base/pred/*.neg.bw | while read f ; do +# ~/kent/src/hg/makeDb/outside/proCapNet/proCapNetPredToFixedStep \ +# --negate /hive/data/genomes/$db/chrom.sizes $f $base/negated/$(basename $f) +# done +# done +# +# Each output was checked against its input: same nBasesCovered, value range +# mirrored, sampled values exactly negated. The originals were then moved to +# previousPred/ and the negated files put in their place, so /gbdb needs no new +# symlinks. previousPred/ is about 197 GB and can be removed once the track has +# been through QA. A rebuild from the downloads does not need any of this: +# proCapNetPredBuild negates the minus strand itself. + cd ~/kent/src/hg/makeDb/outside/proCapNet ./proCapNetTrackDb hs1 proCapNetExperiments.tsv \ ~/kent/src/hg/makeDb/trackDb/human/hs1/transcriptionStart.ra # transcriptionStart.ra is generated; the include line in human/hs1/trackDb.ra # is added by hand: # include transcriptionStart.ra alpha # hs1 is a curated hub assembly, so the stanzas reach the browser through the # hub built under /gbdb/hs1/hubs/$USER by the trackDb make, and are visible only # when curatedHubPrefix in the sandbox hg.conf names that directory. # the downloads are only needed for the re-encoding rm -rf /hive/data/outside/proCapNet/hs1/pred