7713a08da69691ba499d5b9d44379c43e8cdb608
markd
  Fri Sep 18 09:27:52 2026 -0700
Adding Transcription Start container with ENCODE 4 PRO-cap and ProCapNet tracks. refs #35528

New superTrack transcriptionStart in the rna group, holding two faceted
composites: encode4ProCap with PRO-cap measurements and proCapNet with the
model predictions and sequence-contribution scores.  hg38 has all three data
types, hs1 the predictions only.

ENCODE 4 PRO-cap comes from the portal rather than the submitter's hub copy.
For each of the six experiments only the plus and minus strand signal of unique
reads files of that experiment's default analysis are taken, which drops files
superseded by a later reprocessing.  ENCODE publishes no pooled file, so the
per-replicate files are summed per cell line and strand, the same merge the
ProCapNet models were trained on.  Total signal is conserved exactly.

The published ProCapNet prediction bigWigs store one bedGraph interval per base
and hold a literal NaN at every unresolved (N) base on hg38, which makes
autoScale and every summary statistic NaN.  They are re-encoded into fixedStep
sections, a third smaller with no value changed, dropping 164,268,582 NaN bases
of 3,088,269,832 on hg38 and none on hs1.  Losslessness verified against the
originals on random windows across five chromosomes.

The composites are faceted rather than plain because a container multiWig under
a plain composite is flattened away by hgTrackUi and never drawn.  Each cell
line is one row with a checkbox per data type, a Sample class facet, ENCODE
accession links and a Files column linking each bigWig on hgdownload.

Scripts and the cell line configuration are in makeDb/outside/proCapNet; the
trackDb stanzas and the faceted metadata tables are generated, not hand edited.

Claude-Session: https://claude.ai/code/session_01LAB6jWshLvB7eNXQKWVuW5

diff --git src/hg/makeDb/outside/proCapNet/proCapNetDownload src/hg/makeDb/outside/proCapNet/proCapNetDownload
new file mode 100755
index 00000000000..e468c95bb2f
--- /dev/null
+++ src/hg/makeDb/outside/proCapNet/proCapNetDownload
@@ -0,0 +1,42 @@
+#!/bin/bash -e
+# Download the ProCapNet prediction and sequence-contribution bigWigs from the
+# Kundaje lab server for one assembly.  Contribution scores exist for hg38 only.
+#
+# The predictions are re-encoded before use, so they are fetched to a working
+# directory outside the track data directory and can be deleted afterwards.  The
+# contribution scores are used unaltered and are fetched straight to the track
+# data directory under their final name.
+#
+# The hub names some cell lines in upper case where we use the cell name, so the
+# file names come from the cell and hubCell columns of the experiments table.
+#
+# usage: proCapNetDownload hg38|hs1 experimentsTsv predWorkDir [contribDir]
+
+if [ $# -lt 3 ] ; then
+    echo "usage: proCapNetDownload hg38|hs1 experimentsTsv predWorkDir [contribDir]" >&2
+    exit 1
+fi
+db=$1
+experimentsTsv=$2
+predWorkDir=$3
+contribDir=$4
+hubUrl=https://mitra.stanford.edu/kundaje/kcochran/ucsc_track_hubs_public/ProCapNet
+fetchList=$(mktemp)
+trap "rm -f $fetchList" EXIT
+
+mkdir -p $predWorkDir
+awk -F'\t' -v url=$hubUrl/$db/procapnet_preds -v dir=$predWorkDir 'NR>1 {
+        print url "/" $2 ".pos.bigWig\t" dir "/" $1 ".pos.bigWig"
+        print url "/" $2 ".neg.bigWig\t" dir "/" $1 ".neg.bigWig"
+    }' $experimentsTsv > $fetchList
+
+if [ -n "$contribDir" ] ; then
+    mkdir -p $contribDir
+    awk -F'\t' -v url=$hubUrl/$db/procapnet_contrib_scores -v dir=$contribDir 'NR>1 {
+            print url "/" $2 ".profile_contrib_scores.bigWig\t" dir "/" $1 ".proCapNet-contrib.bw"
+        }' $experimentsTsv >> $fetchList
+fi
+
+# --continue-at lets an interrupted run restart without refetching
+awk -F'\t' '{print "curl -sSf --retry 5 --continue-at - -o " $2 " " $1}' $fetchList \
+    | xargs -d '\n' -P 6 -I CMD sh -c CMD