bc527b6264234c33824854e5b596fb6f790983d4
lrnassar
  Tue Aug 4 14:22:28 2026 -0700
Fix release blockers and data errors found during QA of the mouseDevTimecourse tracks. refs #37001

Add maxWindowToDraw 10000000 to the six bigBarChart subtracks. Without it a
whole-chromosome view asked the track to draw 3445 items x 156 bars and took 71
seconds on mm10 and 69 on mm39. The superTrack is on by default, so any mouse
user zooming out could hit it. Now 254 ms and 86 ms.

Correct a 1-bp off-by-one in the bigBarChart chromStart. The hub builder wrote
1-based GTF gene starts into the 0-based BED chromStart field, so every gene sat
one base right of its true start while chromEnd was correct. Measured against
GENCODE on mm10 before the fix, 42081/42093 genes (VM21) and 35323/35333 (VM4)
were start+1 with none exact. Added fixBarChartStarts.sh, which rebuilds the
files and refuses to run on one that has already been corrected. Originals kept
as *.bb.preStartFix. Reported upstream to the hub author.

Correct the replicate numbers on the bigWig signal composite. The biosample TSV
has no replicate column, so generateBigwigTrackDb.py had been deriving one by
sorting biosample accessions alphabetically, which mislabeled 124 of the 312
subtracks and flipped the default-on state of 62 of them. Added
fetchReplicateNumbers.py to read the real biological_replicate_number from the
ENCODE portal.

Rewrite the signal shortLabels. They had been hard-truncated at 20 characters,
which left 20 subtracks with duplicate labels. The generator now emits a
one-letter view code and errors out if two labels match within the 17 characters
hgTracks draws in the left label area, rather than silently truncating.

Make generateBigwigTrackDb.py reproduce the committed .ra. It now emits the
two-digit tissue prefixes that give the author-requested biological order, and
the html setting, instead of depending on a one-off patch applied afterwards.

Add barChartMerge, barChartMetric, labelFields and defaultLabelFields, and Title
Case all shortLabels.

Update the nine description pages: replicate wording to match barChartMerge,
GitHub source links to the makedoc, build scripts and trackDb in Methods, an
mm39 liftOver accounting note, and remove a duplicated sentence from the shared
Display include.

Add curl -f to downloadBigwigs.sh so an HTTP error body is never saved as a
bigWig and then skipped forever by the restart check.

diff --git src/hg/makeDb/doc/mm10.txt src/hg/makeDb/doc/mm10.txt
index b1ce387e5bf..39f5bb09450 100644
--- src/hg/makeDb/doc/mm10.txt
+++ src/hg/makeDb/doc/mm10.txt
@@ -20008,46 +20008,97 @@
 
 cd ~/kent/src/hg/makeDb/trackDb/mouse/mm10
 ~/kent/src/hg/makeDb/scripts/mouseDevTimecourse/generateBigwigTrackDb.py > developmentTimecourseSignalMm10.ra
 # Added "include developmentTimecourseSignalMm10.ra" to trackDb.ra
 
 # 2026-07-14 (Gerardo): Peng noticed the bigWig tracks display in
 # alphabetical order (#36998 note-79). sortOrder compares the tag strings
 # alphabetically; where a tag sits in the subGroup2 line has no effect.
 # Ran the script below to prefix each tissue tag with a two-digit number
 # matching its position in biological order (t01_thymus, t02_spleen, ...
 # t17_neural_tube), so the alphabetical comparison now produces the right
 # order. The display label is unchanged. Before:
 # adrenal_gland=adrenal_gland (tag and label are the same string). After:
 # t07_adrenal_gland=adrenal_gland (tag has the prefix, label doesn't).
 # generateBigwigTrackDb.py was not updated to match; re-running it would
-# regenerate the old alphabetical tags.
+# regenerate the old alphabetical tags. (Folded into the script during QA on
+# 2026-08-04, see below; the one-off snippet here is kept for the record but
+# is no longer needed.)
 
 cd ~/kent/src/hg/makeDb/trackDb/mouse/mm10
 
 python3 <<'EOF'
 import re
 
 path = "developmentTimecourseSignalMm10.ra"
 text = open(path).read()
 
 order = [
     "thymus", "spleen", "liver", "heart", "skeletal_muscle_tissue",
     "urinary_bladder", "adrenal_gland", "kidney", "lung", "stomach",
     "intestine", "limb", "embryonic_facial_prominence", "forebrain",
     "midbrain", "hindbrain", "neural_tube",
 ]
 tag_map = {tag: "t%02d_%s" % (i + 1, tag) for i, tag in enumerate(order)}
 
 # subGroup2 header line: new tag=label pairs (tag prefixed, label unprefixed)
 new_line = "    subGroup2 tissue Tissue " + " ".join(tag_map[t] + "=" + t for t in order)
 text = re.sub(r"^    subGroup2 tissue Tissue .*$", new_line, text, count=1, flags=re.MULTILINE)
 
 # Each subtrack's "subGroups view=... tissue=<tag> age=..." line
 def repl(m):
     return "tissue=" + tag_map[m.group(1)]
 text = re.sub(r"tissue=([a-z_]+)(?= age=)", repl, text)
 
 open(path, "w").write(text)
 EOF
 
+# 2026-08-04 (Lou, QA #37001): three corrections to the bigWig composite, all
+# folded into generateBigwigTrackDb.py so it now reproduces the committed .ra.
+#
+# 1. Replicate numbers were wrong on 124 of the 312 subtracks (31 of the 78
+#    experiments had their two replicates swapped). Diane's biosample TSV has no
+#    replicate column, so the script had been deriving one by sorting biosample
+#    accessions alphabetically, which bears no relation to ENCODE's
+#    biological_replicate_number. Added fetchReplicateNumbers.py to pull the real
+#    values from the ENCODE portal:
+
+~/kent/src/hg/makeDb/scripts/mouseDevTimecourse/fetchReplicateNumbers.py \
+  > /hive/data/outside/woldlab/mouseDevTimecourse/mm10/ENCSR574CRQ_replicates.tsv
+
+# 2. shortLabels were built and then hard-truncated at 20 characters, which cut
+#    the view suffix off entirely for the 10 longest and gave 20 subtracks
+#    duplicate labels ("neuraltube e15.5 r1 " for both the unique-reads and
+#    all-reads subtrack). The script now emits a one-character view code (U/A)
+#    and errors out instead of truncating.
+#
+# 3. Tissue tags and the html line are emitted by the script rather than patched
+#    in afterwards, and all labels are Title Cased.
+
+cd ~/kent/src/hg/makeDb/trackDb/mouse/mm10
+~/kent/src/hg/makeDb/scripts/mouseDevTimecourse/generateBigwigTrackDb.py \
+  > developmentTimecourseSignalMm10.ra
+
+# 2026-08-04 (Lou, QA #37001): corrected a 1-bp off-by-one in the bigBarChart
+# chromStart. The hub builder wrote 1-based GTF gene starts into the 0-based BED
+# chromStart field, so every gene sat one base right of its true start while
+# chromEnd was correct. Verified against GENCODE on mm10 before the fix:
+# 42081/42093 genes (VM21) and 35323/35333 (VM4) were start+1 with none exact
+# and ends matching. Reported upstream to Diane Trout; like the tissue reorder
+# and the color update above, this has to be reapplied after any hub refetch.
+# Originals are kept as *.bb.preStartFix.
+
+~/kent/src/hg/makeDb/scripts/mouseDevTimecourse/fixBarChartStarts.sh mm10 \
+    /hive/data/outside/woldlab/mouseDevTimecourse/mm10/mouse_development_TPM_M4_mm10.bb \
+    /hive/data/outside/woldlab/mouseDevTimecourse/mm10/mouse_development_FPKM_M4_mm10.bb \
+    /hive/data/outside/woldlab/mouseDevTimecourse/mm10/mouse_development_TPM_M21_mm10.bb \
+    /hive/data/outside/woldlab/mouseDevTimecourse/mm10/mouse_development_FPKM_M21_mm10.bb
+
+# Re-verified afterwards: zero genes remain at start+1, and 42073 of the 42093
+# comparable genes match GENCODE VM21 exactly on both start and end (42081 on
+# start, 42079 on end). The ~20 that still differ are genes where the track and
+# wgEncodeGencodeCompVM21 disagree on the gene extent, since Comp excludes some
+# transcript biotypes; they are not coordinate errors. itemCount is unchanged and
+# every non-coordinate field (name, score, strand, name2, expCount, expScores) is
+# byte-identical.
+
 ##############################################################################