bc527b6264234c33824854e5b596fb6f790983d4
lrnassar
  Tue Aug 4 14:22:28 2026 -0700
Fix release blockers and data errors found during QA of the mouseDevTimecourse tracks. refs #37001

Add maxWindowToDraw 10000000 to the six bigBarChart subtracks. Without it a
whole-chromosome view asked the track to draw 3445 items x 156 bars and took 71
seconds on mm10 and 69 on mm39. The superTrack is on by default, so any mouse
user zooming out could hit it. Now 254 ms and 86 ms.

Correct a 1-bp off-by-one in the bigBarChart chromStart. The hub builder wrote
1-based GTF gene starts into the 0-based BED chromStart field, so every gene sat
one base right of its true start while chromEnd was correct. Measured against
GENCODE on mm10 before the fix, 42081/42093 genes (VM21) and 35323/35333 (VM4)
were start+1 with none exact. Added fixBarChartStarts.sh, which rebuilds the
files and refuses to run on one that has already been corrected. Originals kept
as *.bb.preStartFix. Reported upstream to the hub author.

Correct the replicate numbers on the bigWig signal composite. The biosample TSV
has no replicate column, so generateBigwigTrackDb.py had been deriving one by
sorting biosample accessions alphabetically, which mislabeled 124 of the 312
subtracks and flipped the default-on state of 62 of them. Added
fetchReplicateNumbers.py to read the real biological_replicate_number from the
ENCODE portal.

Rewrite the signal shortLabels. They had been hard-truncated at 20 characters,
which left 20 subtracks with duplicate labels. The generator now emits a
one-letter view code and errors out if two labels match within the 17 characters
hgTracks draws in the left label area, rather than silently truncating.

Make generateBigwigTrackDb.py reproduce the committed .ra. It now emits the
two-digit tissue prefixes that give the author-requested biological order, and
the html setting, instead of depending on a one-off patch applied afterwards.

Add barChartMerge, barChartMetric, labelFields and defaultLabelFields, and Title
Case all shortLabels.

Update the nine description pages: replicate wording to match barChartMerge,
GitHub source links to the makedoc, build scripts and trackDb in Methods, an
mm39 liftOver accounting note, and remove a duplicated sentence from the shared
Display include.

Add curl -f to downloadBigwigs.sh so an HTTP error body is never saved as a
bigWig and then skipped forever by the restart check.

diff --git src/hg/makeDb/scripts/mouseDevTimecourse/fixBarChartStarts.sh src/hg/makeDb/scripts/mouseDevTimecourse/fixBarChartStarts.sh
new file mode 100755
index 00000000000..c9e3216561f
--- /dev/null
+++ src/hg/makeDb/scripts/mouseDevTimecourse/fixBarChartStarts.sh
@@ -0,0 +1,73 @@
+#!/bin/bash
+# Correct a 1-bp off-by-one in the chromStart of the Wold Lab mouse development
+# bigBarChart files.
+#
+# The hub's builder wrote 1-based GTF gene starts into the 0-based BED
+# chromStart field, so every gene sits one base to the right of its true start
+# while chromEnd is correct. Verified against GENCODE VM21 and VM4 on mm10:
+# 42081/42093 (M21) and 35323/35333 (M4) genes were start+1, none exact, with
+# ends matching. mm39 inherits the offset through the liftOver. See #37001.
+#
+# Reported upstream to Diane Trout; rerun this after any hub refetch, in the
+# same way the tissue reorder and color update have to be reapplied.
+#
+# Usage: fixBarChartStarts.sh <db> <file.bb> [<file.bb> ...]
+# Originals are kept alongside as <name>.bb.preStartFix
+
+set -euo pipefail
+
+if [ $# -lt 2 ]; then
+    echo "usage: fixBarChartStarts.sh <db> <file.bb> [<file.bb> ...]" >&2
+    exit 1
+fi
+
+DB=$1; shift
+SIZES=/hive/data/genomes/$DB/chrom.sizes
+[ -s "$SIZES" ] || { echo "no chrom.sizes for $DB at $SIZES" >&2; exit 1; }
+
+TMP=$(mktemp -d)
+trap 'rm -rf "$TMP"' EXIT
+
+for BB in "$@"; do
+    [ -s "$BB" ] || { echo "missing: $BB" >&2; exit 1; }
+    NAME=$(basename "$BB")
+    echo "=== $NAME ==="
+
+    # Reuse the file's own schema so the rebuilt file is otherwise identical.
+    bigBedInfo -as "$BB" | sed -n '/^table /,/^    )/p' > "$TMP/schema.as"
+    [ -s "$TMP/schema.as" ] || { echo "could not extract .as from $NAME" >&2; exit 1; }
+
+    bigBedToBed "$BB" "$TMP/in.bed"
+    IN_ROWS=$(wc -l < "$TMP/in.bed")
+
+    # Refuse to run on a file that has already been corrected, which would
+    # underflow chromStart to -1. mt-Tf legitimately sits at chrM:0 after the
+    # fix, so this is the signature of a rerun. Checked in its own pass: awk's
+    # exit status is not observable from the middle of a pipeline.
+    if awk -F'\t' '$2 <= 0 {print; found=1} END {exit !found}' "$TMP/in.bed" \
+         > "$TMP/zero.bed"; then
+        echo "$NAME already has chromStart <= 0 (e.g. $(head -1 "$TMP/zero.bed" \
+            | cut -f1-4 | tr '\t' ' ')) - already corrected, refusing to shift again" >&2
+        exit 1
+    fi
+
+    awk -F'\t' 'BEGIN{OFS="\t"} {$2 = $2 - 1; print}' "$TMP/in.bed" \
+      | LC_ALL=C sort -k1,1 -k2,2n > "$TMP/out.bed"
+
+    OUT_ROWS=$(wc -l < "$TMP/out.bed")
+    [ "$IN_ROWS" = "$OUT_ROWS" ] || {
+        echo "row count changed: $IN_ROWS -> $OUT_ROWS" >&2; exit 1; }
+
+    bedToBigBed -as="$TMP/schema.as" -type=bed6+3 \
+        "$TMP/out.bed" "$SIZES" "$TMP/new.bb"
+
+    NEW_ROWS=$(bigBedInfo "$TMP/new.bb" | awk '/^itemCount/{gsub(",","",$2); print $2}')
+    [ "$NEW_ROWS" = "$OUT_ROWS" ] || {
+        echo "itemCount mismatch: expected $OUT_ROWS got $NEW_ROWS" >&2; exit 1; }
+
+    cp -p "$BB" "$BB.preStartFix"
+    mv "$TMP/new.bb" "$BB"
+    chmod 664 "$BB"
+    echo "  $IN_ROWS items, starts shifted -1, original kept at $NAME.preStartFix"
+    rm -f "$TMP/in.bed" "$TMP/out.bed"
+done