8bb883ed32e24a1007bf9f97ee1793a46bf47bd0
mspeir
  Thu Aug 27 10:00:43 2026 -0700
singleCellSignalsPeaks: classify the SEA-AD and Allen tracks, add BrainVar stages

72 hg38 SEA-AD and 89 mm10 allen-basal-ganglia tracks had lost their broad cell
class and were drawing in dataset colours instead of the shared palette; both
assemblies are now 0 unclassified. Fix is in the hub build (cellBrowser
ucsc/allTracksHub).

Adds an "Other glia" class, since "Other" was doing four unrelated jobs. BrainVar
goes 4 -> 13 hg38 subtracks with the new stage-split pseudobulk. Scaling moves to
"autoScale group" on the composite so selected subtracks share one scale, and
every bigWig now carries a data range -- without one the 13 BrainVar tracks drew
as flat lines against hgTracks' built-in 0:127.

Description pages and makeDocs updated, including removing the stale claim that
SEA-AD is coloured by its own subclass palette.

refs #37914, refs #37820

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

diff --git src/hg/makeDb/doc/hg38/singleCellSignalsPeaks.txt src/hg/makeDb/doc/hg38/singleCellSignalsPeaks.txt
index 9739603fea2..34dac1a5c5f 100644
--- src/hg/makeDb/doc/hg38/singleCellSignalsPeaks.txt
+++ src/hg/makeDb/doc/hg38/singleCellSignalsPeaks.txt
@@ -26,43 +26,59 @@
 # git-controlled). Default: /hive/data/inside/cells/all-tracks-hub-build
 #
 # The per-track source files (abs_path column of manifest.tsv) are the files the
 # Cell Browser datasets already serve; nothing is regenerated here, only copied.
 
 ##############################################################################
 # 2. Copy the data files into place  (bed dir, served via a /gbdb symlink)
 ##############################################################################
 # Every subtrack of the cellBrowserHg38 composite is copied into
 #   /hive/data/genomes/hg38/bed/singleCellSignalsPeaks/<served-relpath>
 # keeping each file's served relative path (e.g.
 #   human-enhancer-atlas/.../Adipocyte.bw ,
 #   allen-brain-science/seaad_MTG/bw/ADNC0Astro.bw ).
 # The served subpath is preserved on purpose: 18 peak-file basenames repeat
 # across datasets (cortex-atac), so a flat directory would clobber them.
-#   936 files total (bigWig + bigBed/bigNarrowPeak), ~206 GB.
+#   934 files total (bigWig + bigBed/bigNarrowPeak), ~274 GB as the copy script
+#   counts it (256 GiB; du reports 511 G because of GPFS block allocation).
+#
+# copySingleCellSignalsPeaksFiles.py skips files already up to date (same size and
+# a destination no older than its source), so a re-run after new data lands moves
+# only the new files -- adding the 9 stage-split brainvar bigWigs took ~1.2 GB and
+# 10 seconds even though the summary line still counts all 934 subtracks.
+#
+# 11 data files in the bed dir are no longer referenced by the .ra (134 MB): the 10
+# cortex-atac interact.old/ bigBeds reclassified to the interact composite and one
+# dropped QC cluster. Harmless, and both are explained above; delete them if the
+# dir is ever tidied.
 #
 # The file list comes straight from the composite's bigDataUrl lines mapped back
 # to manifest abs_paths; copy each abs_path to bed/<relpath> (mkdir -p parents).
 
 ##############################################################################
 # 3. Generate the trackDb .ra
 ##############################################################################
 # makeSingleCellSignalsPeaksRa.py reads the hub's hg38 stanzas, keeps the
 # cellBrowserHg38 subtracks, renames the composite to singleCellSignalsPeaks,
 # repoints every bigDataUrl at the local /gbdb copy, and writes the .ra with
 # group=regulation (ATAC signal/peaks sit with the ENCODE regulatory tracks).
-# Subtrack colors and labels (incl. the SEA-AD subclass colors) carry through.
+# Subtrack colors and labels carry through from the hub stanzas.
+#
+# Colors are the shared broad-cell-class palette, NOT each dataset's own scheme --
+# SEA-AD used to come through in its own subclass colors, and no longer does, so a
+# cell class is one color across every dataset and both assemblies. See
+# doc/mm10/singleCellSignalsPeaks.txt sections 4 and 4b.
 #
 #   scriptDir=$HOME/kent/src/hg/makeDb/scripts/singleCellSignalsPeaks
 #   python3 $scriptDir/makeSingleCellSignalsPeaksRa.py \
 #       --stanzas $CBHUB_OUT/stanzas/hg38.trackDb.txt \
 #       --out $HOME/kent/src/hg/makeDb/trackDb/human/hg38/singleCellSignalsPeaks.ra
 #
 # https://github.com/ucscGenomeBrowser/kent/tree/master/src/hg/makeDb/scripts/singleCellSignalsPeaks
 
 ##############################################################################
 # 4. Facet metadata
 ##############################################################################
 # The faceted composite's metaDataUrl points at a copy of the hub's hg38
 # main-faceted metadata (primaryKey = Track):
 #
 #   cp $CBHUB_OUT/meta/hg38.metadata.tsv \
@@ -84,22 +100,34 @@
 #   - cortex-atac calls peaks three ways and serves all three for each cell type.
 #     The method is a filename suffix on some files (AstroOligo_MACSpeaks.bb) and
 #     the containing directory on others (MACSpeaks/AstroOligo.bb) -- both layouts
 #     appear in the same dataset and the two files are genuinely different peak
 #     sets, so both forms are detected and named in the label.
 #   - human-enhancer-atlas rolls tissue-qualified fibroblast/endothelial clusters
 #     (Fibro_Muscle, Endothelial_General_2) up to one cell type; the source code
 #     goes in the label so the clusters stay distinguishable.
 #   - SEA-AD shortLabels carry a compact region + ADNC token (Astrocyte MTG A0),
 #     since the longLabel distinction alone would leave eight identical short
 #     labels per cell type (2 regions x 4 ADNC levels).
 
 ##############################################################################
 # Counts
 ##############################################################################
-# 936 files resolved across 9 datasets: human-enhancer-atlas (444),
-# sea-ad-brain-atac (184), cortex-atac (91), retina (69), neuro-degen-atac (67),
+# 934 subtracks across 9 datasets: human-enhancer-atlas (444),
+# sea-ad-brain-atac (184), cortex-atac (81), retina (69), neuro-degen-atac (66),
 # multiomic-human-heart (40), cardiogenesis-atac (19), olg-eae-ms (18),
-# brainvar (4). 10 mislabeled interaction bigBeds (cortex-atac interact.old/) are
-# reclassified to the interact composite and 1 QC cluster is dropped, leaving 925
-# subtracks in the track. Facet metadata rows match the subtracks 1:1.
-# All 925 longLabels are unique; shortLabels are <=22 chars with no underscores.
+# brainvar (13). Mislabeled interaction bigBeds (cortex-atac interact.old/) are
+# reclassified to the interact composite and QC clusters are dropped before the
+# .ra is written. Facet metadata rows match the subtracks 1:1.
+# All 934 longLabels are unique; shortLabels are <=22 chars with no underscores.
+# Every subtrack has a broad cell class and a palette color (0 unclassified).
+# One track legitimately shows class "Unknown": neuro-degen-atac's cluster whose
+# source label is literally "Unannotated". That is a real palette class, not a
+# classification failure.
+#
+# brainvar went 4 -> 13 subtracks when the group sent stage-split pseudobulk
+# coverage: 5 prenatal and 4 postnatal added alongside the 4 original
+# combined-stage tracks. Progenitors are prenatal only, so 9 new files, not 10.
+# Their methods text says "a tile size of 1 kb" but the data is 100 bp (every
+# interval is 100 bp wide on 100 bp boundaries, matching the TileSize-100
+# filenames); 100 bp is what the description page states. Worth confirming with
+# them which they intended.