679b48855a32f8ea1e1ea8ca3c095a0c2c7f6c7a
max
  Tue Sep 8 10:08:41 2026 -0700
detailsScript: add a scatterPlot plot type, and use it for pcLAI

Clicking a pcLAI window now shows where that window sits in the ancestry space
it was placed in: a scatterplot of the 1000 Genomes reference haplotypes with
the window's own PCA coordinate and its segment's coordinate marked on it. The
numbers were already on the details page and told a reader almost nothing.

New plot type scatterPlot (hg/js/hgc.scatterPlot.js), driven the same way as
histogram. Background points come from a JSON or TSV file named by dataUrl and
may carry a category, which colors them and builds a legend, and a label, which
is shown on mouseover. The cloud is drawn on a canvas, since these files hold
thousands of points and that many <circle> elements make the page crawl; axes
and the highlighted points stay SVG on top. Point lookup for the mouseover goes
through a cell index so a large file stays smooth.

Two additions serve every plot type, not just this one:

- exportFields, a config key listing further bigBed fields whose values are
passed to the module as a fieldValues object. Without it a plot needing two
coordinates would need them packed into one field, and pcLAI keeps them in
pca and pcaSegment. Only fields that exist in the bigBed are exported, at
most 32, and the JSON types are checked rather than asserted because
jsonListVal and jsonStringVal errAbort and this JSON is written by a hub.

- a config key ending in Url is treated as a file, by the convention
trackSettingIsFile() already uses, and a relative one is resolved against the
track's own bigDataUrl. The module does not fetch it directly; it asks
hgTrackUi for it, the route facetedComposite uses for its metadata. That
checks the canonicalized path against the hubs on the cart and reads it with
udc, so a hub-relative path works even for a hub loaded from a local path
(the GenArk /gbdb hubs), no CORS header is needed, and a file outside a
connected hub cannot be read. Verified that /etc/passwd, file://, a dot-dot
escape, an unattached hub and an unrelated host are all refused with 400.
When the session has file caching off, hgc now exports udcTimeout the way
hgTrackUi does and the module POSTs, so the browser cannot answer from cache.

Fixes a crash reachable from any hub: "detailsScript.<plotType>.<field> null"
segfaulted hgc, because jsonObjectVal returns NULL for a JSON null and the hash
routines dereference it. This hit the shipped histogram type too.

trackDbSettingsGen.py stopped reading a setting's description at the first
"Example:" paragraph and never read <ul> at all, so it dropped everything after
the first example and every list item. That silently truncated 226 of the 264
descriptions, including spectrum's minGrayLevel/scoreMin/scoreMax bullets, and
would have dropped this whole scatterPlot section. It now skips the Example
label instead of stopping, and folds list items in. No setting loses a word and
none gains or loses an example.

pcLAI wiring: the background file is the authors' published reference panel
(github.com/AI-sandbox/hprc-pclai reference_pca_metadata.tsv), converted by
hprc2annotMakePclaiRefPanel.py -- 3122 haplotypes, 21 populations, 94 KB, one
file for the collection since it is the reference space rather than per-assembly
data. The four values pcaSegment takes across all 460 assemblies turn out to be
the four continental cluster centres, so the highlighted segment dot always
lands on one of them.

genark: addContrib now rewrites a "...Url" inside a detailsScript value the same
way it rewrites bigDataUrl, and symlinks the collection's shared root-level data
files next to the docs, so contrib/<name>/<file> resolves in the deeper GenArk
layout. It writes the alpha tier only, leaving the assembly's default hub alone,
and clears any unmarked copy of the collection's stanzas that the assembly build
baked in, which would otherwise leave the hub declaring each track twice.

refs #35415

diff --git src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt
index bc4b75dd394..2e2eeb9c115 100644
--- src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt
+++ src/hg/makeDb/trackDb/contrib/hprc2annot/hprc2annot.trackDb.txt
@@ -1,112 +1,113 @@
 # trackDb stanzas for the hprc2annot GenArk contributed track collection.
 # HPRC Release 2 annotations on the ~462 HPRC assembly hubs, refs #35415.
 #
 # hprc2annotMakeTrackDb.py reads this file, keeps only the stanzas whose
 # bigDataUrl file is present in a given assembly's hub directory, and writes
 # the per-assembly trackDb.txt. Stanzas are separated by blank lines; comment
 # lines are ignored. Paths here are relative to the assembly's hub directory;
 # genark addContrib rewrites them to contrib/hprc2annot/... when it wires the
 # collection into each assembly's hub.txt.
 #
 # dataVersion values come from the HPRC index CSV file names (see the makeDoc,
 # src/hg/makeDb/doc/contrib/hprc2annot.txt); update them when HPRC reissues a
 # track.
 
 track hprcCatGenes
 shortLabel CAT Genes
 longLabel CAT (Comparative Annotation Toolkit) gene annotations
 type bigGenePred
 bigDataUrl catGenes.bb
 group genes
 priority 10
 visibility pack
 labelFields name,name2
 defaultLabelFields name2
 searchIndex name,name2
 baseColorDefault genomicCodons
 dataVersion HPRC Release 2, CAT genes v1.3
 html ../docs/catGenes
 
 track hprcLiftoffGenes
 shortLabel Liftoff Genes
 longLabel Liftoff RefSeq gene annotations
 type bigGenePred
 bigDataUrl liftoffGenes.bb
 group genes
 priority 11
 visibility dense
 labelFields name,name2
 defaultLabelFields name2
 searchIndex name,name2
 baseColorDefault genomicCodons
 dataVersion HPRC Release 2, Liftoff genes v1.0
 html ../docs/liftoffGenes
 
 track hprcCensat
 shortLabel cenSat
 longLabel Centromere satellite annotation (cenSat)
 type bigBed 9 +
 bigDataUrl censat.bb
 group varRep
 priority 20
 visibility pack
 itemRgb on
 searchIndex name
 dataVersion HPRC Release 2, cenSat v1.0
 html ../docs/censat
 
 track hprcActiveCentromeres
 shortLabel Active Centromere
 longLabel Active centromere regions
 type bigBed 3
 bigDataUrl censatCentromeres.bb
 group map
 priority 21
 visibility dense
 color 220,20,60
 dataVersion HPRC Release 2, cenSat centromeres v1.0
 html ../docs/censatCentromeres
 
 track hprcSegdups
 shortLabel Seg Dups
 longLabel Segmental duplications (SEDEF)
 type bigBed 9 +
 bigDataUrl segdups.bb
 group varRep
 priority 22
 visibility dense
 itemRgb on
 filterValues.original True,False
 filterType.original multipleListOr
 filterLabel.original SEDEF original call
 mouseOver Duplicate of ${partner}<br>Identity: ${pctMatch}%<br>Aligned length: ${alnLen} bp<br>Satellite bases: ${satBases}
 dataVersion HPRC Release 2, SEDEF segdups v1.1
 html ../docs/segdups
 
 track hprcPclai
 shortLabel pcLAI Ancestry
 longLabel Pangenome local ancestry inference (pcLAI)
 type bigBed 9 +
 bigDataUrl pclai.bb
 group compGeno
 priority 30
 visibility pack
 itemRgb on
 mouseOver Window ${window}<br>PCA coordinates (PC1,PC2): ${pca}<br>Segment PCA: ${pcaSegment}<br>Confidence: ${score}
 dataVersion HPRC Release 2, pcLAI v1.1
+detailsScript.scatterPlot.pca {"dataUrl":"../pclaiRefPanel.json","exportFields":["pca","pcaSegment"],"title":"Position in ancestry space","xLabel":"PC1","yLabel":"PC2"}
 html ../docs/pclai
 
 track hprcMethylation
 shortLabel Methylation
 longLabel ONT 5mC CpG methylation
 type bigWig 0 100
 bigDataUrl methylation.bw
 group regulation
 priority 40
 visibility full
 autoScale off
 viewLimits 0:100
 maxHeightPixels 100:40:8
 color 0,0,200
 dataVersion HPRC Release 2, ONT methylation v1.0
 html ../docs/methylation