679b48855a32f8ea1e1ea8ca3c095a0c2c7f6c7a max Tue Sep 8 10:08:41 2026 -0700 detailsScript: add a scatterPlot plot type, and use it for pcLAI Clicking a pcLAI window now shows where that window sits in the ancestry space it was placed in: a scatterplot of the 1000 Genomes reference haplotypes with the window's own PCA coordinate and its segment's coordinate marked on it. The numbers were already on the details page and told a reader almost nothing. New plot type scatterPlot (hg/js/hgc.scatterPlot.js), driven the same way as histogram. Background points come from a JSON or TSV file named by dataUrl and may carry a category, which colors them and builds a legend, and a label, which is shown on mouseover. The cloud is drawn on a canvas, since these files hold thousands of points and that many <circle> elements make the page crawl; axes and the highlighted points stay SVG on top. Point lookup for the mouseover goes through a cell index so a large file stays smooth. Two additions serve every plot type, not just this one: - exportFields, a config key listing further bigBed fields whose values are passed to the module as a fieldValues object. Without it a plot needing two coordinates would need them packed into one field, and pcLAI keeps them in pca and pcaSegment. Only fields that exist in the bigBed are exported, at most 32, and the JSON types are checked rather than asserted because jsonListVal and jsonStringVal errAbort and this JSON is written by a hub. - a config key ending in Url is treated as a file, by the convention trackSettingIsFile() already uses, and a relative one is resolved against the track's own bigDataUrl. The module does not fetch it directly; it asks hgTrackUi for it, the route facetedComposite uses for its metadata. That checks the canonicalized path against the hubs on the cart and reads it with udc, so a hub-relative path works even for a hub loaded from a local path (the GenArk /gbdb hubs), no CORS header is needed, and a file outside a connected hub cannot be read. Verified that /etc/passwd, file://, a dot-dot escape, an unattached hub and an unrelated host are all refused with 400. When the session has file caching off, hgc now exports udcTimeout the way hgTrackUi does and the module POSTs, so the browser cannot answer from cache. Fixes a crash reachable from any hub: "detailsScript.<plotType>.<field> null" segfaulted hgc, because jsonObjectVal returns NULL for a JSON null and the hash routines dereference it. This hit the shipped histogram type too. trackDbSettingsGen.py stopped reading a setting's description at the first "Example:" paragraph and never read <ul> at all, so it dropped everything after the first example and every list item. That silently truncated 226 of the 264 descriptions, including spectrum's minGrayLevel/scoreMin/scoreMax bullets, and would have dropped this whole scatterPlot section. It now skips the Example label instead of stopping, and folds list items in. No setting loses a word and none gains or loses an example. pcLAI wiring: the background file is the authors' published reference panel (github.com/AI-sandbox/hprc-pclai reference_pca_metadata.tsv), converted by hprc2annotMakePclaiRefPanel.py -- 3122 haplotypes, 21 populations, 94 KB, one file for the collection since it is the reference space rather than per-assembly data. The four values pcaSegment takes across all 460 assemblies turn out to be the four continental cluster centres, so the highlighted segment dot always lands on one of them. genark: addContrib now rewrites a "...Url" inside a detailsScript value the same way it rewrites bigDataUrl, and symlinks the collection's shared root-level data files next to the docs, so contrib/<name>/<file> resolves in the deeper GenArk layout. It writes the alpha tier only, leaving the assembly's default hub alone, and clears any unmarked copy of the collection's stanzas that the assembly build baked in, which would otherwise leave the hub declaring each track twice. refs #35415 diff --git src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml index 90543346bd7..cf2046dafa2 100644 --- src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml +++ src/hg/htdocs/goldenPath/help/trackDb/trackDbLibrary.shtml @@ -4688,46 +4688,126 @@ <li><code><fieldName></code> — a bigBed extra field (defined in the autoSql schema) whose value is passed to the JavaScript function. The raw value is suppressed from the HTML table and replaced by the visualization.</li> </ul> <p> The <code><jsonConfig></code> value is a JSON object with configuration for the visualization, such as titles and axis labels. Its keys are merged into the data object passed to the JavaScript function. </p> <p> Multiple fields can be visualized by using multiple <code>detailsScript</code> settings with different field names. Fields sharing the same <code><plotType></code> are grouped together and passed to a single JavaScript function call. </p> <p> - The currently available plot type is <code>histogram</code>, which draws an SVG bar chart - from data encoded as space-separated <code>key=value</code> pairs (logfmt format). - The histogram plot type accepts the following JSON config keys: + Two keys in <code><jsonConfig></code> are understood by the browser itself, for + every plot type: + </p> + <ul> + <li><code>exportFields</code> — a JSON list of other bigBed field names. Their + values are passed to the JavaScript function as well, in a <code>fieldValues</code> + object keyed by field name. This lets one setting drive a visualization that needs + several fields, without the values having to be packed into a single field. Names that + are not fields of this bigBed are ignored, and at most 32 are exported. Unlike the field + the setting names, a field listed here still shows its own row in the details table.</li> + <li>any key whose name ends in <code>Url</code> — treated as a file, in the same way + as the <code>bigDataUrl</code> setting. A relative path is resolved against the track's + own <code>bigDataUrl</code>, so it can be written relative to the data file rather than + as a full URL, and this works whether the hub is loaded over http(s) or from a local + path. The file is read by the server, not fetched by the browser, so the host serving it + needs no <code>Access-Control-Allow-Origin</code> header. It does have to sit inside the + directory tree of a hub attached to the session: the browser requests it through + <code>hgTrackUi</code>, which canonicalizes the path and refuses anything outside a + connected hub, so a file elsewhere on the web or on the server cannot be read this + way.</li> + </ul> + <p> + Two plot types are available. + </p> + <p> + <code>histogram</code> draws an SVG bar chart from data encoded as space-separated + <code>key=value</code> pairs (logfmt format). It accepts the JSON config keys <code>title</code> (chart heading) and <code>xLabel</code> (x-axis label). </p> <P><B>Example:</B></P> <pre> detailsScript.histogram.afrHist {"title":"AFR Allele Frequencies","xLabel":"Allele size (repeat copies)"} detailsScript.histogram.eurHist {"title":"EUR Allele Frequencies","xLabel":"Allele size (repeat copies)"} </pre> <p> In this example, the bigBed fields <code>afrHist</code> and <code>eurHist</code> contain logfmt-encoded histograms such as <code>15=0.025 22=0.016 23=0.747 24=0.211</code>. On the details page, each field's table row is replaced by an SVG bar chart drawn by the <code>hgc.histogram.js</code> module. </p> + <p> + <code>scatterPlot</code> draws a cloud of background points and highlights the position + of the clicked item within it, which suits a track whose items carry coordinates in some + precomputed space. It accepts these JSON config keys: + </p> + <ul> + <li><code>dataUrl</code> — the file of background points (required). It must live + inside the hub, as described above.</li> + <li><code>xLabel</code>, <code>yLabel</code> — axis labels.</li> + <li><code>title</code> — replaces the field description in the left-hand column.</li> + <li><code>exportFields</code> — the fields holding the coordinates to highlight, as + described above. Each is drawn as a large dot in its own color, in the order listed, and + labelled in the legend below the plot. A track with only one point to highlight can leave + this out and put the coordinate in the field the setting names.</li> + </ul> + <p> + Each coordinate field holds one point, written either as <code>(x,y)</code> or as logfmt + <code>x=<num> y=<num></code>; in the logfmt form the keys may carry a suffix, + so <code>x1=0.43 y1=-1.41</code> is also read. + </p> + <p> + The <code>dataUrl</code> file holds the background points, as JSON: + </p> +<pre> +[[x,y], ...] +{"points": [[x,y], ...]} +{"points": [{"x":x, "y":y, "l":"mouseover label", "c":"category"}, ...]} +{"labels": ["category A","category B"], "points": [[x,y,categoryIndex,"mouseover label"], ...]} +</pre> + <p> + or, when the URL ends in <code>.tsv</code> or <code>.txt</code>, as a tab-separated file + with a header line naming an <code>x</code> and a <code>y</code> column, plus optional + <code>l</code> and <code>c</code> columns: + </p> +<pre> +x y l c +0.467 -1.368 HG00096_H1 British from England and Scotland +</pre> + <p> + Points carrying a category (<code>c</code>, or an index into <code>labels</code>) are + colored by it and listed in a legend; without one the cloud is drawn in grey. Points + carrying a label (<code>l</code>) show it on mouseover. The form with a separate + <code>labels</code> list keeps the file small when long category names repeat across + thousands of points. The cloud is drawn on a canvas, so a file of tens of thousands of + points still renders quickly. + </p> + <P><B>Example:</B></P> +<pre> +detailsScript.scatterPlot.pca {"dataUrl":"../pclaiRefPanel.json","exportFields":["pca","pcaSegment"],"title":"Position in ancestry space","xLabel":"PC1","yLabel":"PC2"} +</pre> + <p> + Here the bigBed fields <code>pca</code> and <code>pcaSegment</code> each hold a point such + as <code>(0.433,-1.407)</code>. On the details page the <code>pca</code> row is replaced + by a scatterplot of the reference points in <code>pclaiRefPanel.json</code>, with both of + the item's own coordinates marked on it. + </p> </DIV> <!-- - - - - - - - SNP settings - - - - - - - --> <DIV class="snp_intro"> <H3>snp<em>NNN</em>: specialized subclass of BED 6 for dbSNP variants</H3> <P><I>NOT FOR HUBS. Nor are any of the settings in this section.</I></P> <P>This particular variant of bed 6, identified by table name, is for UCSC's subset of <A HREF="https://www.ncbi.nlm.nih.gov/projects/SNP/" TARGET=_BLANK>dbSNP</A>, NCBI's database of short genetic variants.</P> </DIV> <DIV class="snp"><span class="types snp"></span> <div class="format"><code>type bed 6 + # Track name starts with "snp" followed by the 3-digit dbSNP build number </code></div>