b85c12cf9af0ee1a954b8cced961bcbd909b7979
lrnassar
  Wed Apr 29 12:04:36 2026 -0700
Expand dbVar tracks to expose all six nstd186 source studies and add new Somatic and Other composites. refs #37406

Restructure the dbVar supertrack:
- Renamed from "dbVar Common Struct Var" to "dbVar Struct Var".
- dbVar Common SV: added subtracks for Lee, Abel, and Byrska-Bishop (the
three nstd186 source studies that were missing from our Curated Common
track), and for the American/East Asian/South Asian/Other populations.
- dbVar Conflict SV: description page refreshed; subtrack longLabel
clarified.
- dbVar Somatic SV (new): single subtrack pulling somatic_sv.bb from the
dbVar hub. Default hidden.
- dbVar Other SV (new): residual bucket for dbVar SVs not classified as
common, somatic, or clinical, split into Healthy and Phenotype subtracks.
Default hidden. NCBI sometimes calls this "presumed normal"; the
description page notes the equivalence. mergeSpannedItems on for the
dense subtracks (normal_healthy ~5.6M items, normal_phenotype ~410K,
somatic_sv ~67K).
- ClinVar SVs are not duplicated; description pages cross-link to the
existing ClinVar track instead.

Description pages: rewrite dbVarCommon.html and dbVarCurated.html, refresh
dbVarConflict.html, add dbVarSomatic.html and dbVarOther.html. Retire the
unused dbVar_common.html. Methods links now point at NCBI's dbVar Overview
rather than the FTP directory listing. searchTable termRegex widened to
^[den]ssv[0-9]+ so dssv* accessions in normal_healthy resolve.

Otto: stage downloads to release/\${db}.new/, validate per file (size
floor and 10% itemCount delta vs the current live copy), then atomically
swap via directory rename with a one-cycle .prev rollback. On validation
failure, leave .new/ in place for human inspection and exit non-zero so
the wrapper emails. On no-op runs the wrapper now stays silent.
checkNstd175.sh's "update done" message moved inside the update branch so
silence is honoured. New-file detection (via a knownFiles.txt manifest)
emails when NCBI adds a file we don't yet expose. knownFiles.txt itself
lives only at the deployment path under /hive/data/outside/otto/dbVar/,
not in the tree.

diff --git src/hg/utils/otto/dbVar/makefile src/hg/utils/otto/dbVar/makefile
index ae8785808c8..6993862baed 100644
--- src/hg/utils/otto/dbVar/makefile
+++ src/hg/utils/otto/dbVar/makefile
@@ -1,5 +1,9 @@
 PREFIX=/hive/data/outside/otto/dbVar/
+# knownFiles.txt is intentionally not installed from the tree -- it's a
+# per-deployment manifest of acknowledged hub filenames that lives under
+# ${PREFIX} alongside lastUpdate and the release/ data, and is edited in
+# place when a new NCBI hub file is approved for exposure.
 install:
 	cp dbVarWrapper.sh checkDbVar.sh checkNstd175.sh giabSv.as processNstd175.py ${PREFIX}
 clean:
 	rm -f *.bed *.bb *.gz