d07356d76401059fbe6c2d0890b1490546a62a4e
max
  Mon Sep 14 02:54:50 2026 -0700
hg38 Fiber-seq: reissued GM12878 data and a nucleosome density track, refs #36210

The lab reprocessed GM12878 (PM00001) and replaced the files in place under the
same hash directory.  Ten of its twelve files changed; a size sweep over all 41
samples confirmed no other sample is affected.  This fixes the two placeholder
haplotype accessibility bigWigs that covered a single base, so that overlay now
draws real data for the sample that comes up by default.  Its peak calls changed
substantially as well, 429,883 source peaks before and 196,742 now, which is
noted on the description page since figures made from the first version of the
track will not reproduce for GM12878.

The downloader now fetches into <file>.part and moves it into place when
complete.  It used curl -C - straight onto the final file, which is right for an
interrupted transfer and silently corrupting when the server has replaced the
file: it would have appended the tail of the new 5.2 GB hap1 file to the
512-byte stub, and the size check afterwards would have passed.  It also takes
an optional list of accessions now, to refresh one sample without walking all 41.

Nucleosome density (all.nucleosome.coverage.bw) was sent separately and is not
in the lab's own hub.  It is on the server for all 41 samples and is added as a
seventh data type in the compendium.  Unlike every other wiggle here it is a
read depth rather than a percentage, so it cannot take fixed viewLimits: the
genome-wide mean runs from 25 to 142 across samples with sequencing depth and
single loci reach 1.7e5.  It is drawn with autoScale, which the description page
explains, and reads as the complement of the accessibility signal.

diff --git src/hg/makeDb/scripts/fiberSeq/fiberSeqCheck.sh src/hg/makeDb/scripts/fiberSeq/fiberSeqCheck.sh
index e215444d4a9..47b44dec363 100755
--- src/hg/makeDb/scripts/fiberSeq/fiberSeqCheck.sh
+++ src/hg/makeDb/scripts/fiberSeq/fiberSeqCheck.sh
@@ -1,72 +1,73 @@
 #!/bin/bash
 # Verify the mirrored Fiber-seq / CpG files: every expected file present, every
 # one readable as a bigWig or bigBed, and report any that hold no data.  The
 # empty-file report matters: at mirror time PM00001 (GM12878)
 # hap1/hap2.percent.accessible.bw were 512-byte stubs on the server, i.e. valid
 # bigWigs with zero bases covered, so a size check alone does not catch them.
 #
 # Usage: fiberSeqCheck.sh <outDir>
 
 set -o pipefail
 
 outDir=$1
 if [ -z "$outDir" ]; then
     echo "usage: $0 <outDir>" >&2
     exit 1
 fi
 
 scriptDir=$(dirname "$(readlink -f "$0")")
 sampleList=$scriptDir/fiberSeqSamples.tsv
 
 bigWigs="all.percent.accessible.bw
 hap1.percent.accessible.bw
 hap2.percent.accessible.bw
+all.nucleosome.coverage.bw
 cpg.combined.bw
 cpg.hap1.bw
 cpg.hap2.bw
 cpg.diffs_all.bw
 cpg.diffs_p0.01.bw
 cpg.diffs_p0.001.bw
 cpg.diffs_p0.0001.bw"
 bigBeds="fire-peaks.bb
 fire-peaks.ucsc.bb"
 
 missing=0
 broken=0
 empty=0
 
 grep -v '^#' "$sampleList" | while IFS=$'\t' read -r acc sample cellType hash; do
     [ -z "$acc" ] && continue
     for f in $bigWigs; do
         p=$outDir/$acc/$f
         if [ ! -s "$p" ]; then
             echo "MISSING $acc/$f"
             continue
         fi
         info=$(bigWigInfo "$p" 2>&1) || { echo "BROKEN $acc/$f"; continue; }
         bases=$(echo "$info" | awk -F': ' '$1=="basesCovered"{gsub(",","",$2); print $2}')
         # A placeholder bigWig is not necessarily basesCovered 0: the two bad
         # files on the server cover exactly one base with a value of zero.  The
         # smallest legitimate file here covers 4.1 million bases, so anything
         # under a thousand is a stub, not thin coverage.
         if [ "${bases:-0}" -lt 1000 ]; then
             echo "EMPTY $acc/$f (basesCovered ${bases:-0})"
         fi
     done
     for f in $bigBeds; do
         p=$outDir/$acc/$f
         if [ ! -s "$p" ]; then
             echo "MISSING $acc/$f"
             continue
         fi
         info=$(bigBedInfo "$p" 2>&1) || { echo "BROKEN $acc/$f"; continue; }
         n=$(echo "$info" | awk -F': ' '$1=="itemCount"{gsub(",","",$2); print $2}')
         if [ "${n:-0}" = "0" ]; then
             echo "EMPTY $acc/$f (itemCount 0)"
         fi
     done
 done | tee "$outDir/fiberSeqCheck.log"
 
 echo
 echo "problems logged to $outDir/fiberSeqCheck.log"
 echo "no lines above means all $(grep -vc '^#' "$sampleList") samples are complete and non-empty"