Commits for lrnassar
switch to files view, user index
v502_preview2 to v502_base (2026-08-04 to 2026-08-10) v502
Show details
bc527b6264234c33824854e5b596fb6f790983d4 Tue Aug 4 14:22:28 2026 -0700
- Fix release blockers and data errors found during QA of the mouseDevTimecourse tracks. refs #37001
Add maxWindowToDraw 10000000 to the six bigBarChart subtracks. Without it a
whole-chromosome view asked the track to draw 3445 items x 156 bars and took 71
seconds on mm10 and 69 on mm39. The superTrack is on by default, so any mouse
user zooming out could hit it. Now 254 ms and 86 ms.
Correct a 1-bp off-by-one in the bigBarChart chromStart. The hub builder wrote
1-based GTF gene starts into the 0-based BED chromStart field, so every gene sat
one base right of its true start while chromEnd was correct. Measured against
GENCODE on mm10 before the fix, 42081/42093 genes (VM21) and 35323/35333 (VM4)
were start+1 with none exact. Added fixBarChartStarts.sh, which rebuilds the
files and refuses to run on one that has already been corrected. Originals kept
as *.bb.preStartFix. Reported upstream to the hub author.
Correct the replicate numbers on the bigWig signal composite. The biosample TSV
has no replicate column, so generateBigwigTrackDb.py had been deriving one by
sorting biosample accessions alphabetically, which mislabeled 124 of the 312
subtracks and flipped the default-on state of 62 of them. Added
fetchReplicateNumbers.py to read the real biological_replicate_number from the
ENCODE portal.
Rewrite the signal shortLabels. They had been hard-truncated at 20 characters,
which left 20 subtracks with duplicate labels. The generator now emits a
one-letter view code and errors out if two labels match within the 17 characters
hgTracks draws in the left label area, rather than silently truncating.
Make generateBigwigTrackDb.py reproduce the committed .ra. It now emits the
two-digit tissue prefixes that give the author-requested biological order, and
the html setting, instead of depending on a one-off patch applied afterwards.
Add barChartMerge, barChartMetric, labelFields and defaultLabelFields, and Title
Case all shortLabels.
Update the nine description pages: replicate wording to match barChartMerge,
GitHub source links to the makedoc, build scripts and trackDb in Methods, an
mm39 liftOver accounting note, and remove a duplicated sentence from the shared
Display include.
Add curl -f to downloadBigwigs.sh so an HTTP error body is never saved as a
bigWig and then skipped forever by the restart check.
- src/hg/makeDb/doc/mm10.txt - lines changed 52, context: html, text, full: html, text
- src/hg/makeDb/doc/mm39/mouseDevTimecourse.txt - lines changed 22, context: html, text, full: html, text
- src/hg/makeDb/scripts/mouseDevTimecourse/downloadBigwigs.sh - lines changed 3, context: html, text, full: html, text
- src/hg/makeDb/scripts/mouseDevTimecourse/fetchReplicateNumbers.py - lines changed 90, context: html, text, full: html, text
- src/hg/makeDb/scripts/mouseDevTimecourse/fixBarChartStarts.sh - lines changed 73, context: html, text, full: html, text
- src/hg/makeDb/scripts/mouseDevTimecourse/generateBigwigTrackDb.py - lines changed 121, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/developmentTimecourseM21mm10FPKM.html - lines changed 17, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/developmentTimecourseM21mm10TPM.html - lines changed 17, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/developmentTimecourseM4mm10FPKM.html - lines changed 17, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/developmentTimecourseM4mm10TPM.html - lines changed 17, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/developmentTimecourseSignalMm10.html - lines changed 10, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/developmentTimecourseSignalMm10.ra - lines changed 623, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/mouseDevTimecourse.html - lines changed 20, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/mouseDevTimecourse.ra - lines changed 24, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm39/developmentTimecourseM21mm39FPKM.html - lines changed 26, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm39/developmentTimecourseM21mm39TPM.html - lines changed 25, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm39/mouseDevTimecourse.html - lines changed 25, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm39/mouseDevTimecourse.ra - lines changed 12, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouseDevTimecourseDisplay.shared.html - lines changed 7, context: html, text, full: html, text
383a5877276ee70beeac9ff6602737f59614ee6c Tue Aug 4 14:26:10 2026 -0700
- Fix broken relatedTracks entries and apply the new one-way/bidirectional markers. refs #37390
Fixed three relationships that pointed at tracks that do not exist and so
rendered nothing: 'gnomad' is 'gnomadVariants' on hg38 and 'gnomadSuper' on
hg19, and 'liftHg38' is 'liftHg19' on hg38. Also corrected the REVEL lift
reason (chains are from hg19, not hg38) and the hg19 GRC Incident reason
(alignments are between hg19 and hg38, not hg38 and hg38).
printRelatedTracks renders the reason after the linked track's short label,
so 52 of 146 entries were stuttering, e.g. "CADD 1.6: CADD, a similar
deleteriousness score" and "NMD Escape: NMD Escape: predicted regions...".
Dropped the redundant leading track name from every reason and documented
the rendering in the file header so it does not creep back in.
Converted to one-way '>' entries: avada to varaico on hg38 and hg19, nmd to
mane and ncbiRefSeq on hg38, revel to liftHg19 on hg38, and pseudoYale60 to
refSeqComposite on hg19. In each case the reverse link only added clutter to
a heavily used track page or pointed users back at a retired track.
Converted to bidirectional '~' entries where both directions carry the same
reason: primateAi/alphaMissense and wgEncodeReg4/cCREs on hg38, and
encode4Reg/cCREs plus both mouseDevTimecourse pairs on mm10.
Standardized the description of the strVar container, which was described
two different ways in the two entries linking to it, and neither matched
its actual contents.
- src/hg/makeDb/trackDb/relatedTracks.ra - lines changed 101, context: html, text, full: html, text
b5ec4fbbfb04f97ebe23fbf6fbdf091dc120944e Tue Aug 4 14:47:26 2026 -0700
- Remove barChartMerge from the mouseDevTimecourse tracks, it washes out the bar colors. refs #37001
barChartMerge fades a merged group's color toward white when the group has fewer
than 100 members. scaleColorPart() in facetedTable.c blends with maxVal 240 by
(threshold - count)/threshold, with threshold 100. That is built for the
Cell Browser single-cell tracks, where the facet count is a cell count in the
hundreds or thousands.
This track is bulk RNA-seq and its .facets count is 1 per sample, so merging by
tissue and time point gives a group count of 2. The blend is then 0.02*color +
0.98*240, which lands every channel at roughly 235-240: thymus #C8B400 rendered
as #EFEFEB. Measured on a rendered image at chr12:56694976-56714605, the track
went from 40 saturated colors to 0.
Reverting the setting restores the colors on both assemblies, verified against
the .facets palette. Also reverts the description page wording back to one bar
per replicate, and drops "grouped and" from the shared Display include since
grouping was only available through this setting.
The rest of the QA changes are unaffected: maxWindowToDraw, barChartMetric,
labelFields and defaultLabelFields are still in place, whole-chromosome renders
are still 245 ms, and the details page reports 156 of 156 bars again.
- src/hg/makeDb/trackDb/mouse/mm10/developmentTimecourseM21mm10FPKM.html - lines changed 7, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/developmentTimecourseM21mm10TPM.html - lines changed 8, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/developmentTimecourseM4mm10FPKM.html - lines changed 7, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/developmentTimecourseM4mm10TPM.html - lines changed 8, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/mouseDevTimecourse.html - lines changed 5, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm10/mouseDevTimecourse.ra - lines changed 4, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm39/developmentTimecourseM21mm39FPKM.html - lines changed 8, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm39/developmentTimecourseM21mm39TPM.html - lines changed 7, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm39/mouseDevTimecourse.html - lines changed 5, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouse/mm39/mouseDevTimecourse.ra - lines changed 2, context: html, text, full: html, text
- src/hg/makeDb/trackDb/mouseDevTimecourseDisplay.shared.html - lines changed 1, context: html, text, full: html, text
1795df5f669cdeb63144810c141ac41128e93111 Wed Aug 5 10:16:41 2026 -0700
- Split the daily code review per commit when a batch times out, and stop dropping FEEDBACK reviews. refs #38020
A batch that ran out of time was retried on the same budget, so it failed the same
way and the whole author's review was lost with no second window. Now the timeout
scales with the batch, a timeout gets one retry at a larger budget instead of a
repeat of the one that just failed, and a batch that still times out is reviewed
one commit at a time and stitched into a single email. Commits the split cannot
reach are named in the digest as unreviewed rather than going missing. The
per-ticket code review path, which was still on a flat 600s, gets the same scaling.
Separately, and the more serious of the two: the decision to email an author was an
exact case-sensitive match on the digest's status line, so a status written as
"Overall Status: FEEDBACK" or with markdown around it was recorded as approved,
never emailed and never flagged. That decision now reads every status line, sends if
any says FEEDBACK, sends when the status cannot be read, and keeps the old exact
match as a backstop, so it can only ever send more mail than before and never less.
Per-commit verdicts get the same treatment.
Diagnostics, because a failed review used to leave nothing to look at: every review
logs its duration alongside commits, files, lines and binary-file count so the real
cost predictor can be identified from data rather than guessed; discarded review text
is logged with a count and preview; and failed reviews keep their working files
instead of having them swept. Capturing the partial output of a timed-out run was
tried and does not work - the CLI buffers and emits nothing before it is killed - so
there is a note on the stream-json approach that would.
Also: the maintainer alert distinguishes broken, incomplete and never-started runs
instead of blaming an expired login for all three; an email that fails to send now
reaches the maintainer rather than the log; the run has an overall deadline so a bad
night cannot still be going when the next cron fires; and the prompts tell the model
not to read images, which is the likely cause of the one-commit timeout that started
this.
- src/utils/codeReviewAi.py - lines changed 973, context: html, text, full: html, text
2070aedea88da26aee0af3e2fd867b67140b09b5 Wed Aug 5 16:59:19 2026 -0700
- Move the maf sequence-name splitter into jkweb so both summary tools share one copy. refs #37928
mafSplitSrcGetChrom existed as two independent copies, in hgLoadMafSummary.c and
mafToBigMafSummary.c, and they had diverged: only the first had the GenArk accession
rule added by 2db6bab8db0. It now lives in src/lib/maf.c, with the rules documented
in maf.h.
mafToBigMafSummary therefore picks up the GenArk rule it was missing, so a dotted
GenArk name like GCA_009914755.4.CM034974.1 now keeps its accession version instead of
dropping it, which matches what hprc90waySummary already holds.
Also removes mafCompGetSrcDb and mafCompGetSrcName from maf.c and maf.h. Both were
first-dot splitters with no callers anywhere in the tree.
- src/hg/makeDb/hgLoadMaf/hgLoadMafSummary.c - lines changed 67, context: html, text, full: html, text
- src/hg/utils/mafToBigMafSummary/mafToBigMafSummary.c - lines changed 33, context: html, text, full: html, text
a8b15f82bbcff0d7953bae0da7df49363939f85a Wed Aug 5 16:59:54 2026 -0700
- Add tests for every maf sequence-name form, and describe the splitting in the usage messages. refs #37928
The existing testDot and testPipe inputs are identical copies of each other and both use
pipe-form query names, so the dot path was never exercised in either tool. That is how
the break in ae63ce5 shipped even though a test suite existed, and why the suite could
not have caught the mafToBigMafSummary divergence either.
New inputs in both tests directories cover an ordinary db.chrom, a species name with a
dotted chrom, both GenArk accession forms, a GenArk accession with no chrom after the
version, a pipe with a dotted chrom, a name with no separator, a reference assembly
whose own name contains a dot, and a name where the reference db is a strict prefix of
another assembly. Every rule in the splitter is now load-bearing: breaking any one of
them fails a target.
Also adds mafToBigMafSummary to TEST_DIRS in hg/utils, since nothing above the directory
was running its suite, and adds a short note to both usage messages describing how a
sequence name is split into assembly and sequence.
- src/hg/makeDb/hgLoadMaf/hgLoadMafSummary.c - lines changed 4, context: html, text, full: html, text
- src/hg/makeDb/hgLoadMaf/tests/expected/testDottedRef.tab - lines changed 2, context: html, text, full: html, text
- src/hg/makeDb/hgLoadMaf/tests/expected/testGenArkRef.tab - lines changed 3, context: html, text, full: html, text
- src/hg/makeDb/hgLoadMaf/tests/expected/testNameForms.tab - lines changed 8, context: html, text, full: html, text
- src/hg/makeDb/hgLoadMaf/tests/input/testDottedRef.maf - lines changed 6, context: html, text, full: html, text
- src/hg/makeDb/hgLoadMaf/tests/input/testGenArkRef.maf - lines changed 7, context: html, text, full: html, text
- src/hg/makeDb/hgLoadMaf/tests/input/testNameForms.maf - lines changed 12, context: html, text, full: html, text
- src/hg/makeDb/hgLoadMaf/tests/makefile - lines changed 18, context: html, text, full: html, text
- src/hg/utils/mafToBigMafSummary/mafToBigMafSummary.c - lines changed 4, context: html, text, full: html, text
- src/hg/utils/mafToBigMafSummary/tests/expected/testDottedRef.bed - lines changed 2, context: html, text, full: html, text
- src/hg/utils/mafToBigMafSummary/tests/expected/testGenArkRef.bed - lines changed 3, context: html, text, full: html, text
- src/hg/utils/mafToBigMafSummary/tests/expected/testNameForms.bed - lines changed 8, context: html, text, full: html, text
- src/hg/utils/mafToBigMafSummary/tests/input/testDottedRef.maf - lines changed 6, context: html, text, full: html, text
- src/hg/utils/mafToBigMafSummary/tests/input/testGenArkRef.maf - lines changed 7, context: html, text, full: html, text
- src/hg/utils/mafToBigMafSummary/tests/input/testNameForms.maf - lines changed 12, context: html, text, full: html, text
- src/hg/utils/mafToBigMafSummary/tests/makefile - lines changed 18, context: html, text, full: html, text
1de43b26fd18d5be3585829158d9b65944979094 Wed Aug 5 17:19:10 2026 -0700
- Harden popEVE build drivers per code review. refs #37950 refs #37791
- Add "set -o pipefail" to both build scripts so a bedToBigBed failure piped to tail is not
masked (runBuildDense.sh reported the previous file's size on failure).
- runBuild.sh (sparse) now writes popEve_sparse.bb instead of popEve.bb, so re-running it
cannot silently clobber the dense popEve.bb produced by runBuildDense.sh. Makedoc updated.
- src/hg/makeDb/doc/hg38/popEve.txt - lines changed 3, context: html, text, full: html, text
- src/hg/makeDb/scripts/popEve/runBuild.sh - lines changed 5, context: html, text, full: html, text
- src/hg/makeDb/scripts/popEve/runBuildDense.sh - lines changed 1, context: html, text, full: html, text
5c6366432eadc33cfdc7c48c1308bce156479905 Wed Aug 5 17:24:23 2026 -0700
- Harden error handling in trackCountsParse per CR feedback. refs #37975
The report pipeline had no pipefail, so it reported awk's exit status rather than
makeUsageReport's. awk is happy with truncated input, so a report that died partway
through produced a non-empty partial count file with a success status, and the v2
cache then reused that partial file on every later run. The empty-file check only
caught a total failure. Adds pipefail, and removes the partial file when the command
fails so a failed run cannot leave something behind for the cache to pick up.
The mergeStderr=False path added in the last commit dropped stderr from the error
message, since CalledProcessError reports stdout in e.output. That is the one call
site using the flag, the bulk tdbQuery, so a failure there lost its reason entirely.
The RuntimeError now carries stderr too.
Also aborts when tdbQuery returns no trackDb entries at all. tdbQuery exits 0 and
prints nothing for a database it does not know, so a mistyped db name used to run to
completion treating every track as notInTrackDb before failing with a message that
blamed the usage logs. Pins the subshell to bash, which pipefail needs and which the
existing sort -t $'\t' already quietly relied on.
- src/utils/qa/trackCountsParse - lines changed 26, context: html, text, full: html, text
c2087cb0ed189d946439a090d43d2dba5d71a9eb Wed Aug 5 17:28:22 2026 -0700
- Fix error containment and a wasted retry sleep in the VCEP version notifier per CR feedback. refs #37795
ClinGen's version field is a free-form string, so an unparseable value raised
ValueError past the per-VCEP except RuntimeError in checkVcep. That aborted the
whole run, skipping the remaining VCEPs and mailing a traceback instead of the
report. normalizeVersion now reports the offending string as a RuntimeError, and
the version comparison moved inside the try so it is caught, keeping the damage
to a single failure line.
fetchUrl also slept 30 seconds after its final failed attempt, which could not
help. It now only sleeps between attempts.
- src/hg/utils/otto/vcepVersions/checkVcepVersions.py - lines changed 15, context: html, text, full: html, text
switch to files view, user index