v409

Code changes:

• Fix liftOver error suggested by user Simon Brent (#26758). Galt
• FIx bug in multi-region custom regions, which does not allow bed input with size 0 elements (#26772). Galt
• Bring repo copies of otto scripts up to date (#18151). Christopher
• Make hgGateway use previously browsed position when changing databases, make hgHubConnect assembly redirect to use last used position (#14481). Christopher
• LRG track otto updating for hg19 and hg38 (#17877) (#24672). Christopher
• Fixed snake crashing on self-alignments (#25987). Mark
• Updated snake tracks to use latest HAL with performance improvements. Mark
• Stop hgTracks from bringing in uncommon shared library with HAL/HDF5 which made CGIs less portable. Mark
• Added VCF as a format to chromToUcsc (#26672). Max
• Right click on feature menu didn't url encode the text (#26687). Max
• UniProt tracks now get a bigGenePred (#25192). Max
• Modified exon number code to draw only if there is space for the text (#26628). Brian
• Scale fonts used in axis labels in lollipop graphs. Brian
• Enable the viewing of query sequence on bigPsl hgc pages. Brian
• Add "flush tables" after building trackDb table to address "corrupt table" problems. Brian
• Repair Postscript output with new fonts. Brian
• Keep track of the dates of files included from trackDb.txt so trackDb cache can be flushed. Brian
• Work to enable dots in assembly names in the MAF alignment pipeline. Brian
• Change buildTableDescriptions to only look for AS files in kent/src/hg and subdirectories. Brian
• Stuff that touches on the genome browser -Jim
  • - Added lots of single cell sequencing data as cell-type expression barchart graphs.
  • - Made matrixToBarChartBed to attach an expression matrix to a bed file and make a barChart track
  • - Added simple graph display to details page on barChart tracks that doesn't need full sample-by-sample expression matrix
  • - Added barChartStats tag that if present allows counts of sample in barChartDetails.
  • - Refactoring code so that $D variable in URLs is substituted the same way in hubs and built-in tracks.
• General purpose stuff -Jim
  • - Improving tabToTabDir so it can do counts and sums and aggregation stats.
  • - Made a csvToTsv program that respects all the quoting rules around csv and is fast
  • - Made wordLine work when there are more than 16,000 words/line on input.
• Stuff for single cell group: -Jim
  • - Made gencodeVersionForGenes that'll scan all human gencode gene sets and refSeq to find best matching gene sets to a list of genes that can be either symbols or ENSG type identitifiers
  • - Checked in some old changes to Tuatara into the kent/src repo. Note that active development is on GitHub now so this bit not especially relevant.
  • - Made hcaColorCells which will allow you to label a few samples with colors and have the rest of the samples intrepolate colors based on data similarity with the labeled samples.
• Matrix processing stuff -Jim
  • - Wrote matrixClusterColumns to aggregate together a matrix with columns for each cell into a matrix with average or median values for each distinctly labeled category of cells. Used with matrixToBarCharBed on single cell barCharts
  • - Added vMatrix.h/.c to have a standard way to handle a matrix either in memory or on disk with row-at-a-time access
  • - Added matrixMarket.h/.c to handle matrix market format.
  • - Wrote hcaUnpack5 utility to unpack scanPy's hdf5-based matrix format into a tsv matrix
  • - Wrote matrixRelable to change matrix labels without affecting number in matrix
  • - Wrote matrixTrim to take off labels and remove all-zero rows.
  • - Wrote matrixScale to multiply every number in matrix by a constant
  • - Wrote matrixNormalize to make all columns or rows scale to 1.0 (or any other number you chose)

Data changes:

• Updated to gnomAD PLI and Missense constraint tracks request by gnomAD. Christopher
• Updated Phylogeny: Public (sarsCov2PhyloPub) track (#26804). Angie
• Added B.1.1.7 in USA track (#26753). Angie
• Added Nextstrain otto tracks: support new Nextstrain clades. Angie
• Updated hgPhyloPlace tree (#26774). Angie
• Added CADD track for hg19 and hg38 (#18492). Max
• Added an exome-probesets track for hg19 and hg38 -made by team clinical: Ana, Began, Tiana and Prana (#24598). Max
• Added a new antibody escape supertrack for wuhCor1 (#26806). Max
• Updated HGMD to 2020 (#24625). Max
• RefSeq plants assembly hubs available (#23891). Hiram
  • https://hgdownload.soe.ucsc.edu/hubs/plants/
  • as well as other clades updated:
  • https://hgdownload.soe.ucsc.edu/hubs/
• 35-way conservation track in pushQ for mm39 (#26584). Hiram
• Started on Gencode V36 version of knownGene. Brian
• Completed COVID GWAS V4 (October 2020) track (#26616). Kate
• Completed Spike Mutations track for SARS-Cov-2 browser (#26782). Kate