Commits for angie

v411_preview2 to v411_base (2021-02-22 to 2021-03-01) v411

7e58340888377874edaad1dbc5174e20295f890c Mon Feb 22 14:17:33 2021 -0800

• Support upload of more sequences, add TSV file summarizing sample variants and placements. Requested by Joe de Risi (UCSF). Increase timeout to 10 minutes; make TSV with each sample's ID, nuc muts, AA muts, imputed bases and path from root to sample. Also use Yatish's new -K subtree algorithm in usher: one subtree encompassing all uploaded samples, plus the specified number of samples randomly selected from the rest of the tree. Don't show every single sample name in the title because there can be 1000 samples in the same subtree now. :)
  • src/hg/hgPhyloPlace/hgPhyloPlace.c - lines changed 18, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/phyloPlace.c - lines changed 189, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/phyloPlace.h - lines changed 26, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/runUsher.c - lines changed 37, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/treeToAuspiceJson.c - lines changed 20, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/writeCustomTracks.c - lines changed 20, context: html, text, full: html, text

ec58e08d65dcad145657e4c4420988ce5c3e4b18 Mon Feb 22 14:22:40 2021 -0800

• Increase bufLen in genomeTxMismatch to accommodate overlapping blocks for ribosomal frameshift as in SARS-CoV-2.
  • src/hg/lib/variantProjector.c - lines changed 6, context: html, text, full: html, text

7c1221c572248594a25af2c26e99c123415efe26 Tue Feb 23 11:38:29 2021 -0800

• The hardcoded QC thresholds for the green/yellow/red coloring in the summary table are out of date. Loosen the defaults a bit to reflect what is now a normal amount of difference from the reference, and add a config file to make them easily adjustable in the future.
  • src/hg/hgPhyloPlace/phyloPlace.c - lines changed 86, context: html, text, full: html, text

a2a68e8ad4ff5927b604b27898c183a519c9e5f4 Tue Feb 23 20:08:34 2021 -0800

• Instead of replacing small subtrees (usher -k) with the new single subtree option (-K), make both (unless a large number of sequences are uploaded; then just -K).
  • src/hg/hgPhyloPlace/phyloPlace.c - lines changed 42, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/phyloPlace.h - lines changed 1, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/runUsher.c - lines changed 57, context: html, text, full: html, text

a9be614b7730455011e5bfcf1f804eecb81f28ec Tue Feb 23 20:09:52 2021 -0800

• Russ request: by default, color by sample type (uploaded vs. existing) and don't show labels (they hide branches and are unreadable when tree is large). Also update Nextstrain clade colors.
  • src/hg/hgPhyloPlace/treeToAuspiceJson.c - lines changed 6, context: html, text, full: html, text

4311905d25fea6367820a25ef089674680cfa37f Wed Feb 24 13:03:14 2021 -0800

• Nextstrain colors in auspice JSON: make uploaded samples stand out, add 20A.EU2.
  • src/hg/hgPhyloPlace/treeToAuspiceJson.c - lines changed 2, context: html, text, full: html, text

5956c40955854fd83b75627124f6420518a78960 Wed Feb 24 20:51:42 2021 -0800

• Add more columns from summary table to TSV summary file. Move PSL indel-tallying code upstream & store results for summaries.
  • src/hg/hgPhyloPlace/phyloPlace.c - lines changed 128, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/phyloPlace.h - lines changed 13, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/vcfFromFasta.c - lines changed 63, context: html, text, full: html, text

acd9854d3c6537ff237c597534abd5b9afbb7a75 Thu Feb 25 10:40:45 2021 -0800

• Adding cron scripts for nightly update of public tree, VCFs, metadata and protobufs for hgPhyloPlace.
  • src/hg/utils/otto/sarscov2phylo/nextcladeCogUk.sh - lines changed 26, context: html, text, full: html, text
  • src/hg/utils/otto/sarscov2phylo/nextcladeNcbi.sh - lines changed 26, context: html, text, full: html, text
  • src/hg/utils/otto/sarscov2phylo/pangolinNcbi.sh - lines changed 16, context: html, text, full: html, text
  • src/hg/utils/otto/sarscov2phylo/updatePublic.sh - lines changed 44, context: html, text, full: html, text
  • src/hg/utils/otto/sarscov2phylo/updatePublicTree.sh - lines changed 329, context: html, text, full: html, text

3ab3206277c766565ec2f5419e3f51f04064212f Thu Feb 25 16:09:26 2021 -0800

• Allow protobuf.tab's description column to contain filenames so we can point to automatically updated latest public files.
  • src/hg/hgPhyloPlace/phyloPlace.c - lines changed 12, context: html, text, full: html, text

0d28b81239a743f286ccc60328d0743a0fa9acbd Thu Feb 25 16:11:21 2021 -0800

• Make public-latest.* links in archive directory and hgPhyloPlaceData.
  • src/hg/utils/otto/sarscov2phylo/updatePublicTree.sh - lines changed 19, context: html, text, full: html, text

6bb6a4d5b9cd9812ed63d2d2e80580099b3bc557 Thu Feb 25 18:59:31 2021 -0800

• Fix bug in skipping unrecognized protobuf repeated field (put else clause in the right place).
  • src/hg/lib/protobuf.c - lines changed 2, context: html, text, full: html, text

17cdc50389b482ee9f3f1ce649ed297201269d53 Thu Feb 25 20:41:01 2021 -0800

• If usher's output includes clade and lineage assignments, parse them and include them in the summary table.
  • src/hg/hgPhyloPlace/phyloPlace.c - lines changed 49, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/phyloPlace.h - lines changed 2, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/runUsher.c - lines changed 32, context: html, text, full: html, text

0b0ce9ae275e9f648f37535a0fe627fccdf6807e Fri Feb 26 14:30:47 2021 -0800

• COG-UK metadata columns changed again.
  • src/hg/utils/otto/sarscov2phylo/getCogUk.sh - lines changed 1, context: html, text, full: html, text

092adf5dfa56fee387d3836b3493ac46ee2c0281 Fri Feb 26 14:48:50 2021 -0800

• Add 'matUtils annotate' calls to add Nextclade and Pangolin assignments to protobuf for hgPhyloPlace.
  • src/hg/utils/otto/sarscov2phylo/updatePublicTree.sh - lines changed 26, context: html, text, full: html, text

8441e4b8e776dea862c62a466f765c8cd94b1e3d Mon Mar 1 00:55:04 2021 -0800

• Add a TSV download file summarizing S (Spike) protein mutations, emphasizing antibody escape and Receptor Binding Domain mutations. Don't show too many buttons at the top. Try a little harder to find metadata for collapsed nodes in subtrees.
  • src/hg/hgPhyloPlace/phyloPlace.c - lines changed 241, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/phyloPlace.h - lines changed 1, context: html, text, full: html, text

439c90e9ae6471a5e575cdc1a35a722ac040c745 Mon Mar 1 00:55:54 2021 -0800

• Don't offer GISAID clade coloring for public trees since we don't have GISAID clade info for them.
  • src/hg/hgPhyloPlace/treeToAuspiceJson.c - lines changed 3, context: html, text, full: html, text

ea4eb88f661eb1d288f61981093d402830496603 Mon Mar 1 01:01:43 2021 -0800

• matToVcf has been replaced by 'matUtils convert'. Also, use 'matUtils annotate's new -l option to clear old clade annotations before reassigning them on the updated protobuf.
  • src/hg/utils/otto/sarscov2phylo/updatePublicTree.sh - lines changed 6, context: html, text, full: html, text