Commits for angie

v437_preview to v437_preview2 (2022-09-05 to 2022-09-12) v437

fb0c581ba94fd1ca33d57e3253f732cd4c4ea62a Fri Aug 26 10:51:31 2022 -0700

• Support Auspice JSON output for monkeypox: url settings not just files, Nextclade_lineage, non-hardcoded Auspice config header, correct amino acid changes on - strand genes.
  • src/hg/hgPhyloPlace/phyloPlace.c - lines changed 7, context: html, text, full: html, text
  • src/hg/hgPhyloPlace/treeToAuspiceJson.c - lines changed 328, context: html, text, full: html, text

813fe4b4d5b0b37734743735a0e43fe96d9a5674 Fri Aug 26 11:12:22 2022 -0700

• Add -ldl to L for CONDA_BUILD; not sure why it wasn't necessary ~v417 but was ~v426.
  • src/inc/common.mk - lines changed 1, context: html, text, full: html, text

3fe189a61a18660027dddf81d724f95fd09b56c3 Fri Aug 26 11:14:37 2022 -0700

• Fix bug in dynamic resizing (forgot to update db->size); required to support larger-than-SARS* genomes like monkeypox.
  • src/utils/vcfFilter/vcfFilter.c - lines changed 5, context: html, text, full: html, text

6701b09b911c195032edf2090f2363fec47b9464 Fri Aug 26 11:16:13 2022 -0700

• Modernize makefile of obscure program than an MLQ user actually wanted to use. refs #29758
  • src/oneShot/preProcGgfAndy/makefile - lines changed 12, context: html, text, full: html, text

e1df6d7bb91a0f8027129677be73ad720d79449d Fri Aug 26 11:38:42 2022 -0700

• Update nextstrain URL and use curl --compressed (thanks @tsibley).
  • src/hg/utils/otto/nextstrainNcov/doUpdate.sh - lines changed 4, context: html, text, full: html, text

ee396acb1919923fb78c5b91b6b17789efba3ca3 Fri Aug 26 11:40:49 2022 -0700

• More regex-cleaning of outlandish things that people put in GenBank titles.
  • src/hg/utils/otto/sarscov2phylo/util.sh - lines changed 1, context: html, text, full: html, text

428e9023e880b56314b2a1b9725415f269f1e54a Fri Aug 26 11:42:45 2022 -0700

• Use minimap2 -x asm20 --score-N=0 as pangolin does, to ensure consistent alignment to reference before masking & hashing.
  • src/hg/utils/otto/sarscov2phylo/pangoHash.sh - lines changed 1, context: html, text, full: html, text

8a78f2f53d1ee8640a4f326765646466b5dcfa5b Fri Aug 26 11:43:56 2022 -0700

• Updating usage for latest version of nextclade.
  • src/hg/utils/otto/sarscov2phylo/runNextclade.sh - lines changed 9, context: html, text, full: html, text

4a51b1bbc3d9b23dd5a36e4712250ec89f009c2c Fri Aug 26 11:44:17 2022 -0700

• Updating usage for pangolin v4.
  • src/hg/utils/otto/sarscov2phylo/runPangolin.sh - lines changed 5, context: html, text, full: html, text

fda84c0982a6c223bde5a3c295890b8ca6189e05 Fri Aug 26 11:45:28 2022 -0700

• When maintaining consistent mappings of duplicated items, make sure the names really match.
  • src/hg/utils/otto/sarscov2phylo/updateIdMapping.sh - lines changed 9, context: html, text, full: html, text

6e75f968aeec30536982821ebc0779760af6db3d Fri Aug 26 11:47:28 2022 -0700

• Dropped to 28 jobs instead of 32 because we have some dead cluster nodes. This is no longer used -- @yceh's usher-sampled is way faster -- just committing changes for the record.
  • src/hg/utils/otto/sarscov2phylo/usherClusterRun.sh - lines changed 12, context: html, text, full: html, text

879c1d08bc8edc582bf06681567834590824ad4c Fri Aug 26 11:51:10 2022 -0700

• Update branch length filter, annotated clades & lineages using previous day's paths for speed, add Taxonium v2 output, add MSA of public sequences.
  • src/hg/utils/otto/sarscov2phylo/extractPublicTree.sh - lines changed 61, context: html, text, full: html, text

d01d503c098b25411a732c2190fde29911293822 Fri Aug 26 11:51:59 2022 -0700

• Update nextclade column offsets.
  • src/hg/utils/otto/sarscov2phylo/findDropoutContam.pl - lines changed 6, context: html, text, full: html, text

1af2f775e2a9c148f0cbff583910f1d68faf2598 Fri Aug 26 11:54:35 2022 -0700

• Do a little more cleanup and always construct name from components; fasta names are not always updated
  • src/hg/utils/otto/sarscov2phylo/fixNcbiFastaNames.pl - lines changed 39, context: html, text, full: html, text

fa70926eb39b233849ef2a1f0e36f4ed264dc832 Fri Aug 26 11:55:24 2022 -0700

• Bump up maxMissing, sometimes there are just a ton.
  • src/hg/utils/otto/sarscov2phylo/gbMetadataAddBioSample.pl - lines changed 3, context: html, text, full: html, text

e4eb9b464422dc81af82a711fd6470174ed3fb92 Fri Aug 26 11:56:30 2022 -0700

• Add retry for all the URLs. Update to latest nextclade version.
  • src/hg/utils/otto/sarscov2phylo/getCogUk.sh - lines changed 29, context: html, text, full: html, text

72afe80e5fc9ade5df012cfb159e39d5ee953d92 Fri Aug 26 11:59:25 2022 -0700

• Add --debug to datasets command to assist in debugging failures. Patch some corrupted associations between nucleotide IDs and BioSample IDs until the fixes percolate through all the databases. Update to latest versions of nextclade and pangolin.
  • src/hg/utils/otto/sarscov2phylo/getNcbi.sh - lines changed 28, context: html, text, full: html, text

84e8a21220c712339da59a4c53ed481162c64156 Fri Aug 26 12:00:07 2022 -0700

• Add IDs for a bunch of new recombinants.
  • src/hg/utils/otto/sarscov2phylo/includeRecombinants.tsv - lines changed 547, context: html, text, full: html, text

02e715d7eb789c5a3fc5399dc6138e94f4fdaa85 Fri Aug 26 12:02:40 2022 -0700

• Add a hook for a prevDate.masked.useMe.pb file so I can do some major pruning, and let sequences be added back the next day without worrying that I've removed too much from the version used by hgPhyloPlace. Update column indices for latest version of nextclade.
  • src/hg/utils/otto/sarscov2phylo/makeNewMaskedVcf.sh - lines changed 9, context: html, text, full: html, text

b798ee59931597499ea8fe9d4b18e7c2ca26b235 Fri Aug 26 12:04:40 2022 -0700

• Many accumulated tweaks, corrections, for ongoing Omicron lineages & sequencing/assembly errors.
  • src/hg/utils/otto/sarscov2phylo/maskDelta.sh - lines changed 33, context: html, text, full: html, text

6cbc26d41876505e15a244a476f70d9314adc08d Fri Aug 26 12:06:28 2022 -0700

• Add latest Nextstrain clades for Omicrons of concern: 22A=BA.4, 22B=BA.5, 22C=BA.2.12.1, 22D=BA.2.75.
  • src/hg/utils/otto/sarscov2phylo/nextstrain.clade-mutations.tsv - lines changed 4, context: html, text, full: html, text

f0ab7f27d98fd3f3f0506a5da4e1f533b827cff4 Fri Aug 26 12:09:20 2022 -0700

• Add some masked sites to BA.2.10.4 path as new sequences make the lineage root wiggle around a bit. Add new lineages BL.1, BA.2.75.3-5, BA.4.8, BA.5.1.12-14, BF.15-16, BA.5.2.6-7, XAS, XAT, XAU, XAV, renaming proposed* lineages that were designated.
  • src/hg/utils/otto/sarscov2phylo/pango.clade-mutations.tsv - lines changed 17, context: html, text, full: html, text

1c62f40bec46afc1a9ad8ee35909008c47d57713 Fri Aug 26 12:20:28 2022 -0700

• Use @yceh's usher-sampled and updated matOptimize instead of running usher on the cluster. Prune before optimization in addition to afterwards in order to avoid some weird moves caused by low-quality long-branch sequences. Annotate clades & lineages using previous day's paths for speed. Add Taxonium v2 output.
  • src/hg/utils/otto/sarscov2phylo/updateCombinedTree.sh - lines changed 55, context: html, text, full: html, text

806ed36b35f9db392dadd975839426718339304d Fri Sep 2 14:05:50 2022 -0700

• Oops, removing the duplicate declaration for trackHubGetGenomeUndecorated found by Chris in code review. refs #29936
  • src/hg/inc/trackHub.h - lines changed 3, context: html, text, full: html, text

8d6ff9f56618bcc7ddfed8dfb653127f9aec1912 Tue Sep 6 13:11:49 2022 -0700

• Restoring some dyStringPrintfs that were inadvertently changed to sqlDyStringPrintf but are not for sql. refs #29973
  • src/hg/hgVai/hgVai.c - lines changed 5, context: html, text, full: html, text