Commits for angie

v476_preview to v476_preview2 (2024-12-16 to 2025-01-06) v476

f3bb2615e7f7d1df3dc6084ab0c5694679a536e7 Wed Dec 18 14:36:01 2024 -0800

• If unable to connect to what looks like it should be a GenArk hub, don't try to write custom tracks (or else we'll get an error trying to connect to a nonexistent db).
  • src/hg/hgPhyloPlace/phyloPlace.c - lines changed 13, context: html, text, full: html, text

fd1b0e6dff69efaea9cb93285b315fae99ff67ff Thu Dec 19 13:38:13 2024 -0800

• Changing hgdownload.gi back to hgdownload.soe, thanks Jonathan for pointing out that .soe can point to hgdownload or hgdownload2. refs #34932
  • src/hg/hgPhyloPlace/phyloPlace.c - lines changed 1, context: html, text, full: html, text

86e5a0344e886b0c19cd767412757d1f1ebacd17 Fri Dec 20 14:49:39 2024 -0800

• Cron-driven script to check Andersen Lab's github repo with assemblies of USDA H5N1 SRA data & reformat for my tree build.
  • src/hg/utils/otto/fluA/updateAndersenLab.sh - lines changed 68, context: html, text, full: html, text

7c4c221a4d2aac593173c70826731ae0aa621aab Fri Dec 20 14:57:16 2024 -0800

• Include serotype and segment in metadata, suppress the progressbar output when downloading.
  • src/hg/utils/otto/fluA/getNcbiFluA.sh - lines changed 4, context: html, text, full: html, text

01859939a397ed998e752ad25c54cdd22742a49c Fri Dec 20 15:00:05 2024 -0800

• Lots of additions for H5N1 outbreak work. * Add Andersen Lab's assemblies of USDA SRA data from 2024 H5N1 outbreak * Add Bloom Lab metadata from Deep Mutational Scanning (DMS) experiments on H5N1 HA (referenced to American Wigeon 2021 vaccine strain) and PB2 * Add build of concatenated segments from the 2024 H5N1 outbreak * Better handling of serotype and segment in INSDC metadata
  • src/hg/utils/otto/fluA/buildConcatTree.sh - lines changed 302, context: html, text, full: html, text
  • src/hg/utils/otto/fluA/buildTree.sh - lines changed 373, context: html, text, full: html, text
  • src/hg/utils/otto/fluA/concat.config.json - lines changed 31, context: html, text, full: html, text
  • src/hg/utils/otto/fluA/dms_annotate_pb2.py - lines changed 200, context: html, text, full: html, text
  • src/hg/utils/otto/fluA/dms_annotate_wigeon.py - lines changed 202, context: html, text, full: html, text
  • src/hg/utils/otto/fluA/h5n1_ha.config.json - lines changed 28, context: html, text, full: html, text
  • src/hg/utils/otto/fluA/joinSegments.py - lines changed 40, context: html, text, full: html, text
  • src/hg/utils/otto/fluA/pb2.config.json - lines changed 15, context: html, text, full: html, text
  • src/hg/utils/otto/fluA/processMetadata.pl - lines changed 148, context: html, text, full: html, text
  • src/hg/utils/otto/fluA/runDms.sh - lines changed 38, context: html, text, full: html, text
  • src/hg/utils/otto/fluA/runDmsPb2.sh - lines changed 39, context: html, text, full: html, text

932c14f1ebdceab4db69a750b8f2246f016c88a6 Fri Dec 20 15:16:25 2024 -0800

• Mask RSV-A (NC_038235.1) pos 5508 because it's noisy and interferes with consortium lineage A.3.
  • src/hg/utils/otto/rsv/mask.A.bb - lines changed 0, context: html, text, full: html, text (a binary file or whitespace-only change or file-permission change shows no diff)
  • src/hg/utils/otto/rsv/mask.A.vcf - lines changed 3, context: html, text, full: html, text

f0c8671e69275687c87a03ab7e28be2694b79dbb Fri Dec 20 15:17:51 2024 -0800

• Tweak lineage mutations to accommodate tree instability.
  • src/hg/utils/otto/rsv/goya.A.clade-mutations.tsv - lines changed 2, context: html, text, full: html, text

9bbe6deba02704509cb2d7314cb13e83a34b4884 Fri Dec 20 15:18:38 2024 -0800

• Exclude sequences that caused weird branch and/or lineage assignment trouble.
  • src/hg/utils/otto/rsv/exclude.ids - lines changed 82, context: html, text, full: html, text

2d6373e8e8e6c30f0ffe140bed6e7d1c21082f46 Fri Dec 20 15:19:55 2024 -0800

• Tweak filtering params, add masking for RSV-A, annotate RSV Genotype Consensus Consortium lineages instead of Ramaekers set.
  • src/hg/utils/otto/rsv/buildTree.sh - lines changed 56, context: html, text, full: html, text
  • src/hg/utils/otto/rsv/mask.A.bed - lines changed 1, context: html, text, full: html, text

5581e1884c54c583fe68844f8271a3baac1617ae Fri Dec 20 15:21:56 2024 -0800

• Accumulated minor updates.
  • src/hg/utils/otto/mpxv/buildTree.sh - lines changed 11, context: html, text, full: html, text
  • src/hg/utils/otto/mpxv/getNcbiMpxv.sh - lines changed 8, context: html, text, full: html, text

265430be8faea69b3c41caa789e9a58a0ac4ebb8 Fri Dec 20 15:25:04 2024 -0800

• Suppress progressbar when downloading, clean up when done.
  • src/hg/utils/otto/dengue/getNcbiDengue.sh - lines changed 3, context: html, text, full: html, text

5cd81d78c5261be06e848e33cf17c76dfc26ae6b Fri Dec 20 15:33:55 2024 -0800

• Add nextclade-assigned lineages to metadata; install hgPhyloPlace files in /gbdb instead of cgi-bin.
  • src/hg/utils/otto/dengue/buildTree.sh - lines changed 27, context: html, text, full: html, text

d48d0f53ee67f93d7193241cd226c8aa039cde42 Fri Dec 20 15:37:20 2024 -0800

• Uniquify to guard against occasional duplicates.
  • src/hg/utils/otto/sarscov2phylo/getCncb.sh - lines changed 5, context: html, text, full: html, text

65c23c156f13dfd1d7acf81390842bc021989cfe Fri Dec 20 15:37:51 2024 -0800

• cleanup typo fix
  • src/hg/utils/otto/sarscov2phylo/getNcbi.sh - lines changed 1, context: html, text, full: html, text

b03342c05eb5d5c23ee2a503ee5ce77e874e725c Fri Dec 20 15:39:24 2024 -0800

• (finally) mask G26530A reversion in BA.1 because it kept splitting BA.1.18. Unmask 21770 in BA.2.86 (alignment boundary change / nextclade / subst of interest).
  • src/hg/utils/otto/sarscov2phylo/branchSpecificMask.yml - lines changed 4, context: html, text, full: html, text

3c3aea0b7973d9497b04647d7c8ed4ebb0f64f5c Fri Dec 20 15:40:01 2024 -0800

• Tweak to run one more command in parallel.
  • src/hg/utils/otto/sarscov2phylo/gisaidFromChunks.sh - lines changed 4, context: html, text, full: html, text

e6d4c01b643aaaf0aea8d0cbc7fbd1ca7cb76232 Fri Dec 20 15:42:16 2024 -0800

• Use /dev/shm as TMPDIR; new compute nodes don't set it and don't have /data/tmp.
  • src/hg/utils/otto/sarscov2phylo/runPangolin.sh - lines changed 2, context: html, text, full: html, text

8c617473dc72f3a1875453a4c238d5c3b7a7cd10 Fri Dec 20 15:49:06 2024 -0800

• Use .pb.gz instead of .pb; install hgPhyloPlace files in /gbdb not cgi-bin.
  • src/hg/utils/otto/sarscov2phylo/extractPublicTree.sh - lines changed 29, context: html, text, full: html, text

3d604489e76b3d834ebed86f8e9b610f2c9c20d8 Fri Dec 20 15:57:02 2024 -0800

• Call makeNewMaskedMaple.sh instead of makeNewMaskedVcf.sh (no more new*fa to clean up at end); use .pb.gz instead of .pb; install hgPhyloPlace files in /gbdb instead of cgi-bin. Make omicron-only subtree; not sure if we'll keep that though, it's still quite large at half the tree.
  • src/hg/utils/otto/sarscov2phylo/makeNewMaskedMaple.sh - lines changed 337, context: html, text, full: html, text
  • src/hg/utils/otto/sarscov2phylo/updateCombinedTree.sh - lines changed 70, context: html, text, full: html, text

a1a0265a5423d4b6433b85e920974f1c5b4c8790 Fri Dec 20 15:57:36 2024 -0800

• Add 24F - 24I.
  • src/hg/utils/otto/sarscov2phylo/nextstrain.clade-mutations.tsv - lines changed 4, context: html, text, full: html, text

33fffa155ef226b353756253fabf9c79198c44df Fri Dec 20 16:01:29 2024 -0800

• Ranges to mask in M. tuberculosis H37Rv (NC_000962.3). I got the file from Ash O'Farrell who got it from https://github.com/iqbal-lab-org/cryptic_tb_callable_mask/blob/master/R00000039_repregions.bed .
  • src/hg/utils/otto/mtb/R00000039_repregions.bed - lines changed 552, context: html, text, full: html, text

61654162481479361c996500c8dcdae120caae95 Fri Dec 20 16:06:08 2024 -0800

• March 2024 build: start with Lily Karim's tree and Ash O'Farrell's metadata, add GenBank sequences, combine metadata.
  • src/hg/utils/otto/mtb/buildTree.sh - lines changed 166, context: html, text, full: html, text
  • src/hg/utils/otto/mtb/combineMetadata.py - lines changed 114, context: html, text, full: html, text
  • src/hg/utils/otto/mtb/getNcbiMtb.sh - lines changed 81, context: html, text, full: html, text
  • src/hg/utils/otto/mtb/transformAshMetadata.py - lines changed 149, context: html, text, full: html, text