v499

Code changes:

• Convert the bundled HTSlib to a git submodule (src/submodules/htslib) and route remote BAM/VCF/tabix access through HTSlib's hfile scheme-handler mechanism backed by UDC, replacing the legacy knetUdc/knetfile hooks: knetUdcInstall remains as a thin compatibility stub; hfile_add_scheme_handler init order is fixed so kent registers UDC handlers before any hopen; remote .bai/.tbi/.crai index reads now go through UDC instead of being downloaded into the current directory; tabix VCF tests previously gated by a non-existent USE_TABIX make variable are re-enabled; fetchKentSource gains initial submodule-aware checkout; and the outdated UCSC-bundled static libcurl is dropped in favor of system libcurl. Mark
• Upgrade session and cookie IDs from 32-bit to 64-bit so the browser keeps working after 4 billion users are served, including robot scripts (#37718). Jonathan
• Prepare the myVariants feature for a test/v1 release: fix dialog box sizing and other small UI changes, allow tracks to be renamed, and make each row a type (snv, cnv, or transcript) (#33808). Chris
• Ongoing development of the lift-over user request system (#31811). Hiram
• hubSpace: allow UI uploads to accept a hub.txt + genomes.txt + trackDb.txt assembly hub, not just a useOneFile-style hub (#37566). Chris
• New hubtools command-line tool: convert trackDb format to TSV, convert TSV to trackDb format, and create bare-bones hub.txt files from a directory of big* files. Max
• Get the hubtools upload command working now that hubSpace is more stable (#36790). Chris
• Add a BLAT API call (#36315). Max
• hgPhyloPlace: draw a RIVET-like recombination diagram in a pop-up. Angie
• Reduce bot abuse by limiting how many IP addresses can present the same hguid in a short timeframe before requiring a captcha re-up, when captcha is enabled (#37494). Jonathan
• Avoid a crash at exit when a CGI opened a database connection before carefulMemHandler was set up; sqlCleanupAll now lets the system reclaim that memory instead of freeing a mix of allocation types by hand (#37494). Jonathan
• Add a way to toggle the trackDb cache per hgTracks invocation without editing hg.conf, and a harness to compare cached and uncached render times (#37551). Braney
• Fix hgTracks out-of-memory (11+ GB) and timeout when rendering a window with quickLifted bigBed-style tracks (#37536). Braney
• Add a way to control the display order of filter and highlight controls on the track configuration page (#29223). Braney
• Fix hgConvert quickLift re-runs: stop re-lifting tracks that already came from a quickLift hub, append to the existing per-source hub file instead of overwriting it, and add a way to remove an individual track from a quickLift hub (#37535). Braney
• Fix a quickLifted UCSC RefSeq item detail page where the CCDS link errored against the hub-virtual db, and Get Genomic Sequence Near Gene pulled sequence from the wrong assembly (#36125). Braney
• Fix a quickLifted bigBed mouseover tooltip that reported the pre-lift source position while the position bar showed the lifted target position, with nothing flagging the discrepancy (#37615). Braney
• Fix a lolly subtrack item that failed to lift to the target assembly, reusing stale memory, corrupting the subtrack's trackDb pointers, and later surfacing as "Can't find tdb for subtrack ... was getSubtrackTdb called on non-subtrack?" (#37646). Braney
• Fix a stale lastPosition left behind by a quickLift or hgGateway hop to a new assembly, which triggered a spurious "Unable to resolve lastPosition" banner on the destination assembly (#37562). Braney
• Fix a genePred crash when flipping strands on records with no exon frames (#36370). Braney
• Fix the OMIM Genes (omimGene2) detail page failing with a MySQL "Commands out of sync" error (#37565). Braney
• Free a leaked database connection on the OMIM Genes (omimGene2) detail page (#37571). Braney
• Fix faceted composite UI supporting-file fetch that broke when a broken hub was attached, by switching to hubFromIdNoAbort (#36320). Jonathan
• Fix advanced track search when a select and its parent td share a name, causing track search and file search options to show false 0 counts (#37556). Jonathan
• Column titles in the faceted composite UI now respect the capitalization used in the metadata. Jonathan
• Tooltips: fix another spot that wasn't using the data-tooltip attribute; tooltips should now use it everywhere (#37528). Chris
• Establish a convention that /gbdb file or directory names beginning with an underscore are never synced to hgDownload, reducing admin emails. Max
• Change the hgTrackUi link at the top of the page to "Configure track container" to make it easier to find. Max
• Correct the browser system's bounce email address so bounced user emails are now received. Hiram
• Enlarge several fixed-size string buffers in web.c that were too small at 128 bytes. Braney
• Add an automated lifecycle for the tip, beta, and rel Docker QA browser instances on hgwdev and wire them into the weekly autoBuild (#37655). Braney
• redmineCli: add support for posting comments, updates, and attachments as Redmine private notes (#37281). Braney

Data changes:

• New VGP 577-way alignment track. Hiram
• New mouse developmental time course gene-expression track (ENCODE Expression) on mm10 and mm39 from the ENCODE Consortium's Wold Lab at Caltech: TPM and FPKM bigBarChart subtracks across 17 tissues at up to 8 time points (embryonic days 10.5 through 16.5 and postnatal day 0), two replicates per condition. mm10 has four subtracks (GENCODE M4 and M21, TPM and FPKM); mm39 has two M21 lift-over subtracks (TPM and FPKM). Bars are colored by tissue, with a gradient that darkens at later time points (#37001). Gerardo
• New EVA SNP Release 9 for 35+ native assemblies and over 100 GenArks (#37517). Lou
• New mobile element insertion site track collection with additional subtracks (#37524). Max
• New EVE predictions track under the deleteriousness predictions on hg38 (#31804). Max
• New TREDNet enhancer predictions track from the NHGRI Ovcharenko/Collins group. Max
• New UMD TP53 variant track on hg19 and hg38 under a new Locus-Specific track collection (#37648). Lou
• Added NMDetective-AI tracks to the NMD Escape track collection for hg38 (#33737). Lou
• Added PMS2CL Paralog Variants and Pseudogene Caution tracks to the InSiGHT public hub and recommended track set on hg19 and hg38 (#36582). Lou
• Finished the MPRA track collection (#37359). Max
• Non-canonical ORFs: color by Kozak strength and translational efficiency (TE), with a generic bigGenePred annotator tool (#35101). Max
• Renamed the varFreqs track collection to "SNV frequencies" (#36642). Max
• Split the combined subtracks from varFreqs into two: one disease, one healthy. Max
• Updated version of the Varaico Variants in Papers track, with extended literature coverage and improved precision (#37658). Johannes
• Added Lowe Lab tracks as contributed tracks on a set of test archaea browsers. Hiram