e6d1189bea4cc541396f842b65a3392c33c8e734 max Wed Sep 2 02:55:03 2026 -0700 hprc2annot: put the collection in git and fix the QA findings The HPRC Release 2 GenArk contributed track collection (7 tracks x 462 assemblies) had only its one-line betaGenArk.txt enable checked in. Add the makeDoc, the build scripts, the seven track description pages and the trackDb stanzas, and fix the problems QA found. Data fixes, both rebuilt across all 462 assemblies: - liftoff: gff3ToGenePred was naming each genePred after the gene, so every transcript of a gene shared one name, the RefSeq accession was lost and the transcript_biotype lookup never matched (type empty on 99.8% of rows). Pass -rnaNameAttr=ID. Duplicate (chrom,start,end,name) tuples go from 24,969 to 0 and type is now empty on 2,132 of 82,973,730 rows. The same flag is a no-op on the CAT GFF3 (byte-identical output), so both gene tracks now share one code path and CAT needs no rebuild. - segdups: the build read SEDEF column 6, strand1, which is "+" by construction on every row, so every inverted duplication rendered forward. Use column 14, strand2, the orientation of the paralogous copy: 13.8M + and 13.8M - across the collection. Also translate the paralog partner out of PanSN through the GenArk chromAlias, since the browser does not translate a plain text field, and store identity as a percentage so the mouseover can read it. hprc2annotFixBed.sh is not idempotent for pclai: a second run re-parses an already-parsed name and blanks the values. It now refuses to touch a converted file. GCA_041900255.1 was damaged that way and is rebuilt from source. Provenance, all from the QA report: - stats.tsv is appended to rather than truncated on every run, and each run regenerates log/summary.tsv, a per-track roll-up over the collection. - dataVersion on all seven tracks. - Rows are now dropped for exactly two reasons and both are counted: past the end of the sequence, or a sequence name absent from the assembly, which also warns with example names. Only GCA_018472765.3 trips the second, the known upstream contig-version mismatch. genePredToBigGenePred failure is checked and an empty conversion result is a failure, not a valid empty bigBed. Description pages: fix a raw UTF-8 character, rewrite the segdups and pclai display conventions which still described the data before the name field was blanked, add a color legend checked against the data, add the pcLAI preprint (from the Crossref record, since it has no PMID), and correct the stated reason liftoff drops transcripts. Display: title case on the short labels, "Active centromeres" shortened to fit the 17-character limit, pcLAI to pack since it has no readable dense state, liftoff and segdups to dense, and a filter on the segdups original flag. refs #35415 diff --git src/hg/makeDb/trackDb/contrib/hprc2annot/segdups.html src/hg/makeDb/trackDb/contrib/hprc2annot/segdups.html new file mode 100644 index 00000000000..ebc9b3a9c72 --- /dev/null +++ src/hg/makeDb/trackDb/contrib/hprc2annot/segdups.html @@ -0,0 +1,105 @@ +<h2>Description</h2> +<p> +This track shows segmental duplications in this Human Pangenome Reference +Consortium (HPRC) Release 2 assembly. Segmental duplications are large blocks of +genomic sequence (typically longer than 1 kb) that occur in two or more nearly +identical copies. They are hotspots for recurrent structural rearrangements and +harbor many genes that have expanded in the human lineage, so their exact +positions differ from person to person and are of particular interest in a +pangenome. Each entry marks one copy of such a block together with the +paralogous copy it aligns to. +</p> + +<h2>Display Conventions</h2> +<p> +Each item is one segmental-duplication region. Items carry no visible label: +the paralogous region the item aligns to, the percent identity of the +alignment, the aligned length and the number of satellite bases are shown when +the mouse is held over an item, and the full set of per-duplication measures is +on the details page. The paralogous region is given in the form +sequence:start-end. The strand is the orientation of the paralogous copy +relative to this one, so items on the minus strand are inverted duplications. +The score is the fraction identity scaled to 0–1000. +</p> +<p> +Items are colored by percent identity, following the same scheme as the +segmental duplication track on the human reference assemblies: +</p> +<table class="stdTbl"> + <tr><th style="background-color:#FF6700;width:2em"> </th> + <td>Orange — 99% or more identity between the two copies.</td></tr> + <tr><th style="background-color:#CCCC00;width:2em"> </th> + <td>Yellow — 98% to 99% identity.</td></tr> + <tr><th style="background-color:#808080;width:2em"> </th> + <td>Gray — below 98% identity. Within this range the shade tracks + identity, so the darkest gray items are the closest to 98%.</td></tr> +</table> +<p> +SEDEF reports both members of each duplicated pair, so most duplications appear +twice, once at each copy. The <i>original</i> field marks which of the two the +source considered the original call; the track configuration page has a filter +on it if only one member of each pair is wanted. +</p> + +<h2>Methods</h2> +<p> +Segmental duplications were detected with SEDEF, which finds pairs of homologous +genomic segments by seeding on shared <i>k</i>-mers and extending and refining +the alignments, then reports each duplicated segment pair with a set of +alignment statistics (see reference below). The calls were produced by the +Eichler laboratory as part of the HPRC assembly annotation. +</p> +<p> +The annotation files were obtained from the HPRC Release 2 data collection on the +public <tt>s3://human-pangenomics</tt> bucket, indexed at +<a href="https://github.com/human-pangenomics/hprc_intermediate_assembly/tree/main/data_tables/annotation/segdups" target="_blank">the hprc_intermediate_assembly data tables</a>. +The per-assembly SEDEF output was converted to a UCSC bigBed file, keeping the +region coordinates, the paralog partner, and the main alignment statistics. The +sequence name of the paralogous region is translated from the HPRC PanSN naming +to the name shown elsewhere in the browser. The steps are described in the +<a href="https://github.com/ucscGenomeBrowser/kent/blob/master/src/hg/makeDb/doc/contrib/hprc2annot.txt" target="_blank">makeDoc</a>, +the build scripts are in the +<a href="https://github.com/ucscGenomeBrowser/kent/tree/master/src/hg/makeDb/scripts/hprc2annot" target="_blank">kent source tree</a>, +and the track configuration is in +<a href="https://github.com/ucscGenomeBrowser/kent/tree/master/src/hg/makeDb/trackDb/contrib/hprc2annot" target="_blank">trackDb/contrib/hprc2annot</a>. +</p> +<p> +Every duplication call in the source files is kept, with one exception. For +HG00735 haplotype 2 (GCA_018472765.3) the SEDEF calls were computed against an +earlier version of the assembly's contigs, so about 40,000 contig-level calls +name sequences that do not exist in the assembly served here and could not be +placed; the roughly 21,600 chromosome-level calls are unaffected, although about +half of them name one of the missing contigs as their paralogous region, so on +that assembly alone the partner is shown in the HPRC naming rather than +translated. This is a version mismatch in the source data rather than something +introduced in the conversion. It is the only assembly of the 462 where either +happens. +</p> + +<h2>Data Access</h2> +<p> +For automated analysis, the annotation is stored in a bigBed-format file +(<tt>segdups.bb</tt>) that can be read with the UCSC tool <tt>bigBedToBed</tt>. +The original files are available from the HPRC S3 bucket linked above. +</p> + +<h2>Credits</h2> +<p> +Annotations were generated by the Human Pangenome Reference Consortium and the +Eichler laboratory. Thanks to the HPRC production team for making these data +available. +</p> + +<h2>References</h2> + + +<p> +Numanagic I, Gökkaya AS, Zhang L, Berger B, Alkan C, Hach F. +<a href="https://academic.oup.com/bioinformatics/article-lookup/doi/10.1093/bioinformatics/bty586" +target="_blank"> +Fast characterization of segmental duplications in genome assemblies</a>. +<em>Bioinformatics</em>. 2018 Sep 1;34(17):i706-i714. +PMID: <a href="https://www.ncbi.nlm.nih.gov/pubmed/30423092" target="_blank">30423092</a>; PMC: <a +href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6129265/" target="_blank">PMC6129265</a> +</p> +