c0a631c93bb41116a873c46ee13c87c5d0806c46 lrnassar Fri Mar 24 16:18:39 2023 -0700 Adding new FAQ entry for diagnosing slow hgTracks behavior. Refs #30620 diff --git src/hg/htdocs/FAQ/FAQtracks.html src/hg/htdocs/FAQ/FAQtracks.html index 267fb62..31988d8 100755 --- src/hg/htdocs/FAQ/FAQtracks.html +++ src/hg/htdocs/FAQ/FAQtracks.html @@ -1,352 +1,433 @@ <!DOCTYPE html> <!--#set var="TITLE" value="Genome Browser FAQ" --> <!--#set var="ROOT" value=".." --> <!-- Relative paths to support mirror sites with non-standard GB docs install --> <!--#include virtual="$ROOT/inc/gbPageStart.html" --> <h1>Frequently Asked Questions: Genome Browser Tracks</h1> <h2>Topics</h2> <ul> +<li><a href="#speed">Why is my tracks display slow?</a></li> <li><a href="#tracks0">List of tracks available for a specific assembly</a></li> <li><a href="#tracks1">Database/browser start coordinates differ by 1 base</a></li> <li><a href="#tracks2">mRNA-associated results</a></li> <li><a href="#tracks3">Correspondence of Genome Browser mRNA positions to those of OMIM genes</a></li> <li><a href="#tracks4">Position changes of features</a></li> <li><a href="#tracks6">Evaluating possible alternative splices</a></li> <li><a href="#tracks7">Matching exons and protein sequence</a></li> <li><a href="#tracks9">Cause of duplicated gene</a></li> <li><a href="#tracks11">Protein doesn't begin with methionine</a></li> <li><a href="#tracks12">Doing an orthology track analysis of a protein</a></li> <li><a href="#tracks14">Quality benchmarks for predicted genes</a></li> <li><a href="#tracks15">Display conventions for gene prediction tracks</a></li> <li><a href="#tracks16">Viewing detailed displays in conservation tracks</a></li> <li><a href="#tracks17">Negative strand coordinates in PSL files</a></li> <li><a href="#tracks18">Inconsistency in stop codon treatment in GTF tracks</a></li> <li><a href="#tracks19">Obtaining clones referenced in Genome Browser</a></li> <li><a href="#tracks20">Locating centromeres and telomeres</a></li> <li><a href="#tracks21">Determining the table name for an annotation track</a></li> <li><a href="#tracks22">Unexpected results in UCSC RefSeq track</a></li> </ul> <hr> <p> <a href="index.html">Return to FAQ Table of Contents</a></p> +<a name="speed"></a> +<h2>Why is my tracks display slow?</h2> +<p> +There are different reasons why the Genome Browser tracks display can become slow. +Below is a list of common scenarios alongside potential solutions. The most common +reason involves the configuration settings that are saved as users interact with +the Genome Browser, such as track visibilities and custom data. +This can be solved by a reset of all your current Genome Browser session +information although it is important to note that this will reset all +settings including filters, track order and remove all custom data. To do +this, click "Genome Browser" > "Reset +All User Settings" in the top blue bar menu or +<a href="../cgi-bin/cartReset" onclick="return confirm('Warning: Reseting all settings '+ +'will return the Genome Browser to a fresh browsing session. All settings including '+ +'filters, visibilities, and custom data will be lost.')">click here</a>.</p> +<p> +If none of the scenarios below improve +the browsing speed, you may also save a <a href="../cgi-bin/hgSession" +target="_blank">session</a> of a display that is slow to load and send it +to us as genome-www@soe.ucsc.edu so that we may provide further guidance.</p> + +<h6>Problem: Too many tracks on at once, or the viewing window is too large.</h6> +<p> +Drawing the tracks image can become a cumbersome task when too many tracks are +enabled and/or if the viewing window is too large. Setting some track visibilities +to <b>hide</b> or zooming into a smaller window should alleviate this problem. +Another consideration is to reduce the visibilities of tracks by by setting them to +<b>dense</b> visibilitiy instead of full/pack/squish. +That will also speed up the drawing.</p> + +<h6>Problem: Large amount of <a href="../cgi-bin/hgCustom">custom tracks</a>.</h6> +<p> +Having a large number of custom tracks loaded can lead to a slowdown. However, the slowdown +should be minimal unless a large number of tracks are turned on. +Custom tracks can be hidden like any other track, or they can also be entirely removed +from the <a href="../cgi-bin/hgCustom">custom tracks</a> page.</p> +<p> +It is also possible to completely reset the Genome Browser +session, which will remove all settings including filters, track order, +and all custom data. To do this, click "Genome Browser" > "Reset +All User Settings" in the top blue bar menu or +<a href="../cgi-bin/cartReset" onclick="return confirm('Warning: Reseting all settings '+ +'will return the Genome Browser to a fresh browsing session. All settings including '+ +'filters, visibilities, and custom data will be lost.')">click here</a>.</p> + +<h6>Problem: Large amount of big data/remote files, such as +<a href="../cgi-bin/hgHubConnect" target="_blank">hubs</a>.</h6> +<p> +Like custom tracks, this slowdown should only be a problem if a large number of tracks +are turned on. However, with remote data there are other considerations that may speed +up performance. When retrieving remote files, consider that you are sending a request +to UCSC, which is then asking for the data wherever it is hosted. So the data must regularly +travel between the hosting location and the Genome Browser servers in CA (main site), Germany +(European mirror), or Japan (Asian mirror). If the hosting connection is unstable, or the +distance between any of +these steps is great, it can lead to slow data display. Consider that many of our free +<a href="../goldenPath/help/hgTrackHubHelp#Hosting" target="_blank">hosting options</a> can +inherently be slow as they are designed for accessibility and not speed. Likewise if you +are using google/amazon storage consider the performance that was purchased as it could +be a playing factor.</p> +<p> +While improving the connection of the hosting location may help, there are other solutions +which do not require the file to be moved. UCSC hosts mirrors in Europe +(https://genome-euro.ucsc.edu/) as well as Asia (https://genome-asia.ucsc.edu/) which should +be faster. In some +extreme circumstances a local installation of the Genome Browser may suit your needs. This +would allow you to load all the custom annotations locally without having to traverse over +the web. In order to learn more about this option see our +<a href="../goldenPath/help/mirror.html" target="_blank">mirroring page</a>.</p> + +<h6>Problem: Slow or unreliable internet connection.</h6> +<p> +If your connection is slow or unreliable, you may want to consider our mirror sites closer to +your location. +UCSC hosts mirrors in Europe (https://genome-euro.ucsc.edu/) as well as Asia +(https://genome-asia.ucsc.edu/) which should be faster. +In some extreme circumstances a local installation of the Genome Browser may +suit your needs. In order to learn more about this option see our +<a href="../goldenPath/help/mirror.html" target="_blank">mirroring page</a>.</p> + <a name="tracks0"></a> <h2>List of tracks available for a specific assembly</h2> <h6>How can I find out which tracks have been released for the assembly in which I'm interested?</h6> <p> The <a href="/goldenPath/releaseLog.html" target="_blank">Release Log</a> contains lists of the published tracks and release dates for the current set of genome assemblies available on our site. It also shows version information for the assemblies of other species used in comparative genomics tracks.</p> <a name="tracks1"></a> <h2>Database/browser start coordinates differ by 1 base</h2> <h6>I am confused about the start coordinates for items in the refGene table. It looks like you need to add "1" to the starting point in order to get the same start coordinate as is shown by the Genome Browser. Why is this the case?</h6> <p> Our internal database representations of coordinates always have a zero-based start and a one-based end. We add 1 to the start before displaying coordinates in the Genome Browser. Therefore, they appear as one-based start, one-based end in the graphical display. The refGene.txt file is a database file, and consequently is based on the internal representation.</p> <p> We use this particular internal representation because it simplifies coordinate arithmetic, i.e. it eliminates the need to add or subtract 1 at every step. If you use a database dump file but would prefer to see the one-based start coordinates, you will always need to add 1 to each start coordinate.</p> <p> If you submit data to the browser in position format (chr#:##-##), the browser assumes this information is 1-based. If you submit data in any other format (BED (chr# ## ##) or otherwise), the browser will assume it is 0-based. You can see this both in our liftOver utility and in our search bar, by entering the same numbers in position or BED format and observing the results. Similarly, any data returned by the browser in position format is 1-based, while data returned in BED format is 0-based.</p> <p> For a detailed explanation, please see our blog entry for the <a href="http://genome.ucsc.edu/blog/the-ucsc-genome-browser-coordinate-counting-systems/" target="_blank">UCSC Genome Browser coordinate counting systems</a>.</p> <a name="tracks2"></a> <h2>mRNA-associated results</h2> <h6>Sometimes when I type in the name of a gene -- e.g. DAO (D aminoacid oxidase) -- the Genome Browser returns a list that includes the gene entry on the assembly, but also contains links to several other genes and aligned mRNAs. What is the relationship between my gene of interest and these results?</h6> <p> The gene search results are obtained from scanning the RefSeq and Known Genes tracks, which are typically based on non-redundant relatively high quality mRNAs. A small fraction of RefSeqs are based on DNA level annotations. In most cases, there is a HUGO Gene Nomenclature Committee symbol or other biological name associated with the gene. In the case of the RefSeq track, the association between these names and the accession is maintained at NCBI and is also present in the refLink table.</p> <p> The mRNA search results are obtained by scanning data associated with the GenBank record for mRNAs. These are often redundant, but occasionally contain something useful that has not yet made it into RefSeq. The mRNA information is often useful because the people who deposited the mRNA into GenBank are listed in the record. Frequently these same people have written interesting articles on the gene or may serve as a source of information on the gene.</p> <a name="tracks3"></a> <h2>Correspondence of Genome Browser mRNA positions to those of OMIM genes</h2> <h6>If I do a Genome Browser search for an mRNA sequence using its GenBank accession number, will I always get the same cytogenetic location as that given by OMIM for the gene?</h6> <p> Not always. Sometimes the Genome Browser will return more than one location when there are recent duplication or assembly problems in the human genome. In these cases, usually one of the locations will agree with OMIM. In a few rare instances involving not-quite-so-recent duplications in the genome, UCSC will attempt to assign it uniquely, but OMIM will think it belongs someplace just under our threshold. A Blat search of the cDNA is very informative in these cases. In rare cases, UCSC or NCBI may have made a data processing error. For the vast majority of cases, however, the two sites do match.</p> <a name="tracks4"></a> <h2>Position changes of features</h2> <h6>Yesterday I was looking at a contig in a specific location, and today the location has changed. What happened?</h6> <p> Check that you are using the same assembly version that you were using yesterday. Features may change positions within a genome between releases, particularly if they are located in an area of the genome that is still in draft form. See <a href="FAQreleases.html#release7">Coordinate changes between assemblies</a> for more information.</p> <a name="tracks6"></a> <h2>Evaluating possible alternative splices</h2> <h6>When you view results from the Genome Browser, how do you determine whether the alternative splice is real or if it is a sequencing artifact?</h6> <p> It's a very good idea to click into the alignment and check that it looks clean at a detailed level and that the splice sites are reasonable. If you have an alternate exon, it is also good to Blat just that exon. Occasionally you may encounter a recent tandem duplication event that encompasses a single exon, which can masquerade as alternative splicing on the graphical display. If it's an EST, check to see if it is from a RAGE library. If so, alternative promoters are likely to be an artifact of the RAGE process rather than biological. If the alternative splice still looks good after these checks, the next step is to do some RT-PCR in the lab.</p> <a name="tracks7"></a> <h2>Matching exons and protein sequence</h2> <h6>I am working with alternatively spliced forms of an enzyme. How can I use the database to identify exons and exactly match them to protein databases (i.e. identify the exons based on a protein sequence and vice versa)?</h6> <p> If you have a protein sequence, you can use <a href="../cgi-bin/hgBlat">Blat</a> to align your sequence to the desired genome. In the ACTIONS column on the Blat search results page, click the details link to view details of exons blocks. Alternatively, click the browser link to display the search results in the Genome Browser. Look for instances in which a gene from the Blat query track aligns exactly or very similarly to an entry in the Known Genes track. Click on the entry to display details about the gene. The SWISS-PROT link on the details page will lead you to more details about this protein.</p> <p> Follow a similar procedure with an mRNA sequence. If there is no corresponding entry in the Known Genes or RefSeq track, then congratulations, you may have found an unreported new gene. You may want to doublecheck the results using NCBI BLAST.</p> <a name="tracks9"></a> <h2>Cause of duplicated gene</h2> <h6>I have found a gene that has two identical copies on different chromosomes within the Genome Browser. Is this possible?</h6> <p> One of the copies may be an artifactual duplication resulting from unavoidable compromises in the assembly process. However, there do exist very recent authentic duplication events. Frequently these are pericentromeric or subtelomeric.</p> <p> There are several checks you can make to determine whether you are viewing an actual duplication or an assembly process artifact. Create a Blat track from the gene's mRNA and examine the details page for a match that is too perfect. Then, open the Genome Browser with the duplication and gap tracks set to dense mode. Look for problems in the flanking sequence in the duplication track. Also look for suspicious placement of the gene, for example inside the intron of another gene. You may also want to follow the OMIM link to look for hand-curated experimental literature summaries. BLASTing the mRNA against a more recent assembly may provide another line of evidence.</p> <a name="tracks11"></a> <h2>Protein doesn't begin with methionine</h2> <h6>I am looking at a human protein that the Genome Browser associates with a particular gene. According to the Genome Browser, its amino acid sequence doesn't start with M (methionine). I thought nuclear-encoded human proteins always began with methionine?</h6> <p> The UCSC genome browser uses translated mRNA data exactly as supplied to GenBank by the original sequencing authors. Any errors at GenBank propagate through many other databases and tools. To work effectively in a bioinformatic area subject to errors, it is a good idea to seek supporting data for any unusual finding.</p> <p> To further investigate this example, you may want to use Blat or BLAST to recover other close members of this gene family. By using comparative alignment, you may discover that the 5' UTR in the mRNA for this protein was likely misinterpreted as coding sequence and that the protein begins with methionine as expected. The error may also be caused by an underlying mRNA in GenBank that stops short of the initiator methionine. In this case, you could use ESTs, other mRNAs, and Blat or BLAST of paralogs against unfinished genome sequence to extend the mRNA to a more plausible full-length sequence.</p> <a name="tracks12"></a> <h2>Doing an orthology track analysis of a protein</h2> <h6>I am working on a lipase called hormone-sensitive lipase (HSL) gene ID NM_010719. I am trying to see if there is any protein that has the same domain organization as HSL. Will doing an orthology track of the protein help me to get an answer? How do I do the orthology track analysis?</h6> <p> You can accomplish this by using Blat and the Genome Browser Superfamily track. Blat the protein sequence from the NCBI RefSeq record, then choose the choose the Browser display option to view your search results in the Genome Browser window. Set the RefSeq and Superfamily tracks to full display mode. The RefSeq track will contain the entry LIPE, and you will find the corresponding entry ENSP00000244289 in the Superfamily track. Click the Superfamily entry, and then click the Superfamily link on the details page that displays. This will open a browser for the Superfamily site. Click "alpha/beta-Hydrolases" to open the Structural Classification of Proteins (SCOP) page. There you will find multiple families listed under this Superfamily, including the lipase in which you're interested.</p> <a name="tracks14"></a> <h2>Quality benchmarks for predicted genes</h2> <h6>Do you offer any benchmarks of quality and quantity of known and predicted genes shown in the Acembly, Ensembl, Genscan, Fgenesh++, Twinscan, and TIGR Gene Index gene prediction tracks?</h6> <p> These tracks are contributed by institutional programs outside of UCSC. You can access links to their home pages and relevant publications from the description pages associated with the tracks (which can be viewed by clicking on the grey mini-button to the left of the track). You may also obtain supplemental information from the <a href="../goldenPath/help/hgTracksHelp.html">Users Guide</a> and the <a href="../goldenPath/credits.html" target="_blank">Credits page</a>. Methods and quality checks are often described in greater detail there. No uniform benchmarking system exists. Finished chromosomes are commonly used, but even here the experimental work continues today on delineating genes.</p> <p> UCSC does not provide summary statistics for these tracks. However, these may be easily compiled from the appropriate tables in the <a href="../cgi-bin/hgTables">Table Browser</a>. The number of predicted genes and exons are easily compared. Some quality checks can also easily be run, such as how many of the predicted gene models are incomplete (e.g. the transcription start coordinate is the same as the CDS start).</p> <p> Looking at almost any coordinate position within the Genome Browser, you can see that there are discrepancies between the predicted gene tracks, as well as further inconsistencies with respect to experimental data tracks such as spliced ESTs. The RefSeq track also contains genes of uncertain status, e.g. lack of initiator methionine. Thus, it is not clear where one can obtain a gold standard for measuring gene prediction quality. A reference set might be hand-curated out of recent journal articles of exceptional thoroughness. UCSC does not currently maintain such a resource.</p> <a name="tracks15"></a> <h2>Display conventions for gene prediction tracks</h2> <h6>What is the significance of the thinner blocks displayed at the beginning and end of a gene in the browser?</h6> <p> The varying thickness of features in the Genome Browser gene tracks denotes the various structural features of a gene, such as exons, introns, and untranslated regions (UTRs). The thickest parts of the track indicate the coding exon regions within the gene. The slightly thinner portions at the leading and trailing ends of the gene track show the 5' and 3' UTRs. Introns are depicted as lines with arrows indicating the direction of transcription.</p> <p> Some aspects of the graphical representation are inevitably lost upon rescaling. For example, coding exons are given preference at coarse scales. For single exon genes, there is no place to put the strand orientation wedges, and therefore the feature's detail page must be consulted.</p> <p> For more information about annotation track display conventions within the Genome Browser, consult the <a href="../goldenPath/help/hgTracksHelp.html">User's Guide</a>.</p> <a name="tracks16"></a> <h2>Viewing detailed displays in conservation tracks</h2> <h6>When I click on a region in the Human/Mouse Evolutionary Conservation Score track, it doesn't give me detailed information.</h6> <p> The track is defaulting to dense display mode because the size of the track's displayed region is too large. Unfortunately, this particular track doesn't have good visual cues to show you when it's defaulting to dense mode. If you zoom in on the region in which you're having the problem, you should be able to display the details page.</p> <a name="tracks17"></a> <h2>Negative strand coordinates in PSL files</h2> <h6>I've noticed that the blatFugu table has two characters representing the strand. Also, I've noticed that the starting/ending positions of the blocks don't fall within the start/end positions of the chromosome target.</h6> <p> When the second character in the strand is "-", the coordinates of the comma-separated list of tStarts are reverse-complemented relative to tStart, much as qStarts behave when the first letter in the strand is "-".</p> <a name="tracks18"></a> <h2>Inconsistency in stop codon treatment in GTF tracks</h2> <h6>I've been doing some comparative gene set analysis using the gene annotation tracks and I believe I have run into an inconsistency in the way that stop codons are treated in the annotations. Looking at the Human June 2002 assembly, the annotations for Ensembl, Twinscan, SGP, and Geneid appear to exclude the stop codon in the coding region coordinates. All of the other gene annotation sets include the stop codon as part of the coding region. My guess is that this inconsistency is the result of the gene sets being imported from different file formats. The <a href="http://mblab.wustl.edu/GTF2.html" target="_blank">GTF2 format</a> does not include the stop codon in the terminal exon, while the GenBank format does, and the GFF format does not specify what to do.</h6> <p> Your guess is correct. We haven't gotten around to fixing this situation. A while ago, the Twinscan group made a GTF validator. It interpreted the stop codon as <em>not</em> part of the coding region. Prior to that, all GFF and GTF annotations that we received did include the stop codon as part of the coding region; therefore, we didn't have special code in our database to enforce it. In response to the validator, Ensembl, SGP and Geneid switched their handling of stop codons to the way that Twinscan does it, hence the discrepancy.</p> <a name="tracks19"></a> <h2>Obtaining clones referenced in Genome Browser</h2> <h6>Is it possible to purchase the chromosome clones referenced in the Genome Browser?</h6> <p> You can find further information about a specific clone by clicking on the clone name link on the details page for the item. This links to the NCBI Clone Registry website, which lists extensive details about the clone, including distributor information.</p> <a name="tracks20"></a> <h2>Locating centromeres and telomeres</h2> <h6>How do I find the positions of the centromeres and telomeres in a particular assembly?</h6> <p> This information can be found in the "gap" database table. Use the Table Browser to extract it. To do this, select your assembly and the gap table, then click the " filter Create" button. Set the "type" field to <code>centromere telomere</code> (separated by a space). For help using the Table Browser, visit the <a href="../goldenPath/help/hgTablesHelp.html">User's Guide</a>.</p> <a name="tracks21"></a> <h2>Determining the table name for an annotation track</h2> <h6>How do I find the name of the database table that contains the data for a particular annotation track?</h6> <p> Each annotation track in the Genome Browser has one or more database tables associated with it. To find the name of the primary table, navigate to the schema page. You will find the schema page by pressing the "mini-button" to the left of the annotation track display, or clicking the hyper-linked track name in the track controls (below the display). From the resulting description page, follow the "View table schema" link. Finally, on the schema page, you will find the name of the database table near the top of the page listed after the "Primary Table" label.</p> <a name="tracks22"></a> <h2>Unexpected results in UCSC RefSeq track</h2> <h6>Why do some annotations differ in the UCSC RefSeq track compared to NCBI's Reference Sequence Database?</h6> <p> Historically, NCBI RefSeq coordinates were not directly available for building tracks in the UCSC Genome Brower. Instead of using coordinates to map annotations, mappings to the reference assembly were conducted using <a href="../FAQ/FAQblat.html" target="_blank">BLAT</a> (BLAST-like alignment tool) alignment methods. This BLAT alignment method has caused some discrepancies from the NCBI RefSeq database. Most discrepancies arise when the BLAT-generated annotations align to multiple regions where the sequence in the assembly is either identical or nearly identical. In essence, by using BLAT to align the sequence, a single transcript could result in matching to multiple novel places across the genome, or alignments of small exons could differ slightly in final coordinates within the region of a gene rich with repeats. BLAT-generated RefSeq track methods are described in corresponding track description pages (e.g., <a href="../cgi-bin/hgTrackUi?db=hg19&g=refGene" target="_blank">RefSeq track description for hg19)</a>.</p> <p> In 2017, NCBI RefSeq coordinates for hg38 were used for generating non-discrepant RefSeq tracks in the UCSC Genome Browser. This new <a href="../cgi-bin/hgTrackUi?db=hg38&g=refSeqComposite" target="_blank">NCBI RefSeq track</a> in the UCSC Genome Browser displays identical RNA coordinates to annotations in the <a href="https://www.ncbi.nlm.nih.gov/refseq/" target="_blank">NCBI Reference Sequence Database</a>. Please note that when annotations do map to multiple loci, the NCBI RefSeq track displays unique identifiers for each locus, while the UCSC RefSeq track retains the same identifier.</p> <p> For more information and future plans to integrate coordinate-generated NCBI RefSeq tracks for other assemblies, please see the <a href="http://genome.ucsc.edu/blog/the-new-ncbi-refseq-tracks-and-you/" target="_blank">NCBI RefSeq track blog post</a>.</p> <!--#include virtual="$ROOT/inc/gbPageEnd.html" -->