bbabbd5d2566d47d923d51dbe350634783455999 mspeir Sun Oct 26 12:14:52 2025 -0700 change soe to gi, refs #35031 diff --git src/hg/htdocs/goldenPath/help/gbib.html src/hg/htdocs/goldenPath/help/gbib.html index 20ad73e4319..d9cbca8a9e3 100755 --- src/hg/htdocs/goldenPath/help/gbib.html +++ src/hg/htdocs/goldenPath/help/gbib.html @@ -59,31 +59,31 @@ for adiscussion of the advantages/disadvantages of the different methods to customize your genome data display.</p> <h3>Differences between GBiB and the Genome Browser</h3> <p> While GBiB and the Genome Browser are similar in many ways, there are key differences. In particular, GBiB makes it much easier to visualize sensitive or protected data. Prior to the introduction of GBiB, it was necessary to upload your data to the UCSC Genome Browser website or place the data files on a publicly accessible web server and supply the URL to UCSC in order to view your own data with the Genome Browser. GBiB removes these requirements: none of your data must be uploaded to the UCSC servers, allowing you to use the Genome Browser on personal datasets in situations where it's infeasible to load the data onto a public web server.</p> <p> Rather than installing the entire UCSC genome annotation database (several terabytes of data), GBiB instead depends upon remote connections to various UCSC servers for much of its functionality and -data. It connects to the UCSC <a href="http://hgdownload.soe.ucsc.edu/downloads.html" +data. It connects to the UCSC <a href="http://hgdownload.gi.ucsc.edu/downloads.html" target="_blank">download server</a> to obtain genomic sequences, liftOver files, and many of the other large data files, and connects to one of UCSC's <a href="mysql.html">public MariaDB servers</a> to download data displayed by the various annotation tracks. A few Genome Browser tracks are unavailable on the UCSC public MariaDB servers due to agreements with the data distributors (DECIPHER and LOVD Variants), and thus are unavailable for use with GBiB.</p> <p> The majority of protected data use in the research community currently focuses on the human genomes, primarily the hg19 (GRCh37) assembly and with a growing body of annotation on the newer hg38 (GRCh38) assembly. As a result, GBiB is currently optimized for use with the hg19 assembly. Many other recent genome assemblies can also be viewed, but access may be slower than for optimized assemblies. Access speed may also be impacted by your connection distance from the UCSC server. To improve performance in these situations, GBiB includes a simple tool that allows you to download ("mirror") selected genome annotation tracks to your machine. You can find more information about this tool in the <a href="#GbibMirroring">Improving Speed and Performance</a> section.</p> @@ -112,42 +112,42 @@ <li> A compatible version of the <a href="https://www.virtualbox.org/" target="_blank">VirtualBox</a> software (version 4.3.6 or higher) must be installed. This software is free to use in many situations. See the VirtualBox wiki for <a href="https://www.virtualbox.org/wiki/Licensing_FAQ" target="_blank">licensing terms and conditions</a> and <a href="https://www.virtualbox.org/wiki/Downloads" target="_blank">installation</a> instructions. You must have administrator privileges to install VirtualBox on your computer.</li> <li> The computer hard disk must have at least 20 GB of free space (more if you plan to mirror many tracks).</li> <li> Your network firewall must allow outgoing connections to the following servers and ports: <ul> <li>MariaDB connections, used to load tracks not local to your computer: <ul> - <li>US server: Port 3306 on genome-mysql.soe.ucsc.edu (128.114.119.174)</li> + <li>US server: Port 3306 on genome-mysql.gi.ucsc.edu (128.114.119.174)</li> <li>European server: Port 3306 on genome-euro-mysql.soe.ucsc.edu (129.70.40.120)</li> </ul> </li> <li>Rsync, used to download track data: <ul> - <li>US server: TCP port 873 on hgdownload.soe.ucsc.edu (128.114.119.163)</li> + <li>US server: TCP port 873 on hgdownload.gi.ucsc.edu (128.114.119.163)</li> <li>European server: TCP port 873 on hgdownload-euro.soe.ucsc.edu (129.70.40.99)</li> </ul></li> <li>Download HTML descriptions on the fly: <ul> - <li>US server: TCP port 80 on hgdownload.soe.ucsc.edu (128.114.119.163)</li> + <li>US server: TCP port 80 on hgdownload.gi.ucsc.edu (128.114.119.163)</li> <li>European server: TCP port 80 on hgdownload-euro.soe.ucsc.edu (129.70.40.99)</li> </ul> </li> </ul> </ul> <h3>Installation</h3> <ol> <li> Confirm that your system meets the above requirements.</li> <li> Download GBiB from the <a href="https://genome-store.ucsc.edu/" target="_blank">Genome Browser store</a> (see <a href="#License">Licensing Information</a>). Due to the large size of the <em>gbib.zip</em> product file, the download time may range from 30 minutes to a few hours depending on your @@ -273,53 +273,53 @@ we recommend the option <em>Default tracks with conservation tables, but no alignments</em>. When downloading large tracks, keep in mind that you cannot delete these tracks and the related data from GBiB once you have downloaded them. If you find that you've started downloading the wrong track or a track that is too large for your machine, you can cancel the download at any point by clicking <em>Cancel Download Now</em>.</p> <p> Depending on your network bandwidth, the download can take several minutes or up to a few hours over a DSL line. <strong>During the download</strong>, the file <em>gbib-data.vdi</em> will grow in size, and <strong>you will not be able to use GBiB</strong>. Once the download is complete, the default tracks should load in less than three seconds for a typical genomic position.</p> <p> If you are in your GBiB on the command-line you can use a direct rsync command for files of interest. For example, if you knew you wanted all the GENCODE tracks on hg19 you could run either of the two rsync commands for the North American or European hgdownload servers: <pre> -<code>sudo rsync hgdownload.soe.ucsc.edu::mysql/hg19/wgEncodeGencode* /data/mysql/hg19/.</code> +<code>sudo rsync hgdownload.gi.ucsc.edu::mysql/hg19/wgEncodeGencode* /data/mysql/hg19/.</code> </pre> <pre> <code>sudo rsync hgdownload-euro.soe.ucsc.edu::mysql/hg19/wgEncodeGencode* /data/mysql/hg19/.</code> </pre> <p> The above commands will rsync all of the files at the UCSC hgdownload server in the hg19 assembly that start with wgEncodeGencode to your GBiB into the hg19 directory. There are some supporting files in a hgFixed directory, such as for the publication tracks, that could be mirrored with such commands.</p> <p> Here is another example for hg38 where the following commands would download all the supporting encRegTfbs and factorbook tables for the Transcription Factor ChIP-seq Clusters track: <pre> -<code>sudo rsync hgdownload.soe.ucsc.edu::mysql/hg38/encRegTfbs* /data/mysql/hg38/.</code> +<code>sudo rsync hgdownload.gi.ucsc.edu::mysql/hg38/encRegTfbs* /data/mysql/hg38/.</code> </pre> <pre> -<code>sudo rsync hgdownload.soe.ucsc.edu::mysql/hg38/factor* /data/mysql/hg38/.</code> +<code>sudo rsync hgdownload.gi.ucsc.edu::mysql/hg38/factor* /data/mysql/hg38/.</code> </pre> <p> You can also download gbdb files in this manner. <pre> -<code>sudo rsync hgdownload.soe.ucsc.edu::gbdb/hg19/multiz100way/phyloP100way.wib /data/gbdb/hg19/multiz100way/.</code> +<code>sudo rsync hgdownload.gi.ucsc.edu::gbdb/hg19/multiz100way/phyloP100way.wib /data/gbdb/hg19/multiz100way/.</code> </pre> <pre> <code>sudo rsync hgdownload-euro.soe.ucsc.edu::gbdb/hg19/multiz100way/phyloP100way.wib /data/gbdb/hg19/multiz100way/.</code> </pre> <p>The above command would copy the phyloP100way track to display in the GBiB from a local file.</p> <h3>Offline mode</h3> <p> GBiB has an offline mode that is particularly useful when you want to ensure that GBiB no longer connects to the Internet once the initial download and setup are complete (for instance, to comply with corporate IT policy). Before going offline, first mirror all the tracks that you will want to access. Then, in the GBiB terminal window type the command: <code>gbibOffline</code>. This command will remove GBiB's network access to the UCSC MariaDB server and download servers.</p> @@ -486,31 +486,31 @@ <code>sudo</code> for root access, which does not require a password. Because stock Windows computers do not have ssh installed, Windows users will have to use the GBiB terminal or install a third-party ssh client for Windows, such as the free software <a href="http://putty.org" target="_blank">Putty</a>.</p> <h3>Data and track conversion tools</h3> <p> By default, GBiB includes a few of the commonly used UCSC file manipulation tools, such as bedToBigBed, wigToBigWig, samtools and tabix. These tools can be used to convert and manipulate your basic files into formats that can be uploaded to GBiB as custom tracks. If you need additional Genome Browser tools, type the following command into the GBiB terminal window: <code>gbibAddTools</code>. This command downloads and installs the full suite of command line tools provided by UCSC. Many of these extra tools can be used to extract data and other useful information from your files, or to convert them between various file types. A complete listing and description for all of these tools can be found on UCSC's -<a href="http://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64/FOOTER" target="_blank">download +<a href="http://hgdownload.gi.ucsc.edu/admin/exe/linux.x86_64/FOOTER" target="_blank">download server</a>.</p> <p> You can use these tools to convert and extract data from your shared files with the standard "Read-only" settings. However, if you would like to to modify files you've shared with your GBiB, you will have to ensure that the "Read-only" access for VirtualBox is turned off. To do so, follow the directions in Step 2 of the <a href="#LocalTracks">Loading local big data tracks</a> section, but deselect the checkbox next to "Read-only".</p> <p> Please note that the gbibAddTools command requires sudo permissions.</p> <h3><em>Example: Indexing a local BAM file</em></h3> <p> A BAM file must be indexed before it can be loaded into the browser. For example, to index a BAM file in a shared folder "Documents" on your hard disk, type:</p> <pre><code>cd /folders/Documents @@ -779,31 +779,31 @@ resolve this problem:</p> <ol> <li> Start VirtualBox.</li> <li> Select <em>File >> Virtual Media Manager</em> (Ctrl+D or ⌘+D) in the VirtualBox menubar.</li> <li> Select <em>gbib-data.vdi</em> and click "Remove".</li> <li> Double-click the newly downloaded <em>browserbox.vbox</em> file or add it with <em>Machine >> Add</em> menu option.</li> </ol> <p> <strong>Genome Browser Error:</strong> "<em>Couldn't connect to database hg19 on -genome-mysql.soe.ucsc.edu as genomep.</em>" +genome-mysql.gi.ucsc.edu as genomep.</em>" <p> <em>Solution 1:</em> This indicates that the virtual machine could not connect to the UCSC MariaDB server. This error can be caused by a change of the IP address (e.g. on a wifi connection) that has not yet been picked up by the virtual machine. In this situation, you can restart the box or run the command <code>sudo ifup --force eth0</code> to reset the network connection.</p> <p> <em>Solution 2:</em> Alternatively, this error may be generated when the firewall does not allow outgoing TCP data on port 3306/MySQL. In this case, contact your institution's IT support staff to inquire about ways to open this port.</p> <p> <strong>Genome Browser Error:</strong> "<em>Couldn't connect to database hgcentral on localhost as root. Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (13)</em>"</p> <p> <em>Solution:</em> This error can be caused when the virtual machine is downloading data such as @@ -859,35 +859,35 @@ this:</p> <pre><code>httpProxy=http://user:password@someProxyServer:3128</code></pre> <pre><code>httpsProxy=http://user:password@someProxyServer:3128</code></pre> <p> If there are domains or domain-suffices that should not be proxied, use <code>noProxy</code>. <pre><code>noProxy=ucsc.edu,mit.edu,localhost,127.0.0.1</code></pre> <p> The file /usr/local/apache/cgi-bin/hg.conf should already include a line like <code>include hg.conf.local</code> to incorporate the changes in hg.conf.local.</p> <p> <strong>Problem:</strong> "No space left on device" error when running gbibAddTools.</p> <p> In older GBiB's, there is an error in the <code>gbibAddTools</code> command. To fix this command, edit the <code>/home/browser/.bashrc</code> file and change the following line: -<pre><code>alias gbibAddTools='mkdir ~/bin -p; rsync -avP hgdownload.soe.ucsc.edu::genome/admin/exe/linux.x86_64/ ~/bin/'</code></pre> +<pre><code>alias gbibAddTools='mkdir ~/bin -p; rsync -avP hgdownload.gi.ucsc.edu::genome/admin/exe/linux.x86_64/ ~/bin/'</code></pre> </p> <p> to either of the follwing lines: -<pre><code>alias gbibAddTools='sudo mkdir -p /data/tools; sudo rsync -avP hgdownload.soe.ucsc.edu::genome/admin/exe/linux.x86_64/ /data/tools/ && ln -s /data/tools ~/bin'</code></pre> +<pre><code>alias gbibAddTools='sudo mkdir -p /data/tools; sudo rsync -avP hgdownload.gi.ucsc.edu::genome/admin/exe/linux.x86_64/ /data/tools/ && ln -s /data/tools ~/bin'</code></pre> <pre><code>alias gbibAddTools='sudo mkdir -p /data/tools; sudo rsync -avP hgdownload-euro.soe.ucsc.edu::genome/admin/exe/linux.x86_64/ /data/tools/ && ln -s /data/tools ~/bin'</code></pre> </p> <p> The updated commands will install the tools to a location with more disk space available using either the North American or European hgdownload servers. </p> <a name="Commands"></a> <h2>Genome Browser in a Box commands</h2> <!-- Explanation here --> <p> In addition to normal Linux commands, GBiB defines some special commands you may use while inside the GBiB terminal's window. These additional commands are documented in the README.txt file on the GBiB terminal's home directory, which can also be accessed via