09293bd99cefe2305bb7aeb6b8c6f4e537a95921 ccpowell Wed Jul 10 09:42:07 2019 -0700 Switching MySQL mentions with MariaDB, refs #23597 diff --git src/hg/htdocs/goldenPath/help/gbib.html src/hg/htdocs/goldenPath/help/gbib.html index fba00fd..c419294 100755 --- src/hg/htdocs/goldenPath/help/gbib.html +++ src/hg/htdocs/goldenPath/help/gbib.html @@ -48,33 +48,33 @@ <h3>Differences between GBiB and the Genome Browser</h3> <p> While GBiB and the Genome Browser are similar in many ways, there are key differences. In particular, GBiB makes it much easier to visualize sensitive or protected data. Prior to the introduction of GBiB, it was necessary to upload your data to the UCSC Genome Browser website or place the data files on a publicly accessible web server and supply the URL to UCSC in order to view your own data with the Genome Browser. GBiB removes these requirements: none of your data must be uploaded to the UCSC servers, allowing you to use the Genome Browser on personal datasets in situations where it's infeasible to load the data onto a public web server.</p> <p> Rather than installing the entire UCSC genome annotation database (several terabytes of data), GBiB instead depends upon remote connections to various UCSC servers for much of its functionality and data. It connects to the UCSC <a href="http://hgdownload.soe.ucsc.edu/downloads.html" target="_blank">download server</a> to obtain genomic sequences, liftOver files, and many of the -other large data files, and connects to one of UCSC's <a href="mysql.html">public MySQL servers</a> +other large data files, and connects to one of UCSC's <a href="mysql.html">public MariaDB servers</a> to download data displayed by the various annotation tracks. A few Genome Browser tracks are -unavailable on the UCSC public MySQL servers due to agreements with the data distributors (DECIPHER +unavailable on the UCSC public MariaDB servers due to agreements with the data distributors (DECIPHER and LOVD Variants), and thus are unavailable for use with GBiB.</p> <p> The majority of protected data use in the research community currently focuses on the human genomes, primarily the hg19 (GRCh37) assembly and with a growing body of annotation on the newer hg38 (GRCh38) assembly. As a result, GBiB is currently optimized for use with the hg19 assembly. Many other recent genome assemblies can also be viewed, but access may be slower than for optimized assemblies. Access speed may also be impacted by your connection distance from the UCSC server. To improve performance in these situations, GBiB includes a simple tool that allows you to download ("mirror") selected genome annotation tracks to your machine. You can find more information about this tool in the <a href="#GbibMirroring">Improving Speed and Performance</a> section.</p> <p> For more background on GBiB see: <p> Haeussler M, Raney BJ, Hinrichs AS, Clawson H, Zweig AS, Karolchik D, Casper J, Speir ML, Haussler @@ -97,31 +97,31 @@ <li> The computer must support virtualization (common for most PCs sold after 2010).</li> <li> A compatible version of the <a href="https://www.virtualbox.org/" target="_blank">VirtualBox</a> software (version 4.3.6 or higher) must be installed. This software is free to use in many situations. See the VirtualBox wiki for <a href="https://www.virtualbox.org/wiki/Licensing_FAQ" target="_blank">licensing terms and conditions</a> and <a href="https://www.virtualbox.org/wiki/Downloads" target="_blank">installation</a> instructions. You must have administrator privileges to install VirtualBox on your computer.</li> <li> The computer hard disk must have at least 20 GB of free space (more if you plan to mirror many tracks).</li> <li> Your network firewall must allow outgoing connections to the following servers and ports: <ul> - <li>MySQL connections, used to load tracks not local to your computer: + <li>MariaDB connections, used to load tracks not local to your computer: <ul> <li>US server: Port 3306 on genome-mysql.soe.ucsc.edu (128.114.119.174)</li> <li>European server: Port 3306 on genome-euro-mysql.soe.ucsc.edu (129.70.40.120)</li> </ul> </li> <li>Rsync, used to download track data: <ul> <li>US server: TCP port 873 on hgdownload.soe.ucsc.edu (128.114.119.163)</li> <li>European server: TCP port 873 on hgdownload-euro.soe.ucsc.edu (129.70.40.99)</li> </ul></li> <li>Download HTML descriptions on the fly: <ul> <li>US server: TCP port 80 on hgdownload.soe.ucsc.edu (128.114.119.163)</li> <li>European server: TCP port 80 on hgdownload-euro.soe.ucsc.edu (129.70.40.99)</li> </ul> @@ -288,31 +288,31 @@ <code>sudo rsync hgdownload.soe.ucsc.edu::gbdb/hg19/multiz100way/phyloP100way.wib /data/gbdb/hg19/multiz100way/.</code> </pre> <pre> <code>sudo rsync hgdownload-euro.soe.ucsc.edu::gbdb/hg19/multiz100way/phyloP100way.wib /data/gbdb/hg19/multiz100way/.</code> </pre> <p>The above command would copy the phyloP100way track to display in the GBiB from a local file.</p> <h3>Offline mode</h3> <p> GBiB has an offline mode that is particularly useful when you want to ensure that GBiB no longer connects to the Internet once the initial download and setup are complete (for instance, to comply with corporate IT policy). Before going offline, first mirror all the tracks that you will want to access. Then, in the GBiB terminal window type the command: <code>gbibOffline</code>. This command -will remove GBiB's network access to the UCSC MySQL server and download servers.</p> +will remove GBiB's network access to the UCSC MariaDB server and download servers.</p> <p> Once GBiB is in offline mode, the Genome Browser will display an error message if you attempt to access a data file not located on your local disk; therefore, we do not recommend this option for general use. To reactivate Internet access, click on the GBiB terminal window and type the command: <code>gbibOnline</code>.</p> <a name="UpdatingGbib"></a> <h2>Updating Genome Browser in a Box</h2> <p> The software that supports the UCSC Genome Browser is updated every three weeks. These updates include new features and bug fixes for existing features. The track data, on the other hand, are not updated on a regular basis. New tracks and updates to existing tracks are released as they pass UCSC's quality assurance process. The only exceptions to this are GenBank-based tracks, including RefSeq Genes, GenBank mRNAs, and others, which are updated weekly through an automatic process.</p> @@ -666,59 +666,59 @@ <li> Start VirtualBox.</li> <li> Select <em>File >> Virtual Media Manager</em> (Ctrl+D or ⌘+D) in the VirtualBox menubar.</li> <li> Select <em>gbib-data.vdi</em> and click "Remove".</li> <li> Double-click the newly downloaded <em>browserbox.vbox</em> file or add it with <em>Machine >> Add</em> menu option.</li> </ol> <p> <strong>Genome Browser Error:</strong> "<em>Couldn't connect to database hg19 on genome-mysql.soe.ucsc.edu as genomep.</em>" <p> -<em>Solution 1:</em> This indicates that the virtual machine could not connect to the UCSC MySQL +<em>Solution 1:</em> This indicates that the virtual machine could not connect to the UCSC MariaDB server. This error can be caused by a change of the IP address (e.g. on a wifi connection) that has not yet been picked up by the virtual machine. In this situation, you can restart the box or run the command <code>sudo ifup --force eth0</code> to reset the network connection.</p> <p> <em>Solution 2:</em> Alternatively, this error may be generated when the firewall does not allow outgoing TCP data on port 3306/MySQL. In this case, contact your institution's IT support staff to inquire about ways to open this port.</p> <p> <strong>Genome Browser Error:</strong> "<em>Couldn't connect to database hgcentral on localhost as root. Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (13)</em>"</p> <p> <em>Solution:</em> This error can be caused when the virtual machine is downloading data such as when using the <a href="#GbibMirroring">mirror tool</a> to install local copies of tracks to increase speed and performance. Once the download finishes, the message will no longer appear. If one cancels a download in progress, the error can persist, in which case restart your virtual machine and the message should be removed.</p> <p> <strong>Problem:</strong> Error when using BLAT.</p> <p> When you use BLAT on your GBiB machine, it attempts to open outgoing TCP connections to the BLAT servers running at UCSC. Each assembly has two ports that need to be open from your GBiB machine to the UCSC BLAT servers. You will likely need to contact the system administrators at your institution and ask them to open outgoing TCP connections to a list of UCSC hostnames and ports. The commands below will allow you to find out the names of these hosts and ports.</p> <p> <em>Solution:</em> To find the hostname-port combination to open from your GBiB machine, use this -Mysql query:</p> +SQL query:</p> <pre><code>mysql hgcentral -e 'select * from blatServers where db="YOURDB"'</code></pre> <p> For example, if you want to enable BLAT for hg19 and hg38, you can issue this command: <pre><code>mysql hgcentral -e 'select * from blatServers where db="hg38" or db="hg19"'</code></pre> <pre><code>+------+---------------------+-------+---------+--------+ | db | host | port | isTrans | canPcr | +------+---------------------+-------+---------+--------+ | hg19 | blat4a.soe.ucsc.edu | 17779 | 0 | 1 | | hg19 | blat4a.soe.ucsc.edu | 17778 | 1 | 0 | | hg38 | blat4c.soe.ucsc.edu | 17781 | 0 | 1 | | hg38 | blat4c.soe.ucsc.edu | 17780 | 1 | 0 | +------+---------------------+-------+---------+--------+</code></pre> <p> <strong>Problem:</strong> GBiB is functioning incorrectly or stops working.</p> <p> @@ -800,40 +800,40 @@ <td>Enable the "Mirror tracks" tool</td> </tr><tr> <td><code>gbibAddTools</code></td> <td>Download the UCSC genome command line tools into the ~bin directory. Requires sudo permissions</td> </tr> </table> <h6>Advanced commands</h6> <table> <tr> <td><code>gbibCoreUpdate</code> <td>Download the most current update script from UCSC now, this is part of an automatic update</td> </tr><tr> <td><code>gbibFixMysql1</code></td> - <td>Fix all mysql databases, fast version</td> + <td>Fix all MariaDB databases, fast version</td> </tr><tr> <td><code>gbibFixMysql2</code></td> - <td>Fix all mysql databases, intensive version</td> + <td>Fix all MariaDB databases, intensive version</td> </tr><tr> <td><code>gbibResetNetwork</code></td> <td>Reinit the eth0 network interface, in case VirtualBox dropped the network connection</td> </tr><tr> <td><code>gbibUcscLog</code></td> - <td>Show a real-time log of all MySQL queries on the console </td> + <td>Show a real-time log of all SQL queries on the console </td> </tr><tr> <td><code>gbibUcscTablesLog</code></td> <td>Show the tables that had to be loaded through the internet from UCSC</td> </tr><tr> <td><code>gbibUcscTablesReset</code></td> <td>Reset the table counters</td> </tr><tr> <td><code>gbibUcscGbdbLog</code></td> <td>Show the gbdb files that had to be loaded through the internet from UCSC</td> </tr><tr> <td><code>gbibUcscGbdbReset</code></td> <td>Reset the gbdb counters</td> </tr> </table>