c928b75199212c8946d11764acee5e9ac68ce6f4 gperez2 Tue Sep 15 10:52:23 2026 -0700 Adding a Blat and In-Silico PCR section for Docker assembly hubs and fixing several small existing issues on the docker.html page, refs #35979 diff --git src/hg/htdocs/goldenPath/help/docker.html src/hg/htdocs/goldenPath/help/docker.html index edbc28b09b1..40b998257dd 100755 --- src/hg/htdocs/goldenPath/help/docker.html +++ src/hg/htdocs/goldenPath/help/docker.html @@ -3,32 +3,33 @@ <!--#set var="ROOT" value="../.." --> <!-- Relative paths to support mirror sites with non-standard GB docs install --> <!--#include virtual="$ROOT/inc/gbPageStart.html" --> <h1>Docker Help Page</h1> <h2>Contents</h2> <h6><a href="#docker">What is Docker?</a></h6> <h6><a href="#installingDocker">How to Install Docker Desktop?</a></h6> <h6><a href="#dockerUCSCgb">Using Docker Desktop for UCSC Genome Browser</a></h6> <h6><a href="#dockerHub">Using the Prebuilt UCSC Genome Browser Image</a></h6> <h6><a href="#buildImage">Building the Image Yourself</a></h6> <h6><a href="#dockerVolume">Create a Docker Volume for Data Persistence</a></h6> -<h6><a href="#updateGB">Updating the Latest UCSC Genome Browser Version</a></h6> +<h6><a href="#updateGB">Updating the Latest UCSC Genome Browser Software</a></h6> <h6><a href="#hg.conf">Customize a UCSC Genome Browser Docker Container</a></h6> +<h6><a href="#blatHub">Enabling Blat and In-Silico PCR on an Assembly Hub</a></h6> <!-- ========== What is Docker? ============================== --> <a id="docker"></a> <h2>What is Docker?</h2> <p> Docker is a platform for developing, testing and running applications. Docker can be used to run genomics tools and manage software such as the UCSC Genome Browser. Docker offers consistency across different computers and environments by packaging everything needed including specific software versions and configurations into a self-contained unit called a container. <h3>Container</h3> <p> A container is software that packages up code and all its dependencies to run an application quickly and reliably from one computing environment to another. A container is isolated from other @@ -74,31 +75,31 @@ <li>Go to the <a href="https://docs.docker.com/desktop/install/linux-install/#supported-platforms" target=_blank>Install Docker Desktop on Linux</a> page and select Linux distribution</li> <li>Check <a href="https://docs.docker.com/desktop/install/linux-install/#system-requirements" target=_blank>system requirements</a></li> <li>Follow <a href="https://docs.docker.com/desktop/install/linux-install/#generic-installation-steps" target=_blank>Generic installation steps</a> </ul> <a id="dockerUCSCgb"></a> <h2>Using Docker Desktop for UCSC Genome Browser</h2> <p>Start Docker Desktop after installation is complete:</p> <ul> <li><b>Windows</b>: start Docker Desktop from the Start menu <li><b>macOS</b>: start Docker Desktop from the Applications folder <li><b>Linux</b>: start the Docker service by running the following command on the terminal: - <pre><code>sudo systemctl start docker</pre></code> + <pre><code>sudo systemctl start docker</code></pre> </ul> <a id="dockerHub"></a> <h2>Using the Prebuilt UCSC Genome Browser Image</h2> <p> UCSC publishes a ready-made Genome Browser image on Docker Hub as <a href="https://hub.docker.com/r/genomebrowser/server" target=_blank>genomebrowser/server</a>. The image is rebuilt for every Genome Browser release and tagged with the version number, for example <code>v502</code>. The <code>latest</code> tag always points at the most recent release, and a single tag covers both Intel and Apple Silicon machines. Most people should pull this image rather than build the Dockerfile themselves, since pulling takes a few minutes where a build takes considerably longer.</p> <p> The following commands download the image and start a container, mapping port 8080 on the host machine to port 80 in the container:</p> <pre><code>docker pull genomebrowser/server @@ -125,41 +126,41 @@ <h3>Obtaining a UCSC Genome Browser Dockerfile</h3> <p>The UCSC Genome Browser dockerfile can be obtained from the <a href="https://github.com/ucscGenomeBrowser/" target=_blank>UCSC Genome Browser Github</a> by using the wget command:</p> <pre><code>wget https://raw.githubusercontent.com/ucscGenomeBrowser/kent/master/src/product/installer/docker/Dockerfile</code></pre> <h3>Creating a Image</h3> <p> Once the dockerfile has been downloaded, running the docker build with the 't' option allows the naming and the optional tag (format: "name:tag") of the image. The image can be created by running the following command in the same directory where the dockerfile is located:</p> <pre><code>docker build . -t user_name/ucsc_genomebrowser_image</code></pre> <h3>Creating a Container</h3> <p>After the image has been created, running the docker run command and the image with the -d option allows the container to be run in the background, whereas the default runs the container in the foreground. The -p option publishes a container's port(s) to the host. The following command -maps port 8080 on the host machine to port 80 in the container and names the container using the -name option: +maps port 8080 on the host machine to port 80 in the container and names the container using the --name option: <pre><code>docker run -d --name ucsc_genomebrowser_container -p 8080:80 user_name/ucsc_genomebrowser_image</code></pre></p> <p>Accessing the running container via http://localhost:8080</p> <p>Running the following command will list the running container: <pre><code>docker container ls</code></pre></p> -<p>Running the following command stops the running container:: +<p>Running the following command stops the running container: <pre><code>docker stop <container_name_or_id></code></pre></p> -<p>Running the following command removes the existing container:: +<p>Running the following command removes the existing container: <pre><code>docker rm <container_name_or_id></code></pre></p> <h3>Using Docker Desktop to Create a Container</h3> <p>The Docker Desktop user interface can be used to run the container by going to the images tab and clicking the run button under Actions:</p> <div class="text-center"> <img alt="Docker Desktop Images tab showing the Genome Browser mirror container image" src="../../images/docker_image.png" style="width:50%;max-width:1083px"> </div> <p>Click Optional settings in the "Run a new container" pop-up window:</p> <div class="text-center"> <img alt="Docker run dialog with the Optional Settings section expanded" src="../../images/docker_optional_settings.png" style="width:50%;max-width:1083px"> </div> @@ -202,34 +203,34 @@ <a id="updateGB"></a> <h2>Updating the Latest UCSC Genome Browser Software</h2> <h3>Access the Docker Container's Shell</h3> <p>Updating the latest UCSC Genome Browser version will require access to the Docker container running shell (command-line interface) of the UCSC Genome Browser. The execute command can be run inside a running Docker container with the -it options. The -i or --interactive option allows interaction with the command being executed and keeps STDIN open even if not attached. This will allow input to be provided for the command. The -t or --tty option allocates a pseudo-TTY and allows for a more interactive experience. The following example shows how to run exec command and the -it options: <pre><code>docker exec -it <container_name_or_id> /bin/bash</code></pre></p> <h3>Update the Genome Browser Software</h3> <p>Running the following command updates the Genome Browser software: -<pre><code> bash root/browserSetup.sh cgiUpdate</code></pre></p> +<pre><code>bash /root/browserSetup.sh cgiUpdate</code></pre></p> -<h2>Customize a UCSC Genome Browser Docker Container</h2> <a id="hg.conf"></a> +<h2>Customize a UCSC Genome Browser Docker Container</h2> <h3>Editing hg.conf</h3> <p>The hg.conf file is a file that has information on how to connect to MariaDB, the location of the other directories and various other settings.</p> <p> The hg.conf file can be edited by running the execute command inside a running Docker container with the -it options. The -i or --interactive option allows interaction with the command being executed and keeps STDIN open even if not attached. This will allow input to be provided for the command. The -t or --tty option allocates a pseudo-TTY and allows for a more interactive experience. Any common text editors such as vi, nano, and vim can be used with the execute command and the -it options. The following example shows how to edit the hg.conf file using vi: <pre><code>docker exec -it <container_name_or_id> vi /usr/local/apache/cgi-bin/hg.conf</code></pre></p> <a id="defaults"></a> @@ -257,21 +258,103 @@ <li>Load the defaultCart.sql file as a table by running the following query:</p> <pre> mysql hgcentral < defaultCart.sql </pre></li> <li>Insert the session to the default cart table by using the user name and the session name, which was the session saved in the earlier step, and run the following query (add userName): <pre> mysql hgcentral -Ne "insert into defaultCart select contents from namedSessionDb where sessionName='nameOfSession' and userName='nameOfUser'" </pre></li> <li>Finally, make sure the following line is in your hg.conf file. This file is found in the cgi-bin directory, e.g. cgi-bin/hg.conf. <pre> defaultCartName=defaultCart </pre></li> </ul> - - - - +<a id="blatHub"></a> +<h2>Enabling Blat and In-Silico PCR on an Assembly Hub</h2> +<p>An <a href="assemblyHubHelp.html" target="_blank">assembly hub</a> can support Blat and +In-Silico PCR by running gfServer inside the running UCSC Genome Browser Docker container, then +pointing the hub's <a href="assemblyHubHelp.html#genomesTxt" target="_blank">genomes.txt</a> file +at gfServer.</p> +<p>gfServer only runs on Linux x86_64. On an arm64 host (such as an Apple Silicon Mac), start the +container with <code>--platform linux/amd64</code> instead of the normal command:</p> +<pre><code>docker run -d -p 8080:80 --platform linux/amd64 genomebrowser/server</code></pre> +<p>The UCSC Genome Browser and Blat software are free for academic, nonprofit, and personal use. +Commercial download and installation of the Blat and In-Silico PCR software may be licensed +through <a href="http://www.kentinformatics.com">Kent Informatics</a>.</p> + +<h3>Downloading the Example Assembly Hub</h3> +<p>An example plant assembly hub can be copied into a directory served by the container's Apache. +Open the container's shell and download the hub into +<code>/usr/local/apache/htdocs/folders</code>:</p> +<pre><code>docker exec -it <container_name_or_id> /bin/bash +mkdir -p /usr/local/apache/htdocs/folders +cd /usr/local/apache/htdocs/folders +wget -r --no-parent --reject "index.html*" -nH --cut-dirs=3 http://genome.ucsc.edu/goldenPath/help/examples/hubExamples/hubAssembly/plantAraTha1/</code></pre> + +<p>The hub can now be attached and loaded in one step, using whichever host port the container is +running on:</p> +<pre><code><a href="http://localhost:8080/cgi-bin/hgTracks?genome=araTha1&hubUrl=http://localhost/folders/hubExamples/hubAssembly/plantAraTha1/hub.txt&pix=800" +target="_blank">http://localhost:8080/cgi-bin/hgTracks?genome=araTha1&hubUrl=http://localhost/folders/hubExamples/hubAssembly/plantAraTha1/hub.txt&pix=800</a></code></pre> +<p>Note: <code>hubUrl</code> uses plain <code>localhost</code>, not <code>localhost:8080</code>. +This URL isn't loaded by your browser. It's loaded by the container itself, internally, where +Apache runs on port 80.</p> + +<h3>Installing gfServer Inside the Container</h3> +<p>The <a href="assemblyHubHelp.html#configuringAssemblyHubs" target="_blank">gfServer</a> utility is +the Blat server used by hgBlat and hgPcr. From the container's shell, the following commands create a +bin directory and install the tool:</p> +<pre><code>mkdir -p /root/bin +rsync -avP hgdownload.gi.ucsc.edu::genome/admin/exe/linux.x86_64/blat/gfServer /root/bin/ +export PATH=/root/bin:$PATH</code></pre> +<p>The last line adds <code>/root/bin</code> to the list of places the container's shell looks for +commands, so <code>gfServer</code> can be run by name.</p> + +<h3>Editing the Hub's genomes.txt</h3> +<p>The example hub has the Blat configuration lines commented out. Open the genomes.txt +file inside the container with a text editor such as vi, nano, or vim:</p> +<pre><code>cd /usr/local/apache/htdocs/folders/hubExamples/hubAssembly/plantAraTha1/ +vi genomes.txt</code></pre> +<p>Uncomment the following lines so the hub knows which ports to query for Blat, translated Blat, +and In-Silico PCR:</p> +<pre><code>blat localhost 17779 +transBlat localhost 17777 +isPcr localhost 17779</code></pre> +<p>All three point to <code>localhost</code> since the gfServer instances run inside this same +container.</p> +<p>If the hub has already been attached in the browser, it can take up to five minutes (300 +seconds) for the browser to pick up changes to genomes.txt. Appending +<code>&udcTimeout=10</code> to the URL shortens this delay. See the +<a href="hgTrackHubHelp.html#Debug" target="_blank">Debugging and Updating</a> section of the +Track Hub User Guide for more information.</p> + +<h3>Starting the gfServer Instances</h3> +<p>Two gfServer instances are required: one for translated (protein) Blat and one for untranslated +(DNA) Blat and In-Silico PCR. Change into the directory containing the assembly's 2bit file and +start both servers in the background:</p> +<pre><code>cd /usr/local/apache/htdocs/folders/hubExamples/hubAssembly/plantAraTha1/araTha1 +gfServer start localhost 17777 -trans -mask araTha1.2bit & +gfServer start localhost 17779 -stepSize=5 araTha1.2bit &</code></pre> + +<h3>Using Blat and In-Silico PCR</h3> +<p>With the hub connected as described above, the <em>Arabidopsis thaliana</em> assembly is now +available on the Blat and PCR pages within the same browser. On the Blat page, +<code><a href="http://localhost:8080/cgi-bin/hgBlat" target="_blank">http://localhost:8080/cgi-bin/hgBlat</a></code>, +the assembly can be searched with plant amino acid sequences such as +<code>IYQTRENKYIIGEIQITESERDRRRSSLPGNH</code> or DNA sequences such as +<code>TAAGTAAAAAATAATATGATTAAGACTAATAAATCTTAATAGTTAATACT</code>.</p> +<p>On the PCR page, +<code><a href="http://localhost:8080/cgi-bin/hgPcr" target="_blank">http://localhost:8080/cgi-bin/hgPcr</a></code>, +the same assembly can be searched with a forward primer such as +<code>TAGGTCTGCACCTGTGGTTCAAAATTTT</code> and a reverse primer such as +<code>CAATACAAGTCAACATTTTAGCGCCGAGA</code>, by clicking the "Flip Reverse Primer" box and +then clicking submit.</p> + +<h3>Keeping gfServer Running Across Container Restarts</h3> +<p>Stopping the container with <code>docker stop</code> also stops gfServer. After starting the +container again, re-enter the container's shell with <code>docker exec -it</code> and rerun the +two <code>gfServer start</code> commands, using the full path <code>/root/bin/gfServer</code> +instead of just <code>gfServer</code>, since the new shell will not have the earlier +<code>export PATH=/root/bin:$PATH</code> from the gfServer installation step.</p> <!--#include virtual="$ROOT/inc/gbPageEnd.html" -->