16a0bb49dfcbf27c039d99e4ab40a41c59814400 max Fri Dec 11 05:56:03 2020 -0800 a quickstart section for the bigBed.html page, may be not optimal yet, but to illustrate the idea more diff --git src/hg/htdocs/goldenPath/help/bigBed.html src/hg/htdocs/goldenPath/help/bigBed.html index d86e687..05f6bee 100755 --- src/hg/htdocs/goldenPath/help/bigBed.html +++ src/hg/htdocs/goldenPath/help/bigBed.html @@ -19,104 +19,120 @@ access to a web-accessible server and need hosting space for your bigBed files, please see the <a href="hgTrackHubHelp.html#Hosting">Hosting</a> section of the Track Hub Help documentation.</p> <p> Additional indices can be created for the items in a bigBed file to support item search in track hubs. See <a href="#Ex3">Example #3</a> below for an example of how to build an additional index.</p> <p> See <a href="http://genomewiki.ucsc.edu/index.php/Selecting_a_graphing_track_data_format" target="_blank">this wiki page</a> for help in selecting the graphing track data format that is most appropriate for your type of data.</p> <p> Note that the <code>bedToBigBed</code> utility uses a substantial amount of memory: approximately 25% more RAM than the uncompressed BED input file.</p> <p> -<h2>Creating a bigBed track</h2> +<h2>Quickstart example commands</h2> + +It is not hard to create a bigBed file. The following UNIX commands create one on a Linux machine and are +explained in more detail in the following sections on this page: + +<pre><code>wget https://genome.ucsc.edu/goldenPath/help/examples/bedExample.txt +wget https://genome.ucsc.edu/goldenPath/help/hg19.chrom.sizes +wget http://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64/bedToBigBed +chmod a+x bedTod +bedToBigBed bedExample.txt hg19.chrom.sizes myBigBed.bb +mv myBigBed.bb ~/public_html/ +</code></pre> + +The last step assumes that your ~/public_html/ directory is accessible from the internet. This may not be the case +on your server. You may have to copy the file to another server and web-accessible location at your University. +Once you know the URL to the file myBigBed.bb, you can paste this URL into the <a href="../../cgi-bin/hgCustom">custom track</a> +box on the UCSC Genome Browser to display the file. + +<h2>Overview of the steps to create a bigBed track</h2> <p> -To create a bigBed track, follow these steps:<p> +To create a bigBed track, follow these steps (for concrete Unix commands, see the examples below on this page):<p> <p> <strong>Step 1.</strong> Create a BED format file following the directions <a href="../../FAQ/FAQformat.html#format1">here</a>. When converting a BED file to a bigBed file, you are limited to one track of data in your input file; therefore, you must create a separate BED file for each data track. Your BED file must be sorted first on the <code>chrom</code> field, and -secondarily on the <code>chromStart</code> field. You can use utility <code>bedSort</code> -available <a href="http://hgdownload.soe.ucsc.edu/admin/exe/" target="_blank">here</a> -or the following UNIX <code>sort</code> -command to do this: </p> -<pre><code>sort -k1,1 -k2,2n unsorted.bed > input.bed</code></pre> +secondarily on the <code>chromStart</code> field. </p> <p> -<strong>Step 2.</strong> -Remove any existing "track" or "browser" lines from your BED file so that it +If your BED file was originally a custom track, remove any existing +"track" or "browser" lines from your BED file so that it contains only data.</p> + <p> <strong>Step 3.</strong> Download the <code>bedToBigBed</code> program from the -<a href="http://hgdownload.soe.ucsc.edu/admin/exe/">binary utilities directory</a>.</p> +<a href="http://hgdownload.soe.ucsc.edu/admin/exe/">binary utilities directory</a>. +Example #2 below shows the exact Unix command.</p> + <p> +The <code>bedToBigBed</code> program can be run with several additional options. Some of these, +such as the <code>-as</code> and <code>-type</code> options, are used in examples below. The +<code>-type</code> option, describes the size of the bigBed file, <code>-type=bedN[+[P]]</code>, +where <code>N</code> is an integer between 3 and 12 and the optional <code>+[P]</code> parameter +specifies the number of extra fields, not required, but preferred. Describing the size of the bigBed file +is needed for access to extra fields like name, itemRgb, etc. +Examples:<code>-type=bed6</code> or <code>-type=bed6+</code> or <code>-type=bed6+3 </code>. +For a full list of the available options, type <code>bedToBigBed</code> (with no arguments) on the +command line to display the usage message.</p> + <strong>Step 4.</strong> Use the <code>fetchChromSizes</code> script from the <a href="http://hgdownload.soe.ucsc.edu/admin/exe/">same directory</a> to create the <em>chrom.sizes</em> file for the UCSC database you are working with (e.g., hg19). If the assembly <code>genNom</code> is hosted by UCSC, chrom.sizes can be a URL like: -<code>http://hgdownload.soe.ucsc.edu/goldenPath/genNom/bigZips/genNom.chrom.sizes</code></p> +<code>http://hgdownload.soe.ucsc.edu/goldenPath/genNom/bigZips/genNom.chrom.sizes</code>. +</p> <p> <strong>Step 5.</strong> Use the <code>bedToBigBed</code> utility to create a bigBed file from your sorted BED file, using the <em>input.bed</em> file and <em>chrom.sizes</em> files created in <em>Steps 1</em> and <em>4</em>:</p> <pre><code><strong>bedToBigBed</strong> input.bed chrom.sizes myBigBed.bb</code></pre> <p> <strong>Step 6.</strong> Move the newly created bigBed file (<em>myBigBed.bb</em>) to a web-accessible http, https, or ftp location. At this point you should have a URL to your data, such as "https://institution.edu/myBigBed.bb", and the file should be accessible outside of your institution/hosting providers network. For more information on where to host your data, please see the <a href="hgTrackHubHelp.html#Hosting">Hosting</a> section of the Track Hub Help documentation.</p> <p> <strong>Step 7.</strong> If the file name ends with a <em>.bigBed</em> or <em>.bb</em> suffix, you can paste the URL of the file directly into the <a href="../../cgi-bin/hgCustom">custom track</a> management page, click "submit" and view the file as a track in the Genome Browser. By default, the file name will be used to name the track. To configure the track name and descriptions, you must create a "<a href="hgTracksHelp.html#TRACK">track line</a>", as shown in <em>Step 8</em>.</p> <p> -<strong>Step 8.</strong> -Construct a <a href="hgTracksHelp.html#CustomTracks">custom track</a> using a single track line. +Alternatively, if you want to set the track labels and other options yourself, +construct a <a href="hgTracksHelp.html#CustomTracks">custom track</a> using a single track line. Note that any of the track attributes listed <a href="customTrack.html#TRACK">here</a> are applicable to tracks of type bigBed. The most basic version of the track line will look something like this:</p> <pre><code><strong>track type=</strong>bigBed <strong>name=</strong>"My Big Bed" <strong>description=</strong>"A Graph of Data from My Lab" <strong>bigDataUrl=</strong>http://myorg.edu/mylab/myBigBed.bb</code></pre> <p> Paste this custom track line into the text box on the <a href="../../cgi-bin/hgCustom">custom track</a> management page.</p> -<p> -The <code>bedToBigBed</code> program can be run with several additional options. Some of these, -such as the <code>-as</code> and <code>-type</code> options, are used in examples below. The -<code>-type</code> option, describes the size of the bigBed file, <code>-type=bedN[+[P]]</code>, -where <code>N</code> is an integer between 3 and 12 and the optional <code>+[P]</code> parameter -specifies the number of extra fields, not required, but preferred. Describing the size of the bigBed file -is needed for access to extra fields like name, itemRgb, etc. -Examples:<code>-type=bed6</code> or <code>-type=bed6+</code> or <code>-type=bed6+3 </code>. -For a full list of the available options, type <code>bedToBigBed</code> (with no arguments) on the -command line to display the usage message.</p> - - <h2>Examples</h2> <h3>Example #1</h3> <p> -In this example, you will create a bigBed custom track using an existing bigBed file, +In this example, you will load an existing bigBed file, <em>bigBedExample.bb</em>, on the UCSC http server. This file contains chromosome 21 data on the human hg19 assembly.</p> <p> To create a custom track using this bigBed file:</p> <ol> <li> Paste the URL <code>http://genome.ucsc.edu/goldenPath/help/examples/bigBedExample.bb</code> into the <a href="../../cgi-bin/hgCustom">custom track</a> management page for the human assembly hg19 (Feb. 2009).</li> <li> Click the "submit" button. </li> <li> On the next page that displays, click the "chr21" link in the custom track listing to view the bigBed track in the Genome Browser. </li> </ol> @@ -141,30 +157,36 @@ <h3>Example #2</h3> <p> In this example, you will create your own bigBed file from an existing bed file.</p> <ol> <li> Save the BED file <a href="examples/bedExample.txt"><em>bedExample.txt</em></a> to a server, ideally one that is accessible from the internet. (<em>Steps 1</em> and <em>2</em> in <em>Creating a bigBed track</em>, above).<br> <pre><code>wget https://genome.ucsc.edu/goldenPath/help/examples/bedExample.txt</code></pre> </li> <li> Save the file <a href="hg19.chrom.sizes"><em>hg19.chrom.sizes</em></a> to your computer. It contains the chrom.sizes data for the human (hg19) assembly (<em>Step 4</em>, above).<br> <pre><code>wget https://genome.ucsc.edu/goldenPath/help/hg19.chrom.sizes</code></pre> </li> + <li>If you use your own file, it has to be sorted. You can use the utility <code>bedSort</code> +available <a href="http://hgdownload.soe.ucsc.edu/admin/exe/" target="_blank">here</a> +or the following UNIX <code>sort</code> +command to do this: </p> +<pre><code>sort -k1,1 -k2,2n unsorted.bed > input.bed</code></pre> + </li> <li> Download the <code>bedToBigBed</code> utility (<em>Step 3</em>, above). Replace "linux" below with "macOSX" if your server is a Mac. <pre><code>wget http://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64/bedToBigBed chmod a+x bedToBigBed</code></pre> </li> <li> Run the utility to create the bigBed output file (<em>Step 5</em>, above): <pre><code><strong>bedToBigBed</strong> bedExample.txt hg19.chrom.sizes myBigBed.bb</code></pre></li> <li> Place the bigBed file you just created (<em>myBigBed.bb</em>) on a web-accessible server (<em>Step 6</em>, above).<br> <pre><code>mv myBigBed.bb ~/public_html/</code></pre> At some Universities, this involves using the commands ftp, scp or rsync to copy the file to a different server, one that is accessible from the internet. We have <a href="hgTrackHubHelp.html#Hosting">documentation</a> how to find such a server. @@ -247,32 +269,38 @@ <li> When you click on features, the contents of all extra fields are shown as a table. You can modify the layout of the resulting page with the trackDb settings "<a href="trackDb/trackDbHub.html#skipFields">skipFields</a>", "<a href="trackDb/trackDbHub.html#sepFields">sepFields</a>" and "<a href="trackDb/trackDbHub.html#skipEmptyFields">skipEmptyFields</a>", and transform text fields into links with the "<a href="trackDb/trackDbHub.html#urls">urls</a>" trackDb setting.</li> <li> Extra fields that start with the character "_" are reserved for internal use (special display code); their contents are not shown on the details page.</li> </ol> <h2>Sharing Your Data with Others</h2> <p> -If you would like to share your bigBed data track with a colleague, learn how to create a URL by -looking at Example #6 on <a href="customTrack.html#EXAMPLE6">this page</a>.</p> +If you would like to share your bigBed data track with a colleague, the best solution is to +save your current view as a stable <a href="hgSessionHelp.html">Genome Browser Session Link</a>. +This will save the position and all settings that you made, all track visibilities, filters, +highlights, etc. +</p> +<p> +If you want to create URLs programmatically from software, look at Example #6 on <a href="customTrack.html#EXAMPLE6">this page</a>. +</p> <h2>Extracting Data from the bigBed Format</h2> <p> Because the bigBed files are indexed binary files, it can be difficult to extract data from them. UCSC has developed the following programs to assist in working with bigBed formats, available from the <a href="http://hgdownload.soe.ucsc.edu/admin/exe/">binary utilities directory</a>:</p> <ul> <li> <code>bigBedToBed</code> — converts a bigBed file to ASCII BED format.</li> <li> <code>bigBedSummary</code> — extracts summary information from a bigBed file.</li> <li> <code>bigBedInfo</code> — prints out information about a bigBed file.</li> </ul> <p>