49c8c187d62eefad13f63e8e01fe12f820eb0ef9 lrnassar Tue May 28 15:02:26 2019 -0700 Updating api.html based on Ann suggestions #18869 diff --git src/hg/htdocs/goldenPath/help/api.html src/hg/htdocs/goldenPath/help/api.html index 5dfafdd..d043118 100755 --- src/hg/htdocs/goldenPath/help/api.html +++ src/hg/htdocs/goldenPath/help/api.html @@ -1,366 +1,366 @@ <!DOCTYPE html> <!--#set var="TITLE" value="JSON API data interface" --> <!--#set var="ROOT" value="../.." --> <!-- Relative paths to support mirror sites with non-standard GB docs install --> <!--#include virtual="$ROOT/inc/gbPageStart.html" --> <h1>JSON API data interface</h1> <h2>Contents</h2> <h6><a href="#Intro">What is JSON data?</a></h6> <h6><a href="#Access">What is the access URL?</a></h6> <h6><a href="#Return">What type of data can be accessed?</a></h6> <h6><a href="#Endpoint">Endpoint functions</a></h6> <h6><a href="#Parameters">Parameters to endpoint functions</a></h6> <h6><a href="#Parameter_use">Required and optional parameters</a></h6> <h6><a href="#Track_types">Supported track types</a></h6> <h6><a href="#list_examples">Example data access, list functions</a></h6> <h6><a href="#getData_examples">Example data access, getData functions</a></h6> <h6><a href="#Error_examples">Error return examples</a></h6> <h6><a href="#Practical_examples">Practical examples</a></h6> <!-- ========== What is JSON data? ============================== --> <a id="Intro"></a> <h2>What is JSON data?</h2> <p> JSON data is a data transfer syntax from a data provider to a data consumer. See also: <a href="https://www.w3schools.com/js/js_json_intro.asp" target=_blank>JSON Introduction</a> </p> <!-- ========== What is the access URL? ============================== --> <a id="Access"></a> <h2>What is the access URL?</h2> <p> -This access url: <b>https://api.genome.ucsc.edu/</b> is used to access +This access URL: <b>https://api.genome.ucsc.edu/</b> is used to access the endpoint functions. For example: <pre> wget -O- 'https://api.genome.ucsc.edu/list/publicHubs' </pre> </p> <!-- ========== What type of data can be accessed? ===================== --> <a id="Return"></a> <h2>What type of data can be accessed?</h2> <p> The following data sets can be accessed at this time: <ul> <li>List of available public hubs</li> <li>List of available UCSC Genome Browser genome assemblies</li> <li>List of genomes from a specified assembly or track hub</li> <li>List of available data tracks from a specified hub or UCSC Genome Browser genome assembly (see also: <a href='trackDb/trackDbHub.html' target=_blank>track definition help</a>)</li> <li>List of chromosomes contained in an assembly hub or UCSC Genome Browser genome assembly</li> <li>List of chromosomes contained in a specific track of an assembly or track hub, or UCSC Genome Browser genome assembly</li> <li>Return DNA sequence from an assembly hub 2bit file, or UCSC Genome Browser assembly</li> <li>Return track data from a specified assembly or track hub, or UCSC Genome Browser assembly</li> </ul> </p> <!-- ========== Endpoint functions ======================= --> <a id="Endpoint"></a> <h2>Endpoint functions to return data</h2> <p> -The url <b>https://api.genome.ucsc.edu/</b> is used to access +The URL <b>https://api.genome.ucsc.edu/</b> is used to access the endpoint functions. For example: <pre> curl -L 'https://api.genome.ucsc.edu/list/ucscGenomes' </pre> </p> <p> <ul> <li><b>/list/publicHubs</b> - list public hubs</li> <li><b>/list/ucscGenomes</b> - list UCSC Genome Browser database genomes</li> <li><b>/list/hubGenomes</b> - list genomes from specified hub</li> <li><b>/list/tracks</b> - list data tracks available in specified hub or database genome (see also: <a href='trackDb/trackDbHub.html' target=_blank>track definition help</a>)</li> <li><b>/list/chromosomes</b> - list chromosomes from a data track in specified hub or database genome</li> <li><b>/getData/sequence</b> - return sequence from specified hub or database genome</li> <li><b>/getData/track</b> - return data from specified track in hub or database genome</li> </ul> </p> <!-- ========== Parameters to endpoint functions ======================= --> <a id="Parameters"></a> <h2>Parameters to endpoint functions</h2> <p> <ul> <li>hubUrl=<url> - specify track hub or assembly hub URL</li> <li>genome=<name> - specify genome assembly in UCSC Genome Browser or track/assembly hub</li> <li>track=<trackName> - specify data track in track/assembly hub or UCSC database genome assembly</li> <li>chrom=<chrN> - specify chromosome name for sequence or track data</li> <li>start=<123> - specify start coordinate (0 relative) for data from track or sequence retrieval (start and end required together). See also: <a href='http://genome.ucsc.edu/blog/the-ucsc-genome-browser-coordinate-counting-systems/' target=_blank>UCSC browser coordinate counting systems</a></li> <li>end=<456> - specify end coordinate (1 relative) for data from track or sequence retrieval (start and end required together). See also: <a href='http://genome.ucsc.edu/blog/the-ucsc-genome-browser-coordinate-counting-systems/' target=_blank>UCSC browser coordinate counting systems</a></li> <li>maxItemsOutput=1000 - limit number of items to output, default: 1,000, maximum limit: 1,000,000 (use <em>-1</em> to get maximum output)</li> <li>trackLeavesOnly=1 - on <em>/list/tracks</em> function, only show tracks, do not show composite container information</li> <li>jsonOutputArrays=1 - on <em>/getData/track</em> function, JSON format is array type for each item of data, instead of the default object type</li> </ul> </p> <p> The parameters are added to the endpoint URL beginning with a question mark <b>?</b>, and multiple parameters are separated with the semi-colon <b>;</b>. For example: <pre> https://api.genome.ucsc.edu/getData/sequence?genome=hg38;chrom=chrM </pre> </p> <!-- ========== Required and optional parameters ======================= --> <a id="Parameter_use"></a> <h2>Required and optional parameters</h2> <p> <table> <tr><th>Endpoint function</th><th>Required</th><th>Optional</th></tr> <tr><th>/list/publicHubs</th><td>(none)</td><td>(none)</td></tr> <tr><th>/list/ucscGenomes</th><td>(none)</td><td>(none)</td></tr> <tr><th>/list/hubGenomes</th><td>hubUrl</td><td>(none)</td></tr> <tr><th>/list/tracks</th><td>genome or (hubUrl and genome)</td><td>trackLeavesOnly=1</td></tr> <tr><th>/list/chromosomes</th><td>genome or (hubUrl and genome)</td><td>track</td></tr> <tr><th>/getData/sequence</th><td>(genome or (hubUrl and genome)) and chrom</td><td>start and end</td></tr> <tr><th>/getData/track</th><td>(genome or (hubUrl and genome)) and track</td><td>chrom, (start and end), maxItemsOutput, jsonOutputArrays</td></tr> </table> </p> <p> The <b>hubUrl</b> and <b>genome</b> parameters are required together to specify a unique genome in an assembly or track hub. The <b>genome</b> for a track hub will usually be a UCSC database genome. Assembly hubs will have their own unique <b>genome</b> sequences. Specify <b>genome</b> without a <b>hubUrl</b> to refer to a UCSC Genome Browser assembly. </p> <p> Using the <b>chrom=<name></b> parameter will limit the request to the single specified chromosome. To limit the request to a specific position, both <b>start=4321</b> and <b>end=5678</b> must be given together. </p> <p> Any extra parameters not allowed in a function will be flagged as an error. </p> <!-- ========== Supported track types ======================= --> <a id="Track_types"></a> -<h2>Supported track types, at this time (April 2019), for getData functions</h2> +<h2>Supported track types for getData functions</h2> <p> <ul> <li><a href='/FAQ/FAQformat.html#format1' target=_blank>bed</a></li> <li><a href='/FAQ/FAQformat.html#format1.5' target=_blank>bigBed</a></li> <!-- not yet <li><a href='/FAQ/FAQformat.html#format9.1' target=_blank>bigGenePred</a></li> --> <li><a href='/FAQ/FAQformat.html#format9.5' target=_blank>bigNarrowPeak</a></li> <li><a href='/FAQ/FAQformat.html#format6.1' target=_blank>bigWig</a></li> <li><a href='/FAQ/FAQformat.html#format9' target=_blank>genePred</a></li> <li><a href='/FAQ/FAQformat.html#format9.5' target=_blank>narrowPeak</a></li> <li><a href='/FAQ/FAQformat.html#format2' target=_blank>psl</a></li> <li><a href='http://hgdownload.cse.ucsc.edu/goldenPath/hg38/database/rmsk.sql' target=_blank>rmsk - repeat masker</a></li> <li><a href='/FAQ/FAQformat.html#format6' target=_blank>wig</a></li> <li>(Work is underway to support additional track types)</li> </ul> </p> <!-- ========== Example data access ======================= --> <a id="list_examples"></a> <h2>Example data access</h2> <p> Your WEB browser can be configured to interpret JSON data and format in a convenient browsing format. Firefox has this function built in, other browsers have add-ons that can be turned on to format JSON data. With your browser thus configured, the following links can demonstrate the functions of the API interface. </p> <h3>Listing functions</h3> <ol> <li><a href='https://api.genome.ucsc.edu/list/publicHubs' target=_blank>list public hubs</a> - <b>api.genome.ucsc.edu/list/publicHubs</b></li> <li><a href='https://api.genome.ucsc.edu/list/ucscGenomes' target=_blank>list UCSC database genomes</a> - <b>api.genome.ucsc.edu/list/ucscGenomes</b></li> <li><a href='https://api.genome.ucsc.edu/list/hubGenomes?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt' target=_blank>list genomes from specified hub</a> - <b>api.genome.ucsc.edu/list/hubGenomes?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt</b></li> <li><a href='https://api.genome.ucsc.edu/list/tracks?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ' target=_blank>list tracks from specified hub and genome</a> - <b>api.genome.ucsc.edu/list/tracks?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ</b></li> <li><a href='https://api.genome.ucsc.edu/list/tracks?genome=hg38' target=_blank>list tracks from UCSC database genome</a> - <b>api.genome.ucsc.edu/list/tracks?genome=hg38</b></li> <li><a href='https://api.genome.ucsc.edu/list/chromosomes?genome=hg38' target=_blank>list chromosomes from UCSC database genome</a> - <b>api.genome.ucsc.edu/list/chromosomes?genome=hg38</b></li> <li><a href='https://api.genome.ucsc.edu/list/chromosomes?genome=hg38;track=gold' target=_blank>list chromosomes from specified track in UCSC database genome</a> - <b>api.genome.ucsc.edu/list/chromosomes?genome=hg38;track=gold</b></li> <li><a href='https://api.genome.ucsc.edu/list/chromosomes?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ' target=_blank>list chromosomes from assembly hub genome</a> - <br><b>api.genome.ucsc.edu/list/chromosomes?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ</b></li> <li><a href='https://api.genome.ucsc.edu/list/chromosomes?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=assembly' target=_blank>list chromosomes from specified track in assembly hub genome</a> - <br><b>api.genome.ucsc.edu/list/chromosomes?hubUrl=hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=assembly</b></li> </ol> <a id="getData_examples"></a> <h3>getData functions</h3> <ol> <li><a href='https://api.genome.ucsc.edu/getData/sequence?genome=hg38;chrom=chrM' target=_blank>Get DNA sequence from specified chromosome in UCSC database genome</a> - <br><b>api.genome.ucsc.edu/getData/sequence?genome=hg38;chrom=chrM</b></li> <li><a href='https://api.genome.ucsc.edu/getData/sequence?genome=hg38;chrom=chrM;start=4321;end=5678' target=_blank>Get DNA sequence from specified chromosome and start,end coordinates in UCSC database genome</a> - <br><b>api.genome.ucsc.edu/getData/sequence?genome=hg38;chrom=chrM;start=4321;end=5678</b></li> <li><a href='https://api.genome.ucsc.edu/getData/sequence?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=mm10;chrom=chrM;start=4321;end=5678' target=_blank>Get DNA sequence from a track hub where 'genome' is a UCSC database</a> - <br><b>api.genome.ucsc.edu/getData/sequence?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=mm10;chrom=chrM;start=4321;end=5678</b></li> <li><a href='https://api.genome.ucsc.edu/getData/sequence?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;chrom=chr1;start=4321;end=5678' target=_blank>Get DNA sequence from specified chromosome and start,end coordinates in an assembly hub genome</a> - <br><b>api.genome.ucsc.edu/getData/sequence?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;chrom=chr1;start=4321;end=5678</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?genome=hg38;track=gold;maxItemsOutput=100' target=_blank>Get track data for specified track in UCSC database genome</a> - <br><b>api.genome.ucsc.edu/getData/track?genome=hg38;track=gold;maxItemsOutput=100</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?genome=hg38;track=gold;chrom=chrM' target=_blank>Get track data for specified track and chromosome in UCSC database genome</a> - <br><b>api.genome.ucsc.edu/getData/track?genome=hg38;track=gold;chrom=chrM</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?genome=hg38;track=gold;chrom=chr1;start=47000;end=48000' target=_blank>Get track data for specified track, chromosome and start,end coordinates in UCSC database genome</a> - <br><b>api.genome.ucsc.edu/getData/track?genome=hg38;track=gold;chrom=chr1;start=47000;end=48000</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=assembly' target=_blank>Get track data for specified track in an assembly hub genome</a> - <br><b>api.genome.ucsc.edu/getData/track?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=assembly</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?hubUrl=http://genome-test.gi.ucsc.edu/~hiram/hubs/Plants/hub.txt;genome=araTha1;track=assembly;chrom=chrCp' target=_blank>Get track data for specified track and chromosome in an assembly hub genome</a> - <br><b>api.genome.ucsc.edu/getData/track?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=assembly;chrom=chr1</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=ensGene' target=_blank>Get track data for specified track in a track hub</a> - <br><b>api.genome.ucsc.edu/getData/track?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=ensGene</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=ensGene;chrom=chr1' target=_blank>Get track data for specified track and chromosome in a track hub</a> - <br><b>api.genome.ucsc.edu/getData/track?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=ensGene;chrom=chr1</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=gc5Base;chrom=chr1;start=4321;end=5678' target=_blank>Wiggle track data for specified track, chromosome with start and end limits in an assembly hub genome</a> - <br><b>api.genome.ucsc.edu/getData/track?hubUrl=hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=gc5Base;chrom=chr1;start=4321;end=5678</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?genome=galGal6;track=gc5BaseBw;maxItemsOutput=100' target=_blank>Wiggle track data for specified track in a UCSC database genome</a> - <br><b>api.genome.ucsc.edu/getData/track?genome=galGal6;track=gc5BaseBw;maxItemsOutput=100</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?genome=galGal6;track=ncbiRefSeqOther;chrom=chr1;start=750000;end=55700000' target=_blank>bigBed data from a UCSC database, chrom and start,end limits</a> - <br><b>api.genome.ucsc.edu/getData/track?genome=galGal6;track=ncbiRefSeqOther;chrom=chr1;start=750000;end=55700000</b></li> </ol> <p> </p> <a id="Error_examples"></a> <h3>Error return examples</h3> <p> <ol> <li><a href='https://api.genome.ucsc.edu/getData/track?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=assembly;chrom=chrI;start=43521;end=54321' target=_blank>Request track data for non-existent chromosome in an assembly hub genome</a> - <br><b>api.genome.ucsc.edu/getData/track?hubUrl=http://hgdownload.soe.ucsc.edu/hubs/mouseStrains/hub.txt;genome=CAST_EiJ;track=assembly;chrom=chrI;start=43521;end=54321</b></li> <li><a href='https://api.genome.ucsc.edu/getData/track?genome=hg19;track=decipherSnvs' target=_blank>Request track data from a restricted track</a>. See <a href='../../FAQ/FAQdownloads.html#download40' target=_blank>FAQ</a> - <br><b>api.genome.ucsc.edu/getData/track?genome=hg19;track=decipherSnvs</b></li> </ol> </p> <!-- ========== Practical Examples ======================= --> <a id="Practical_examples"></a> <h2>Practical examples</h2> <h3>Looking up the schema of a specific track</h3> <p> When querying track data with the <b>/getData/track</b> function, the <b>jsonOutputArrays</b> can be used in conjunction to see the track schema. This includes a description of each field present in the track. The data will also be returned in JSON array type.</p> <p> <a href='https://api.genome.ucsc.edu/getData/track?genome=hg38;track=gold;chrom=chrM;jsonOutputArrays=1' target=_blank>Request data from hg38 gold track in array type</a> - <br><b>api.genome.ucsc.edu/getData/track?genome=hg38;track=gold;chrom=chrM;jsonOutputArrays=1</b></p> <h3>Hide track container information with trackLeavesOnly parameter</h3> <p> When using the <b>/list/tracks</b> function to see the available tracks in an assembly, it can be useful to return all tracks in the same hierarchical level. By default, composite and supertracks will have the subtracks nested below, however, the <b>trackLeavesOnly=1</b> parameter can be passed to hide the container information and display all tracks and subtracks at the same level.</p> <p> In the following example, the first link does not include the <b>trackLeavesOnly</b> parameter. The output can be compared to the second link to see the difference, which can be observed in the conservation track. In the first link, the <i>multiz20way</i> track is nested within the <i>cons20way</i> track. In the second link, however, the <i>multiz20way</i> subtrack is seen at an equivalent level with all other tracks, and the container, <i>cons20way</i>, is not present in the list.</p> <p> <a href='https://api.genome.ucsc.edu/list/tracks?genome=rn6' target=_blank>Request available tracks in the rn6 genome</a> - <br><b>api.genome.ucsc.edu/list/tracks?genome=rn6</b></p> <p> <a href='https://api.genome.ucsc.edu/list/tracks?genome=rn6;trackLeavesOnly=1' target=_blank>Request available tracks in the rn6 genome, hiding container information</a> - <br><b>api.genome.ucsc.edu/list/tracks?genome=rn6;trackLeavesOnly=1</b></p> <h3>Requesting track data with over one million (1M) items in output</h3> <p> Certain tracks may contain over 1M items. When these tracks are queried using the <b>/getData/track</b> function, only the first million items are returned. The API assumes this default value of 1M unless a different value (less than 1M) is specified with the parameter <b>maxItemsOutput</b>.</p> <p> One of these tracks is the <i>knownGene</i> track for hg19. Removing the <b>maxItemsOutput</b> parameter from the following link will lead to a 384Mb download, and may cause certain web browsers to time out.</p> <p> <a href='https://api.genome.ucsc.edu/getData/track?genome=hg19;track=knownGene;maxItemsOutput=5' target=_blank>Request items in knownGene track of hg19, remove maxItemsOutput parameter for 1M max return</a> - <br><b>api.genome.ucsc.edu/getData/track?genome=hg19;track=knownGene;maxItemsOutput=5</b></p> <p> There are different ways around this item limit, depending on how many items are in the track. For the <i>knownGene</i> track, breaking it down to component chromosome queries using the <b>chrom</b> parameter will suffice. In order to get a listing of the chrom names, and what chroms have data for that track, the <b>/list/chromosomes</b> function can be used.</p> <p> <a href='https://api.genome.ucsc.edu/list/chromosomes?genome=hg19;track=knownGene' target=_blank>Request listing of chroms that have data for the knownGene track in hg19</a> - <br><b>api.genome.ucsc.edu/list/chromosomes?genome=hg19;track=knownGene</b></p> <p> With the list of chrom names that have data, the <b>/getData/track</b> function can be used -again while specifying the <b>chrom</b> parameter. In the following example chr1 is queried -and the <i>itemsReturned</i> field shows a total of 7967 items in the output. Well below the +again while specifying the <b>chrom</b> parameter. In the following example, chr1 is queried +and the <i>itemsReturned</i> field shows a total of 7967 items in the output, well below the 1M limit, meaning all data for chr1 has been extracted. This can then be repeated for all chroms of interest.</p> <p> <a href='https://api.genome.ucsc.edu/getData/track?genome=hg19;track=knownGene;chrom=chr1' -target=_blank>Request iems in knownGene track of hg19, only for chr1</a> - +target=_blank>Request items in knownGene track of hg19, only for chr1</a> - <br><b>api.genome.ucsc.edu/getData/track?genome=hg19;track=knownGene;chrom=chr1</b></p> <p> For tracks that have additional items, such as SNP tracks, the query can be further broken down using the additional <b>start</b> and <b>end</b> parameters.</p> <!--#include virtual="$ROOT/inc/gbPageEnd.html" -->