2353ec21c6649e1de33b43bd1b3b304603ac71f1 mspeir Thu Oct 1 15:31:50 2026 -0700 Document how to set the order of the bars on a barChart track, refs #37619 Adds a "Setting the order of the bars" section to the barChart format page. It explains that the order is a property of the data file rather than of the display, that barChartBars, barChartCategoryUrl, barChartColors and the barChartStatsUrl rows are all positional and have to move together, and then gives three recipes: barChartReorder on a barChart file, the same wrapped in bigBedToBed and bedToBigBed for a bigBarChart, and expMatrixToBarchartBed --groupOrderFile for anyone who is still at the matrix stage and can get the order right the first time. The barChartBars entry in trackDbLibrary.shtml gains a line saying the labels are positional, with a link to the new section. Also corrects the bedToBigBed command in Example 4 to pass -sort. expMatrixToBarchartBed writes its rows in item-name order, since it joins on name, so bedToBigBed rejects the file as unsorted as soon as the two orders disagree. The example has always worked on the three-row file we host, whose rows happen to be in position order already, and fails on real data. Checked the claims against the code rather than against the existing prose: barChartTrack.c walks expScores and the category list in step at increasing x with no sort anywhere, createCategs numbers the categories in label order, facetedTableSelectOffsets hands back the stats row index as the expScores offset, and getSampleValsFromFile matches samples to categories by name, which is why the matrix and sample files need no changes. Every command and both example orderings were run against the files already published under goldenPath/help/examples/barChart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> diff --git src/hg/htdocs/goldenPath/help/barChart.html src/hg/htdocs/goldenPath/help/barChart.html index 88be11e9118..21dcb19ba63 100755 --- src/hg/htdocs/goldenPath/help/barChart.html +++ src/hg/htdocs/goldenPath/help/barChart.html @@ -306,33 +306,36 @@ <ol> <li>If not already completed, follow steps 1-4 from <a href="#example3">Example 3</a> above, or download the example bed6+5 file <a href="examples/barChart/hg38.gtexTranscripts.bed">here</a> </li> <li>Download the <em>fetchChromSizes</em> and <em>bedToBigBed</em> programs from the <a href="http://hgdownload.gi.ucsc.edu/admin/exe">utilities directory</a> appropriate to your operating system. <li>Use <em>fetchChromSizes</em> to create a <em>chrom.sizes</em> file for the UCSC database you are working with (hg38 for these examples). Alternatively, you can download the <em>chrom.sizes</em> file for any assembly hosted at UCSC from our <a href="http://hgdownload.gi.ucsc.edu/downloads.html">downloads</a> page (click on "Full data set" for any assembly). For example, the <em>hg38.chrom.sizes</em> file for the hg38 database is located at <a href="http://hgdownload.gi.ucsc.edu/goldenPath/hg38/bigZips/hg38.chrom.sizes" target="_blank">http://hgdownload.gi.ucsc.edu/goldenPath/hg38/bigZips/hg38.chrom.sizes</a>.</li> <li>Save the autoSql file <a href="examples/barChart/barChartBed.as">barChartBed.as</a> to your computer.</li> - <li>Run bedToBigBed to create the bigBarChart file: + <li>Run bedToBigBed to create the bigBarChart file. <em>expMatrixToBarchartBed</em> writes + its rows in item-name order rather than chromosome order, so include the <code>-sort</code> + option and let bedToBigBed put them in position order for you. Without it the program stops + with "is not sorted at line ..." as soon as the two orders disagree: <pre><code> -bedToBigBed -as=barChartBed.as -type=bed6+5 inputBed hg38.chrom.sizes output.bigBed +bedToBigBed -sort -as=barChartBed.as -type=bed6+5 inputBed hg38.chrom.sizes output.bigBed </code></pre></li> <li>Move the newly constructed bigBarChart file to a web accessible http, https, or ftp location.</li> <li>Construct a custom track line with a bigDataUrl parameter pointing to the newly created bigBarChart file. If the matrix and category files used to make the precursor barChart file are also moved to an http, https, or ftp location, we can point to them on the custom track line as well (all settings must be on the same line): <pre><code> track type=bigBarChart name="bigBarChart Example One" description="A bigBarChart file" barChartBars="adiposeSubcut breastMamTissue colonTransverse muscleSkeletal wholeBlood" barChartMetric=median barChartUnit=RPKM bigDataUrl=http://genome.ucsc.edu/goldenPath/help/examples/barChart/hg38.gtexTranscripts.bb barChartMatrixUrl=http://genome.ucsc.edu/goldenPath/help/examples/barChart/exampleMatrix.txt barChartSampleUrl=http://genome.ucsc.edu/goldenPath/help/examples/barChart/exampleSampleData.txt visibility=pack </code></pre> @@ -547,30 +550,119 @@ simpleBarChartBed.as</a> can be used with the <code>bedToBigBed</code> command to create the bigBarChart file. <pre> bedToBigBed myTissueComp.bed hg38.chrom.sizes myTissueComp.bb -type=bed6+3 -as=simpleBarChartBed.as pass1 - making usageList (1 chroms): 15 millis pass2 - checking and writing primary data (5 records, 9 fields): 1 millis </pre> <!-- <p> You can then view the bigBarChart custom track with the following track line: </p> <pre> track name="My singleCell barChart" type=bigBarChart bigDataUrl=https://hgwdev.gi.ucsc.edu/goldenPath/help/examples/barChart/singleCell/myTissueComp.bb </pre> --> +<a name="barOrder"></a> +<h2>Setting the order of the bars</h2> +<p> +Bars are drawn left to right in the order their values appear in the expScores field. Nothing +re-sorts them in the browser image, so the order is a property of the data file itself. +The labels are positional in the same way: the first name in the +<a href="trackDb/trackDbHub.html#barChartBars" target="_blank">barChartBars</a> setting, or the +first line of the file named by +<a href="trackDb/trackDbHub.html#barChartCategoryUrl" target="_blank">barChartCategoryUrl</a>, +belongs to the first value in expScores, the second to the second, and so on. The colors in +<a href="trackDb/trackDbHub.html#barChartColors" target="_blank">barChartColors</a> and the rows +of the statistics file named by +<a href="trackDb/trackDbHub.html#barChartStatsUrl" target="_blank">barChartStatsUrl</a> are read +the same way, one per bar.</p> +<p> +Datasets often read better in an order other than the alphabetical one most tools produce, for +instance tissues grouped by the biology rather than by the first letter of their names. Changing +the order means moving the values inside the expScores field of every row, and moving the labels, +colors and statistics to match. The <em>barChartReorder</em> program does the first part and +prints the <code>barChartBars</code> line for the second. It works on any barChart BED file, +whatever produced it, so it does not matter whether you started from an expression matrix, from +<em>matrixToBarChartBed</em>, from your own pipeline, or from a file somebody sent you.</p> + +<h6 id="barOrderScript">The barChartReorder utility</h6> +<p> +Download <em>barChartReorder</em> from the +<a href="http://hgdownload.gi.ucsc.edu/admin/exe" target="_blank">utilities directory</a> +appropriate for your operating system, and make sure it is on your PATH as described in +<a href="#example3">Example #3</a>. Run it with no arguments for a usage message.</p> +<p> +It takes two files that list the bar names, one name per line: the order the file is in now, and +the order you want. Both must name the same bars. Pass the BED file to rewrite and the file to +write, and optionally a <em>.categories</em> file to put into the same order:</p> +<pre><code>cat oldOrder.txt +adiposeSubcut +breastMamTissue +colonTransverse +muscleSkeletal +wholeBlood + +cat newOrder.txt +wholeBlood +muscleSkeletal +adiposeSubcut +breastMamTissue +colonTransverse + +barChartReorder oldOrder.txt newOrder.txt input.bed reordered.bed \ + --categories old.categories --outCategories new.categories +barChartBars wholeBlood muscleSkeletal adiposeSubcut breastMamTissue colonTransverse +</code></pre> +<p> +The last line is the program's output. Paste it into the track, and put +<code>barChartColors</code> and the +<a href="trackDb/trackDbHub.html#barChartStatsUrl" target="_blank">barChartStatsUrl</a> file into +the same order by hand. The program stops with an error rather than writing a half-correct file if +the two order files disagree, if a row does not hold as many values as there are names, or if the +categories file is missing one of the bars.</p> + +<h6 id="barOrderBigBed">Reordering a bigBarChart file</h6> +<p> +A bigBarChart file is a binary index, so convert it to text first and rebuild it afterwards. Use +the same schema you started from, which you can recover from the file itself with +<code>bigBedInfo -as input.bb</code>:</p> +<pre><code>bigBedToBed input.bb input.bed +barChartReorder oldOrder.txt newOrder.txt input.bed reordered.bed +bedToBigBed -sort -as=barChartBed.as -type=bed6+5 reordered.bed hg38.chrom.sizes output.bb +</code></pre> + +<h6 id="barOrderBuild">Choosing the order when you first build the file</h6> +<p> +If you are building from an expression matrix as in <a href="#example3">Example #3</a>, you can +get the order right at the start instead and skip the step above. +<em>expMatrixToBarchartBed</em> groups the samples alphabetically by category name unless you give +it <code>--groupOrderFile</code>, which takes the same kind of file as <em>newOrder.txt</em> +above:</p> +<pre><code>expMatrixToBarchartBed --groupOrderFile newOrder.txt \ + exampleSampleData.txt exampleMatrix.txt hg38.gtexTranscripts out.bed +</code></pre> +<p> +List every category that appears in the sample file: one left out is dropped from the output +without any warning, and a name in the order file that is not in the sample file stops the program +with an error. The order it used becomes the header line of the output file, and +<code>--verbose</code> prints it to the screen as well.</p> +<p> +In either case the matrix file and the sample file are left alone. The samples in them are matched +to categories by name rather than by position, so the boxplot on the details page needs no +changes.</p> + <h2>Sharing your data with others</h2> <p> If you would like to share your barChart/bigBarChart data track with a colleague, learn how to create a URL by looking at Example 6 on <a href="customTrack.html#EXAMPLE6">this page</a>.</p> <h2>Extracting data from the bigBarChart format</h2> <p> Because bigBarChart files are an extension of bigBed files, which are indexed binary files, it can be difficult to extract data from them. UCSC has developed the following programs to assist in working with bigBed formats, available from the <a href="http://hgdownload.gi.ucsc.edu/admin/exe/">binary utilities directory</a>.</p> <ul> <li> <code>bigBedToBed</code> — converts a bigBed file to ASCII BED format.</li> <li>