295b9dbab341deac96c7f6166bea56f0fc7fa770 max Fri Feb 12 05:38:32 2021 -0800 changes after code review, refs #26951. Since all README files were merged into mirrorManual.txt in 2016, to avoid further confusion, I am removing them now. I manually merged all changes made since then (just 3-4 changes) into mirrorManual.txt now. mirrorManual.txt is automatically converted to https://genome.ucsc.edu/goldenPath/help/mirrorManual.html by the makefile in mirrorDocs. This makes sure that we have installation instructions that can be read with vi or less and at the same time we have a nice html file that is easy to read, looks like official documentation and is indexed by Google well. The big no.1 advantage of mirrorManual.txt is that the sections have an order in increasing complexity and the user knows where to start. The old README files had no order at all and the names were misleading (e.g. quickstart was not about a quickstart) diff --git src/product/README.install src/product/README.install deleted file mode 100644 index 6347476..0000000 --- src/product/README.install +++ /dev/null @@ -1,229 +0,0 @@ - -Installing the UCSC Genome browser - -This file is from: - http://genome-source.soe.ucsc.edu/gitlist/kent.git/blob/master/src/product/README.install - -Scripts to perform all of the functions below can be found in -the directory here: ./scripts/ -Please see the ./scripts/README file for more information. - -1. Confirm the following: - a. Apache WEB server is installed and working - http://localhost/ - Provides the Apache default home page from your machine - NOTE: The browser static html WEB pages require the Apache - XBitHack option to be enabled to allow: - <!--#include ... --> statements to function. - Add 'Options +Includes' for your html directory, your - httpd.conf file entry looks like: -XBitHack on -<Directory /usr/local/apache/htdocs> - Options +Includes -</Directory> - - You can test your Apache cgi-bin/ directory with the script: - scripts/printEnv.pl - - b. MySQL database is installed and working - $ mysql -u browser -pgenome -e 'show tables;' mysql - MySQL can be run from the command line, and - the tables from the database mysql can be displayed. - See also: README.mysql.setup - c. MySQL development package is installed (mysql-devel) - The directory: /usr/include/mysql/ has the mysql .h files - And the library: /usr/lib/mysql/libmysqlclient.a exists - (your exact pathnames may vary depending upon your installation) - -2. Set MySQL database access permissions. The examples mentioned - in the README.mysql.setup instructions will allow this - setup to function as described here. - - To setup the example user accounts as mentioned in these - instructions, run the script: - - ex.MySQLUserPerms.sh - -3. Find the location of your Apache WEB server DocumentRoot - and cgi-bin directory. - Typical locations are: /var/www and /usr/local/apache - /var/www/html - /var/www/cgi-bin - The directory where these are located is referred to as - WEBROOT in this documentation: - WEBROOT=/var/www - export WEBROOT - The browser WEB pages and cgi-bin binaries expect these - two directories to be next to each other in ${WEBROOT} - since referrals in html are often: "../cgi-bin" - - The browser should function even if WEBROOT is in a different - directory from the primary Apache web root. In this case, - the three directories: html cgi-bin and trash should be - at the same level in this other WEBROOT. For example: - /some/other/directory/path/html/ - /some/other/directory/path/cgi-bin/ - /some/other/directory/path/trash/ - Symlinks to the trash directory should exist from the html - directory. As so: - /some/other/directory/path/html/trash -> ../trash - The actual trash directory can be somewhere else. If it is - not in your Apache /var/www/trash/ directory, then create - that symlink as well as the html/trash symlink. For example - /var/www/trash -> /some/other/directory/trash - /var/www/html/trash -> /some/other/directory/trash - -4. Create html, cgi-bin and trash directories: - mkdir ${WEBROOT}/html - mkdir ${WEBROOT}/cgi-bin - chmod 755 ${WEBROOT}/cgi-bin - (this chmod 755 will prevent suexec failures that are indicated - by "Premature end of script headers" errors in the Apache - error_log. Your cgi binaries should also be 755 permissions.) - - mkdir ${WEBROOT}/trash - chmod 777 ${WEBROOT}/trash - ln -s ${WEBROOT}/trash ${WEBROOT}/html/trash - - The browser creates .png (and other) files in the trash directory. - The 'chmod 777' allows the Apache WEB server to write into - that directory. - - A cron job should be set to periodically clean the files in trash. - See also, the two scripts here: - scripts/trashCleanMonitor.csh - scripts/trashCleaner.csh - -5. Download static WEB page content: - See also: scripts/updateHtml.sh - -6. Copy CGI binaries: - rsync -avP rsync://hgdownload.soe.ucsc.edu/cgi-bin/ ${WEBROOT}/cgi-bin/ - This set of binaries are for x86_64 types of Linux machines. - If you need to instead build binaries for your platform, - follow the instructions in: README.building.source - See also: scripts/kentSrcUpdate.sh - -7. Create hgcentral database and tables. This is the primary gateway - database that allows the browser to find specific organism - databases. See also: scripts/fetchHgCentral.sh to fetch - a current copy of hgcentral.sql - - mysql -u browser -pgenome -e "create database hgcentral;" - mysql -u browser -pgenome hgcentral < hgcentral.sql - - Please note, it is possible to create alternative hgcentral - databases. For example, for test purposes. In this - case use a unique name for the hgcentral database, such - as "hgcentraltest", and it can be specified in the hg.conf - file as mentioned in the next step. To create a second copy - of the hgcentral database: - - mysql -u browser -pgenome -e "create database hgcentraltest;" - mysql -u browser -pgenome hgcentraltest < hgcentral.sql - -8. Create the hg.conf file in ${WEBROOT}/cgi-bin/hg.conf - to allow the CGI binaries to find the hgcentral database - - Use the file here: ex.hg.conf - as the beginning template for your system: - - Copy the sample hg.conf: - cp ex.hg.conf ${WEBROOT}/cgi-bin/hg.conf - - Please edit this hg.conf file and set any parameters required - for your installation. Use the comments in that file as your guide. - - Browser developers will want a copy of this file in - their home directory with mode 600 and named: - ~/.hg.conf - These copies may have different db.user specification - to allow developers write access to the database. - See also: README.mysql.setup - -10. Load databases of interest. See also: README.QuickStart - scripts/activeDbList.sh - scripts/minimal.db.list.txt - scripts/loadDb.sh - And discussion in scripts/README about whether you can use directly - the MySQL binary database files, or if you need to download goldenPath - database text dumps and load them into the database. - - An alternative to loading the database tables from text files, - is to directly rsync the MySQL tables themselves and place them - in your MySQL /var/ directory. These tables are much larger - than the text files due to the sizes of indexes created during a - table load, but it can save a lot of time since the data loading - step is quite compute intensive. A typical rsync command for an - entire database (e.g. ce4) would be something like: - rsync -avP --delete --max-delete=20 \ - rsync://hgdownload.soe.ucsc.edu/mysql/ce4/ \ - /var/lib/mysql/ce4/ - -11. Download extra databases to work with a full genome assembly - such as human/hg38: hgFixed go140213 proteins140122 sp140122 - Construct symlinks in your MySQL data directory to use database - names: go proteome uniProt for these database directories: - -$ ls -og proteome go uniProt -lrwxrwxrwx 1 8 Feb 26 11:39 go -> go140213 -lrwxrwxrwx 1 14 Mar 27 12:01 proteome -> proteins140122 -lrwxrwxrwx 1 8 Mar 27 12:01 uniProt -> sp140122 - -$ ls -ld go140213 proteins140122 sp140122 -drwx------ 2 mysql mysql 4096 Feb 26 10:57 go140213 -drwx------ 2 mysql mysql 4096 Aug 19 08:08 proteins140122 -drwx------ 2 mysql mysql 4096 Mar 26 13:01 sp140122 - - These file names are data stamped YYMMDD to indicate changes - over time as they are updated with new builds of the UCSC gene track. - When a new UCSC gene track is released, fetch new databases and - change the symlink. - -12. Copy the gbdb data to /gbdb - See also: - scripts/fetchFullGbdb.sh - scripts/fetchMinimalGbdb.sh - -13. The browser should now appear at the URL: - http://localhost/ - - Check your Apache error_log file for hints to solving problems. - -14. blat server setup - The blatServers table in the database hgcentral needs to - have a fully qualified host name specified in the 'host' column. - Educational and non-profit institutions are allowed to use - blat free of charge. Commercial installations of the browser - require a license for blat. See also: http://kentinformatics.com/index.html - and: http://genome.ucsc.edu/license/ - In the source tree: src/gfServer/README.blat - -15. Useful links: - - http://genomewiki.ucsc.edu/index.php/Category:Mirror_Site_FAQ - - There are numerous README files in the source tree on - a variety of specific subjects, e.g.: - ./src/README - ./src/product/README.* - ./src/hg/makeDb/trackDB/README - ./src/hg/makeDb/doc/make*.txt - -16. Apache configuration: - - To lock down your trash directory from scanning via "indexes" - enter the following in your httpd.conf: - -<Directory "/var/www/html/trash"> - Options MultiViews - AllowOverride None - Order allow,deny - Allow from all -</Directory> - - The specified directory name is your apache: DocumentRoot/trash - e.g. /usr/local/apache/htdocs/trash - -==================================================================== -Information last updated: 19 August 2014 -====================================================================