295b9dbab341deac96c7f6166bea56f0fc7fa770 max Fri Feb 12 05:38:32 2021 -0800 changes after code review, refs #26951. Since all README files were merged into mirrorManual.txt in 2016, to avoid further confusion, I am removing them now. I manually merged all changes made since then (just 3-4 changes) into mirrorManual.txt now. mirrorManual.txt is automatically converted to https://genome.ucsc.edu/goldenPath/help/mirrorManual.html by the makefile in mirrorDocs. This makes sure that we have installation instructions that can be read with vi or less and at the same time we have a nice html file that is easy to read, looks like official documentation and is indexed by Google well. The big no.1 advantage of mirrorManual.txt is that the sections have an order in increasing complexity and the user knows where to start. The old README files had no order at all and the names were misleading (e.g. quickstart was not about a quickstart) diff --git src/product/README.building.source src/product/README.building.source deleted file mode 100644 index 73df1e1..0000000 --- src/product/README.building.source +++ /dev/null @@ -1,289 +0,0 @@ - -Building the kent source tree. - -Note the scripts available in the ./scripts/ directory here -that can run all of these steps for you with the -script: scripts/kentSrcUpdate.sh -Following the instructions in scripts/README - -1. Check your shell environment. It should - contain a MACHTYPE variable. Typical values, for example: - i386 i686 sparc alpha x86_64 ppc - If there is none, set it to indicate your - CPU architecture, for example, - MACHTYPE=i686 - export MACHTYPE - You need to do this if your existing MACHTYPE is a long string - such as: i686-pc-linux-gnu - since this is used as an argument on the command line to gcc - which will not work with the - signs in the strings. - Typical values: i386 i686 x86_64 sparc ppc - - To determine the machine cpu type of your computer, try - one of these uname command options: - $ uname -m - $ uname -p - $ uname -a - - Remember to set this MACHTYPE in your .bashrc or .tcshrc home directory - environment files so it will be set properly at your next login. - - The browser code has not been tested by UCSC outside a - CentOS Linux on Intel or AMD Opteron environment. - Other users do report successful operation on other systems. - - Depending upon what libraries you have installed locally, you may want - to set environment variable USE_SAMTABIX. - - See also: - http://genomewiki.ucsc.edu/index.php/Build_Environment_Variables - -2. Create a directory where binaries will be moved to during - the build of the source tree: - $ mkdir -p ~/bin/${MACHTYPE} - -3a. ALTERNATE PATH: - If you are going to do a full build anyway, skip this and proceed straight to step 3 below. - Otherwise, to make a minimal utility build without mysql: - There are some utilities that depend only on jkweb.a and not jkhgap.a - which means they can be compiled without needing mysql client installed. - To make a utility like pslCDnaFilter without installing mysql client: - # create jkweb.a - cd kent/src/lib - make - # create stringify utility required by some makefiles - cd kent/src/utils/stringify - make - # create pslCDnaFilter utility program - cd kent/src/hg/pslCDnaFilter - make - Proceed similarly for any other such utilities. - You are done and can stop here. - -3. Create the following shell environment variables: - - NOTE: As of mid-October 2013 the makefile build system in the source - tree will attempt to configure MYSQLINC and MYSQLLIBS with - the mysql_config command. To use that automatic configuration, - eliminate any MYSQLINC and MYSQLLIBS definitions from your shell - environment and make sure mysql_config can be found in your PATH. - This option is not perfect and may not function correctly. - It usually will not use a static linked library which can lead - to a known issue on the Mac OSX. See 'Known Problems' item 10 below. - - In the case the automatic configuration does not function, you - can set these variables in your shell environment to override - the automatic configuration: - - MYSQLINC=/usr/include/mysql - MYSQLLIBS="/usr/lib/mysql/libmysqlclient.a -lz" - export MYSQLINC MYSQLLIBS - Your setting may vary depending upon where your - mysql is installed. The above example is typical. - If your system does not have this set of include files - or this static client.a file, you may need to install - the mysql-devel package to obtain the mysql development - environment. (http://dev.mysql.com/downloads/) - With that package installed, this command: - mysql_config --libs - will display the appropriate libraries to link with - for your system configuration. And: - mysql_config --include - will display the appropriate MYSQLINC directory. - The -lz requires a link to the libraries installed in the - zlib-devel rpm. - -4a. Required SSL support: - In order for the libraries to be able to use SSL, - for instance to support fetching HTTPS URLs, - install openssl. We are currently using these packages: - openssl-1.0.1e - openssl-devel-1.0.1e - SSL is also useful for accessing bigWig and bigBed over HTTPS. - -4b. optional SAM/BAM and tabix (for VCF) support: - If you want to display BAM and VCF tracks, download and build samtabix - (a merge of Heng Li's samtools and tabix libraries): - git clone git://genome-source.soe.ucsc.edu/samtabix.git samtabix - cd samtabix - make - Add the following environment variables to .bashrc or .tcshrc, - OR alternatively add these lines at the top of kent/src/inc/common.mk, - replacing "/path/to/samtabix" with your local samtabix directory: - USE_SAMTABIX=1 - SAMTABIXDIR=/path/to/samtabix - -4. In the source tree, perform the following: - $ cd kent/src - $ make libs - At this point, you can build utilities in other parts of - the source tree if that is your goal. Go to the directory - of the utility command you want to build and run a 'make' - in that directory. The resulting binary will be placed - in $HOME/bin/$MACHTYPE - -5. To continue building the CGI binaries - $ cd kent/src/hg - $ make compile - $ make install DESTDIR=/destination/prefix - - Where "/destination/prefix" is your choice, and - this will install the cgi binaries in: - /destination/prefix/usr/local/apache/cgi-bin - - If your cgi-bin directory is something different than - /usr/local/apache/cgi-bin then use the CGI_BIN variable, e.g.: - make install DESTDIR=/var/www CGI_BIN=/cgi-bin \ - DOCUMENTROOT=/var/www/html - - You can save yourself time and trouble if your Apache is somewhere - other than at /usr/local/apache by creating that directory and making - symlinks to your actual apache directories. For example: - mkdir /usr/local/apache - ln -s /var/www/cgi-bin /usr/local/apache/cgi-bin - ln -s /var/www/html /usr/local/apache/htdocs - ln -s /var/www/cgi-bin-${LOGNAME} /usr/local/apache/cgi-bin - - With those symlinks in place, a simple 'make cgi' can be used - instead of the 'make compile; make install DESTDIR=...' business. - - If your apache DocumentRoot is something different than - /usr/local/apache/htdocs then use the DOCUMENTROOT variable. - This value should be a full path and should agree with the - browser.documentRoot setting in hg.conf; see README.mysql.setup - for more details. - - $ make install DESTDIR=/destination/prefix CGI_BIN=/cgi-bin/path DOCUMENTROOT=/usr/local/apache/htdocs - - to install binaries in "/destination/prefix/cgi-bin/path" - - [NOTE: These 'make' commands assume gnu make is being used] - -6. After source tree updates, the make sequence is: - $ cd kent/src - $ make clean - $ make libs - $ cd hg - $ make compile - $ make install DESTDIR=/destination/prefix - -==================================================================== -Known problems: - -1. A compile/link of a utility has complaints about undefined symbols - such as: bind, accept, listen, gethostbyname, socket, setsockopt, - connect, inet_pton, inet_ntop or h_errno - To fix this, you need to add '-lsocket -lnsl' to your MYSQLLIBS - environment variable. This is typically the case on Solaris systems: - MYSQLLIBS="/usr/lib/mysql/libmysqlclient.a -lsocket -lnsl -lz" - -2. The build fails immediately in the first src/lib/ directory with - the compiler issuing a warning about unused variables. - Some newer versions of gcc issue these warnings and the - src/inc/common.mk file specifies -Werror which causes it to - exit on these warnings. Either remove the -Werror specifications - in src/inc/common.mk or add the -Wno-unused-variable to instruct - the compiler to allow these warnings without an exit. - -3. The build fails during a link of any program under the src/hg/ - hierarchy with an error something like: - undefined reference to `SSL_CTX_free' - undefined reference to `ERR_get_error_line_data' - undefined reference to `SSL_read' - undefined reference to `SSL_get_error' - undefined reference to `SSL_write' - This error is due to your mysql libraries have been compiled with SSL - functionality enabled. To fix this build problem, add '-lssl' - to your MYSQLLIBS environment variable to satisify these SSL - library functions. - -4. Build fails on Macintosh with an error: -aliType.c:5: warning: `rcsid' defined but not used -make: *** [aliType.o] Error 1 - - The OSTYPE environment variable needs to be set to "darwin". If your - shell is bash, it is a shell local variable instead of an environment - variable as with tcsh. - - To avoid this error, place an export statement in your $HOME/.bashrc - environment: - export OSTYPE - This causes the OSTYPE variable to be recognized during the - make and the conditional statements in src/inc/common.mk will - include the -Wno-unused-variable option in the build. - -5. The build fails with an error: - gifdecomp.c:274: error: call to function 'gif_out_line' without - a real prototype - - This happens with newer versions of the gcc compiler. To avoid the - build stopping at this point, eliminate the -Werror argument from the - src/inc/common.mk file. We will fix this error as soon as we - get our hands on a newer version of the gcc compiler. - -6. The build fails with an error of this type: - pipeline.c: In function waitOnExec: - pipeline.c:351: error: ignoring return value of read, declared with - attribute warn_unused_result - - This happens with newer versions of the gcc compiler. To avoid the - build stopping at this point, eliminate the -Werror argument from the - src/inc/common.mk file. We will fix this error as soon as we - get our hands on a newer version of the gcc compiler. - -7. Build fails complaining about missing png.h file: - pngwrite.c:7:87: error: png.h: No such file or directory - - Find out where your png.h file is and set PNGINCL to something like: - export PNGINCL="-I/opt/local/include/libpng14" - pointing to the directory where png.h exists. - -8. Build fails at the point of building a CGI binary with the error: - Undefined symbols: - "_png_write_png", referenced from: - _mgSaveToPng in jkweb.a(pngwrite.o) - ... other png symbols ... - - You need to find out where your libpng.a is located. For example - it may be in /opt/local/lib/libpng.a When found, use the PNGLIB - variable to specify it, for example: - export PNGLIB=/opt/local/lib/libpng.a - or equivalent: - export PNGLIB='-L/opt/local/lib -lpng' - -9. Build fails with complains about missing functions dlclose, dlopen: - /usr/lib/x86_64-linux-gnu/libmysqlclient.a(client_plugin.c.o): In function `add_plugin': - (.text+0x1ed): undefined reference to `dlclose' - /usr/lib/x86_64-linux-gnu/libmysqlclient.a(client_plugin.c.o): In function `mysql_client_plugin_deinit': - (.text+0x28b): undefined reference to `dlclose' - /usr/lib/x86_64-linux-gnu/libmysqlclient.a(client_plugin.c.o): In function `mysql_load_plugin_v': - (.text+0x51e): undefined reference to `dlopen' - - To add the library that includes these functions, add '-ldl' to the - MYSQLLIBS string: MYSQLLIBS="/usr/lib/mysql/libmysqlclient.a -ldl -lz" - -10. Mac OSX dynamic link with the MySQL libraries results in the following - run-time error for CGI binaries or command line programs: - - dyld: Library not loaded: libmysqlclient.18.dylib - - This is a known problem with the dynamic library for MySQL, - with potential work-arounds offered: - http://bugs.mysql.com/bug.php?id=61243 - - Or, you can set the MYSQLINC and MYSQLLIBS shell environment variables - to the static MySQL library as mentioned in Step 3 above. - -11. Genome browser issues error: - - "PDF format not available" when trying to export to pdf. - - The browser can't find the command 'ps2pdf' which is usually - found in /usr/bin/ps2pdf and is installed with the 'ghostscript' - package. Install the 'ghostscript' package and verify ps2pdf is - in: /usr/bin/ps2pdf - -==================================================================== -For more details see: -http://genome-source.soe.ucsc.edu/gitlist/kent.git/blob/master/src/product/README.building.source -====================================================================