Commits for kent

v282_preview2 to v282_base (2013-04-09 to 2013-04-16) v282

• Adding makeSymLink
  • src/inc/portable.h - lines changed 3, context: html, text, full: html, text
  • src/lib/osunix.c - lines changed 8, context: html, text, full: html, text
• Added sqlQuickLongLong
  • src/hg/inc/jksql.h - lines changed 3, context: html, text, full: html, text
  • src/hg/lib/jksql.c - lines changed 15, context: html, text, full: html, text
• Moving some code from edwSubmit to lib so as to share. Adding edwGetLocalFile routine.
  • src/hg/encode3/encodeDataWarehouse/edwSubmit/edwSubmit.c - lines changed 154, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/inc/edwLib.h - lines changed 33, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/edwLib.c - lines changed 277, context: html, text, full: html, text
• Adding a bunch of new tables for automatic QA efforts.
  • src/hg/encode3/encodeDataWarehouse/inc/encodeDataWarehouse.h - lines changed 224, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.as - lines changed 48, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.c - lines changed 528, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.sql - lines changed 52, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/redoSchema - lines changed 6, context: html, text, full: html, text
• Added new assembly loader for encode data warehouse
  • src/hg/encode3/encodeDataWarehouse/edwAddAssembly/edwAddAssembly.c - lines changed 65, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwAddAssembly/makefile - lines changed 4, context: html, text, full: html, text
• Making reset script start up database with hg19 and mm9
  • src/hg/encode3/encodeDataWarehouse/lib/resetEdw - lines changed 2, context: html, text, full: html, text
• Fixed bug where bed loader was segfaulting on beds with less than 9 fields.
  • src/lib/basicBed.c - lines changed 1, context: html, text, full: html, text
• Adding cgiParsedVars - something that makes it easier to use the cgi var parser.
  • src/inc/cheapcgi.h - lines changed 15, context: html, text, full: html, text
  • src/lib/cheapcgi.c - lines changed 34, context: html, text, full: html, text
• Polishing bits of schema to do with encodeQa, and refactoring for qa pipeline.
  • src/hg/encode3/encodeDataWarehouse/edwAddAssembly/edwAddAssembly.c - lines changed 5, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwSubmit/edwSubmit.c - lines changed 20, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/inc/encodeDataWarehouse.h - lines changed 468, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.as - lines changed 43, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.c - lines changed 1168, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.sql - lines changed 44, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/redoSchema - lines changed 2, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/resetEdw - lines changed 6, context: html, text, full: html, text
• Getting started on first pass of after-upload validation.
  • src/hg/encode3/encodeDataWarehouse/edwMakeValidFile/edwMakeValidFile.c - lines changed 162, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwMakeValidFile/makefile - lines changed 4, context: html, text, full: html, text
• Program to add subscriber - a program called when files are added.
  • src/hg/encode3/encodeDataWarehouse/edwAddSubscriber/makefile - lines changed 4, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwAddSubscriber/test.ra - lines changed 2, context: html, text, full: html, text
• Program to load a new enrichment target bed to database.
  • src/hg/encode3/encodeDataWarehouse/edwAddQaEnrichTarget/edwAddQaEnrichTarget.c - lines changed 94, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwAddQaEnrichTarget/makefile - lines changed 4, context: html, text, full: html, text
• Putting in a bunch of new edw-related programs.
  • src/hg/encode3/encodeDataWarehouse/makefile - lines changed 5, context: html, text, full: html, text
• Learning pipeline api again.
  • src/hg/oneShot/freen/freen.c - lines changed 21, context: html, text, full: html, text
• Adding mapRatio field to validFile
  • src/hg/encode3/encodeDataWarehouse/inc/encodeDataWarehouse.h - lines changed 2, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.as - lines changed 1, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.c - lines changed 15, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.sql - lines changed 1, context: html, text, full: html, text
• Adding suffix or two back to edwFileName so auto uncompression works.
  • src/hg/encode3/encodeDataWarehouse/inc/edwLib.h - lines changed 4, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/edwLib.c - lines changed 45, context: html, text, full: html, text
• Adding suffix or two back to edwFileName so auto uncompression works.
  • src/hg/encode3/encodeDataWarehouse/edwSubmit/edwSubmit.c - lines changed 8, context: html, text, full: html, text
• Making makeValidFile count up items and create subsample files for fastq's and bams.
  • src/hg/encode3/encodeDataWarehouse/edwMakeValidFile/edwMakeValidFile.c - lines changed 197, context: html, text, full: html, text
• Starting to work on next pass - the part that calculates enrichments. Possibly have it running bwa twice in pipe to produce a sam file.
  • src/hg/encode3/encodeDataWarehouse/edwMakeEnrichments/edwMakeEnrichments.c - lines changed 144, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwMakeEnrichments/makefile - lines changed 4, context: html, text, full: html, text
• Adding samToBed and samToOpenBed functions.
  • src/inc/bamFile.h - lines changed 6, context: html, text, full: html, text
  • src/lib/bamFile.c - lines changed 54, context: html, text, full: html, text
• Made rTempName tolerate a trailing slash in dir name.
  • src/lib/osunix.c - lines changed 3, context: html, text, full: html, text
• Renaming samplePath to sampleBed since that's what it always is now. Adding flag 'gotMapRatio' so you can tell when that field is valid.
  • src/hg/encode3/encodeDataWarehouse/inc/encodeDataWarehouse.h - lines changed 4, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.as - lines changed 3, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.c - lines changed 37, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.sql - lines changed 3, context: html, text, full: html, text
• Adding edwValDataDir global.
  • src/hg/encode3/encodeDataWarehouse/inc/edwLib.h - lines changed 1, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/edwLib.c - lines changed 1, context: html, text, full: html, text
• Switched to just system() calls with temp files from Mark's pipeline after some difficulty getting pipeline to behave robustly. Seems to work now.
  • src/hg/encode3/encodeDataWarehouse/edwMakeValidFile/edwMakeValidFile.c - lines changed 95, context: html, text, full: html, text
• Look like have it calculating enrichments for bam and fastq files, but still need to save them in database.
  • src/hg/encode3/encodeDataWarehouse/edwMakeEnrichments/edwMakeEnrichments.c - lines changed 158, context: html, text, full: html, text
• Making scan through sam file keep a little more information.
  • src/hg/encode3/encodeDataWarehouse/edwMakeValidFile/edwMakeValidFile.c - lines changed 93, context: html, text, full: html, text
• Computing sample coverage.
  • src/hg/encode3/encodeDataWarehouse/edwMakeValidFile/edwMakeValidFile.c - lines changed 8, context: html, text, full: html, text
• Seems to work for fastq and bam.
  • src/hg/encode3/encodeDataWarehouse/edwMakeEnrichments/edwMakeEnrichments.c - lines changed 89, context: html, text, full: html, text
• Adding some more bed3 routines - for the minimalists.
  • src/inc/basicBed.h - lines changed 12, context: html, text, full: html, text
  • src/lib/basicBed.c - lines changed 53, context: html, text, full: html, text
• Removing gotMapRatio field, now we always have it.
  • src/hg/encode3/encodeDataWarehouse/inc/encodeDataWarehouse.h - lines changed 2, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.as - lines changed 1, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.c - lines changed 17, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.sql - lines changed 1, context: html, text, full: html, text
• Adding a few views to db init script.
  • src/hg/encode3/encodeDataWarehouse/lib/resetEdw - lines changed 4, context: html, text, full: html, text
• Adding very light clipping to sampleBed so there are no negative chromStarts.
  • src/hg/encode3/encodeDataWarehouse/edwMakeValidFile/edwMakeValidFile.c - lines changed 2, context: html, text, full: html, text
• Making enrichments work for bigBed files.
  • src/hg/encode3/encodeDataWarehouse/edwMakeEnrichments/edwMakeEnrichments.c - lines changed 200, context: html, text, full: html, text
• Making so a few more bigBeds get labeled as such in manifest - the ones made from GTF and BEDs in particular.
  • src/hg/encode3/importEncode2/encode2MakeEncode3/encode2MakeEncode3.c - lines changed 2, context: html, text, full: html, text
• Made bigBedInfo -extraIndex give you item count and field offset too.
  • src/utils/bigBedInfo/bigBedInfo.c - lines changed 6, context: html, text, full: html, text
• Sorting before building index.
  • src/utils/bedToBigBed/bedToBigBed.c - lines changed 2, context: html, text, full: html, text
• Adding files view and mouse targets for enrichment analysis.
  • src/hg/encode3/encodeDataWarehouse/lib/resetEdw - lines changed 5, context: html, text, full: html, text
• Making it so mouse and human have separate enrichment targets. Adding bigWig enrichment support.
  • src/hg/encode3/encodeDataWarehouse/edwMakeEnrichments/edwMakeEnrichments.c - lines changed 118, context: html, text, full: html, text
• Bringing structure commenting up to standards.
  • src/hg/encode3/encodeDataWarehouse/edwMakeEnrichments/edwMakeEnrichments.c - lines changed 4, context: html, text, full: html, text
• Changing 'regulatory' enrichment to 'open.'
  • src/hg/encode3/importEncode2/encode2Manifest/encode2Manifest.c - lines changed 4, context: html, text, full: html, text
• Adding mustRename wrapper around rename().
  • src/inc/portable.h - lines changed 3, context: html, text, full: html, text
  • src/lib/osunix.c - lines changed 8, context: html, text, full: html, text
• Improving docs a hair.
  • src/hg/encode3/importEncode2/README - lines changed 4, context: html, text, full: html, text
• adding database parameter to command line.
  • src/hg/encode3/importEncode2/encode2Meta/encode2Meta.c - lines changed 20, context: html, text, full: html, text
• Removing licensePlate from edwFile - it doesn't happen until validation.
  • src/hg/encode3/encodeDataWarehouse/inc/encodeDataWarehouse.h - lines changed 3, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.as - lines changed 2, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.c - lines changed 42, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/encodeDataWarehouse.sql - lines changed 2, context: html, text, full: html, text
• Implementing a number of Stanford's suggestions including not assigning license plate until later (which means files need another name before validation), and changing what internally used files are plated and what are not. Now submission files are not plated, but files used in QA pipeline are
  • src/hg/encode3/encodeDataWarehouse/edwAddAssembly/edwAddAssembly.c - lines changed 18, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwAddQaEnrichTarget/edwAddQaEnrichTarget.c - lines changed 39, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwMakeEnrichments/edwMakeEnrichments.c - lines changed 32, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwMakeValidFile/edwMakeValidFile.c - lines changed 130, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwSubmit/edwSubmit.c - lines changed 18, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/inc/edwLib.h - lines changed 21, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/edwLib.c - lines changed 139, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/resetEdw - lines changed 10, context: html, text, full: html, text
• Improving commenting.
  • src/hg/encode3/encodeDataWarehouse/edwSubmit/edwSubmit.c - lines changed 8, context: html, text, full: html, text
• Starting to do replicate QA.
  • src/hg/encode3/encodeDataWarehouse/edwMakeReplicateQa/edwMakeReplicateQa.c - lines changed 105, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/edwMakeReplicateQa/makefile - lines changed 4, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/makefile - lines changed 2, context: html, text, full: html, text
• Adding some functions for fetching edwFile and edwValidFile from database.
  • src/hg/encode3/encodeDataWarehouse/inc/edwLib.h - lines changed 17, context: html, text, full: html, text
  • src/hg/encode3/encodeDataWarehouse/lib/edwLib.c - lines changed 68, context: html, text, full: html, text
• Making edwSubmit not start fetching a file it is already fetching in another thread so often
  • src/hg/encode3/encodeDataWarehouse/edwSubmit/edwSubmit.c - lines changed 8, context: html, text, full: html, text
• Making it avoid validating something already validated.
  • src/hg/encode3/encodeDataWarehouse/edwMakeValidFile/edwMakeValidFile.c - lines changed 66, context: html, text, full: html, text
• Handling gtf and gff file enrichments.
  • src/hg/encode3/encodeDataWarehouse/edwMakeEnrichments/edwMakeEnrichments.c - lines changed 23, context: html, text, full: html, text