title: eHive: an artificial intelligence workflow system for genomic analysis.
abstract: BACKGROUND
  The Ensembl project produces updates to its comparative genomics resources with each of its several releases per year. During each release cycle approximately two weeks are allocated to generate all the genomic alignments and the protein homology predictions. The number of calculations required for this task grows approximately quadratically with the number of species. We currently support 50 species in Ensembl and we expect the number to continue to grow in the future.
  
  
  RESULTS
  We present eHive, a new fault tolerant distributed processing system initially designed to support comparative genomic analysis, based on blackboard systems, network distributed autonomous agents, dataflow graphs and block-branch diagrams. In the eHive system a MySQL database serves as the central blackboard and the autonomous agent, a Perl script, queries the system and runs jobs as required. The system allows us to define dataflow and branching rules to suit all our production pipelines. We describe the implementation of three pipelines: (1) pairwise whole genome alignments, (2) multiple whole genome alignments and (3) gene trees with protein homology inference. Finally, we show the efficiency of the system in real case scenarios.
  
  
  CONCLUSIONS
  eHive allows us to produce computationally demanding results in a reliable and efficient way with minimal supervision and high throughput. Further documentation is available at: http://www.ensembl.org/info/docs/eHive/.
authors.affiliation: European Bioinformatics Institute, Wellcome Trust Genome Campus, Cambridge, UK.
authors.forename: Jessica
authors.initials: J
authors.lastname: Severin
authors.forename: Kathryn
authors.initials: K
authors.lastname: Beal
authors.forename: Albert J
authors.initials: AJ
authors.lastname: Vilella
authors.forename: Stephen
authors.initials: S
authors.lastname: Fitzgerald
authors.forename: Michael
authors.initials: M
authors.lastname: Schuster
authors.forename: Leo
authors.initials: L
authors.lastname: Gordon
authors.forename: Abel
authors.initials: A
authors.lastname: Ureta-Vidal
authors.forename: Paul
authors.initials: P
authors.lastname: Flicek
authors.forename: Javier
authors.initials: J
authors.lastname: Herrero
mesh_terms: D030541:Databases, Genetic; D016678:Genome; D023281:Genomics / Q000379:methods*; D012984:Software
references.citation: Genome Res. 2008 Nov;18(11):1829-43
references.pmid: 18849525
references.citation: Genome Res. 2009 Feb;19(2):327-35
references.pmid: 19029536
references.citation: Nucleic Acids Res. 2006 Jul 1;34(Web Server issue):W729-32
references.pmid: 16845108
references.citation: Proc Natl Acad Sci U S A. 2003 Sep 30;100(20):11484-9
references.pmid: 14500911
references.citation: BMC Genomics. 2009 Jan 14;10:22
references.pmid: 19144180
references.citation: Genome Res. 2002 Apr;12(4):656-64
references.pmid: 11932250
references.citation: J Mol Biol. 1990 Oct 5;215(3):403-10
references.pmid: 2231712
references.citation: Nature. 2007 May 10;447(7141):167-77
references.pmid: 17495919
references.citation: Genome Res. 2008 Nov;18(11):1814-28
references.pmid: 18849524
references.citation: Nucleic Acids Res. 2009 Jan;37(Database issue):D690-7
references.pmid: 19033362
references.citation: Genome Res. 2005 Jul;15(7):901-13
references.pmid: 15965027
references.citation: Nucleic Acids Res. 2008 Jan;36(Database issue):D735-40
references.pmid: 18056084
references.citation: Methods Mol Biol. 2007;395:221-36
references.pmid: 17993677
references.citation: Genome Res. 2003 Apr;13(4):721-31
references.pmid: 12654723
references.citation: Curr Biol. 2006 Mar 7;16(5):460-71
references.pmid: 16458514