title: EGASP: the human ENCODE Genome Annotation Assessment Project.
abstract: BACKGROUND
  We present the results of EGASP, a community experiment to assess the state-of-the-art in genome annotation within the ENCODE regions, which span 1% of the human genome sequence. The experiment had two major goals: the assessment of the accuracy of computational methods to predict protein coding genes; and the overall assessment of the completeness of the current human genome annotations as represented in the ENCODE regions. For the computational prediction assessment, eighteen groups contributed gene predictions. We evaluated these submissions against each other based on a 'reference set' of annotations generated as part of the GENCODE project. These annotations were not available to the prediction groups prior to the submission deadline, so that their predictions were blind and an external advisory committee could perform a fair assessment.
  
  
  RESULTS
  The best methods had at least one gene transcript correctly predicted for close to 70% of the annotated genes. Nevertheless, the multiple transcript accuracy, taking into account alternative splicing, reached only approximately 40% to 50% accuracy. At the coding nucleotide level, the best programs reached an accuracy of 90% in both sensitivity and specificity. Programs relying on mRNA and protein sequences were the most accurate in reproducing the manually curated annotations. Experimental validation shows that only a very small percentage (3.2%) of the selected 221 computationally predicted exons outside of the existing annotation could be verified.
  
  
  CONCLUSIONS
  This is the first such experiment in human DNA, and we have followed the standards established in a similar experiment, GASP1, in Drosophila melanogaster. We believe the results presented here contribute to the value of ongoing large-scale annotation projects and should guide further experimental methods when being scaled up to the entire human genome sequence.
authors.affiliation: Centre de Regulació Genòmica, Institut Municipal d'Investigació Mèdica-Universitat Pompeu Fabra, E08003 Barcelona, Catalonia, Spain. rguigo@imim.es
authors.forename: Roderic
authors.initials: R
authors.lastname: Guigó
authors.forename: Paul
authors.initials: P
authors.lastname: Flicek
authors.forename: Josep F
authors.initials: JF
authors.lastname: Abril
authors.forename: Alexandre
authors.initials: A
authors.lastname: Reymond
authors.forename: Julien
authors.initials: J
authors.lastname: Lagarde
authors.forename: France
authors.initials: F
authors.lastname: Denoeud
authors.forename: Stylianos
authors.initials: S
authors.lastname: Antonarakis
authors.forename: Michael
authors.initials: M
authors.lastname: Ashburner
authors.forename: Vladimir B
authors.initials: VB
authors.lastname: Bajic
authors.forename: Ewan
authors.initials: E
authors.lastname: Birney
authors.forename: Robert
authors.initials: R
authors.lastname: Castelo
authors.forename: Eduardo
authors.initials: E
authors.lastname: Eyras
authors.forename: Catherine
authors.initials: C
authors.lastname: Ucla
authors.forename: Thomas R
authors.initials: TR
authors.lastname: Gingeras
authors.forename: Jennifer
authors.initials: J
authors.lastname: Harrow
authors.forename: Tim
authors.initials: T
authors.lastname: Hubbard
authors.forename: Suzanna E
authors.initials: SE
authors.lastname: Lewis
authors.forename: Martin G
authors.initials: MG
authors.lastname: Reese
mesh_terms: D017398:Alternative Splicing; D000818:Animals; D019295:Computational Biology / Q000379:methods / Q000592:standards*; D030541:Databases, Genetic; D005796:Genes; D015894:Genome, Human; D023281:Genomics / Q000379:methods / Q000592:standards*; D006801:Humans; D051379:Mice; D012333:RNA, Messenger / Q000032:analysis; D017422:Sequence Analysis, DNA; D017423:Sequence Analysis, RNA
chemical_list: D012333:RNA, Messenger