EGASP: the human ENCODE Genome Annotation Assessment Project.
BACKGROUND
We present the results of EGASP, a community experiment to assess the state-of-the-art in genome annotation within the ENCODE regions, which span 1% of the human genome sequence. The experiment had two major goals: the assessment of the accuracy of computational methods to predict protein coding genes; and the overall assessment of the completeness of the current human genome annotations as represented in the ENCODE regions. For the computational prediction assessment, eighteen groups contributed gene predictions. We evaluated these submissions against each other based on a 'reference set' of annotations generated as part of the GENCODE project. These annotations were not available to the prediction groups prior to the submission deadline, so that their predictions were blind and an external advisory committee could perform a fair assessment.


RESULTS
The best methods had at least one gene transcript correctly predicted for close to 70% of the annotated genes. Nevertheless, the multiple transcript accuracy, taking into account alternative splicing, reached only approximately 40% to 50% accuracy. At the coding nucleotide level, the best programs reached an accuracy of 90% in both sensitivity and specificity. Programs relying on mRNA and protein sequences were the most accurate in reproducing the manually curated annotations. Experimental validation shows that only a very small percentage (3.2%) of the selected 221 computationally predicted exons outside of the existing annotation could be verified.


CONCLUSIONS
This is the first such experiment in human DNA, and we have followed the standards established in a similar experiment, GASP1, in Drosophila melanogaster. We believe the results presented here contribute to the value of ongoing large-scale annotation projects and should guide further experimental methods when being scaled up to the entire human genome sequence.
Guigó
Roderic
R
Centre de Regulació Genòmica, Institut Municipal d'Investigació Mèdica-Universitat Pompeu Fabra, E08003 Barcelona, Catalonia, Spain. rguigo@imim.es
Flicek
Paul
P
Abril
Josep F
JF
Reymond
Alexandre
A
Lagarde
Julien
J
Denoeud
France
F
Antonarakis
Stylianos
S
Ashburner
Michael
M
Bajic
Vladimir B
VB
Birney
Ewan
E
Castelo
Robert
R
Eyras
Eduardo
E
Ucla
Catherine
C
Gingeras
Thomas R
TR
Harrow
Jennifer
J
Hubbard
Tim
T
Lewis
Suzanna E
SE
Reese
Martin G
MG
D017398:Alternative Splicing; D000818:Animals; D019295:Computational Biology / Q000379:methods / Q000592:standards*; D030541:Databases, Genetic; D005796:Genes; D015894:Genome, Human; D023281:Genomics / Q000379:methods / Q000592:standards*; D006801:Humans; D051379:Mice; D012333:RNA, Messenger / Q000032:analysis; D017422:Sequence Analysis, DNA; D017423:Sequence Analysis, RNA
D012333:RNA, Messenger