title: Evaluation of predictions in the CASP10 model refinement category.
abstract: Here we report on the assessment results of the third experiment to evaluate the state of the art in protein model refinement, where participants were invited to improve the accuracy of initial protein models for 27 targets. Using an array of complementary evaluation measures, we find that five groups performed better than the naïve (null) method-a marked improvement over CASP9, although only three were significantly better. The leading groups also demonstrated the ability to consistently improve both backbone and side chain positioning, while other groups reliably enhanced other aspects of protein physicality. The top-ranked group succeeded in improving the backbone conformation in almost 90% of targets, suggesting a strategy that for the first time in CASP refinement is successful in a clear majority of cases. A number of issues remain unsolved: the majority of groups still fail to improve the quality of the starting models; even successful groups are only able to make modest improvements; and no prediction is more similar to the native structure than to the starting model. Successful refinement attempts also often go unrecognized, as suggested by the relatively larger improvements when predictions not submitted as model 1 are also considered.
authors.affiliation: Department of Computer Science Bioinformatics Group, University College London, London, WC1E 6BT, United Kingdom.
authors.forename: Timothy
authors.initials: T
authors.lastname: Nugent
authors.forename: Domenico
authors.initials: D
authors.lastname: Cozzetto
authors.forename: David T
authors.initials: DT
authors.lastname: Jones
mesh_terms: D000465:Algorithms; D019295:Computational Biology / Q000379:methods / Q000592:standards; D008958:Models, Molecular; D015233:Models, Statistical; D011487:Protein Conformation; D011506:Proteins / Q000737:chemistry*; D016415:Sequence Alignment
chemical_list: D011506:Proteins
keywords: CASP; model quality evaluation; protein model refinement; protein structure prediction; template-based modeling
references.citation: Proteins. 2011;79 Suppl 10:147-60
references.pmid: 22069036
references.citation: Proteins. 2007;69 Suppl 8:38-56
references.pmid: 17894352
references.citation: Acta Crystallogr D Biol Crystallogr. 2010 Jan;66(Pt 1):12-21
references.pmid: 20057044
references.citation: Proteins. 2013 Jan;81(1):119-31
references.pmid: 22927229
references.citation: Nucleic Acids Res. 2003 Jul 1;31(13):3370-4
references.pmid: 12824330
references.citation: Acta Crystallogr D Biol Crystallogr. 2012 Aug;68(Pt 8):935-52
references.pmid: 22868759
references.citation: Nucleic Acids Res. 2007 Jul;35(Web Server issue):W407-10
references.pmid: 17517781
references.citation: Nucleic Acids Res. 2012 Jul;40(Web Server issue):W294-7
references.pmid: 22649060
references.citation: Nucleic Acids Res. 2013 Jan;41(Database issue):D475-82
references.pmid: 23193259
references.citation: J Chem Theory Comput. 2013 Feb 12;9(2):1294-1303
references.pmid: 23526422
references.citation: J Chem Phys. 2012 May 21;136(19):195101
references.pmid: 22612113
references.citation: J Appl Crystallogr. 2007 Aug 1;40(Pt 4):658-674
references.pmid: 19461840
references.citation: Proteins. 2013 Mar;81(3):469-78
references.pmid: 23070940
references.citation: Proteins. 2012 Jul;80(7):1744-54
references.pmid: 22411697
references.citation: J Comput Chem. 2013 Mar 5;34(6):445-50
references.pmid: 23065773
references.citation: Proteins. 2009;77 Suppl 9:29-49
references.pmid: 19731372
references.citation: Proteins. 2012 Aug;80(8):2071-9
references.pmid: 22513870
references.citation: Nucleic Acids Res. 2012 Jul;40(Web Server issue):W323-8
references.pmid: 22564897
references.citation: J Chem Theory Comput. 2012 Oct 9;8(10):3985-3991
references.pmid: 25530735
references.citation: Proteins. 2014 Feb;82 Suppl 2:7-13
references.pmid: 24038551
references.citation: J Mol Biol. 1994 May 20;238(5):777-93
references.pmid: 8182748
references.citation: J Phys Chem B. 2012 Mar 1;116(8):2365-75
references.pmid: 22260550
references.citation: Proteins. 2011;79 Suppl 10:74-90
references.pmid: 22069034
references.citation: Proteins. 2011 Sep;79(9):2725-34
references.pmid: 21755541
references.citation: Acta Crystallogr D Biol Crystallogr. 2001 Oct;57(Pt 10):1373-82
references.pmid: 11567148
references.citation: J Comput Chem. 2011 Mar;32(4):709-17
references.pmid: 20925085
references.citation: Structure. 2011 Dec 7;19(12):1784-95
references.pmid: 22153501
references.citation: J Comput Chem. 2011 Nov 15;32(14):3014-22
references.pmid: 21793008
references.citation: Proteins. 2012 Aug;80(8):1974-86
references.pmid: 22488760
references.citation: Proteins. 2009;77 Suppl 9:66-80
references.pmid: 19714776