title: Shape analysis, a field in need of careful validation.
abstract: In the last two decades, the statistical analysis of shape has become an actively studied field and finds applications in a wide range of areas. In addition to algorithmic development, many researchers have distributed end-user orientated toolboxes, which further enable the utilization of the algorithms in an "off the shelf" fashion. However, there is little work on the evaluation and validation of these techniques, which poses a rather serious challenge when interpreting their results. To address this lack of validation, we design a validation framework and then use it to test some of the most widely used toolboxes. Our initial results show inconsistencies and disagreement among four different methods. We believe this type of analysis to be critical not only for the community of algorithm designers but also perhaps more importantly to researchers who use these tools without knowing the algorithm details and seek objective criteria for tool selection.
authors.affiliation: Department of Electrical and Computer Engineering, The University of Alabama at Birmingham, Birmingham, Alabama; Neuro-Oncology Program, Comprehensive Cancer Center, The University of Alabama at Birmingham, Birmingham, Alabama.
authors.forename: Yi
authors.initials: Y
authors.lastname: Gao
authors.forename: Tammy
authors.initials: T
authors.lastname: Riklin-Raviv
authors.forename: Sylvain
authors.initials: S
authors.lastname: Bouix
mesh_terms: D000465:Algorithms; D001931:Brain Mapping; D019295:Computational Biology; D003342:Corpus Striatum / Q000033:anatomy & histology*; D066264:Datasets as Topic; D005163:Factor Analysis, Statistical; D007839:Functional Laterality; D006801:Humans; D015203:Reproducibility of Results
keywords: algorithm evaluation; reproducibility; statistical shape analysis
references.citation: Med Image Anal. 2009 Aug;13(4):543-63
references.pmid: 19525140
references.citation: Hum Brain Mapp. 2014 Feb;35(2):659-72
references.pmid: 23124690
references.citation: Evolution. 2009 Apr;63(4):1003-16
references.pmid: 19154365
references.citation: Neuroimage. 2001 Jul;14(1 Pt 1):21-36
references.pmid: 11525331
references.citation: Proc Workshop Math Methods Biomed Image Analysis. 2012;:25-32
references.pmid: 23783079
references.citation: Proc Natl Acad Sci U S A. 1996 Oct 29;93(22):12132-6
references.pmid: 8901545
references.citation: IEEE Trans Med Imaging. 1996;15(6):785-95
references.pmid: 18215958
references.citation: Neuroimage. 2007 Feb 1;34(3):1149-59
references.pmid: 17185000
references.citation: Biol Psychiatry. 2011 Dec 1;70(11):1083-90
references.pmid: 21903200
references.citation: Science. 2000 Dec 22;290(5500):2323-6
references.pmid: 11125150
references.citation: Med Image Anal. 2003 Sep;7(3):207-20
references.pmid: 12946464
references.citation: Am J Psychiatry. 2009 Jan;166(1):74-82
references.pmid: 19015232
references.citation: Schizophr Res. 2009 May;110(1-3):127-39
references.pmid: 19328654
references.citation: Neuroimage. 2000 Jun;11(6 Pt 1):805-21
references.pmid: 10860804
references.citation: IEEE Trans Pattern Anal Mach Intell. 2008 Nov;30(11):2023-39
references.pmid: 18787249
references.citation: Insight J. 2006;(1071):242-250
references.pmid: 21941375
references.citation: Hum Brain Mapp. 2013 Nov;34(11):3075-85
references.pmid: 22815197
references.citation: IEEE Trans Med Imaging. 1999 Aug;18(8):712-21
references.pmid: 10534053
references.citation: IEEE Trans Med Imaging. 2008 Aug;27(8):1143-51
references.pmid: 18672431
references.citation: IEEE Trans Pattern Anal Mach Intell. 2008 Aug;30(8):1385-99
references.pmid: 18566493
references.citation: Med Image Comput Comput Assist Interv. 2005;8(Pt 2):459-67
references.pmid: 16685992
references.citation: Hum Brain Mapp. 2014 Mar;35(3):792-809
references.pmid: 23281100
references.citation: Comput Aided Des. 2009 Oct 1;41(10):739-755
references.pmid: 20161035
references.citation: Med Image Anal. 2005 Jun;9(3):209-21
references.pmid: 15854842
references.citation: Med Image Comput Comput Assist Interv. 2008;11(Pt 1):477-85
references.pmid: 18979781
references.citation: Med Image Comput Comput Assist Interv. 2007;10(Pt 1):959-67
references.pmid: 18051151
references.citation: Proc Natl Acad Sci U S A. 1998 Apr 14;95(8):4121-5
references.pmid: 9539700
references.citation: Med Image Anal. 2008 Oct;12(5):626-37
references.pmid: 18658005
references.citation: Neuroimage. 2004;23 Suppl 1:S19-33
references.pmid: 15501089
references.citation: Med Image Anal. 2011 Aug;15(4):622-39
references.pmid: 20688559
references.citation: Hum Brain Mapp. 1998;6(5-6):348-57
references.pmid: 9788071
references.citation: Neuroimage. 2009 Mar;45(1 Suppl):S61-72
references.pmid: 19041946
references.citation: Inf Process Med Imaging. 2005;19:381-92
references.pmid: 17354711
references.citation: Med Image Comput Comput Assist Interv. 2007;10(Pt 1):850-7
references.pmid: 18051138