title: Validation of an integrated method for determining cognitive ability: Implications for routine assessments and clinical trials.
abstract: BACKGROUND
  Although accurate diagnosis of deficit of mild intensity is critical, various methods are used to assess, dichotomize and integrate performance, with no validated gold standard. This study described and validated a framework for the analysis of cognitive performance.
  
  
  METHODS
  This study was performed by using the Groupe de Réflexion sur L'Evaluation des Fonctions EXécutives (GREFEX) database (724 controls and 461 patients) examined by 7 tests assessing executive functions. The first phase determined the criteria for the cutoff scores, the second phase, the effect of test number on diagnostic accuracy and the third phase, the best methods for combining test scores into an overall summary score. Four validation criteria were used: determination of impaired performance as compared to expected one, false-positive rate ≤5%, detection of both single and multiple impairments with optimal sensitivity.
  
  
  RESULTS
  The procedure based on 5th percentile cutoffs determined from standardized residuals was the most appropriate procedure. Although area under the curve (AUC) increased with the number of scores (p = .0001), the false-positive rate also increased (p = .0001), resulting in suboptimal sensitivity for detecting selective impairment. Two overall summary scores, the average of the seven process scores and the Item Response Theory (IRT) score, had significantly (p = .0001) higher AUCs, even for patients with a selective impairment, and provided higher resulting prevalence of dysexecutive disorders (p = .0001).
  
  
  CONCLUSIONS
  The present study provides and validates a generative framework for the interpretation of cognitive data. Two overall summary score met all 4 validation criteria. A practical consequence is the need to profoundly modify the analysis and interpretation of cognitive assessments for both routine use and clinical research.
authors.affiliation: Department of Neurology and Laboratory of Functional Neurosciences University Hospital of Amiens, France. Electronic address: godefroy.olivier@chu-amiens.fr.
authors.forename: Olivier
authors.initials: O
authors.lastname: Godefroy
authors.affiliation: Department of General Internal Medicine, University of Washington, Harborview Medical Center, Seattle, WA, USA.
authors.forename: Laura
authors.initials: L
authors.lastname: Gibbons
authors.affiliation: Department of Biostatistics, University Hospital of Amiens, France.
authors.forename: Momar
authors.initials: M
authors.lastname: Diouf
authors.affiliation: Hauenstein Neuroscience Center, Saint Mary's Health Care, Grand Rapids, MI, USA.
authors.forename: David
authors.initials: D
authors.lastname: Nyenhuis
authors.affiliation: Department of Neurology and Laboratory of Functional Neurosciences University Hospital of Amiens, France.
authors.forename: Martine
authors.initials: M
authors.lastname: Roussel
authors.affiliation: Brill Chair in Neurology, Dept of Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.
authors.forename: Sandra
authors.initials: S
authors.lastname: Black
authors.affiliation: Department of Neurology and Laboratory of Functional Neurosciences University Hospital of Amiens, France.
authors.forename: Jean Marc
authors.initials: JM
authors.lastname: Bugnicourt
mesh_terms: D000328:Adult; D000368:Aged; D002986:Clinical Trials as Topic; D003071:Cognition / Q000502:physiology*; D003072:Cognition Disorders / Q000175:diagnosis* / Q000523:psychology; D056344:Executive Function / Q000502:physiology*; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D009483:Neuropsychological Tests
keywords: Dementia; Diagnostic accuracy; Executive functions; Mild cognitive impairment; Stroke; sensitivity and specificity