title: Using classical test theory, item response theory, and Rasch measurement theory to evaluate patient-reported outcome measures: a comparison of worked examples.
abstract: OBJECTIVE
  To provide comparisons and a worked example of item- and scale-level evaluations based on three psychometric methods used in patient-reported outcome development-classical test theory (CTT), item response theory (IRT), and Rasch measurement theory (RMT)-in an analysis of the National Eye Institute Visual Functioning Questionnaire (VFQ-25).
  
  
  METHODS
  Baseline VFQ-25 data from 240 participants with diabetic macular edema from a randomized, double-masked, multicenter clinical trial were used to evaluate the VFQ at the total score level. CTT, RMT, and IRT evaluations were conducted, and results were assessed in a head-to-head comparison.
  
  
  RESULTS
  Results were similar across the three methods, with IRT and RMT providing more detailed diagnostic information on how to improve the scale. CTT led to the identification of two problematic items that threaten the validity of the overall scale score, sets of redundant items, and skewed response categories. IRT and RMT additionally identified poor fit for one item, many locally dependent items, poor targeting, and disordering of over half the response categories.
  
  
  CONCLUSIONS
  Selection of a psychometric approach depends on many factors. Researchers should justify their evaluation method and consider the intended audience. If the instrument is being developed for descriptive purposes and on a restricted budget, a cursory examination of the CTT-based psychometric properties may be all that is possible. In a high-stakes situation, such as the development of a patient-reported outcome instrument for consideration in pharmaceutical labeling, however, a thorough psychometric evaluation including IRT or RMT should be considered, with final item-level decisions made on the basis of both quantitative and qualitative results.
authors.affiliation: Novartis AG, Basel, Switzerland. Electronic address: lmcleod@rti.org.
authors.forename: Jennifer
authors.initials: J
authors.lastname: Petrillo
authors.affiliation: Plymouth University Peninsula Schools of Medicine and Dentistry, Plymouth, UK.
authors.forename: Stefan J
authors.initials: SJ
authors.lastname: Cano
authors.affiliation: RTI Health Solutions, Research Triangle Park, NC, USA.
authors.forename: Lori D
authors.initials: LD
authors.lastname: McLeod
authors.affiliation: RTI Health Solutions, Research Triangle Park, NC, USA.
authors.forename: Cheryl D
authors.initials: CD
authors.lastname: Coon
mesh_terms: D003930:Diabetic Retinopathy / Q000175:diagnosis / Q000453:epidemiology; D006801:Humans; D008269:Macular Edema / Q000175:diagnosis / Q000453:epidemiology; D063868:Patient Outcome Assessment; D011594:Psychometrics / Q000379:methods / Q000592:standards; D057566:Self Report / Q000592:standards*; D011795:Surveys and Questionnaires / Q000592:standards*
keywords: Rasch measurement theory; classical test theory; item response theory; patient-reported outcome; psychometrics