title: Neurology objective structured clinical examination reliability using generalizability theory.
abstract: OBJECTIVE
  This study examines factors affecting reliability, or consistency of assessment scores, from an objective structured clinical examination (OSCE) in neurology through generalizability theory (G theory).
  
  
  METHODS
  Data include assessments from a multistation OSCE taken by 194 medical students at the completion of a neurology clerkship. Facets evaluated in this study include cases, domains, and items. Domains refer to areas of skill (or constructs) that the OSCE measures. G theory is used to estimate variance components associated with each facet, derive reliability, and project the number of cases required to obtain a reliable (consistent, precise) score.
  
  
  RESULTS
  Reliability using G theory is moderate (Φ coefficient = 0.61, G coefficient = 0.64). Performance is similar across cases but differs by the particular domain, such that the majority of variance is attributed to the domain. Projections in reliability estimates reveal that students need to participate in 3 OSCE cases in order to increase reliability beyond the 0.70 threshold.
  
  
  CONCLUSIONS
  This novel use of G theory in evaluating an OSCE in neurology provides meaningful measurement characteristics of the assessment. Differing from prior work in other medical specialties, the cases students were randomly assigned did not influence their OSCE score; rather, scores varied in expected fashion by domain assessed.
authors.affiliation: From Rush University Medical College (A.D.B.); Department of Medical Education (Y.S.P.), The University of Illinois at Chicago; and Department of Neurology (R.V.L., J.R.B.), The University of Chicago, IL. angela_blood@rush.edu.
authors.forename: Angela D
authors.initials: AD
authors.lastname: Blood
authors.affiliation: From Rush University Medical College (A.D.B.); Department of Medical Education (Y.S.P.), The University of Illinois at Chicago; and Department of Neurology (R.V.L., J.R.B.), The University of Chicago, IL.
authors.forename: Yoon Soo
authors.initials: YS
authors.lastname: Park
authors.affiliation: From Rush University Medical College (A.D.B.); Department of Medical Education (Y.S.P.), The University of Illinois at Chicago; and Department of Neurology (R.V.L., J.R.B.), The University of Chicago, IL.
authors.forename: Rimas V
authors.initials: RV
authors.lastname: Lukas
authors.affiliation: From Rush University Medical College (A.D.B.); Department of Medical Education (Y.S.P.), The University of Illinois at Chicago; and Department of Neurology (R.V.L., J.R.B.), The University of Chicago, IL.
authors.forename: James R
authors.initials: JR
authors.lastname: Brorson
mesh_terms: D002982:Clinical Clerkship; D002983:Clinical Competence / Q000592:standards*; D004504:Education, Medical, Undergraduate; D006801:Humans; D009460:Neurologic Examination / Q000592:standards*; D009462:Neurology / Q000193:education*; D016544:Patient Simulation; D015203:Reproducibility of Results; D013223:Statistics as Topic; D013337:Students, Medical