title: Reliability analysis of the objective structured clinical examination using generalizability theory.
abstract: BACKGROUND
  The objective structured clinical examination (OSCE) is a widely used method for assessing clinical competence in health sciences education. Studies using this method have shown evidence of validity and reliability. There are no published studies of OSCE reliability measurement with generalizability theory (G-theory) in Latin America. The aims of this study were to assess the reliability of an OSCE in medical students using G-theory and explore its usefulness for quality improvement.
  
  
  METHODS
  An observational cross-sectional study was conducted at National Autonomous University of Mexico (UNAM) Faculty of Medicine in Mexico City. A total of 278 fifth-year medical students were assessed with an 18-station OSCE in a summative end-of-career final examination. There were four exam versions. G-theory with a crossover random effects design was used to identify the main sources of variance. Examiners, standardized patients, and cases were considered as a single facet of analysis.
  
  
  RESULTS
  The exam was applied to 278 medical students. The OSCE had a generalizability coefficient of 0.93. The major components of variance were stations, students, and residual error. The sites and the versions of the tests had minimum variance.
  
  
  CONCLUSIONS
  Our study achieved a G coefficient similar to that found in other reports, which is acceptable for summative tests. G-theory allows the estimation of the magnitude of multiple sources of error and helps decision makers to determine the number of stations, test versions, and examiners needed to obtain reliable measurements.
authors.affiliation: Secretariat of Medical Education, UNAM Faculty of Medicine, México City, México; andretreme@gmail.com.
authors.forename: Juan Andrés
authors.initials: JA
authors.lastname: Trejo-Mejía
authors.affiliation: Secretariat of Medical Education, UNAM Faculty of Medicine, México City, México.
authors.forename: Melchor
authors.initials: M
authors.lastname: Sánchez-Mendiola
authors.affiliation: Institute of Applied Mathematics and Systems, UNAM, México City, México.
authors.forename: Ignacio
authors.initials: I
authors.lastname: Méndez-Ramírez
authors.affiliation: Secretariat of Medical Education, UNAM Faculty of Medicine, México City, México.
authors.forename: Adrián
authors.initials: A
authors.lastname: Martínez-González
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D003430:Cross-Sectional Studies; D004501:Education, Medical / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D008800:Mexico; D008962:Models, Theoretical; D015203:Reproducibility of Results
keywords: Mexico; OSCE; clinical competence; generalizability theory; reliability