The ophthalmic clinical evaluation exercise: reliability determination.
OBJECTIVE
Reliable and valid tools must be developed to assess the core residency competencies identified by the Accreditation Council for Graduate Medical Education. The Ophthalmic Clinical Evaluation Exercise (OCEX) is a tool designed to assess the ophthalmology resident's competence in patient care. The OCEX has been shown to have face and content validity. This study will determine the degree to which the OCEX is reliable and has construct validity.


METHODS
Ninety-four academic ophthalmology teaching faculty from ophthalmology residency programs across the country.


METHODS
Participants reviewed a video compact disc of the same resident and new patient encounter and then completed the OCEX. A scoring rubric was provided.


RESULTS
Results indicate that the OCEX is a reliable tool for faculty to use to assess residency competency. The coefficient alpha statistic (a measure of reliability/internal consistency) for the OCEX as a whole was 0.81. The alpha statistics for 3 of 4 subscales that comprise the OCEX (i.e., interviewing skills = 0.65, interpersonal skills/professionalism = 0.73, case presentation = 0.70) were lower than the OCEX as a whole, but were acceptable for new scales. However, the alpha for the examination subscale (i.e., 0.27) was extremely low. Interrater reliability assessment shows that of 33 individual OCEX items, 31 (94%) had at least 85% of the raters rating the student in 1 of 2 consecutive rating categories.


CONCLUSIONS
The OCEX shows both reliability and validity and, therefore, meets the Accreditation Council for Graduate Medical Education criteria for an acceptable assessment tool.
Golnik
Karl C
KC
Department of Ophthalmology, University of Cincinnati and The Cincinnati Eye Institute, Cincinnati, Ohio, USA. kgolnik@cinci.rr.com
Goldenhar
Linda
L
D000068:Accreditation / Q000592:standards; D002983:Clinical Competence / Q000592:standards*; D004503:Education, Medical, Graduate / Q000592:standards*; D004521:Educational Measurement / Q000592:standards*; D006801:Humans; D007396:Internship and Residency / Q000592:standards*; D009885:Ophthalmology / Q000193:education*; D015397:Program Evaluation; D011787:Quality of Health Care; D015203:Reproducibility of Results; D013044:Specialty Boards / Q000592:standards; D014481:United States