Validation of a method to measure resident doctors' reflections on quality improvement.
OBJECTIVE
Resident reflection on the clinical learning environment is prerequisite to identifying quality improvement (QI) opportunities and demonstrating competence in practice-based learning. However, residents' abilities to reflect on QI opportunities are unknown. Therefore, we developed and determined the validity of the Mayo Evaluation of Reflection on Improvement Tool (MERIT) for assessing resident reflection on QI opportunities.


METHODS
The content of MERIT, which consists of 18 items structured on 4-point scales, was based on existing literature and input from national experts. Using MERIT, six faculty members rated 50 resident reflections. Factor analysis was used to examine the dimensionality of MERIT instrument scores. Inter-rater and internal consistency reliabilities were calculated.


RESULTS
Factor analysis revealed three factors (eigenvalue; number of items): Reflection on Personal Characteristics of QI (8.5; 7); Reflection on System Characteristics of QI (1.9; 6), and Problem of Merit (1.5; 5). Inter-rater reliability was very good (intraclass correlation coefficient range: 0.73-0.89). Internal consistency reliability was excellent (Cronbach's alpha 0.93 overall and 0.83-0.91 for factors). Item mean scores were highest for Problem of Merit (3.29) and lowest for Reflection on System Characteristics of QI (1.99).


CONCLUSIONS
Validity evidence supports MERIT as a meaningful measure of resident reflection on QI opportunities. Our findings suggest that dimensions of resident reflection on QI opportunities may include personal, system and Problem of Merit factors. Additionally, residents may be more effective at reflecting on 'problems of merit' than personal and systems factors.
Wittich
Christopher M
CM
Department of Internal Medicine, Mayo Clinic College of Medicine, Mayo Clinic, Rochester, Minnesota 5590, USA.
Beckman
Thomas J
TJ
Drefahl
Monica M
MM
Mandrekar
Jayawant N
JN
Reed
Darcy A
DA
Krajicek
Bryan J
BJ
Haddad
Rudy M
RM
McDonald
Furman S
FS
Kolars
Joseph C
JC
Thomas
Kris G
KG
D002983:Clinical Competence / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D005163:Factor Analysis, Statistical; D006801:Humans; D007396:Internship and Residency / Q000379:methods / Q000592:standards; D011786:Quality Control; D015203:Reproducibility of Results