Blink or think: can further reflection improve initial diagnostic impressions?
OBJECTIVE
Experienced clinicians derive many diagnoses intuitively, because most new problems they see closely resemble problems they've seen before. The majority of these diagnoses, but not all, will be correct. This study determined whether further reflection regarding initial diagnoses improves diagnostic accuracy during a high-stakes board exam, a model for studying clinical decision making.


METHODS
Keystroke response data were used from 500 residents who took the 2010 American Board of Internal Medicine (ABIM) Internal Medicine Certification Examination. Data included time to initial response on each question, whether the answer was correct, and whether or not the resident changed her or his initial response. The focus was on 80 diagnosis questions that comprised realistic clinical vignettes with multiple-choice single-best answers. Cognitive skill (ability) was measured using overall exam scores. Case complexity was determined using item difficulty (proportion of examinees that correctly answered the question). A hierarchical generalized linear model was used to assess the relationship between time spent on initial responses and the probability of correctly answering the questions.


RESULTS
On average, residents changed their responses on 12% of all diagnosis questions (or 9.6 questions out of 80). Changing an answer from incorrect to correct was almost twice as likely as changing an answer from correct to incorrect. The relationship between response time and accuracy was complex.


CONCLUSIONS
Further reflection appears to be beneficial to diagnostic accuracy, especially for more complex cases.
Dr. Hess is a consultant, Hess Consulting, St-Nicolas, Québec, Canada. Dr. Lipner is senior vice president of evaluation, research and development, American Board of Internal Medicine, Philadelphia, Pennsylvania. Dr. Thompson is professor of cognitive psychology, University of Saskatchewan, Saskatoon, Saskatchewan, Canada. Dr. Holmboe is senior vice president of milestone development and evaluation, Accreditation Council for Graduate Medical Education, Chicago, Illinois. Dr. Graber is senior research fellow, RTI International, Research Triangle Park, North Carolina, and professor emeritus, at SUNY Stony Brook University School of Medicine, Stony Brook, New York.
Brian J
BJ
Hess
Rebecca S
RS
Lipner
Valerie
V
Thompson
Eric S
ES
Holmboe
Mark L
ML
Graber
D002755:Choice Behavior; D003657:Decision Making; D003933:Diagnosis; D004521:Educational Measurement; D005260:Female; D006801:Humans; D007388:Internal Medicine; D007396:Internship and Residency; D016014:Linear Models; D008297:Male; D013044:Specialty Boards; D013850:Thinking; D013997:Time Factors; D014481:United States