Doctors and numbers: an assessment of the critical risk interpretation test.
Risk interpretation affects decision making. Yet, there is no valid assessment of how clinicians interpret the risk data that they commonly encounter.



To establish the reliability and validity of a 20-item test of clinicians' risk interpretation.



The Critical Risk Interpretation Test (CRIT) measures clinicians' abilities to 1) modify the interpretation based on meaningful differences in the outcome (e.g., disease specific v. all-cause mortality) and time period (e.g., lifetime v. 10-year mortality), 2) maintain a stable interpretation for different risk framings (e.g., relative v. absolute risk), and 3) correctly interpret how diagnostic testing modifies risk. There were 658 clinicians and medical trainees who participated: 116 nurse practitioners (NPs) at a national conference, 273 medical students at 1 institution, 148 residents in internal medicine at 2 institutions, and 121 internists at 1 institution. Participants completed a self-administered paper test during educational conferences. Seventeen evidence-based medicine experts took the test online and formally assessed content validity. Eighteen second-year medical students were recruited to take the test and a retest 3 weeks later to explore test-retest correlation.



Expert review supported test clarity and content validity. Factor analysis supported that the CRIT identifies at least 3 separable areas of clinician knowledge. Test-retest correlation was fair (intraclass correlation coefficient = 0.65; standard error = 0.15). Scores on our test correlated with other tests of related abilities. Mean test scores varied among groups, with differences in prior evidence-based medicine training and experience (93 for NPs, 101 for medical students, 101 for residents, 103 for academic internists, and 110 for physician experts; P < 0.001).



Our results provide supporting evidence for the reliability and validity of the CRIT as an index of critical risk interpretation abilities, which is acceptable and feasible to administer in an educational setting.
Veterans Affairs Center for Clinical Management Research, Ann Arbor Veterans Affairs Health System, University of Michigan Medical School, Ann Arbor, MI (TJC, AF)
Tanner J
TJ
Caverly
Internal Medicine, University of Colorado School of Medicine, Aurora, CO (AVP, BPC, SRM, JSK, IB, DDM)|Ambulatory Care, Denver Veterans Affairs Medical Center, Denver, CO (AVP, JM, SP)
Allan V
AV
Prochazka
Internal Medicine, University of Colorado School of Medicine, Aurora, CO (AVP, BPC, SRM, JSK, IB, DDM)
Brandon P
BP
Combs
Department of Medicine, John H. Stroger Jr. Hospital of Cook County, Chicago, IL (BPL).
Brian P
BP
Lucas
Internal Medicine, University of Colorado School of Medicine, Aurora, CO (AVP, BPC, SRM, JSK, IB, DDM)
Shane R
SR
Mueller
Internal Medicine, University of Colorado School of Medicine, Aurora, CO (AVP, BPC, SRM, JSK, IB, DDM)
Jean S
JS
Kutner
Internal Medicine, University of Colorado School of Medicine, Aurora, CO (AVP, BPC, SRM, JSK, IB, DDM)
Ingrid
I
Binswanger
Veterans Affairs Center for Clinical Management Research, Ann Arbor Veterans Affairs Health System, University of Michigan Medical School, Ann Arbor, MI (TJC, AF)
Angela
A
Fagerlin
Ambulatory Care, Denver Veterans Affairs Medical Center, Denver, CO (AVP, JM, SP)
Jacqueline
J
McCormick
Ambulatory Care, Denver Veterans Affairs Medical Center, Denver, CO (AVP, JM, SP)
Shirley
S
Pfister
Internal Medicine, University of Colorado School of Medicine, Aurora, CO (AVP, BPC, SRM, JSK, IB, DDM)
Daniel D
DD
Matlock
D002983:Clinical Competence; D003627:Data Interpretation, Statistical; D003657:Decision Making; D019317:Evidence-Based Medicine; D005163:Factor Analysis, Statistical; D017144:Focus Groups; D057220:Health Literacy / Q000379:methods* / Q000592:standards*; D006282:Health Personnel / Q000523:psychology; D006801:Humans; D016014:Linear Models; D010820:Physicians / Q000523:psychology*; D015203:Reproducibility of Results; D018570:Risk Assessment / Q000379:methods*
medical decision making; numeracy; risk communication; risk perception