Interpretation of evidence in data by untrained medical students: a scenario-based study.
BACKGROUND
To determine which approach to assessment of evidence in data - statistical tests or likelihood ratios - comes closest to the interpretation of evidence by untrained medical students.


METHODS
Empirical study of medical students (N = 842), untrained in statistical inference or in the interpretation of diagnostic tests. They were asked to interpret a hypothetical diagnostic test, presented in four versions that differed in the distributions of test scores in diseased and non-diseased populations. Each student received only one version. The intuitive application of the statistical test approach would lead to rejecting the null hypothesis of no disease in version A, and to accepting the null in version B. Application of the likelihood ratio approach led to opposite conclusions - against the disease in A, and in favour of disease in B. Version C tested the importance of the p-value (A: 0.04 versus C: 0.08) and version D the importance of the likelihood ratio (C: 1/4 versus D: 1/8).


RESULTS
In version A, 7.5% concluded that the result was in favour of disease (compatible with p value), 43.6% ruled against the disease (compatible with likelihood ratio), and 48.9% were undecided. In version B, 69.0% were in favour of disease (compatible with likelihood ratio), 4.5% against (compatible with p value), and 26.5% undecided. Increasing the p value from 0.04 to 0.08 did not change the results. The change in the likelihood ratio from 1/4 to 1/8 increased the proportion of non-committed responses.


CONCLUSIONS
Most untrained medical students appear to interpret evidence from data in a manner that is compatible with the use of likelihood ratios.
Perneger
Thomas V
TV
Division of Clinical Epidemiology, Geneva University Hospitals, Geneva, Switzerland and Faculty of Medicine, University of Geneva, Geneva, Switzerland. thomas.perneger@hcuge.ch
Courvoisier
Delphine S
DS
D000328:Adult; D001530:Belgium; D006302:Health Services Research; D006801:Humans; D007407:Interviews as Topic; D008875:Middle Aged; D010821:Physicians, Family / Q000523:psychology*; D019338:Polypharmacy; D012424:Rural Population; D014505:Urban Population
JAMA. 2007 Sep 5;298(9):1010-22
17785646
Ann Intern Med. 1999 Jun 15;130(12):1005-13
10383350
Int J Epidemiol. 2003 Oct;32(5):687-91
14559729
J Clin Epidemiol. 2006 Sep;59(9):1009-14
16895826
BMJ. 2001 Jan 27;322(7280):226-31
11159626
J Gen Intern Med. 2002 Nov;17(11):839-44
12406355
Epidemiology. 2001 May;12(3):288-90
11337598
Lancet. 2005 Apr 23-29;365(9469):1500-5
15850636
Acad Med. 1998 May;73(5):538-40
9609869
Cogn Process. 2010 May;11(2):103-21
19784854
Epidemiology. 2001 May;12(3):291-4
11337599
JAMA. 1987 May 8;257(18):2459-63
3573245
JAMA. 1994 Mar 2;271(9):703-7
8309035
BMC Med Res Methodol. 2010 May 19;10:44
20482841
BMJ. 2005 May 7;330(7499):1080-3
15879401
Science. 1974 Sep 27;185(4157):1124-31
17835457
Ann Intern Med. 1999 Jun 15;130(12):995-1004
10383371
BMJ. 1999 Aug 21;319(7208):508-12
10454409