title: Interpretation of evidence in data by untrained medical students: a scenario-based study.
abstract: BACKGROUND
  To determine which approach to assessment of evidence in data - statistical tests or likelihood ratios - comes closest to the interpretation of evidence by untrained medical students.
  
  
  METHODS
  Empirical study of medical students (N = 842), untrained in statistical inference or in the interpretation of diagnostic tests. They were asked to interpret a hypothetical diagnostic test, presented in four versions that differed in the distributions of test scores in diseased and non-diseased populations. Each student received only one version. The intuitive application of the statistical test approach would lead to rejecting the null hypothesis of no disease in version A, and to accepting the null in version B. Application of the likelihood ratio approach led to opposite conclusions - against the disease in A, and in favour of disease in B. Version C tested the importance of the p-value (A: 0.04 versus C: 0.08) and version D the importance of the likelihood ratio (C: 1/4 versus D: 1/8).
  
  
  RESULTS
  In version A, 7.5% concluded that the result was in favour of disease (compatible with p value), 43.6% ruled against the disease (compatible with likelihood ratio), and 48.9% were undecided. In version B, 69.0% were in favour of disease (compatible with likelihood ratio), 4.5% against (compatible with p value), and 26.5% undecided. Increasing the p value from 0.04 to 0.08 did not change the results. The change in the likelihood ratio from 1/4 to 1/8 increased the proportion of non-committed responses.
  
  
  CONCLUSIONS
  Most untrained medical students appear to interpret evidence from data in a manner that is compatible with the use of likelihood ratios.
authors.affiliation: Division of Clinical Epidemiology, Geneva University Hospitals, Geneva, Switzerland and Faculty of Medicine, University of Geneva, Geneva, Switzerland. thomas.perneger@hcuge.ch
authors.forename: Thomas V
authors.initials: TV
authors.lastname: Perneger
authors.forename: Delphine S
authors.initials: DS
authors.lastname: Courvoisier
mesh_terms: D000328:Adult; D001530:Belgium; D006302:Health Services Research; D006801:Humans; D007407:Interviews as Topic; D008875:Middle Aged; D010821:Physicians, Family / Q000523:psychology*; D019338:Polypharmacy; D012424:Rural Population; D014505:Urban Population
references.citation: JAMA. 2007 Sep 5;298(9):1010-22
references.pmid: 17785646
references.citation: Ann Intern Med. 1999 Jun 15;130(12):1005-13
references.pmid: 10383350
references.citation: Int J Epidemiol. 2003 Oct;32(5):687-91
references.pmid: 14559729
references.citation: J Clin Epidemiol. 2006 Sep;59(9):1009-14
references.pmid: 16895826
references.citation: BMJ. 2001 Jan 27;322(7280):226-31
references.pmid: 11159626
references.citation: J Gen Intern Med. 2002 Nov;17(11):839-44
references.pmid: 12406355
references.citation: Epidemiology. 2001 May;12(3):288-90
references.pmid: 11337598
references.citation: Lancet. 2005 Apr 23-29;365(9469):1500-5
references.pmid: 15850636
references.citation: Acad Med. 1998 May;73(5):538-40
references.pmid: 9609869
references.citation: Cogn Process. 2010 May;11(2):103-21
references.pmid: 19784854
references.citation: Epidemiology. 2001 May;12(3):291-4
references.pmid: 11337599
references.citation: JAMA. 1987 May 8;257(18):2459-63
references.pmid: 3573245
references.citation: JAMA. 1994 Mar 2;271(9):703-7
references.pmid: 8309035
references.citation: BMC Med Res Methodol. 2010 May 19;10:44
references.pmid: 20482841
references.citation: BMJ. 2005 May 7;330(7499):1080-3
references.pmid: 15879401
references.citation: Science. 1974 Sep 27;185(4157):1124-31
references.pmid: 17835457
references.citation: Ann Intern Med. 1999 Jun 15;130(12):995-1004
references.pmid: 10383371
references.citation: BMJ. 1999 Aug 21;319(7208):508-12
references.pmid: 10454409