title: The reliability of assessing the appropriateness of requested diagnostic tests.
abstract: Despite a poor reliability, peer assessment is the traditional method to assess the appropriateness of health care activities. This article describes the reliability of the human assessment of the appropriateness of diagnostic tests requests. The authors used a random selection of 1217 tests from 253 request forms submitted by general practitioners in the Maastricht region of The Netherlands. Three reviewers independently assessed the appropriateness of each requested test. Interrater kappa values ranged from 0.33 to 0.42, and kappa values of intrarater agreement ranged from 0.48 to 0.68. The joint reliability coefficient of the 3 reviewers was 0.66. This reliability is sufficient to review test ordering over a series of cases but is not sufficient to make case-by-case assessments. Sixteen reviewers are needed to obtain a joint reliability of 0.95. The authors conclude that there is substantial variation in assessment concerning what is an appropriately requested diagnostic test and that this feedback method is not reliable enough to make a case-by-case assessment. Computer support maybe beneficial to support and make the process of peer review more uniform.
authors.affiliation: Department of Medical Informatics, University of Mastricht, The Netherlands. R.Bindels@mi.unimaas.nl
authors.forename: Rianne
authors.initials: R
authors.lastname: Bindels
authors.forename: Arie
authors.initials: A
authors.lastname: Hasman
authors.forename: Jan W J
authors.initials: JW
authors.lastname: van Wersch
authors.forename: Peter
authors.initials: P
authors.lastname: Pop
authors.forename: Ron A G
authors.initials: RA
authors.lastname: Winkens
mesh_terms: D020000:Decision Support Systems, Clinical; D019937:Diagnostic Techniques and Procedures / Q000706:statistics & numerical data*; D005194:Family Practice; D006801:Humans; D009426:Netherlands; D015588:Observer Variation; D018024:Peer Review, Health Care / Q000592:standards*; D017410:Practice Guidelines as Topic; D010818:Practice Patterns, Physicians'; D015203:Reproducibility of Results; D014600:Utilization Review / Q000592:standards*