title: Inter-rater reliability of explicit indicators of prescribing appropriateness.
abstract: OBJECTIVE
  To assess the inter-rater reliability of 14 explicit indicators of appropriate long-term prescribing.
  
  
  METHODS
  All available data required for the assessment of 59 long-term prescriptions started during a hospital admission for 25 patients were transcribed from the patients' medical records. These transcripts were presented in a standardised format and random order to four raters (two doctors and two pharmacists) who used the indicators to judge the appropriateness of each prescription. Debriefing interviews were held with each rater. An a priori level of acceptable agreement between the raters was set at a weighed kappa of 0.70.
  
  
  RESULTS
  There was no apparent difference between pharmacists and doctors for all findings, so data were combined. Two indicators showed poor agreement, three showed moderate agreement, and nine showed substantial or near perfect agreement, exceeding a weighted kappa of 0.70. There was excellent positive agreement as to which prescriptions were judged appropriate by the indicators, but much worse negative agreement as to which prescriptions were judged to be inappropriate. In the interviews, the raters remarked on the difficulty of applying explicit indicators when they routinely made implicit judgements about data in the medical records.
  
  
  CONCLUSIONS
  Nine of the indicators achieved the required level of reliability and the negative agreement levels showed that this was the area that required greater improvement in future developments of the indicators. Further work needs to be conducted to investigate ways of the improving instructions on how to make explicit judgements and reducing the need for implicit or subjective assessments.
authors.affiliation: School of Pharmacy and Pharmaceutical Sciences, University of Manchester, Manchester, UK. Mary.Tully@manchester.ac.uk
authors.forename: Mary P
authors.initials: MP
authors.lastname: Tully
authors.forename: Judith A
authors.initials: JA
authors.lastname: Cantrill
mesh_terms: D003266:Continuity of Patient Care / Q000706:statistics & numerical data; D011307:Drug Prescriptions / Q000706:statistics & numerical data*; D004363:Drug Utilization / Q000706:statistics & numerical data; D006739:Hospital Administration / Q000706:statistics & numerical data; D006801:Humans; D008499:Medical Records; D015588:Observer Variation; D019984:Quality Indicators, Health Care / Q000706:statistics & numerical data*; D015203:Reproducibility of Results