title: Profiling quality of care: Is there a role for peer review?
abstract: BACKGROUND
  We sought to develop a more reliable structured implicit chart review instrument for use in assessing the quality of care for chronic disease and to examine if ratings are more reliable for conditions in which the evidence base for practice is more developed.
  
  
  METHODS
  We conducted a reliability study in a cohort with patient records including both outpatient and inpatient care as the objects of measurement. We developed a structured implicit review instrument to assess the quality of care over one year of treatment. 12 reviewers conducted a total of 496 reviews of 70 patient records selected from 26 VA clinical sites in two regions of the country. Each patient had between one and four conditions specified as having a highly developed evidence base (diabetes and hypertension) or a less developed evidence base (chronic obstructive pulmonary disease or a collection of acute conditions). Multilevel analysis that accounts for the nested and cross-classified structure of the data was used to estimate the signal and noise components of the measurement of quality and the reliability of implicit review.
  
  
  RESULTS
  For COPD and a collection of acute conditions the reliability of a single physician review was quite low (intra-class correlation = 0.16-0.26) but comparable to most previously published estimates for the use of this method in inpatient settings. However, for diabetes and hypertension the reliability is significantly higher at 0.46. The higher reliability is a result of the reviewers collectively being able to distinguish more differences in the quality of care between patients (p < 0.007) and not due to less random noise or individual reviewer bias in the measurement. For these conditions the level of true quality (i.e. the rating of quality of care that would result from the full population of physician reviewers reviewing a record) varied from poor to good across patients.
  
  
  CONCLUSIONS
  For conditions with a well-developed quality of care evidence base, such as hypertension and diabetes, a single structured implicit review to assess the quality of care over a period of time is moderately reliable. This method could be a reasonable complement or alternative to explicit indicator approaches for assessing and comparing quality of care. Structured implicit review, like explicit quality measures, must be used more cautiously for illnesses for which the evidence base is less well developed, such as COPD and acute, short-course illnesses.
authors.affiliation: Veterans Affairs Health Services Research and Development Center of Excellence, Veterans Affairs Ann Arbor Healthcare System, Michigan, USA. thofer@umich.edu
authors.forename: Timothy P
authors.initials: TP
authors.lastname: Hofer
authors.forename: Steven M
authors.initials: SM
authors.lastname: Asch
authors.forename: Rodney A
authors.initials: RA
authors.lastname: Hayward
authors.forename: Lisa V
authors.initials: LV
authors.lastname: Rubenstein
authors.forename: Mary M
authors.initials: MM
authors.lastname: Hogan
authors.forename: John
authors.initials: J
authors.lastname: Adams
authors.forename: Eve A
authors.initials: EA
authors.lastname: Kerr
mesh_terms: D000208:Acute Disease / Q000628:therapy; D002908:Chronic Disease / Q000145:classification / Q000628:therapy; D015331:Cohort Studies; D003266:Continuity of Patient Care; D003920:Diabetes Mellitus / Q000628:therapy*; D019468:Disease Management; D019317:Evidence-Based Medicine; D006300:Health Services Misuse; D006801:Humans; D006973:Hypertension / Q000628:therapy*; D007388:Internal Medicine / Q000592:standards*; D015141:Los Angeles; D008499:Medical Records; D008824:Michigan; D015588:Observer Variation; D010043:Outcome and Process Assessment, Health Care; D018024:Peer Review, Health Care / Q000379:methods*; D011320:Primary Health Care / Q000592:standards*; D029424:Pulmonary Disease, Chronic Obstructive / Q000628:therapy*; D011785:Quality Assurance, Health Care / Q000379:methods*; D014728:Veterans