title: Use of volunteer student abstractors for a retrospective cohort analysis: a study of inter-rater reliability.
abstract: BACKGROUND
  Little is known about the reliability of data collected by abstractors without professional medical training. This investigation sought to determine the level of agreement among untrained volunteer abstractors as part of a study to evaluate the risk assessment of venous thromboembolism in patients who have undergone trauma.
  
  
  METHODS
  Forty-nine paper charts were chosen randomly from a volunteer-reviewed cohort of 2,339 and were compared with those of a single experienced abstractor. Inter-rater agreement was assessed using percent agreement, Cohen's kappa, and prevalence-adjusted bias-adjusted kappa (PABAK).
  
  
  RESULTS
  Of the 71 data points, 28 had perfect agreement. The average agreement across all charts was 97%. Data with imperfect agreement had kappa values between .27 and .96 (mean, .75), with one additional value at zero even though it was associated with an agreement of 94%. PABAK values ranged from .67 to .98 (mean, .91), an average increase of .17 compared with kappa values.
  
  
  CONCLUSIONS
  The performance of volunteers showed outstanding inter-rater reliability; however, limitations of interpretation can influence reliability.
authors.affiliation: Legacy Emanuel Medical Center, 2801 N. Gantenbein Ave. MOB No. 130, Portland, OR 97227, USA.
authors.forename: Yaroslav
authors.initials: Y
authors.lastname: Gritsiouk
authors.forename: Damian
authors.initials: D
authors.lastname: Hegsted
authors.forename: Stuart
authors.initials: S
authors.lastname: Gardiner
authors.forename: Lisa
authors.initials: L
authors.lastname: Merriman
authors.forename: Kelly Dean
authors.initials: KD
authors.lastname: Gubler
mesh_terms: D000043:Abstracting and Indexing / Q000706:statistics & numerical data*; D000293:Adolescent; D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D015331:Cohort Studies; D006801:Humans; D008499:Medical Records; D008875:Middle Aged; D015588:Observer Variation; D012189:Retrospective Studies; D018570:Risk Assessment; D013337:Students, Medical; D013338:Students, Nursing; D054556:Venous Thromboembolism / Q000209:etiology; D014838:Volunteers; D014947:Wounds and Injuries / Q000150:complications; D055815:Young Adult