title: Interrater reliability for the Four Habits Coding Scheme as part of a randomized controlled trial.
abstract: OBJECTIVE
  To describe the process for developing interrater reliability (IRR) for the Four Habits Coding Scheme (4HCS) for a heterogeneous material as part of a randomized controlled trial.
  
  
  METHODS
  Videotapes from 497 hospital encounters involving 71 doctors from most clinical specialties were collected. Four experienced psychology students were trained as raters. We calculated Pearson's r and the intraclass correlation (ICC) on the total score across consecutive samples of twenty videos, and Pearson's r on single videos across items in the initial coding phase.
  
  
  RESULTS
  After 18h of training and one rating session, the total score Pearson's r and ICC exceeded .70 for all pairs of raters. Across items within single videos, the Pearson's r was never below 0.60 after the first 50 videos. At item and habit level Pearson's r remained unsatisfactory for some rater pairs mostly due to low variance on some items.
  
  
  CONCLUSIONS
  Based on the evaluation of the effect of communication skills training via a total score, IRR was satisfactory for the 4HCS as applied to heterogeneous material. However, good reliability at item level was difficult to achieve.
  
  
  CONCLUSIONS
  4HCS may be used as an outcome measure for clinical communication skills in randomized controlled trials.
authors.affiliation: HØKH, Research Centre, Akershus University Hospital, Lørenskog, Norway. b.f.jensen@medisin.uio.no
authors.forename: Bård
authors.initials: B
authors.lastname: Fossli Jensen
authors.forename: Pål
authors.initials: P
authors.lastname: Gulbrandsen
authors.forename: Jurate Saltyte
authors.initials: JS
authors.lastname: Benth
authors.forename: Fredrik A
authors.initials: FA
authors.lastname: Dahl
authors.forename: Edward
authors.initials: E
authors.lastname: Krupat
authors.forename: Arnstein
authors.initials: A
authors.lastname: Finset
mesh_terms: D000293:Adolescent; D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D002648:Child; D002675:Child, Preschool; D059019:Clinical Coding / Q000592:standards*; D003142:Communication; D018592:Cross-Over Studies; D005260:Female; D006184:Habits; D006769:Hospitals, General; D006801:Humans; D007223:Infant; D007231:Infant, Newborn; D008297:Male; D008875:Middle Aged; D009664:Norway; D015588:Observer Variation; D010819:Physician's Role / Q000523:psychology; D010817:Physician-Patient Relations; D011361:Professional Competence / Q000592:standards*; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires; D014743:Videotape Recording; D055815:Young Adult