title: Automating annotation of information-giving for analysis of clinical conversation.
abstract: OBJECTIVE
  Coding of clinical communication for fine-grained features such as speech acts has produced a substantial literature. However, annotation by humans is laborious and expensive, limiting application of these methods. We aimed to show that through machine learning, computers could code certain categories of speech acts with sufficient reliability to make useful distinctions among clinical encounters.
  
  
  METHODS
  The data were transcripts of 415 routine outpatient visits of HIV patients which had previously been coded for speech acts using the Generalized Medical Interaction Analysis System (GMIAS); 50 had also been coded for larger scale features using the Comprehensive Analysis of the Structure of Encounters System (CASES). We aggregated selected speech acts into information-giving and requesting, then trained the machine to automatically annotate using logistic regression classification. We evaluated reliability by per-speech act accuracy. We used multiple regression to predict patient reports of communication quality from post-visit surveys using the patient and provider information-giving to information-requesting ratio (briefly, information-giving ratio) and patient gender.
  
  
  RESULTS
  Automated coding produces moderate reliability with human coding (accuracy 71.2%, κ=0.57), with high correlation between machine and human prediction of the information-giving ratio (r=0.96). The regression significantly predicted four of five patient-reported measures of communication quality (r=0.263-0.344).
  
  
  CONCLUSIONS
  The information-giving ratio is a useful and intuitive measure for predicting patient perception of provider-patient communication quality. These predictions can be made with automated annotation, which is a practical option for studying large collections of clinical encounters with objectivity, consistency, and low cost, providing greater opportunity for training and reflection for care providers.
authors.affiliation: Language Technologies Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA.
authors.forename: Elijah
authors.initials: E
authors.lastname: Mayfield
authors.forename: M Barton
authors.initials: MB
authors.lastname: Laws
authors.forename: Ira B
authors.initials: IB
authors.lastname: Wilson
authors.forename: Carolyn
authors.initials: C
authors.lastname: Penstein Rosé
mesh_terms: D001185:Artificial Intelligence; D059019:Clinical Coding / Q000379:methods*; D003142:Communication; D001330:Electronic Data Processing; D015658:HIV Infections; D006801:Humans; D008499:Medical Records / Q000145:classification; D013060:Speech
keywords: Automated Annotation; Computational Linguistics; Machine Learning; Patient-provider Communication
references.citation: J Palliat Med. 2010 May;13(5):595-603
references.pmid: 20491550
references.citation: Patient Educ Couns. 2009 Oct;77(1):60-7
references.pmid: 19395222
references.citation: J Am Med Inform Assoc. 2010 Sep-Oct;17(5):536-9
references.pmid: 20819859
references.citation: J Med Internet Res. 2009 Nov 27;11(4):e48
references.pmid: 19945947
references.citation: AIDS Behav. 2011 May;15(4):805-11
references.pmid: 20066486
references.citation: Patient Educ Couns. 2009 May;75(2):169-77
references.pmid: 19097842
references.citation: BMJ. 2004 May 15;328(7449):1166
references.pmid: 15142921
references.citation: CMAJ. 1995 May 1;152(9):1423-33
references.pmid: 7728691
references.citation: Soc Sci Med. 2009 Feb;68(3):533-43
references.pmid: 19019520
references.citation: Patient Educ Couns. 2006 Mar;60(3):301-12
references.pmid: 16051459
references.citation: Ann Fam Med. 2004 Nov-Dec;2(6):595-608
references.pmid: 15576546
references.citation: Med Care. 1999 May;37(5):510-7
references.pmid: 10335753
references.citation: Patient Educ Couns. 2013 May;91(2):192-9
references.pmid: 23391684
references.citation: Milbank Q. 1999;77(3):305-39, 274
references.pmid: 10526547
references.citation: Soc Sci Med. 2000 Oct;51(7):1087-110
references.pmid: 11005395
references.citation: J Am Med Inform Assoc. 2013 Jan 1;20(1):84-94
references.pmid: 22859645
references.citation: Med Care. 2009 Aug;47(8):826-34
references.pmid: 19584762
references.citation: Med Care. 1990 Jun;28(6):473-88
references.pmid: 2355754
references.citation: J Am Med Inform Assoc. 2011 Sep-Oct;18(5):568-73
references.pmid: 21724741
references.citation: J Gen Intern Med. 1988 Sep-Oct;3(5):448-57
references.pmid: 3049968
references.citation: J Gen Intern Med. 2002 Apr;17(4):243-52
references.pmid: 11972720
references.citation: J Gen Intern Med. 1989 Nov-Dec;4(6):506-11
references.pmid: 2585158
references.citation: AIDS Behav. 2013 Jan;17(1):148-59
references.pmid: 22290609
references.citation: J Am Med Inform Assoc. 2011 Sep-Oct;18(5):580-7
references.pmid: 21709161
references.citation: J Am Board Fam Pract. 2002 Jan-Feb;15(1):25-38
references.pmid: 11841136