title: Natural language processing in the electronic medical record: assessing clinician adherence to tobacco treatment guidelines.
abstract: BACKGROUND
  Comprehensively assessing care quality with electronic medical records (EMRs) is not currently possible because much data reside in clinicians' free-text notes.
  
  
  METHODS
  We evaluated the accuracy of MediClass, an automated, rule-based classifier of the EMR that incorporates natural language processing, in assessing whether clinicians: (1) asked if the patient smoked; (2) advised them to stop; (3) assessed their readiness to quit; (4) assisted them in quitting by providing information or medications; and (5) arranged for appropriate follow-up care (i.e., the 5A's of smoking-cessation care).
  
  
  METHODS
  We analyzed 125 medical records of known smokers at each of four HMOs in 2003 and 2004. One trained abstractor at each HMO manually coded all 500 records according to whether or not each of the 5A's of smoking cessation care was addressed during routine outpatient visits.
  
  
  METHODS
  For each patient's record, we compared the presence or absence of each of the 5A's as assessed by each human coder and by MediClass. We measured the chance-corrected agreement between the human raters and MediClass using the kappa statistic.
  
  
  RESULTS
  For "ask" and "assist," agreement among human coders was indistinguishable from agreement between humans and MediClass (p>0.05). For "assess" and "advise," the human coders agreed more with each other than they did with MediClass (p<0.01); however, MediClass performance was sufficient to assess quality in these areas. The frequency of "arrange" was too low to be analyzed.
  
  
  CONCLUSIONS
  MediClass performance appears adequate to replace human coders of the 5A's of smoking-cessation care, allowing for automated assessment of clinician adherence to one of the most important, evidence-based guidelines in preventive health care.
authors.affiliation: Center for Health Research, Kaiser Permanente, Portland, Oregon 97227, USA. brian.hazlehurst@kpchr.org
authors.forename: Brian
authors.initials: B
authors.lastname: Hazlehurst
authors.forename: Dean F
authors.initials: DF
authors.lastname: Sittig
authors.forename: Victor J
authors.initials: VJ
authors.lastname: Stevens
authors.forename: K Sabina
authors.initials: KS
authors.lastname: Smith
authors.forename: Jack F
authors.initials: JF
authors.lastname: Hollis
authors.forename: Thomas M
authors.initials: TM
authors.lastname: Vogt
authors.forename: Jonathan P
authors.initials: JP
authors.lastname: Winickoff
authors.forename: Russ
authors.initials: R
authors.lastname: Glasgow
authors.forename: Ted E
authors.initials: TE
authors.lastname: Palen
authors.forename: Nancy A
authors.initials: NA
authors.lastname: Rigotti
mesh_terms: D003376:Counseling / Q000379:methods*; D017408:Guidelines as Topic; D008485:Medical Audit; D016347:Medical Records Systems, Computerized; D009323:Natural Language Processing; D010818:Practice Patterns, Physicians' / Q000592:standards*; D016540:Smoking Cessation; D012985:Software Design; D014481:United States