title: Text classification for assisting moderators in online health communities.
abstract: OBJECTIVE
  Patients increasingly visit online health communities to get help on managing health. The large scale of these online communities makes it impossible for the moderators to engage in all conversations; yet, some conversations need their expertise. Our work explores low-cost text classification methods to this new domain of determining whether a thread in an online health forum needs moderators' help.
  
  
  METHODS
  We employed a binary classifier on WebMD's online diabetes community data. To train the classifier, we considered three feature types: (1) word unigram, (2) sentiment analysis features, and (3) thread length. We applied feature selection methods based on χ² statistics and under sampling to account for unbalanced data. We then performed a qualitative error analysis to investigate the appropriateness of the gold standard.
  
  
  RESULTS
  Using sentiment analysis features, feature selection methods, and balanced training data increased the AUC value up to 0.75 and the F1-score up to 0.54 compared to the baseline of using word unigrams with no feature selection methods on unbalanced data (0.65 AUC and 0.40 F1-score). The error analysis uncovered additional reasons for why moderators respond to patients' posts.
  
  
  CONCLUSIONS
  We showed how feature selection methods and balanced training data can improve the overall classification performance. We present implications of weighing precision versus recall for assisting moderators of online health communities. Our error analysis uncovered social, legal, and ethical issues around addressing community members' needs. We also note challenges in producing a gold standard, and discuss potential solutions for addressing these challenges.
  
  
  CONCLUSIONS
  Social media environments provide popular venues in which patients gain health-related information. Our work contributes to understanding scalable solutions for providing moderators' expertise in these large-scale, social media environments.
authors.affiliation: Department of Telecommunication, Information Studies, and Media, Michigan State University, 404 Wilson Rd, Rm 409, East Lansing, MI 48864, USA. Electronic address: jinahuh@msu.edu.
authors.forename: Jina
authors.initials: J
authors.lastname: Huh
authors.forename: Meliha
authors.initials: M
authors.lastname: Yetisgen-Yildiz
authors.forename: Wanda
authors.initials: W
authors.lastname: Pratt
mesh_terms: D020407:Internet; D012944:Social Support; D060145:Text Messaging
keywords: Consumer health; Health information seeking; Human–computer interaction; Online health communities; Text mining
references.citation: J Med Internet Res. 2011 Aug 16;13(3):e62
references.pmid: 21846635
references.citation: Arch Gen Psychiatry. 1979 Nov;36(12):1331-7
references.pmid: 496553
references.citation: J Biomed Inform. 2011 Apr;44(2):277-88
references.pmid: 21256977
references.citation: J Am Med Inform Assoc. 2013 Dec;20(e2):e253-9
references.pmid: 23851443
references.citation: SAR QSAR Environ Res. 2005 Dec;16(6):517-29
references.pmid: 16428129
references.citation: AMIA Annu Symp Proc. 2013 Nov 16;2013:627-36
references.pmid: 24551364
references.citation: Diabetes Care. 2005 Mar;28(3):626-31
references.pmid: 15735199
references.citation: Stud Health Technol Inform. 2008;136:407-12
references.pmid: 18487765
references.citation: J Med Internet Res. 2010 Oct 05;12(4):e45
references.pmid: 20923755
references.citation: J Am Med Inform Assoc. 2013 Sep-Oct;20(5):887-90
references.pmid: 23543111
references.citation: J Gen Psychol. 2007 Oct;134(4):405-13
references.pmid: 18183737
references.citation: J Med Internet Res. 2009 Jul 22;11(3):e25
references.pmid: 19632978
references.citation: J Biomed Inform. 2011 Dec;44(6):1032-8
references.pmid: 21856442
references.citation: Proc SIGCHI Conf Hum Factor Comput Syst. 2012;2012:923-926
references.pmid: 24634893
references.citation: AMIA Annu Symp Proc. 2006;:469-73
references.pmid: 17238385
references.citation: J Am Med Inform Assoc. 2010 May-Jun;17(3):229-36
references.pmid: 20442139
references.citation: Psychol Sci. 2003 Jan;14(1):60-5
references.pmid: 12564755
references.citation: Stud Health Technol Inform. 2003;95:463-8
references.pmid: 14664030