title: ChemTok: A New Rule Based Tokenizer for Chemical Named Entity Recognition.
abstract: Named Entity Recognition (NER) from text constitutes the first step in many text mining applications. The most important preliminary step for NER systems using machine learning approaches is tokenization where raw text is segmented into tokens. This study proposes an enhanced rule based tokenizer, ChemTok, which utilizes rules extracted mainly from the train data set. The main novelty of ChemTok is the use of the extracted rules in order to merge the tokens split in the previous steps, thus producing longer and more discriminative tokens. ChemTok is compared to the tokenization methods utilized by ChemSpot and tmChem. Support Vector Machines and Conditional Random Fields are employed as the learning algorithms. The experimental results show that the classifiers trained on the output of ChemTok outperforms all classifiers trained on the output of the other two tokenizers in terms of classification performance, and the number of incorrectly segmented entities.
authors.affiliation: Computer Engineering Department, Eastern Mediterranean University, Famagusta, Northern Cyprus, Mersin 10, Turkey.
authors.forename: Abbas
authors.initials: A
authors.lastname: Akkasi
authors.affiliation: Computer Engineering Department, Eastern Mediterranean University, Famagusta, Northern Cyprus, Mersin 10, Turkey.
authors.forename: Ekrem
authors.initials: E
authors.lastname: Varoğlu
authors.affiliation: Information Technology Department, Eastern Mediterranean University, Famagusta, Northern Cyprus, Mersin 10, Turkey.
authors.forename: Nazife
authors.initials: N
authors.lastname: Dimililer
mesh_terms: D000465:Algorithms; D055598:Chemical Phenomena; D057225:Data Mining / Q000379:methods*; D000069550:Machine Learning; D009323:Natural Language Processing; D012984:Software; D060388:Support Vector Machine; D009626:Terminology as Topic