title: Active Learning-based corpus annotation--the PathoJen experience.
abstract: We report on basic design decisions and novel annotation procedures underlying the development of PathoJen, a corpus of Medline abstracts annotated for pathological phenomena, including diseases as a proper subclass. This named entity type is known to be hard to delineate and capture by annotation guidelines. We here propose a two-category encoding schema where we distinguish short from long mention spans, the first covering standardized terminology (e.g. diseases), the latter accounting for less structured descriptive statements about norm-deviant states, as well as criteria and observations that might signal pathologies. The second design decision relates to the way annotation instances are sampled. Here we subscribe to an Active Learning-based approach which is known to save annotation costs without sacrificing annotation quality by means of a sample bias. By design, Active Learning picks up 'hard' to annotate instances for human annotators, whereas 'easier' ones are passed over to the automatic classifier whose models already incorporate and gradually improve with previous annotation experience.
authors.affiliation: Jena University Language & Information Engineering (JULIE) Lab, Friedrich-Schiller-Universität Jena, Fürstengraben 30, D-07743 Jena, Germany. udo.hahn@uni-jena.de
authors.forename: Udo
authors.initials: U
authors.lastname: Hahn
authors.forename: Elena
authors.initials: E
authors.lastname: Beisswanger
authors.forename: Ekaterina
authors.initials: E
authors.lastname: Buyko
authors.forename: Erik
authors.initials: E
authors.lastname: Faessler
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D006801:Humans; D016239:MEDLINE; D010336:Pathology / Q000145:classification*; D018794:Problem-Based Learning