title: Overview of the interactive task in BioCreative V.
abstract: Fully automated text mining (TM) systems promote efficient literature searching, retrieval, and review but are not sufficient to produce ready-to-consume curated documents. These systems are not meant to replace biocurators, but instead to assist them in one or more literature curation steps. To do so, the user interface is an important aspect that needs to be considered for tool adoption. The BioCreative Interactive task (IAT) is a track designed for exploring user-system interactions, promoting development of useful TM tools, and providing a communication channel between the biocuration and the TM communities. In BioCreative V, the IAT track followed a format similar to previous interactive tracks, where the utility and usability of TM tools, as well as the generation of use cases, have been the focal points. The proposed curation tasks are user-centric and formally evaluated by biocurators. In BioCreative V IAT, seven TM systems and 43 biocurators participated. Two levels of user participation were offered to broaden curator involvement and obtain more feedback on usability aspects. The full level participation involved training on the system, curation of a set of documents with and without TM assistance, tracking of time-on-task, and completion of a user survey. The partial level participation was designed to focus on usability aspects of the interface and not the performance per se In this case, biocurators navigated the system by performing pre-designed tasks and then were asked whether they were able to achieve the task and the level of difficulty in completing the task. In this manuscript, we describe the development of the interactive task, from planning to execution and discuss major findings for the systems tested.Database URL: http://www.biocreative.org.
authors: Center for Bioinformatics and Computational Biology, University of Delaware, Newark, DE, 19711, USA Department of Computer and Information Sciences, University of Delaware, Newark, DE, 19711, USA.
authors: Qinghua
authors: Q
authors: Wang
authors: International Centre of Health Information Technology, Taipei Medical University, Taipei, Taiwan.
authors: Shabbir
authors: S
authors: S Abdul
authors: DETI/IEETA, University of Aveiro, Campus Universitário de Santiago, Aveiro 3810-193, Portugal.
authors: Lara
authors: L
authors: Almeida
authors: National Centre for Text Mining, University of Manchester, Manchester, UK.
authors: Sophia
authors: S
authors: Ananiadou
authors: Facultad de Ciencias, Universidad Nacional Autónoma de México, 04510 Ciudad de México, México.
authors: Yalbi I
authors: YI
authors: Balderas-Martínez
authors: National Centre for Text Mining, University of Manchester, Manchester, UK.
authors: Riza
authors: R
authors: Batista-Navarro
authors: BMD Software, Aveiro, Portugal.
authors: David
authors: D
authors: Campos
authors: Northern Institute for Cancer Research, Newcastle University, New Castle, UK.
authors: Lucy
authors: L
authors: Chilton
authors: Rutgers University-Camden, Camden, NJ 08102, USA.
authors: Hui-Jou
authors: HJ
authors: Chou
authors: Centro de Ciencias Genómicas, Universidad Nacional Autónoma de México, 04510 Ciudad de México, México.
authors: Gabriela
authors: G
authors: Contreras
authors: Department of Botany and Plant Pathology, Oregon State University Corvallis, OR 97331, USA.
authors: Laurel
authors: L
authors: Cooper
authors: Department of Computer Science and Information Engineering, National Taitung University, Taitung, Taiwan.
authors: Hong-Jie
authors: HJ
authors: Dai
authors: College of Agriculture and Natural Resources, University of Delaware, Newark, DE 19711, USA.
authors: Barbra
authors: B
authors: Ferrell
authors: Fraunhofer Institute for Algorithms and Scientific Computing, Schloss Birlinghoven, 53754 St. Augustin, Germany.
authors: Juliane
authors: J
authors: Fluck
authors: Centro de Ciencias Genómicas, Universidad Nacional Autónoma de México, 04510 Ciudad de México, México.
authors: Socorro
authors: S
authors: Gama-Castro
authors: SourceData, EMBO, Heidelberg, Germany.
authors: Nancy
authors: N
authors: George
authors: College of Medical and Dental Sciences, Institute of Cancer and Genomic Sciences, Centre for Computational Biology, University of Birmingham, Birmingham B15 2TT, UK Institute of Translational Medicine, University Hospitals Birmingham NHS Foundation Trust, Birmingham B15 2TT, UK.
authors: Georgios
authors: G
authors: Gkoutos
authors: Life Science Informatics, University of Bonn, Bonn, Germany.
authors: Afroza K
authors: AK
authors: Irin
authors: Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.
authors: Lars J
authors: LJ
authors: Jensen
authors: Blue Brain Project, École Polytechnique Fédérale de Lausanne (EPFL) Biotech Campus, Geneva, Switzerland.
authors: Silvia
authors: S
authors: Jimenez
authors: Prince of Wales Clinical School, University of New South Wales NSW, Sydney, New South Wales, Australia.
authors: Toni R
authors: TR
authors: Jue
authors: SRI International, Menlo Park, CA 94025, USA.
authors: Ingrid
authors: I
authors: Keseler
authors: Fraunhofer Institute for Algorithms and Scientific Computing, Schloss Birlinghoven, 53754 St. Augustin, Germany.
authors: Sumit
authors: S
authors: Madan
authors: DETI/IEETA, University of Aveiro, Campus Universitário de Santiago, Aveiro 3810-193, Portugal.
authors: Sérgio
authors: S
authors: Matos
authors: Oxford e-Research Centre, University of Oxford, Oxford, UK.
authors: Peter
authors: P
authors: McQuilton
authors: Department of Informatics and Bio-Computing, Ontario Institute for Cancer Research, Toronto, ON M5G0A3, Canada.
authors: Marija
authors: M
authors: Milacic
authors: HGMD, Institute of Medical Genetics, Cardiff University, Heath Park, Cardiff, UK.
authors: Matthew
authors: M
authors: Mort
authors: Department of Bioinformatics, Bharathiar University, Coimbatore, Tamil Nadu, India.
authors: Jeyakumar
authors: J
authors: Natarajan
authors: Institute of Marine Biology, Biotechnology and Aquaculture, Hellenic Centre for Marine Research, Heraklion, Crete, Greece.
authors: Evangelos
authors: E
authors: Pafilis
authors: Microbial Genomics and Bioinformatics Group, Max Planck Institute for Marine Microbiology, Bremen, Germany.
authors: Emiliano
authors: E
authors: Pereira
authors: Innovation Center for Biomedical Informatics (ICBI), Georgetown University, Washington, DC 20007, USA.
authors: Shruti
authors: S
authors: Rao
authors: Institute of Computational Linguistics, University of Zurich, Zurich, Switzerland.
authors: Fabio
authors: F
authors: Rinaldi
authors: Department of Informatics and Bio-Computing, Ontario Institute for Cancer Research, Toronto, ON M5G0A3, Canada.
authors: Karen
authors: K
authors: Rothfels
authors: GMGF, Aix-Marseille Universite, 13385 Marseille, France Inserm, UMR_S 910, 13385 Marseille, France.
authors: David
authors: D
authors: Salgado
authors: Department of Medical Sciences, iBiMED & IEETA, University of Aveiro, 3810-193 Aveiro, Portugal.
authors: Raquel M
authors: RM
authors: Silva
authors: Taipei Medical University Graduate Institute of Biomedical informatics, Taipei, Taiwan.
authors: Onkar
authors: O
authors: Singh
authors: Department of Genetics, University of Cambridge, Cambridge, UK.
authors: Raymund
authors: R
authors: Stefancsik
authors: Institute of Information Science, Academia Sinica, Taipei, Taiwan.
authors: Chu-Hsien
authors: CH
authors: Su
authors: Department of Bioinformatics, Bharathiar University, Coimbatore, Tamil Nadu, India.
authors: Suresh
authors: S
authors: Subramani
authors: Freelance Scientific Curator, Cleveland, OH, USA.
authors: Hamsa D
authors: HD
authors: Tadepally
authors: Institute of Sport and Physical Activity Research (ISPAR), University of Bedfordshire, Bedford, UK.
authors: Loukia
authors: L
authors: Tsaprouni
authors: Ontology Development Group, Oregon Health & Science University, Portland, OR 97239, USA.
authors: Nicole
authors: N
authors: Vasilevsky
authors: WormBase Consortium, Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA 91125, USA.
authors: Xiaodong
authors: X
authors: Wang
authors: Institute for Research in Immunology and Cancer, Université de Montréal, Montréal, Canada.
authors: Andrew
authors: A
authors: Chatr-Aryamontri
authors: Medical College of Wisconsin, Milwaukee, WI 53226, USA.
authors: Stanley J F
authors: SJ
authors: Laulederkind
authors: Reed Elsevier, Philadelphia, PA 19103, USA.
authors: Sherri
authors: S
authors: Matis-Mitchell
authors: European Bioinformatics Institute (EMBL-EBI), Hinxton, UK.
authors: Johanna
authors: J
authors: McEntyre
authors: European Bioinformatics Institute (EMBL-EBI), Hinxton, UK.
authors: Sandra
authors: S
authors: Orchard
authors: European Bioinformatics Institute (EMBL-EBI), Hinxton, UK.
authors: Sangya
authors: S
authors: Pundir
authors: Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd, Basel, Switzerland.
authors: Raul
authors: R
authors: Rodriguez-Esteban
authors: WormBase Consortium, Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA 91125, USA.
authors: Kimberly
authors: K
authors: Van Auken
authors: National Center for Biotechnology Information (NCBI), National Institutes of Health, Bethesda, MD 20894, USA.
authors: Zhiyong
authors: Z
authors: Lu
authors: MaizeGDB USDA ARS and University of Missouri, Columbia, MO 65211, USA.
authors: Mary
authors: M
authors: Schaeffer
authors: Center for Bioinformatics and Computational Biology, University of Delaware, Newark, DE, 19711, USA Department of Computer and Information Sciences, University of Delaware, Newark, DE, 19711, USA.
authors: Cathy H
authors: CH
authors: Wu
authors: The MITRE Corporation, Bedford, MA 01730, USA.
authors: Lynette
authors: L
authors: Hirschman
authors: Center for Bioinformatics and Computational Biology, University of Delaware, Newark, DE, 19711, USA Department of Computer and Information Sciences, University of Delaware, Newark, DE, 19711, USA arighi@dbi.udel.edu.
authors: Cecilia N
authors: CN
authors: Arighi
mesh_terms: D066289:Data Curation / Q000379:methods*; D057225:Data Mining / Q000379:methods*; D001330:Electronic Data Processing / Q000379:methods*