The BEL information extraction workflow (BELIEF): evaluation in the BioCreative V BEL and IAT track.
Network-based approaches have become extremely important in systems biology to achieve a better understanding of biological mechanisms. For network representation, the Biological Expression Language (BEL) is well designed to collate findings from the scientific literature into biological network models. To facilitate encoding and biocuration of such findings in BEL, a BEL Information Extraction Workflow (BELIEF) was developed. BELIEF provides a web-based curation interface, the BELIEF Dashboard, that incorporates text mining techniques to support the biocurator in the generation of BEL networks. The underlying UIMA-based text mining pipeline (BELIEF Pipeline) uses several named entity recognition processes and relationship extraction methods to detect concepts and BEL relationships in literature. The BELIEF Dashboard allows easy curation of the automatically generated BEL statements and their context annotations. Resulting BEL statements and their context annotations can be syntactically and semantically verified to ensure consistency in the BEL network. In summary, the workflow supports experts in different stages of systems biology network building. Based on the BioCreative V BEL track evaluation, we show that the BELIEF Pipeline automatically extracts relationships with an F-score of 36.4% and fully correct statements can be obtained with an F-score of 30.8%. Participation in the BioCreative V Interactive task (IAT) track with BELIEF revealed a systems usability scale (SUS) of 67. Considering the complexity of the task for new users-learning BEL, working with a completely new interface, and performing complex curation-a score so close to the overall SUS average highlights the usability of BELIEF.Database URL: BELIEF is available at http://www.scaiview.com/belief/.
Fraunhofer Institute for Algorithms and Scientific Computing, Schloss Birlinghoven, Sankt Augustin, Germany sumit.madan@scai.fraunhofer.de juliane.fluck@scai.fraunhofer.de.
Sumit
S
Madan
Fraunhofer Institute for Algorithms and Scientific Computing, Schloss Birlinghoven, Sankt Augustin, Germany.
Sven
S
Hodapp
Fraunhofer Institute for Algorithms and Scientific Computing, Schloss Birlinghoven, Sankt Augustin, Germany.
Philipp
P
Senger
Philip Morris International R&D, Philip Morris Products S.A, Quai Jeanrenaud 5, Neuchâtel, 2000, Switzerland.
Sam
S
Ansari
Philip Morris International R&D, Philip Morris Products S.A, Quai Jeanrenaud 5, Neuchâtel, 2000, Switzerland.
Justyna
J
Szostak
Philip Morris International R&D, Philip Morris Products S.A, Quai Jeanrenaud 5, Neuchâtel, 2000, Switzerland.
Julia
J
Hoeng
Philip Morris International R&D, Philip Morris Products S.A, Quai Jeanrenaud 5, Neuchâtel, 2000, Switzerland.
Manuel
M
Peitsch
Fraunhofer Institute for Algorithms and Scientific Computing, Schloss Birlinghoven, Sankt Augustin, Germany sumit.madan@scai.fraunhofer.de juliane.fluck@scai.fraunhofer.de.
Juliane
J
Fluck
D057225:Data Mining / Q000379:methods*; D000069550:Machine Learning; D008954:Models, Biological; D011381:Programming Languages
BMC Bioinformatics. 2008 Apr 11;9 Suppl 3:S6
18426551
Nat Rev Genet. 2011 Jan;12(1):56-68
21164525
Drug Discov Today. 2012 May;17(9-10):413-8
22155224
Genome Biol. 2008;9 Suppl 2:S2
18834493
Pac Symp Biocomput. 2008;:652-63
18229723
Database (Oxford). 2015 Apr 17;2015:bav030
25887162
BMC Bioinformatics. 2014 Jul 11;15:238
25015298
Drug Discov Today. 2014 Feb;19(2):193-8
24444544
Nature. 2008 Sep 4;455(7209):47-50
18769432
Database (Oxford). 2014 Aug 25;2014:
25157073
Pac Symp Biocomput. 2003;:403-14
12603045
Database (Oxford). 2016 Jul 09;2016:
27402677
Genome Biol. 2008;9 Suppl 2:S3
18834494
Database (Oxford). 2015 Jun 17;2015:bav057
26200752
BMC Bioinformatics. 2009 Jul 21;10:228
19622167
BMC Syst Biol. 2011 Oct 19;5:168
22011616
PLoS Comput Biol. 2012;8(12):e1002820
23300411
BMC Bioinformatics. 2012 Jun 26;13 Suppl 11:S4
22759458
Database (Oxford). 2014 May 22;2014:
24852177
BMC Bioinformatics. 2013 Mar 22;14:104
23517090
BMC Bioinformatics. 2005;6 Suppl 1:S14
15960826
Database (Oxford). 2016 Aug 23;2016:
27554092
Nat Biotechnol. 2010 Sep;28(9):935-42
20829833
Front Neuroinform. 2014 Apr 29;8:40
24808858
Database (Oxford). 2009;2009:bap019
20157492