Parallel workflow manager for non-parallel bioinformatic applications to solve large-scale biological problems on a supercomputer.
Rapid expansion of online resources providing access to genomic, structural, and functional information associated with biological macromolecules opens an opportunity to gain a deeper understanding of the mechanisms of biological processes due to systematic analysis of large datasets. This, however, requires novel strategies to optimally utilize computer processing power. Some methods in bioinformatics and molecular modeling require extensive computational resources. Other algorithms have fast implementations which take at most several hours to analyze a common input on a modern desktop station, however, due to multiple invocations for a large number of subtasks the full task requires a significant computing power. Therefore, an efficient computational solution to large-scale biological problems requires both a wise parallel implementation of resource-hungry methods as well as a smart workflow to manage multiple invocations of relatively fast algorithms. In this work, a new computer software mpiWrapper has been developed to accommodate non-parallel implementations of scientific algorithms within the parallel supercomputing environment. The Message Passing Interface has been implemented to exchange information between nodes. Two specialized threads - one for task management and communication, and another for subtask execution - are invoked on each processing unit to avoid deadlock while using blocking calls to MPI. The mpiWrapper can be used to launch all conventional Linux applications without the need to modify their original source codes and supports resubmission of subtasks on node failure. We show that this approach can be used to process huge amounts of biological data efficiently by running non-parallel programs in parallel mode on a supercomputer. The C++ source code and documentation are available from http://biokinet.belozersky.msu.ru/mpiWrapper .
* Lomonosov Moscow State University, Belozersky Institute of Physicochemical Biology and Faculty of Bioengineering and Bioinformatics, Leninskiye Gory 1-73, Moscow 119991, Russia.
Dmitry
D
Suplatov
† Lomonosov Moscow State University, Faculty of Computational Mathematics and Cybernetics, Leninskiye Gory 1-52, Moscow 119991, Russia.
Nina
N
Popova
‡ Lomonosov Moscow State University, Research Computing Center, Leninskiye Gory 1-4, Moscow 119991, Russia.
Sergey
S
Zhumatiy
† Lomonosov Moscow State University, Faculty of Computational Mathematics and Cybernetics, Leninskiye Gory 1-52, Moscow 119991, Russia.|‡ Lomonosov Moscow State University, Research Computing Center, Leninskiye Gory 1-4, Moscow 119991, Russia.
Vladimir
V
Voevodin
* Lomonosov Moscow State University, Belozersky Institute of Physicochemical Biology and Faculty of Bioengineering and Bioinformatics, Leninskiye Gory 1-73, Moscow 119991, Russia.
Vytas
V
Švedas
D000465:Algorithms; D019295:Computational Biology / Q000295:instrumentation / Q000379:methods*; D003205:Computing Methodologies; D011381:Programming Languages; D016415:Sequence Alignment / Q000379:methods; D012984:Software; D057188:Workflow
Bioinformatics; Message Passing Interface; Penicillin acylase; computational biology; large-scale data analysis; mpiWrapper; programming languages; resource-intensive applications; supercomputer cluster