title: Solving global shallow water equations on heterogeneous supercomputers.
abstract: The scientific demand for more accurate modeling of the climate system calls for more computing power to support higher resolutions, inclusion of more component models, more complicated physics schemes, and larger ensembles. As the recent improvements in computing power mostly come from the increasing number of nodes in a system and the integration of heterogeneous accelerators, how to scale the computing problems onto more nodes and various kinds of accelerators has become a challenge for the model development. This paper describes our efforts on developing a highly scalable framework for performing global atmospheric modeling on heterogeneous supercomputers equipped with various accelerators, such as GPU (Graphic Processing Unit), MIC (Many Integrated Core), and FPGA (Field Programmable Gate Arrays) cards. We propose a generalized partition scheme of the problem domain, so as to keep a balanced utilization of both CPU resources and accelerator resources. With optimizations on both computing and memory access patterns, we manage to achieve around 8 to 20 times speedup when comparing one hybrid GPU or MIC node with one CPU node with 12 cores. Using a customized FPGA-based data-flow engines, we see the potential to gain another 5 to 8 times improvement on performance. On heterogeneous supercomputers, such as Tianhe-1A and Tianhe-2, our framework is capable of achieving ideally linear scaling efficiency, and sustained double-precision performances of 581 Tflops on Tianhe-1A (using 3750 nodes) and 3.74 Pflops on Tianhe-2 (using 8644 nodes). Our study also provides an evaluation on the programming paradigm of various accelerator architectures (GPU, MIC, FPGA) for performing global atmospheric simulation, to form a picture about both the potential performance benefits and the programming efforts involved.
authors.affiliation: Ministry of Education Key Laboratory for Earth System Modeling, and Department of Earth System Science, Tsinghua University, Beijing, China.|Joint Center for Global Change Studies (JCGCS), Beijing, China.|National Supercomputing Center in Wuxi, Wuxi, China.
authors.forename: Haohuan
authors.initials: H
authors.lastname: Fu
authors.affiliation: Ministry of Education Key Laboratory for Earth System Modeling, and Department of Earth System Science, Tsinghua University, Beijing, China.|Joint Center for Global Change Studies (JCGCS), Beijing, China.|Department of Computer Science and Technology, Tsinghua University, Beijing, China.|National Supercomputing Center in Wuxi, Wuxi, China.
authors.forename: Lin
authors.initials: L
authors.lastname: Gan
authors.affiliation: Institute of Software, Chinese Academy of Science, Beijing, China.
authors.forename: Chao
authors.initials: C
authors.lastname: Yang
authors.affiliation: Ministry of Education Key Laboratory for Earth System Modeling, and Department of Earth System Science, Tsinghua University, Beijing, China.|Joint Center for Global Change Studies (JCGCS), Beijing, China.|Department of Computer Science and Technology, Tsinghua University, Beijing, China.|National Supercomputing Center in Wuxi, Wuxi, China.
authors.forename: Wei
authors.initials: W
authors.lastname: Xue
authors.affiliation: Joint Center for Global Change Studies (JCGCS), Beijing, China.|College of Global Change and Earth System Science, Beijing Normal University, Beijing, China.|National Supercomputing Center in Wuxi, Wuxi, China.
authors.forename: Lanning
authors.initials: L
authors.lastname: Wang
authors.affiliation: Department of Computer Science and Technology, Tsinghua University, Beijing, China.
authors.forename: Xinliang
authors.initials: X
authors.lastname: Wang
authors.affiliation: Ministry of Education Key Laboratory for Earth System Modeling, and Department of Earth System Science, Tsinghua University, Beijing, China.|Joint Center for Global Change Studies (JCGCS), Beijing, China.|National Supercomputing Center in Wuxi, Wuxi, China.
authors.forename: Xiaomeng
authors.initials: X
authors.lastname: Huang
authors.affiliation: Ministry of Education Key Laboratory for Earth System Modeling, and Department of Earth System Science, Tsinghua University, Beijing, China.|Joint Center for Global Change Studies (JCGCS), Beijing, China.|Department of Computer Science and Technology, Tsinghua University, Beijing, China.|National Supercomputing Center in Wuxi, Wuxi, China.
authors.forename: Guangwen
authors.initials: G
authors.lastname: Yang
mesh_terms: D000465:Algorithms; D002980:Climate; D003198:Computer Simulation; D003201:Computers; D014867:Water / Q000737:chemistry
chemical_list: D014867:Water