title: Anatomically accurate high resolution modeling of human whole heart electromechanics: A strongly scalable algebraic multigrid solver method for nonlinear deformation.
abstract: Electromechanical (EM) models of the heart have been used successfully to study fundamental mechanisms underlying a heart beat in health and disease. However, in all modeling studies reported so far numerous simplifications were made in terms of representing biophysical details of cellular function and its heterogeneity, gross anatomy and tissue microstructure, as well as the bidirectional coupling between electrophysiology (EP) and tissue distension. One limiting factor is the employed spatial discretization methods which are not sufficiently flexible to accommodate complex geometries or resolve heterogeneities, but, even more importantly, the limited efficiency of the prevailing solver techniques which are not sufficiently scalable to deal with the incurring increase in degrees of freedom (DOF) when modeling cardiac electromechanics at high spatio-temporal resolution. This study reports on the development of a novel methodology for solving the nonlinear equation of finite elasticity using human whole organ models of cardiac electromechanics, discretized at a high para-cellular resolution. Three patient-specific, anatomically accurate, whole heart EM models were reconstructed from magnetic resonance (MR) scans at resolutions of 220 μm, 440 μm and 880 μm, yielding meshes of approximately 184.6, 24.4 and 3.7 million tetrahedral elements and 95.9, 13.2 and 2.1 million displacement DOF, respectively. The same mesh was used for discretizing the governing equations of both electrophysiology (EP) and nonlinear elasticity. A novel algebraic multigrid (AMG) preconditioner for an iterative Krylov solver was developed to deal with the resulting computational load. The AMG preconditioner was designed under the primary objective of achieving favorable strong scaling characteristics for both setup and solution runtimes, as this is key for exploiting current high performance computing hardware. Benchmark results using the 220 μm, 440 μm and 880 μm meshes demonstrate efficient scaling up to 1024, 4096 and 8192 compute cores which allowed the simulation of a single heart beat in 44.3, 87.8 and 235.3 minutes, respectively. The efficiency of the method allows fast simulation cycles without compromising anatomical or biophysical detail.
authors.affiliation: Institute of Biophysics, Medical University of Graz, Graz, Austria.
authors.forename: Christoph M
authors.initials: CM
authors.lastname: Augustin
authors.affiliation: Institute of Biophysics, Medical University of Graz, Graz, Austria.
authors.forename: Aurel
authors.initials: A
authors.lastname: Neic
authors.affiliation: Institute for Mathematics and Scientific Computing, Karl-Franzens-University Graz, Graz, Austria.
authors.forename: Manfred
authors.initials: M
authors.lastname: Liebmann
authors.affiliation: Institute of Biophysics, Medical University of Graz, Graz, Austria.
authors.forename: Anton J
authors.initials: AJ
authors.lastname: Prassl
authors.affiliation: Dept. Biomedical Engineering, Division of Imaging Sciences and Biomedical Engineering, King's College of London, London, United Kingdom.
authors.forename: Steven A
authors.initials: SA
authors.lastname: Niederer
authors.affiliation: Institute for Mathematics and Scientific Computing, Karl-Franzens-University Graz, Graz, Austria.
authors.forename: Gundolf
authors.initials: G
authors.lastname: Haase
authors.affiliation: Institute of Biophysics, Medical University of Graz, Graz, Austria.
authors.forename: Gernot
authors.initials: G
authors.lastname: Plank
keywords: Algebraic Multigrid; Cardiac Electromechanics; Parallel Computing; Whole Heart Model
references.citation: Circ Arrhythm Electrophysiol. 2009 Aug;2(4):433-40
references.pmid: 19808500
references.citation: IEEE Trans Med Imaging. 2013 Jan;32(1):130-8
references.pmid: 23221814
references.citation: Proc Natl Acad Sci U S A. 2007 May 8;104(19):7922-6
references.pmid: 17468396
references.citation: Math Biosci. 2009 Aug;220(2):89-101
references.pmid: 19447119
references.citation: Eur J Mech A Solids. 2014 Nov;48:38-47
references.pmid: 25267880
references.citation: J Biomech Eng. 1996 Nov;118(4):464-72
references.pmid: 8950649
references.citation: Europace. 2016 Mar;18(3):376-83
references.pmid: 25842272
references.citation: Circ Res. 1993 Apr;72(4):744-56
references.pmid: 8443866
references.citation: IEEE Trans Biomed Eng. 2012 Aug;59(8):2281-90
references.pmid: 22692867
references.citation: Circ Cardiovasc Imaging. 2011 Nov;4(6):636-40
references.pmid: 21917782
references.citation: Med Image Anal. 2011 Dec;15(6):801-13
references.pmid: 21788150
references.citation: J Biomech. 1992 Oct;25(10):1129-40
references.pmid: 1400513
references.citation: J Physiol. 2012 Sep 15;590(18):4553-69
references.pmid: 22615436
references.citation: Circ Res. 1982 Mar;50(3):342-51
references.pmid: 7060230
references.citation: Philos Trans A Math Phys Eng Sci. 2008 Sep 28;366(1879):3381-409
references.pmid: 18603526
references.citation: IEEE Trans Med Imaging. 2013 Jan;32(1):18-27
references.pmid: 23192521
references.citation: Ann Biomed Eng. 2007 Jan;35(1):1-18
references.pmid: 17111210
references.citation: Prog Biophys Mol Biol. 2008 Jun-Jul;97(2-3):543-61
references.pmid: 18417196
references.citation: Int J Numer Method Biomed Eng. 2012 Jan;28(1):52-71
references.pmid: 25830205
references.citation: Philos Trans A Math Phys Eng Sci. 2009 Jun 13;367(1896):2257-92
references.pmid: 19414455
references.citation: Am J Physiol Heart Circ Physiol. 2007 Jul;293(1):H875-80
references.pmid: 17369453
references.citation: Med Image Anal. 2013 Jul;17 (5):525-37
references.pmid: 23602918
references.citation: Biomech Model Mechanobiol. 2011 Jun;10(3):295-306
references.pmid: 20589408
references.citation: Prog Biophys Mol Biol. 2014 Aug;115(2-3):198-212
references.pmid: 25117497
references.citation: Am J Physiol Heart Circ Physiol. 2010 Feb;298(2):H699-718
references.pmid: 19933417
references.citation: Med Image Comput Comput Assist Interv. 2007;10(Pt 2):402-10
references.pmid: 18044594
references.citation: Ann Biomed Eng. 2003 May;31(5):577-88
references.pmid: 12757201
references.citation: J Mol Cell Cardiol. 2010 Jan;48(1):112-21
references.pmid: 19835882
references.citation: Circ Res. 2010 Jan 8;106(1):185-92
references.pmid: 19893011
references.citation: J Physiol. 1976 Feb;255(2):335-46
references.pmid: 1255523
references.citation: Front Physiol. 2011 Apr 09;2:14
references.pmid: 21516246
references.citation: Circ Res. 1979 May;44(5):701-12
references.pmid: 428066
references.citation: Math Biosci. 2005 Apr;194(2):233-48
references.pmid: 15854678
references.citation: IEEE Trans Biomed Eng. 2009 May;56(5):1318-30
references.pmid: 19203877
references.citation: Europace. 2005 Sep;7 Suppl 2:118-27
references.pmid: 16102509
references.citation: Comput Methods Biomech Biomed Engin. 2013;16(7):802-5
references.pmid: 23734785
references.citation: IEEE Trans Biomed Eng. 2007 Apr;54(4):585-96
references.pmid: 17405366
references.citation: J Biomech. 1995 Oct;28(10):1167-77
references.pmid: 8550635
references.citation: Biomech Model Mechanobiol. 2015 Aug;14(4):829-49
references.pmid: 25567753
references.citation: Prog Biophys Mol Biol. 1998;69(2-3):157-83
references.pmid: 9785937
references.citation: Cardiovasc Res. 2011 Feb 1;89(2):336-43
references.pmid: 20952413
references.citation: Magn Reson Med. 2004 Jul;52(1):141-6
references.pmid: 15236377
references.citation: IEEE Trans Biomed Eng. 2011 Apr;58(4):1055-65
references.pmid: 20699206
references.citation: Ann Biomed Eng. 2009 May;37(5):860-73
references.pmid: 19263223
references.citation: Int J Numer Methods Eng. 2014 Jul 27;99(4):290-312
references.pmid: 26751957
references.citation: IEEE Trans Biomed Eng. 2011 Jul;58(7):2109-19
references.pmid: 21536516
references.citation: Circulation. 2007 Apr 17;115(15):2006-14
references.pmid: 17389270
references.citation: Med Biol Eng Comput. 2012 May;50(5):461-72
references.pmid: 22411321
references.citation: Comput Cardiol (2010). 2013 Oct 1;40:373-376
references.pmid: 24729986
references.citation: Ann Biomed Eng. 2005 May;33(5):590-602
references.pmid: 15981860
references.citation: J Electrocardiol. 2003;36 Suppl:69-74
references.pmid: 14716595
references.citation: Ann Biomed Eng. 2012 Oct;40(10 ):2243-54
references.pmid: 22648575
references.citation: Bull Math Biol. 2007 Oct;69(7):2199-225
references.pmid: 17453303
references.citation: Ann Biomed Eng. 2000 Jul;28(7):781-92
references.pmid: 11016415
references.citation: IEEE Trans Biomed Eng. 1999 Sep;46(9):1166-8
references.pmid: 10493080
references.citation: Prog Biophys Mol Biol. 1999;71(1):91-138
references.pmid: 10070213
references.citation: Philos Trans A Math Phys Eng Sci. 2011 Nov 13;369(1954):4331-51
references.pmid: 21969679
references.citation: IEEE Trans Biomed Eng. 2012 Feb;59(2):311-22
references.pmid: 21926009
references.citation: J Mol Cell Cardiol. 1997 Jun;29(6):1511-23
references.pmid: 9220338
references.citation: J Physiol. 2012 Sep 15;590(18):4515-35
references.pmid: 22753546
references.citation: J Physiol. 1998 Jan 15;506 ( Pt 2):431-44
references.pmid: 9490870
references.citation: PLoS Comput Biol. 2009 Apr;5(4):e1000371
references.pmid: 19390615
references.citation: Am J Physiol Heart Circ Physiol. 2011 Aug;301(2):H279-86
references.pmid: 21572017
references.citation: Philos Trans A Math Phys Eng Sci. 2009 Sep 13;367(1902):3445-75
references.pmid: 19657007
references.citation: J Biomech. 2003 May;36(5):737-48
references.pmid: 12695004
references.citation: Am J Physiol. 1991 Apr;260(4 Pt 2):H1365-78
references.pmid: 2012234
references.citation: Exp Physiol. 2006 Mar;91(2):307-21
references.pmid: 16407474
references.citation: J Physiol. 2015 Mar 1;593(5):1083-111
references.pmid: 25480801
references.citation: Circ Res. 1998 Nov 30;83(11):1165-77
references.pmid: 9831710
references.citation: J Biomech. 1972 May;5(3):309-11
references.pmid: 4666535
references.citation: J Comput Phys. 2012 May 20;231(10):3946-3962
references.pmid: 24976644
references.citation: Prog Biophys Mol Biol. 2008 Jan-Apr;96(1-3):3-18
references.pmid: 17900668
references.citation: Am J Physiol Heart Circ Physiol. 2011 Nov;301(5):H1974-83
references.pmid: 21856904
references.citation: Biomech Model Mechanobiol. 2014 Jun;13(3):627-41
references.pmid: 23990017