title: Reproducibility in density functional theory calculations of solids.
abstract: The widespread popularity of density functional theory has given rise to an extensive range of dedicated codes for predicting molecular and crystalline properties. However, each code implements the formalism in a different way, raising questions about the reproducibility of such predictions. We report the results of a community-wide effort that compared 15 solid-state codes, using 40 different potentials or basis set types, to assess the quality of the Perdew-Burke-Ernzerhof equations of state for 71 elemental crystals. We conclude that predictions from recent codes and pseudopotentials agree very well, with pairwise differences that are comparable to those between different high-precision experiments. Older methods, however, have less precise agreement. Our benchmark provides a framework for users and developers to document the precision of new applications and methodological improvements.
authors.affiliation: Center for Molecular Modeling, Ghent University, Technologiepark 903, BE-9052 Zwijnaarde, Belgium.
authors.forename: Kurt
authors.initials: K
authors.lastname: Lejaeghere
authors.affiliation: Peter Grünberg Institute and Institute for Advanced Simulation, Forschungszentrum Jülich and JARA (Jülich Aachen Research Alliance), D-52425 Jülich, Germany.
authors.forename: Gustav
authors.initials: G
authors.lastname: Bihlmayer
authors.affiliation: Department of Physics, Åbo Akademi, FI-20500 Turku, Finland. Centre of Excellence in Computational Nanoscience (COMP) and Department of Applied Physics, Aalto University School of Science, Post Office Box 11100, FI-00076 Aalto, Finland.
authors.forename: Torbjörn
authors.initials: T
authors.lastname: Björkman
authors.affiliation: Institute of Materials Chemistry, Vienna University of Technology, Getreidemarkt 9/165-TC, A-1060 Vienna, Austria.
authors.forename: Peter
authors.initials: P
authors.lastname: Blaha
authors.affiliation: Peter Grünberg Institute and Institute for Advanced Simulation, Forschungszentrum Jülich and JARA (Jülich Aachen Research Alliance), D-52425 Jülich, Germany.
authors.forename: Stefan
authors.initials: S
authors.lastname: Blügel
authors.affiliation: Department of Mechanical Engineering and Materials Science, Duke University, Durham, NC 27708, USA.
authors.forename: Volker
authors.initials: V
authors.lastname: Blum
authors.affiliation: Université Grenoble Alpes, Institut Nanosciences et Cryogénie-Modeling and Material Exploration Department (INAC-MEM), Laboratoire de Simulation Atomistique (L_Sim), F-38042 Grenoble, France. Commissariat à l'Énergie Atomique et aux Énergies Alternatives (CEA), INAC-MEM, L_Sim, F-38054 Grenoble, France.
authors.forename: Damien
authors.initials: D
authors.lastname: Caliste
authors.affiliation: Theory and Simulation of Materials (THEOS) and National Centre for Computational Design and Discovery of Novel Materials (MARVEL), École Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland.
authors.forename: Ivano E
authors.initials: IE
authors.lastname: Castelli
authors.affiliation: Department of Physics, University of Durham, Durham DH1 3LE, UK.
authors.forename: Stewart J
authors.initials: SJ
authors.lastname: Clark
authors.affiliation: International School for Advanced Studies (SISSA) and DEMOCRITOS, Consiglio Nazionale delle Ricerche-Istituto Officina dei Materiali (CNR-IOM), Via Bonomea 265, I-34136 Trieste, Italy.
authors.forename: Andrea
authors.initials: A
authors.lastname: Dal Corso
authors.affiliation: International School for Advanced Studies (SISSA) and DEMOCRITOS, Consiglio Nazionale delle Ricerche-Istituto Officina dei Materiali (CNR-IOM), Via Bonomea 265, I-34136 Trieste, Italy.
authors.forename: Stefano
authors.initials: S
authors.lastname: de Gironcoli
authors.affiliation: Université Grenoble Alpes, Institut Nanosciences et Cryogénie-Modeling and Material Exploration Department (INAC-MEM), Laboratoire de Simulation Atomistique (L_Sim), F-38042 Grenoble, France. Commissariat à l'Énergie Atomique et aux Énergies Alternatives (CEA), INAC-MEM, L_Sim, F-38054 Grenoble, France.
authors.forename: Thierry
authors.initials: T
authors.lastname: Deutsch
authors.affiliation: Max-Planck-Institut für Mikrostrukturphysik, Weinberg 2, D-06120 Halle, Germany.
authors.forename: John Kay
authors.initials: JK
authors.lastname: Dewhurst
authors.affiliation: Department of Physics and Astronomy, Division of Materials Theory, Uppsala University, Post Office Box 516, SE-75120 Uppsala, Sweden.
authors.forename: Igor
authors.initials: I
authors.lastname: Di Marco
authors.affiliation: Institut für Physik and Integrative Research Institute for the Sciences (IRIS)-Adlershof, Humboldt-Universität zu Berlin, Zum Großen Windkanal 6, D-12489 Berlin, Germany. Fritz-Haber-Institut der Max-Planck-Gesellschaft, Faradayweg 4-6, D-14195 Berlin, Germany.
authors.forename: Claudia
authors.initials: C
authors.lastname: Draxl
authors.affiliation: Center for Atomic-Scale Materials Design, Department of Physics, Technical University of Denmark, DK-2800 Kongens Lyngby, Denmark.
authors.forename: Marcin
authors.initials: M
authors.lastname: Dułak
authors.affiliation: Department of Physics and Astronomy, Division of Materials Theory, Uppsala University, Post Office Box 516, SE-75120 Uppsala, Sweden.
authors.forename: Olle
authors.initials: O
authors.lastname: Eriksson
authors.affiliation: Max-Planck-Institut für Mikrostrukturphysik, Weinberg 2, D-06120 Halle, Germany.
authors.forename: José A
authors.initials: JA
authors.lastname: Flores-Livas
authors.affiliation: Material Measurement Laboratory, National Institute of Standards and Technology, 100 Bureau Drive, Stop 8553, Gaithersburg, MD 20899, USA.
authors.forename: Kevin F
authors.initials: KF
authors.lastname: Garrity
authors.affiliation: Université Grenoble Alpes, Institut Nanosciences et Cryogénie-Modeling and Material Exploration Department (INAC-MEM), Laboratoire de Simulation Atomistique (L_Sim), F-38042 Grenoble, France. Commissariat à l'Énergie Atomique et aux Énergies Alternatives (CEA), INAC-MEM, L_Sim, F-38054 Grenoble, France.
authors.forename: Luigi
authors.initials: L
authors.lastname: Genovese
authors.affiliation: Department of Mathematics, Computer Science, and Physics, University of Udine, Via delle Scienze 206, I-33100 Udine, Italy.
authors.forename: Paolo
authors.initials: P
authors.lastname: Giannozzi
authors.affiliation: Institute of Condensed Matter and Nanosciences-Nanoscopic Physics (NAPS), Université Catholique de Louvain, Chemin des Étoiles 8, BE-1348 Louvain-la-Neuve, Belgium.
authors.forename: Matteo
authors.initials: M
authors.lastname: Giantomassi
authors.affiliation: Institut für Physik, Universität Basel, Klingelbergstrasse 82, CH-4056 Basel, Switzerland.
authors.forename: Stefan
authors.initials: S
authors.lastname: Goedecker
authors.affiliation: Institute of Condensed Matter and Nanosciences-Nanoscopic Physics (NAPS), Université Catholique de Louvain, Chemin des Étoiles 8, BE-1348 Louvain-la-Neuve, Belgium.
authors.forename: Xavier
authors.initials: X
authors.lastname: Gonze
authors.affiliation: Department of Physics and Astronomy, Division of Materials Theory, Uppsala University, Post Office Box 516, SE-75120 Uppsala, Sweden. School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA.
authors.forename: Oscar
authors.initials: O
authors.lastname: Grånäs
authors.affiliation: Max-Planck-Institut für Mikrostrukturphysik, Weinberg 2, D-06120 Halle, Germany.
authors.forename: E K U
authors.initials: EK
authors.lastname: Gross
authors.affiliation: Institut für Physik and Integrative Research Institute for the Sciences (IRIS)-Adlershof, Humboldt-Universität zu Berlin, Zum Großen Windkanal 6, D-12489 Berlin, Germany. Fritz-Haber-Institut der Max-Planck-Gesellschaft, Faradayweg 4-6, D-14195 Berlin, Germany.
authors.forename: Andris
authors.initials: A
authors.lastname: Gulans
authors.affiliation: Department of Computer Science, University of California-Davis, Davis, CA 95616, USA.
authors.forename: François
authors.initials: F
authors.lastname: Gygi
authors.affiliation: Department of Physics and Astronomy, Rutgers University, Piscataway, NJ 08854-8019, USA. Mat-Sim Research, Post Office Box 742, Murray Hill, NJ 07974, USA.
authors.forename: D R
authors.initials: DR
authors.lastname: Hamann
authors.affiliation: Department of Physics, University of York, Heslington, York YO10 5DD, UK.
authors.forename: Phil J
authors.initials: PJ
authors.lastname: Hasnip
authors.affiliation: Department of Physics, Wake Forest University, Winston-Salem, NC 27109, USA.
authors.forename: N A W
authors.initials: NA
authors.lastname: Holzwarth
authors.affiliation: Department of Physics and Astronomy, Division of Materials Theory, Uppsala University, Post Office Box 516, SE-75120 Uppsala, Sweden.
authors.forename: Diana
authors.initials: D
authors.lastname: Iuşan
authors.affiliation: Scientific Computing Department, Science and Technology Facilities Council, Rutherford Appleton Laboratory, Didcot OX11 0QX, UK.
authors.forename: Dominik B
authors.initials: DB
authors.lastname: Jochym
authors.affiliation: CEA, DAM, DIF, F-91297 Arpajon, France.
authors.forename: François
authors.initials: F
authors.lastname: Jollet
authors.affiliation: Department of Materials, University of Oxford, 16 Parks Road, Oxford OX1 3PH, UK.
authors.forename: Daniel
authors.initials: D
authors.lastname: Jones
authors.affiliation: Faculty of Physics and Center for Computational Materials Science, University of Vienna, Sensengasse 8/12, A-1090 Vienna, Austria.
authors.forename: Georg
authors.initials: G
authors.lastname: Kresse
authors.affiliation: Leibniz‑Institut für Festkörper- und Werkstoffforschung (IFW) Dresden, Post Office Box 270 116, D-01171 Dresden, Germany. Dresden Center for Computational Materials Science (DCMS), Technische Universität Dresden, D-01069 Dresden, Germany.
authors.forename: Klaus
authors.initials: K
authors.lastname: Koepernik
authors.affiliation: Theory and Simulation of Materials (THEOS) and National Centre for Computational Design and Discovery of Novel Materials (MARVEL), École Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland. International School for Advanced Studies (SISSA) and DEMOCRITOS, Consiglio Nazionale delle Ricerche-Istituto Officina dei Materiali (CNR-IOM), Via Bonomea 265, I-34136 Trieste, Italy.
authors.forename: Emine
authors.initials: E
authors.lastname: Küçükbenli
authors.affiliation: Department of Physics and Astronomy, Division of Materials Theory, Uppsala University, Post Office Box 516, SE-75120 Uppsala, Sweden.
authors.forename: Yaroslav O
authors.initials: YO
authors.lastname: Kvashnin
authors.affiliation: Department of Physics and Astronomy, Division of Materials Theory, Uppsala University, Post Office Box 516, SE-75120 Uppsala, Sweden. Institute for Molecules and Materials, Radboud University, Heyendaalseweg 135, 6525 AJ Nijmegen, Netherlands.
authors.forename: Inka L M
authors.initials: IL
authors.lastname: Locht
authors.affiliation: Institut für Physik and Integrative Research Institute for the Sciences (IRIS)-Adlershof, Humboldt-Universität zu Berlin, Zum Großen Windkanal 6, D-12489 Berlin, Germany.
authors.forename: Sven
authors.initials: S
authors.lastname: Lubeck
authors.affiliation: Faculty of Physics and Center for Computational Materials Science, University of Vienna, Sensengasse 8/12, A-1090 Vienna, Austria.
authors.forename: Martijn
authors.initials: M
authors.lastname: Marsman
authors.affiliation: Theory and Simulation of Materials (THEOS) and National Centre for Computational Design and Discovery of Novel Materials (MARVEL), École Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland.
authors.forename: Nicola
authors.initials: N
authors.lastname: Marzari
authors.affiliation: Leibniz‑Institut für Festkörper- und Werkstoffforschung (IFW) Dresden, Post Office Box 270 116, D-01171 Dresden, Germany.
authors.forename: Ulrike
authors.initials: U
authors.lastname: Nitzsche
authors.affiliation: Department of Physics and Astronomy, Division of Materials Theory, Uppsala University, Post Office Box 516, SE-75120 Uppsala, Sweden.
authors.forename: Lars
authors.initials: L
authors.lastname: Nordström
authors.affiliation: Institute for Solid State Physics, The University of Tokyo, Kashiwa 277-8581, Japan.
authors.forename: Taisuke
authors.initials: T
authors.lastname: Ozaki
authors.affiliation: Institut de Minéralogie, de Physique des Matériaux, et de Cosmochimie (IMPMC), Sorbonne Universités-Pierre and Marie Curie University Paris 06, Centre National de la Recherche Scientifique (CNRS) Unité Mixte de Recherche (UMR) 7590, Muséum National d'Histoire Naturelle, Institut de Recherche pour le Développement (IRD) Unité de Recherche 206, 4 Place Jussieu, F-75005 Paris, France.
authors.forename: Lorenzo
authors.initials: L
authors.lastname: Paulatto
authors.affiliation: Department of Materials Science and Metallurgy, University of Cambridge, 27 Charles Babbage Road, Cambridge CB3 0FS, UK.
authors.forename: Chris J
authors.initials: CJ
authors.lastname: Pickard
authors.affiliation: Center for Molecular Modeling, Ghent University, Technologiepark 903, BE-9052 Zwijnaarde, Belgium. High Performance Computing Unit, Ghent University, Krijgslaan 281 S9, BE-9000 Ghent, Belgium.
authors.forename: Ward
authors.initials: W
authors.lastname: Poelmans
authors.affiliation: Department of Physics, University of York, Heslington, York YO10 5DD, UK.
authors.forename: Matt I J
authors.initials: MI
authors.lastname: Probert
authors.affiliation: Department of Physics, Royal Holloway, University of London, Egham TW20 0EX, UK. ISIS Facility, Science and Technology Facilities Council, Rutherford Appleton Laboratory, Didcot OX11 0QX, UK.
authors.forename: Keith
authors.initials: K
authors.lastname: Refson
authors.affiliation: Leibniz‑Institut für Festkörper- und Werkstoffforschung (IFW) Dresden, Post Office Box 270 116, D-01171 Dresden, Germany. Dresden Center for Computational Materials Science (DCMS), Technische Universität Dresden, D-01069 Dresden, Germany.
authors.forename: Manuel
authors.initials: M
authors.lastname: Richter
authors.affiliation: Institute of Condensed Matter and Nanosciences-Nanoscopic Physics (NAPS), Université Catholique de Louvain, Chemin des Étoiles 8, BE-1348 Louvain-la-Neuve, Belgium.
authors.forename: Gian-Marco
authors.initials: GM
authors.lastname: Rignanese
authors.affiliation: Institut für Physik, Universität Basel, Klingelbergstrasse 82, CH-4056 Basel, Switzerland.
authors.forename: Santanu
authors.initials: S
authors.lastname: Saha
authors.affiliation: Fritz-Haber-Institut der Max-Planck-Gesellschaft, Faradayweg 4-6, D-14195 Berlin, Germany. Department of Chemistry and Biochemistry and Materials Department, University of California-Santa Barbara, Santa Barbara, CA 93106-5050, USA.
authors.forename: Matthias
authors.initials: M
authors.lastname: Scheffler
authors.affiliation: Department of Computer Science, University of California-Davis, Davis, CA 95616, USA.
authors.forename: Martin
authors.initials: M
authors.lastname: Schlipf
authors.affiliation: Institute of Materials Chemistry, Vienna University of Technology, Getreidemarkt 9/165-TC, A-1060 Vienna, Austria.
authors.forename: Karlheinz
authors.initials: K
authors.lastname: Schwarz
authors.affiliation: Max-Planck-Institut für Mikrostrukturphysik, Weinberg 2, D-06120 Halle, Germany.
authors.forename: Sangeeta
authors.initials: S
authors.lastname: Sharma
authors.affiliation: Material Measurement Laboratory, National Institute of Standards and Technology, 100 Bureau Drive, Stop 8553, Gaithersburg, MD 20899, USA.
authors.forename: Francesca
authors.initials: F
authors.lastname: Tavazza
authors.affiliation: Institute for Solid State Physics, Vienna University of Technology, A-1040 Vienna, Austria.
authors.forename: Patrik
authors.initials: P
authors.lastname: Thunström
authors.affiliation: Fritz-Haber-Institut der Max-Planck-Gesellschaft, Faradayweg 4-6, D-14195 Berlin, Germany. Physics and Materials Science Research Unit, University of Luxembourg, L-1511 Luxembourg.
authors.forename: Alexandre
authors.initials: A
authors.lastname: Tkatchenko
authors.affiliation: CEA, DAM, DIF, F-91297 Arpajon, France.
authors.forename: Marc
authors.initials: M
authors.lastname: Torrent
authors.affiliation: Department of Physics and Astronomy, Rutgers University, Piscataway, NJ 08854-8019, USA.
authors.forename: David
authors.initials: D
authors.lastname: Vanderbilt
authors.affiliation: Institute of Condensed Matter and Nanosciences-Nanoscopic Physics (NAPS), Université Catholique de Louvain, Chemin des Étoiles 8, BE-1348 Louvain-la-Neuve, Belgium.
authors.forename: Michiel J
authors.initials: MJ
authors.lastname: van Setten
authors.affiliation: Center for Molecular Modeling, Ghent University, Technologiepark 903, BE-9052 Zwijnaarde, Belgium.
authors.forename: Veronique
authors.initials: V
authors.lastname: Van Speybroeck
authors.affiliation: Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, USA.
authors.forename: John M
authors.initials: JM
authors.lastname: Wills
authors.affiliation: Department of Materials, University of Oxford, 16 Parks Road, Oxford OX1 3PH, UK.
authors.forename: Jonathan R
authors.initials: JR
authors.lastname: Yates
authors.affiliation: Institute of Theoretical and Simulational Chemistry, School of Chemistry and Chemical Engineering, Harbin Institute of Technology, Harbin 150001, People's Republic of China.
authors.forename: Guo-Xu
authors.initials: GX
authors.lastname: Zhang
authors.affiliation: Center for Molecular Modeling, Ghent University, Technologiepark 903, BE-9052 Zwijnaarde, Belgium. Department of Materials Science and Engineering, Ghent University, Technologiepark 903, BE-9052 Zwijnaarde, Belgium.
authors.forename: Stefaan
authors.initials: S
authors.lastname: Cottenier