title: On effective and optical resolutions of diffraction data sets.
abstract: In macromolecular X-ray crystallography, diffraction data sets are traditionally characterized by the highest resolution dhigh of the reflections that they contain. This measure is sensitive to individual reflections and does not refer to the eventual data incompleteness and anisotropy; it therefore does not describe the data well. A physically relevant and robust measure that provides a universal way to define the `actual' effective resolution deff of a data set is introduced. This measure is based on the accurate calculation of the minimum distance between two immobile point scatterers resolved as separate peaks in the Fourier map calculated with a given set of reflections. This measure is applicable to any data set, whether complete or incomplete. It also allows characterizion of the anisotropy of diffraction data sets in which deff strongly depends on the direction. Describing mathematical objects, the effective resolution deff characterizes the `geometry' of the set of measured reflections and is irrelevant to the diffraction intensities. At the same time, the diffraction intensities reflect the composition of the structure from physical entities: the atoms. The minimum distance for the atoms typical of a given structure is a measure that is different from and complementary to deff; it is also a characteristic that is complementary to conventional measures of the data-set quality. Following the previously introduced terms, this value is called the optical resolution, dopt. The optical resolution as defined here describes the separation of the atomic images in the `ideal' crystallographic Fourier map that would be calculated if the exact phases were known. The effective and optical resolution, as formally introduced in this work, are of general interest, giving a common `ruler' for all kinds of crystallographic diffraction data sets.
authors.affiliation: Architecture et Réactivité de l'ARN, UPR 9002 CNRS, IBMC (Institute of Molecular and Cellular Biology), 15 Rue René Descartes, 67084 Strasbourg, France; Université Louis Pasteur de Strasbourg, 67000 Strasbourg, France.
authors.forename: Ludmila
authors.initials: L
authors.lastname: Urzhumtseva
authors.forename: Bruno
authors.initials: B
authors.lastname: Klaholz
authors.forename: Alexandre
authors.initials: A
authors.lastname: Urzhumtsev
mesh_terms: D016880:Anisotropy; D003198:Computer Simulation; D018360:Crystallography, X-Ray / Q000379:methods*; D005583:Fourier Analysis; D046911:Macromolecular Substances / Q000737:chemistry; D008956:Models, Chemical; D016011:Normal Distribution; D039881:Prokaryotic Initiation Factor-2 / Q000737:chemistry; D014961:X-Ray Diffraction / Q000379:methods*
chemical_list: D046911:Macromolecular Substances; D039881:Prokaryotic Initiation Factor-2
keywords: anisotropy; incomplete data sets; minimum distance; point scatterers; resolution; typical atoms