Too good to be true: when overwhelming evidence fails to convince.
Is it possible for a large sequence of measurements or observations, which support a hypothesis, to counterintuitively decrease our confidence? Can unanimous support be too good to be true? The assumption of independence is often made in good faith; however, rarely is consideration given to whether a systemic failure has occurred. Taking this into account can cause certainty in a hypothesis to decrease as the evidence for it becomes apparently stronger. We perform a probabilistic Bayesian analysis of this effect with examples based on (i) archaeological evidence, (ii) weighing of legal evidence and (iii) cryptographic primality testing. In this paper, we investigate the effects of small error rates in a set of measurements or observations. We find that even with very low systemic failure rates, high confidence is surprisingly difficult to achieve; in particular, we find that certain analyses of cryptographically important numerical tests are highly optimistic, underestimating their false-negative rate by as much as a factor of 280.
School of Electrical and Electronic Engineering, The University of Adelaide , Adelaide 5005, Australia.
Lachlan J
0000-0003-1767-7897
LJ
Gunn
Laboratoire Angevin de Recherche en Ingénierie des Systèmes (LARIS) , University of Angers , 62 avenue Notre Dame du Lac, Angers 49000, France.
François
F
Chapeau-Blondeau
School of Electrical and Electronic Engineering, The University of Adelaide, Adelaide 5005, Australia; School of Information Technology and Mathematical Sciences, University of South Australia, Mawson Lakes, South Australia 5095, Australia.
Mark D
MD
McDonnell
School of Electrical and Electronic Engineering, The University of Adelaide , Adelaide 5005, Australia.
Bruce R
BR
Davis
School of Electrical and Electronic Engineering, The University of Adelaide , Adelaide 5005, Australia.
Andrew
A
Allison
School of Electrical and Electronic Engineering, The University of Adelaide , Adelaide 5005, Australia.
Derek
D
Abbott
Bayesian; criminology; cryptography