The paradox of verbal autopsy in cause of death assignment: symptom question unreliability but predictive accuracy.
We believe that it is important that governments understand the reliability of the mortality data which they have at their disposable to guide policy debates. In many instances, verbal autopsy (VA) will be the only source of mortality data for populations, yet little is known about how the accuracy of VA diagnoses is affected by the reliability of the symptom responses. We previously described the effect of the duration of time between death and VA administration on VA validity. In this paper, using the same dataset, we assess the relationship between the reliability and completeness of symptom responses and the reliability and accuracy of cause of death (COD) prediction.



The study was based on VAs in the Population Health Metrics Research Consortium (PHMRC) VA Validation Dataset from study sites in Bohol and Manila, Philippines and Andhra Pradesh, India. The initial interview was repeated within 3-52 months of death. Question responses were assessed for reliability and completeness between the two survey rounds. COD was predicted by Tariff Method.



A sample of 4226 VAs was collected for 2113 decedents, including 1394 adults, 349 children, and 370 neonates. Mean question reliability was unexpectedly low (kappa= 0.447): 42.5 % of responses positive at the first interview were negative at the second, and 47.9 % of responses positive at the second had been negative at the first. Question reliability was greater for the short form of the PHMRC instrument (kappa= 0.497) and when analyzed at the level of the individual decedent (kappa= 0.610). Reliability at the level of the individual decedent was associated with COD predictive reliability and predictive accuracy.



Families give coherent accounts of events leading to death but the details vary from interview to interview for the same case. Accounts are accurate but inconsistent; different subsets of symptoms are identified on each occasion. However, there are sufficient accurate and consistent subsets of symptoms to enable the Tariff Method to assign a COD. Questions which contributed most to COD prediction were also the most reliable and consistent across repeat interviews; these have been included in the short form VA questionnaire. Accuracy and reliability of diagnosis for an individual death depend on the quality of interview. This has considerable implications for the progressive roll out of VAs into civil registration and vital statistics (CRVS) systems.
Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA USA.
Peter
P
Serina
School of Public Health, University of Queensland, Brisbane, Australia ; Melbourne School of Population and Global Health, The University of Melbourne, Building 379, 207 Bouverie St, Carlton, 3053 VIC Australia.
Ian
0000-0001-5322-7848
I
Riley
Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA USA.
Bernardo
B
Hernandez
Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA USA.
Abraham D
AD
Flaxman
The George Institute for Global Health, Hyderabad, India.
Devarsetty
D
Praveen
Research Institute for Tropical Medicine, Muntinlupa City, Philippines.
Veronica
V
Tallo
The George Institute for Global Health, University of Sydney, Level 10, King George V Building 83-117 Missenden Rd, PO Box M201, Camperdown, 2050 NSW Australia.
Rohina
R
Joshi
Research Institute for Tropical Medicine, Muntinlupa City, Philippines.
Diozele
D
Sanvictores
Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA USA.
Andrea
A
Stewart
Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA USA.
Meghan D
MD
Mooney
Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA USA.
Christopher J L
CJ
Murray
Melbourne School of Population and Global Health, The University of Melbourne, Building 379, 207 Bouverie St, Carlton, 3053 VIC Australia.
Alan D
AD
Lopez
D000328:Adult; D001344:Autopsy; D002423:Cause of Death; D002648:Child; D003643:Death; D018450:Disease Progression; D005190:Family; D006801:Humans; D007231:Infant, Newborn; D010679:Philippines; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires / Q000592:standards*; D014798:Vital Statistics
Cause of death; Reliability; Validity; Verbal autopsy