Direct estimation of cause-specific mortality fractions from verbal autopsies: multisite validation study using clinical diagnostic gold standards

The King and Lu (KL) method for directly estimating the fraction of all deaths in a population due to a given cause has been used to interpret verbal autopsies (VAs) in areas with incomplete vital registration systems. New research from IHME and the Department of Health Services at the University of Washington, as part of the Population Health Metrics Research Consortium (PHMRC), validates the KL method with a large dataset of VAs for which the underlying causes of death are known.

 

The study, Direct estimation of cause-specific mortality fractions from verbal autopsies: multisite validation study using clinical diagnostic gold standards, shows that the KL method should only be used to estimate cause-specific mortality fractions (the fractions of deaths in a population due to different causes) in populations with a short list of possible causes of death, when individual-level cause of death predictions are not needed.

The KL method is an attractive method for VA analysis. Like all computer methods for VA analysis, the KL method is faster and cheaper than methods that require review by physicians, known as physician-certified verbal autopsy (PCVA). Other studies have shown impressive results from the KL method in estimating cause-specific mortality fractions. However, the previous validation studies used short lists of possible causes of death and did not assess standardized metrics to compare the performance of different VA methods, which limits the comparison of the KL method to other methods like PCVA. This study addressed these issues by validating the performance of the KL method using deaths for which the causes of death are known, collected as part of the PHMRC gold standard verbal autopsy validation study, and by using standardized metrics so that the KL method can be compared to PCVA.

 

The study was designed to assess the accuracy of the KL method in interpreting cause-specific mortality fractions from adult, child, and neonatal VAs and to compare the quality of the KL method to PCVA. It is part of ongoing work by IHME to determine the most accurate and efficient methods of predicting causes of death using VA.

KL method performance was found to be similar to PCVA for cause-specific mortality fraction accuracy, attaining values of 0.669, 0.698, and 0.795 for adults, children, and neonates, respectively. Unlike PCVA, the KL method is not reliant on information about health care experience, which includes any information the caretaker has about the patient's medical treatment, including whether health workers provided documentation for the cause of hospitalization or cause of death. Without health care experience information included in VAs, the KL method outperformed PCVA for all age groups.

 

Quality of prediction varied among age groups, with generally higher accuracy for neonates. For adults, the KL method was most effective in predicting cause-specific mortality fractions for maternal causes and causes that are due to injuries, such as drowning. In children, the most accurately predicted causes of death were measles, malaria, bite of venomous animal, and violent death. For neonates, the KL method was most accurate for stillbirth and preterm delivery.

 

Increasing the number of causes to which the KL method can assign VA deaths had a dramatic effect on estimation quality. Cause-specific mortality fraction accuracy decreased substantially as the length of the list of causes increased.

The KL method was applied to 12,542 adult, child, and neonatal VAs from the PHMRC gold standard VA validation study, for which gold standard cause of death was established using strict clinical diagnostic criteria. In order to evaluate the KL method in a real-world situation, the researchers tested KL method estimations for test datasets with varying distributions of causes of death. The quality of cause-specific mortality fraction accuracy for the KL method was validated against the known causes of death from the PHMRC study and compared with the results of PCVA.

Understanding the population-level patterns of mortality is critical for policymakers to inform planning and allocate resources. Automated methods to directly estimate population-level mortality, like the KL method, are affordable and do not require physician certification. This was the first large-scale validation of the KL method compared to a gold standard cause of death assignment, and the KL method performed about as well as PCVA in terms of cause-specific mortality fraction accuracy. The authors recommend that the KL method could be used instead of PCVA in settings where populations are expected to have little exposure to health care (since including health care experience information did not markedly affect the cause-specific mortality fraction accuracy of the KL method), for neonatal VAs, and in populations with a short list of possible causes of death.

 

The KL method is less accurate than other automated methods for interpreting VAs, such as Simplified Symptom Pattern, Tariff, and Random Forests, and unlike the KL method, these methods estimate individual causes of death. For these reasons, the authors suggest that one of these methods be used to analyze VAs in adults and children.