Guidelines on constructing funnel plots for quality indicators: A case study on mortality in intensive care unit patients.
Funnel plots are graphical tools to assess and compare clinical performance of a group of care professionals or care institutions on a quality indicator against a benchmark. Incorrect construction of funnel plots may lead to erroneous assessment and incorrect decisions potentially with severe consequences. We provide workflow-based guidance for data analysts on constructing funnel plots for the evaluation of binary quality indicators, expressed as proportions, risk-adjusted rates or standardised rates. Our guidelines assume the following steps: (1) defining policy level input; (2) checking the quality of models used for case-mix correction; (3) examining whether the number of observations per hospital is sufficient; (4) testing for overdispersion of the values of the quality indicator; (5) testing whether the values of quality indicators are associated with institutional characteristics; and (6) specifying how the funnel plot should be constructed. We illustrate our guidelines using data from the Dutch National Intensive Care Evaluation registry. We expect that our guidelines will be useful to data analysts preparing funnel plots and to registries, or other organisations publishing quality indicators. This is particularly true if these people and organisations wish to use standard operating procedures when constructing funnel plots, perhaps to comply with the demands of certification.
1 Academic Medical Center, University of Amsterdam, Department of Medical Informatics, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands.|2 Dutch National Intensive Care Registry (NICE), Amsterdam, The Netherlands.
Ilona Wm
IW
Verburg
1 Academic Medical Center, University of Amsterdam, Department of Medical Informatics, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands.|2 Dutch National Intensive Care Registry (NICE), Amsterdam, The Netherlands.|3 Clinical Research Unit, Academic Medical Center, University of Amsterdam, The Netherlands.
Rebecca
R
Holman
4 Health eResearch Centre, Division of Imaging, Informatics, and Data Science, University of Manchester, Manchester, UK.
Niels
N
Peek
1 Academic Medical Center, University of Amsterdam, Department of Medical Informatics, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands.
Ameen
A
Abu-Hanna
1 Academic Medical Center, University of Amsterdam, Department of Medical Informatics, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands.|2 Dutch National Intensive Care Registry (NICE), Amsterdam, The Netherlands.
Nicolette F
NF
de Keizer
D019985:Benchmarking / Q000379:methods / Q000706:statistics & numerical data; D003422:Critical Care; D003953:Diagnosis-Related Groups; D017408:Guidelines as Topic; D017052:Hospital Mortality; D006801:Humans; D007362:Intensive Care Units; D015233:Models, Statistical; D019984:Quality Indicators, Health Care
Funnel plot; benchmarking; case-mix correction; intensive care unit; mortality; overdispersion; prediction models; quality indicators; sample size; workflow diagram