title: Benchmarking clinical practice in surgery: looking beyond traditional mortality rates.
abstract: This paper proposes two new measures to assess performance of surgical practice based on observed mortality: reliability, measured as the area under the ROC curve and a living score, the sum of individual risk among surviving patients, divided by the total number of patients. A Monte Carlo simulation of surgeons' practice was used for conceptual validation and an analysis of a real-world hospital department was used for managerial validation. We modelled surgical practice as a bivariate distribution function of risk and final state. We sampled 250 distributions, varying the maximum risk each surgeon faced, the distribution of risk among dead patients, the mortality rate and the number of surgeries performed yearly. We applied the measures developed to a Portuguese cardiothoracic department. We found that the joint use of the reliability and living score measures overcomes the limitations of risk adjusted mortality rates, as it enables a different valuation of deaths, according to their risk levels. Reliability favours surgeons with casualties, predominantly, in high values of risk and penalizes surgeons with deaths in relatively low levels of risk. The living score is positively influenced by the maximum risk for which a surgeon yields surviving patients. These measures enable a deeper understanding of surgical practice and, as risk adjusted mortality rates, they rely only on mortality and risk scores data. The case study revealed that the performance of the department analysed could be improved with enhanced policies of risk management, involving the assignment of surgeries based on surgeon's reliability and living score.
authors.affiliation: Faculdade de Engenharia da Universidade do Porto: Rua Dr. Roberto Frias, s/n, 4200-465, Porto, Portugal. ricardo.alves.castro@fe.up.pt.
authors.forename: Ricardo A S
authors.initials: RA
authors.lastname: Castro
authors.affiliation: Instituto de Ciências Biomédicas Abel Salazar da Universidade do Porto: Rua de Jorge Viterbo Ferreira, no. 228, 4050-313, Porto, Portugal.
authors.forename: Pedro N
authors.initials: PN
authors.lastname: Oliveira
authors.affiliation: CEGE - Centros de Estudos em Gestão e Economia da Universidade, Católica Portuguesa - Centro Regional do Porto: Rua Diogo Botelho, no. 1327, 4169-005, Porto, Portugal.
authors.forename: Conceição
authors.initials: C
authors.lastname: Silva Portela
authors.affiliation: Faculdade de Engenharia da Universidade do Porto: Rua Dr. Roberto Frias, s/n, 4200-465, Porto, Portugal.
authors.forename: Ana S
authors.initials: AS
authors.lastname: Camanho
authors.affiliation: ICS - Instituto de Ciências da Saúde da Universidade Católica Portuguesa - Centro Regional do Porto: Rua Dr. Ant´onio Bernardino de Almeida, s/n, 4200-072, Porto, Portugal.
authors.forename: João
authors.initials: J
authors.lastname: Queiroz e Melo
mesh_terms: D019985:Benchmarking / Q000379:methods*; D002983:Clinical Competence; D003198:Computer Simulation; D017052:Hospital Mortality; D006801:Humans; D009010:Monte Carlo Method; D019982:Organizational Case Studies; D011174:Portugal / Q000453:epidemiology; D012372:ROC Curve; D015203:Reproducibility of Results; D018570:Risk Assessment / Q000379:methods*; D013903:Thoracic Surgery; D019616:Thoracic Surgical Procedures / Q000401:mortality / Q000592:standards
keywords: Patient outcomes; Quality measurement; ROC curves; Simulation and modelling