Evaluating the reliability and reproducibility of the Ottawa Thoracic Morbidity and Mortality classification system.
BACKGROUND
Minimizing adverse events after surgery is widely recognized as an important indicator of quality; yet no consensus has been reached on how to standardize the reporting of adverse events after surgical procedures. Our objectives were to develop a standardized classification system to monitor both the presence and severity of thoracic morbidity and mortality, and to evaluate its reliability and reproducibility among a national cohort of thoracic surgeons.


METHODS
To assess the Thoracic Morbidity and Mortality classification system (based on the Clavien-Dindo classification of adverse events), a 31-item questionnaire was sent to all members of the Canadian Association of Thoracic Surgeons in August 2009, consisting of a general description of the Thoracic Morbidity and Mortality severity grades, 20 case-based questions of postoperative adverse events to be classified, and questions regarding personal judgments. We derived descriptive and quantitative information using weighted Kappa statistics.


RESULTS
Fifty-two (54.7%) thoracic surgeons completed the questionnaire; 41 (78.8%) of the respondents were affiliated with an academic teaching hospital. A total of 1,326 individual weighted Kappa statistics were calculated for all distinct pairs of raters, of which 1,152 (87%) were greater than 0.81, a range that is interpreted as "almost perfect agreement." A further 174 (13%) were in the range between 0.61 and 0.8, interpreted as "substantial agreement." All results were statistically significant (p < 0.0001). The classification system was regarded as straightforward (98% of the respondents), reproducible (94%), logical (92%), and useful (98%).


CONCLUSIONS
The modified classification system appears to offer objective, reliable, and reproducible reporting of thoracic morbidity and mortality, and thus may assist continuous quality improvement in thoracic surgery.
Division of Thoracic Surgery, Department of Surgery, University of Ottawa, Ontario, Canada.
Jelena
J
Ivanovic
Ahmed
A
Al-Hussaini
Derar
D
Al-Shehab
Jennifer
J
Threader
Patrick James
PJ
Villeneuve
Tim
T
Ramsay
Donna E
DE
Maziak
Sebastian
S
Gilbert
Farid M
FM
Shamji
R Sudhir
RS
Sundaresan
Andrew J E
AJ
Seely
D006801:Humans; D009864:Ontario / Q000453:epidemiology; D011159:Population Surveillance; D011183:Postoperative Complications / Q000145:classification* / Q000401:mortality*; D058996:Quality Improvement / Q000706:statistics & numerical data*; D015203:Reproducibility of Results; D012107:Research Design / Q000592:standards*; D011795:Surveys and Questionnaires; D013903:Thoracic Surgery / Q000706:statistics & numerical data*; D019616:Thoracic Surgical Procedures / Q000145:classification* / Q000401:mortality*; D017749:Total Quality Management / Q000458:organization & administration / Q000706:statistics & numerical data