title: Development of a consensus algorithm to improve interobserver agreement and accuracy in the determination of tricuspid regurgitation severity.
abstract: BACKGROUND
  Multiparametric scoring of valvular regurgitation may compromise interobserver agreement, as readers weight parameters differently. The aims of this study were to quantify interobserver variability in the grading of chronic tricuspid regurgitation (TR), develop an algorithm for grading TR, and assess the effect of this algorithm on concordance and accuracy.
  
  
  METHODS
  On the basis of current guidelines, two experts graded the severity of TR by consensus in 40 patients with a spectrum of TR severity. A subgroup of patients (n = 18) also had TR severity assessed by cardiac magnetic resonance. Sixteen cardiologists independently graded the first 20 cases as severe or nonsevere TR. After group review, a grading algorithm to differentiate severe and nonsevere TR was devised by consensus. The same observers used the algorithm to grade the second set of cases.
  
  
  RESULTS
  Baseline differentiation of severe from nonsevere TR showed modest reliability and accuracy compared with an expert read (multirater κ = 0.55; overall agreement, 78%; accuracy, 81%). The consensus algorithm for severe TR was a suggestive color jet and at least one of (1) right atrial area > 18 cm(2) and inferior vena cava diameter > 2.5 cm; (2) vena contracta width > 0.7 cm and jet area > 10 cm(2); (3) a dense, triangular TR Doppler profile; and (4) holosystolic reversal of hepatic vein flow. Application of this algorithm improved the multirater κ coefficient to 0.80, the level of agreement to 90% (P = .033), and mean reader accuracy to 92% (P = .001).
  
  
  CONCLUSIONS
  Only modest baseline agreement was found between readers on the distinction of severe and nonsevere TR. An objective, structured grading algorithm improved both interrater agreement and accuracy.
authors.affiliation: Department of Cardiovascular Sciences, University of Calgary, Calgary, Alberta, Canada; Heart and Vascular Institute, Cleveland Clinic, Cleveland, Ohio.
authors.forename: Andrew D M
authors.initials: AD
authors.lastname: Grant
authors.affiliation: Department of Cardiology, University Health Network, University of Toronto, Toronto, Ontario, Canada; Heart and Vascular Institute, Cleveland Clinic, Cleveland, Ohio.
authors.forename: Paaladinesh
authors.initials: P
authors.lastname: Thavendiranathan
authors.affiliation: Heart and Vascular Institute, Cleveland Clinic, Cleveland, Ohio.
authors.forename: L Leonardo
authors.initials: LL
authors.lastname: Rodriguez
authors.affiliation: Heart and Vascular Institute, Cleveland Clinic, Cleveland, Ohio.
authors.forename: Deborah
authors.initials: D
authors.lastname: Kwon
authors.affiliation: Heart and Vascular Institute, Cleveland Clinic, Cleveland, Ohio; Menzies Research Institute Tasmania, Hobart, Australia. Electronic address: tom.marwick@utas.edu.au.
authors.forename: Thomas H
authors.initials: TH
authors.lastname: Marwick
mesh_terms: D000465:Algorithms; D004452:Echocardiography / Q000379:methods / Q000592:standards; D005260:Female; D006801:Humans; D007089:Image Enhancement / Q000379:methods*; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012720:Severity of Illness Index; D014262:Tricuspid Valve Insufficiency / Q000000981:diagnostic imaging*
keywords: Accuracy; Agreement; Tricuspid regurgitation