title: Optimizing study design for interobserver reliability: IUGA-ICS classification of complications of prostheses and graft insertion.
abstract: OBJECTIVE
  Results of interobserver reliability studies for the International Urogynecological Association-International Continence Society (IUGA-ICS) Complication Classification coding can be greatly influenced by study design factors such as participant instruction, motivation, and test-question clarity. We attempted to optimize these factors.
  
  
  METHODS
  After a 15-min instructional lecture with eight clinical case examples (including images) and with classification/coding charts available, those clinicians attending an IUGA Surgical Complications workshop were presented with eight similar-style test cases over 10 min and asked to code them using the Category, Time and Site classification. Answers were compared to predetermined correct codes obtained by five instigators of the IUGA-ICS prostheses and grafts complications classification. Prelecture and postquiz participant confidence levels using a five-step Likert scale were assessed.
  
  
  RESULTS
  Complete sets of answers to the questions (24 codings) were provided by 34 respondents, only three of whom reported prior use of the charts. Average score [n (%)] out of eight, as well as median score (range) for each coding category were: (i) Category: 7.3 (91 %); 7 (4-8); (ii) Time: 7.8 (98 %); 7 (6-8); (iii) Site: 7.2 (90 %); 7 (5-8). Overall, the equivalent calculations (out of 24) were 22.3 (93 %) and 22 (18-24). Mean prelecture confidence was 1.37 (out of 5), rising to 3.85 postquiz. Urogynecologists had the highest correlation with correct coding, followed closely by fellows and general gynecologists.
  
  
  CONCLUSIONS
  Optimizing training and study design can lead to excellent results for interobserver reliability of the IUGA-ICS Complication Classification coding, with increased participant confidence in complication-coding ability.
authors.affiliation: St Vincent's Clinic, Suite 904, 438 Victoria Street, Darlinghurst, NSW, Australia, bernard@haylen.co.
authors.forename: Bernard T
authors.initials: BT
authors.lastname: Haylen
authors.forename: Joseph
authors.initials: J
authors.lastname: Lee
authors.forename: Chris
authors.initials: C
authors.lastname: Maher
authors.forename: Jan
authors.initials: J
authors.lastname: Deprest
authors.forename: Robert
authors.initials: R
authors.lastname: Freeman
mesh_terms: D059019:Clinical Coding; D005260:Female; D006176:Gynecology; D006801:Humans; D015588:Observer Variation; D011183:Postoperative Complications / Q000145:classification / Q000209:etiology; D019919:Prosthesis Implantation / Q000009:adverse effects* / Q000706:statistics & numerical data*; D015203:Reproducibility of Results; D012107:Research Design / Q000592:standards*; D012955:Societies, Medical; D014180:Transplantation / Q000009:adverse effects* / Q000706:statistics & numerical data*; D014572:Urology