title: Audio-digital recordings used for independent confirmation of site-based MADRS interview scores.
abstract: Signal detection requires ratings reliability throughout a clinical trial. The confirmation of site-based rater scores by a second, independent and blinded rater is a reasonable metric of ratings reliability. We used audio-digital pens to record site-based interviews of the Montgomery-Asberg Depression Rating Scale (MADRS) in a double-blind, placebo controlled trial of a novel antidepressant in treatment resistant depressed patients. Blinded, site-independent raters generated "dual" scores that revealed high correlations between site-based and site-independent raters (r=0.940 for all ratings) and high sensitivity, specificity, predictive values, and kappa coefficients for treatment response and non-response outcomes using the site-based rater scores as the standard. The blinded raters achieved an 89.4% overall accuracy and 0.786 kappa for matching the treatment response or non-response outcomes of the site-based raters. A limitation of this method is that independent ratings depend on the quality of site-based interviews and patient responses to the site-based interviewers. Nonetheless, this quality assurance strategy may have broad applicability for studies that use subjective measures and wherever ratings reliability is a concern. "Dual" scoring of recorded site-based ratings can be a relatively unobtrusive surveillance strategy to confirm scores and to identify and remediate rater "outliers" during a study.
authors.affiliation: Clintara LLC, Boston, MA, USA. Electronic address: sdtargum@yahoo.com.
authors.forename: Steven D
authors.initials: SD
authors.lastname: Targum
authors.affiliation: Clintara LLC, Boston, MA, USA.
authors.forename: J Cara
authors.initials: JC
authors.lastname: Pendergrass
authors.affiliation: Clintara LLC, Boston, MA, USA.
authors.forename: Chelsea
authors.initials: C
authors.lastname: Toner
authors.affiliation: UCB Biosciences Inc. (formerly CeNeRx Pharmaceuticals), Raleigh, NC, USA.
authors.forename: Mahnaz
authors.initials: M
authors.lastname: Asgharnejad
authors.affiliation: PPD Inc. (formerly CeNeRx Pharmaceuticals), Wilmington, NC, USA.
authors.forename: Daniel J
authors.initials: DJ
authors.lastname: Burch
mesh_terms: D000928:Antidepressive Agents / Q000627:therapeutic use; D003308:Copying Processes / Q000379:methods*; D061218:Depressive Disorder, Treatment-Resistant / Q000188:drug therapy*; D004311:Double-Blind Method; D006571:Heterocyclic Compounds / Q000627:therapeutic use; D006801:Humans; D007407:Interviews as Topic / Q000379:methods*; D008996:Monoamine Oxidase Inhibitors / Q000627:therapeutic use; D011569:Psychiatric Status Rating Scales; D011594:Psychometrics; D016032:Randomized Controlled Trials as Topic / Q000379:methods*; D015203:Reproducibility of Results
chemical_list: D000928:Antidepressive Agents; C551866:CX157; D006571:Heterocyclic Compounds; D008996:Monoamine Oxidase Inhibitors
keywords: Audio-digital recording; Inter-rater reliability; Major Depressive Disorder; Predictive value; Ratings precision; Site-independent ratings