title: Interrater reliability of Violence Risk Appraisal Guide scores provided in Canadian criminal proceedings.
abstract: Published research suggests that most violence risk assessment tools have relatively high levels of interrater reliability, but recent evidence of inconsistent scores among forensic examiners in adversarial settings raises concerns about the "field reliability" of such measures. This study specifically examined the reliability of Violence Risk Appraisal Guide (VRAG) scores in Canadian criminal cases identified in the legal database, LexisNexis. Over 250 reported cases were located that made mention of the VRAG, with 42 of these cases containing 2 or more scores that could be submitted to interrater reliability analyses. Overall, scores were skewed toward higher risk categories. The intraclass correlation (ICCA1) was .66, with pairs of forensic examiners placing defendants into the same VRAG risk "bin" in 68% of the cases. For categorical risk statements (i.e., low, moderate, high), examiners provided converging assessment results in most instances (86%). In terms of potential predictors of rater disagreement, there was no evidence for adversarial allegiance in our sample. Rater disagreement in the scoring of 1 VRAG item (Psychopathy Checklist-Revised; Hare, 2003), however, strongly predicted rater disagreement in the scoring of the VRAG (r = .58). (PsycINFO Database Record
authors.affiliation: Department of Psychology, Texas A&M University.
authors.forename: John F
authors.initials: JF
authors.lastname: Edens
authors.affiliation: Department of Psychology, Texas A&M University.
authors.forename: Brittany N
authors.initials: BN
authors.lastname: Penson
authors.affiliation: Department of Psychology, Texas A&M University.
authors.forename: Jared R
authors.initials: JR
authors.lastname: Ruchensky
authors.affiliation: Department of Psychology, University of Alabama.
authors.forename: Jennifer
authors.initials: J
authors.lastname: Cox
authors.affiliation: Department of Psychology, Francis Marion University.
authors.forename: Shannon Toney
authors.initials: ST
authors.lastname: Smith
mesh_terms: D000328:Adult; D002170:Canada; D057189:Checklist; D057237:Criminals / Q000523:psychology*; D005555:Forensic Psychiatry; D006801:Humans; D008297:Male; D015588:Observer Variation; D011581:Psychological Tests; D015203:Reproducibility of Results; D018570:Risk Assessment; D014754:Violence / Q000523:psychology*