title: Inter-rater reliability in child sexual abuse diagnosis among expert reviewers.
abstract: OBJECTIVE
  To determine how well experts agree when assessing child sexual abuse cases.
  
  
  METHODS
  A total of twelve physician subjects were recruited and voluntarily enrolled from an existing peer review network. Experts from the network had been chosen for their experience in the field and their affiliation with children's advocacy centers. Each expert submitted three cases of prepubertal female genital examinations clearly demonstrable of the case findings. Submitted cases included demographics, history, physical and genital exam findings, photodocumentation, and diagnosis. Experts reviewed each submitted case and labeled the case negative for physical finding(s), positive for physical finding(s), or indeterminate. Cases were analyzed to determine the level of agreement.
  
  
  RESULTS
  Thirty-six cases were submitted for use in this study; one case was excluded prior to starting the review process. After all experts completed their reviews the authors reviewed the cases and results. Two additional cases were excluded, one due to poor quality photodocumentation and one for not meeting the study criteria. Thirty-three cases were used for data analysis. All 12 expert reviewers agreed in 15 of the cases. Overall, in 22 of 33 (67%) cases at least 11 of the 12 reviewers agreed with the original diagnosis. Six of 33 (18%) cases had variable agreement (8-10 reviewers agreed with original diagnosis) among reviewers; 5 of 33 (15%) cases had poor or mixed agreement (7 or less reviewers agreed with original diagnosis).
  
  
  CONCLUSIONS
  Experts exhibit consensus in cases where the findings clearly are normal and abnormal, but demonstrate much more variability in cases where the diagnostic decisions are less obvious. Most of the diagnostic variability is due to interpretation of the findings as normal, abnormal or indeterminate, not on the perception of the examination findings themselves. More research should be done to develop a national consensus on the accurate interpretation of anogenital examination findings. Photographic image quality plays an important role in this quality review process and universally needs to be improved.
authors: Eastern Virginia Medical School, Children's Hospital of The King's Daughters Child Abuse Program, 935 Redgate Avenue, Norfolk, VA 23507, USA.
authors: Suzanne P
authors: SP
authors: Starling
authors: Lori D
authors: LD
authors: Frasier
authors: Kristi
authors: K
authors: Jarvis
authors: Anne
authors: A
authors: McDonald
mesh_terms: D000293:Adolescent; D002648:Child; D002650:Child Abuse, Sexual / Q000175:diagnosis*; D002675:Child, Preschool; D032921:Consensus; D005260:Female; D006801:Humans; D008297:Male; D015588:Observer Variation; D010380:Peer Review; D011785:Quality Assurance, Health Care; D015203:Reproducibility of Results; D014481:United States
keywords: Child sexual abuse; Inter-rater reliability; Peer review; Quality assurance