Interrater and intrarater reliability in the diagnosis and staging of endometriosis.
OBJECTIVE
To estimate the interrater and intrarater reliability of endometriosis diagnosis and severity of disease among gynecologic surgeons viewing operative digital images.


METHODS
The study population comprised a random sample (n=148 [36%]) of women who participated in the Endometriosis: Natural History, Diagnosis and Outcomes study. Four academic expert and four local, specialized expert surgeons reviewed the images, diagnosed the presence or absence of endometriosis for each woman, and rated severity using the revised American Society for Reproductive Medicine (ASRM) criteria. Interrater-level and intrarater-level agreement were calculated for both endometriosis diagnosis and staging.


RESULTS
The interrater reliability for endometriosis diagnosis among the eight surgeons was substantial: Fleiss κ=0.69 (95% confidence interval [CI] 0.64-0.74). Surgeons agreed on revised ASRM endometriosis staging criteria after experienced assessment in a majority of cases (mean 61%, range 52-75%) with moderate interrater reliability: Fleiss κ=0.44 (95% CI 0.41-0.47). The intrarater reliability for experienced assessment compared with computer-assisted revised ASRM staging was almost perfect (mean weighted κ=0.95, range 0.89-0.99).


CONCLUSIONS
Substantial reliability was found for revised ASRM endometriosis diagnosis, whereas moderate reliability was observed for staging. Almost perfect reliability was observed for surgeons' rating of disease severity compared with computerized-assisted, checklist-based staging. Findings suggest that reliability in endometriosis diagnosis is not greatly altered by location or composition of surgeons, supporting the conduct of multisite studies or compilation of endometriosis data across clinical centers. Although surgeons appear to be skilled at assessing endometriosis stage intuitively, how staging of disease burden correlates with clinical outcomes remains to be developed.
Department of Family and Preventive Medicine and the Divisions of Reproductive Endocrinology and Infertility and Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, and the Division of Maternal-Fetal Medicine, University of Utah, Salt Lake City, UT, USA.
Karen C
KC
Schliep
Joseph B
JB
Stanford
Zhen
Z
Chen
Bo
B
Zhang
Jessie K
JK
Dorais
Erica
E
Boiman Johnstone
Ahmad O
AO
Hammoud
Michael W
MW
Varner
Germaine M Buck
GM
Louis
C Matthew
CM
Peterson
D000328:Adult; D004715:Endometriosis / Q000175:diagnosis*; D005260:Female; D006801:Humans; D015203:Reproducibility of Results; D012720:Severity of Illness Index