title: A crossed random effects modeling approach for estimating diagnostic accuracy from ordinal ratings without a gold standard.
abstract: In diagnostic studies without a gold standard, the assumption on the dependence structure of the multiple tests or raters plays an important role in model performance. In case of binary disease status, both conditional independence and crossed random effects structure have been proposed and their performance investigated. Less attention has been paid to the situation where the true disease status is ordinal. In this paper, we propose crossed subject-specific and rater-specific random effects to account for the dependence structure and assess the robustness of the proposed model to misspecification in the random effects distributions. We applied the models to data from the Physician Reliability Study, which focuses on assessing the diagnostic accuracy in a population of raters for the staging of endometriosis, a gynecological disorder in women. Using this new methodology, we estimate the probability of a correct classification and show that regional experts can more easily classify the intermediate stage than resident physicians.
authors.affiliation: Division of Epidemiology Statistics and Prevention Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, Bethesda, MD 20892, U.S.A.
authors.forename: Yunlong
authors.initials: Y
authors.lastname: Xie
authors.forename: Zhen
authors.initials: Z
authors.lastname: Chen
authors.forename: Paul S
authors.initials: PS
authors.lastname: Albert
mesh_terms: D000465:Algorithms; D003198:Computer Simulation; D003627:Data Interpretation, Statistical; D003955:Diagnostic Tests, Routine / Q000592:standards*; D004715:Endometriosis / Q000000981:diagnostic imaging; D005260:Female; D006801:Humans; D015233:Models, Statistical; D009010:Monte Carlo Method; D015588:Observer Variation; D010820:Physicians; D011859:Radiography; D015203:Reproducibility of Results
keywords: MCEM algorithm; endometriosis; random effects models