title: Screening for data clustering in multicenter studies: the residual intraclass correlation.
abstract: BACKGROUND
  In multicenter studies, center-specific variations in measurements may arise for various reasons, such as low interrater reliability, differences in equipment, deviations from the protocol, sociocultural characteristics, and differences in patient populations due to e.g. local referral patterns. The aim of this research is to derive measures for the degree of clustering. We present a method to detect heavily clustered variables and to identify physicians with outlying measurements.
  
  
  METHODS
  We use regression models with fixed effects to account for patient case-mix and a random cluster intercept to study clustering by physicians. We propose to use the residual intraclass correlation (RICC), the proportion of residual variance that is situated at the cluster level, to detect variables that are influenced by clustering. An RICC of 0 indicates that the variance in the measurements is not due to variation between clusters. We further suggest, where appropriate, to evaluate RICC in combination with R2, the proportion of variance that is explained by the fixed effects. Variables with a high R2 may have benefits that outweigh the disadvantages of clustering in terms of statistical analysis. We apply the proposed methods to a dataset collected for the development of models for ovarian tumor diagnosis. We study the variability in 18 tumor characteristics collected through ultrasound examination, 4 patient characteristics, and the serum marker CA-125 measured by 40 physicians on 2407 patients.
  
  
  RESULTS
  The RICC showed large variation between variables: from 2.2% for age to 25.1% for the amount of fluid in the pouch of Douglas. Seven variables had an RICC above 15%, indicating that a considerable part of the variance is due to systematic differences at the physician level, rather than random differences at the patient level. Accounting for differences in ultrasound machine quality reduced the RICC for a number of blood flow measurements.
  
  
  CONCLUSIONS
  We recommend that the degree of data clustering is addressed during the monitoring and analysis of multicenter studies. The RICC is a useful tool that expresses the degree of clustering as a percentage. Specific applications are data quality monitoring and variable screening prior to the development of a prediction model.
authors.forename: Laure
authors.initials: L
authors.lastname: Wynants
authors.forename: Dirk
authors.initials: D
authors.lastname: Timmerman
authors.forename: Tom
authors.initials: T
authors.lastname: Bourne
authors.forename: Sabine
authors.initials: S
authors.lastname: Van Huffel
authors.affiliation: KU Leuven Department of Development and Regeneration, Leuven, Belgium. ben.vancalster@med.kuleuven.be.
authors.forename: Ben
authors.initials: B
authors.lastname: Van Calster
mesh_terms: D018394:CA-125 Antigen / Q000097:blood; D016000:Cluster Analysis; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D015233:Models, Statistical; D015337:Multicenter Studies as Topic / Q000379:methods*; D015588:Observer Variation; D010051:Ovarian Neoplasms / Q000097:blood / Q000000981:diagnostic imaging; D012044:Regression Analysis; D015203:Reproducibility of Results; D014463:Ultrasonography / Q000592:standards
chemical_list: D018394:CA-125 Antigen