title: The meaning and suitability of various effect sizes for structured Rater x Ratee designs.
abstract: Four types of analysis are commonly applied to data from structured Rater x Ratee designs. These types are characterized by the unit of analysis, which is either raters or ratees, and by the design used, which is either between-units or within-unit design. The 4 types of analysis are quite different, and therefore they give rise to effect sizes that differ in their substantive interpretations. In most cases, effect sizes based on between-ratee analysis have the least ambiguous meaning and will best serve the aims of meta-analysts and primary researchers. Effect sizes that arise from within-unit designs confound the strength of an effect with its homogeneity. Nonetheless, the authors identify how a range of effect-size types such as these serve the aims of meta-analysis appropriately.
authors.affiliation: Technische Universitat Chemnitz, Institut fur Psychologie, Chemnitz, Germany. johannes.hoenekopp@phil.tu-chemnitz.de
authors.forename: Johannes
authors.initials: J
authors.lastname: Hönekopp
authors.forename: Betsy Jane
authors.initials: BJ
authors.lastname: Becker
authors.forename: Frederick L
authors.initials: FL
authors.lastname: Oswald
mesh_terms: D000328:Adult; D001504:Beauty; D003627:Data Interpretation, Statistical; D005145:Face; D005260:Female; D006801:Humans; D007600:Judgment; D016013:Likelihood Functions; D008297:Male; D015201:Meta-Analysis as Topic; D015588:Observer Variation; D012016:Reference Values; D015203:Reproducibility of Results; D012107:Research Design / Q000706:statistics & numerical data*