title: Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide No. 68.
abstract: BACKGROUND
  Generalizability theory (G theory) is a statistical method to analyze the results of psychometric tests, such as tests of performance like the Objective Structured Clinical Examination, written or computer-based knowledge tests, rating scales, or self-assessment and personality tests. It is a generalization of classical reliability theory, which examines the relative contribution of the primary variable of interest, the performance of subjects, compared to error variance. In G theory, various sources of error contributing to the inaccuracy of measurement are explored. G theory is a valuable tool in judging the methodological quality of an assessment method and improving its precision.
  
  
  OBJECTIVE
  Starting from basic statistical principles, we gradually develop and explain the method. We introduce tools to perform generalizability analysis, and illustrate the use of generalizability analysis with a series of common, practical examples in educational practice.
  
  
  CONCLUSIONS
  We realize that statistics and mathematics can be either boring or fearsome to many physicians and educators, yet we believe that some foundations are necessary for a better understanding of generalizability analysis. Consequently, we have tried, wherever possible, to keep the use of equations to a minimum and to use a conversational and slightly "off-serious" style.
authors.affiliation: Faculty of Health Sciences, McMaster University, 1280 Main Street West, Hamilton, ON, L8S 4K1, Canada. blochr@mcmaster.ca
authors.forename: Ralph
authors.initials: R
authors.lastname: Bloch
authors.forename: Geoffrey
authors.initials: G
authors.lastname: Norman
mesh_terms: D003627:Data Interpretation, Statistical; D004521:Educational Measurement / Q000379:methods*; D017408:Guidelines as Topic; D006801:Humans; D015203:Reproducibility of Results; D012107:Research Design