title: Using generalizability analysis to estimate parameters for anatomy assessments: A multi-institutional study.
abstract: With integrated curricula and multidisciplinary assessments becoming more prevalent in medical education, there is a continued need for educational research to explore the advantages, consequences, and challenges of integration practices. This retrospective analysis investigated the number of items needed to reliably assess anatomical knowledge in the context of gross anatomy and histology. A generalizability analysis was conducted on gross anatomy and histology written and practical examination items that were administered in a discipline-based format at Indiana University School of Medicine and in an integrated fashion at the University of Alabama School of Medicine and Rush University Medical College. Examination items were analyzed using a partially nested design s×(i:o) in which items were nested within occasions (i:o) and crossed with students (s). A reliability standard of 0.80 was used to determine the minimum number of items needed across examinations (occasions) to make reliable and informed decisions about students' competence in anatomical knowledge. Decision study plots are presented to demonstrate how the number of items per examination influences the reliability of each administered assessment. Using the example of a curriculum that assesses gross anatomy knowledge over five summative written and practical examinations, the results of the decision study estimated that 30 and 25 items would be needed on each written and practical examination to reach a reliability of 0.80, respectively. This study is particularly relevant to educators who may question whether the amount of anatomy content assessed in multidisciplinary evaluations is sufficient for making judgments about the anatomical aptitude of students. Anat Sci Educ 10: 109-119. © 2016 American Association of Anatomists.
authors.affiliation: Department of Anatomy and Cell Biology, Indiana University School of Medicine, Indianapolis, Indiana.
authors.forename: Jessica N
authors.initials: JN
authors.lastname: Byram
authors.affiliation: Department of Anatomy and Cell Biology, Indiana University School of Medicine, Indianapolis, Indiana.
authors.forename: Mark F
authors.initials: MF
authors.lastname: Seifert
authors.affiliation: Department of Cell, Developmental, and Integrative Biology, University of Alabama at Birmingham School of Medicine, Birmingham, Alabama.
authors.forename: William S
authors.initials: WS
authors.lastname: Brooks
authors.affiliation: Department of Cell, Developmental, and Integrative Biology, University of Alabama at Birmingham School of Medicine, Birmingham, Alabama.
authors.forename: Laura
authors.initials: L
authors.lastname: Fraser-Cotlin
authors.affiliation: Department of Physical Therapy, University of Illinois at Chicago, Chicago, Illinois.
authors.forename: Laura E
authors.initials: LE
authors.lastname: Thorp
authors.affiliation: Department of Anatomy and Cell Biology, Rush University, Chicago, Illinois.
authors.forename: James M
authors.initials: JM
authors.lastname: Williams
authors.affiliation: Department of Anatomy and Cell Biology, Rush University, Chicago, Illinois.
authors.forename: Adam B
authors.identifier: http://orcid.org/0000-0002-1221-5602
authors.initials: AB
authors.lastname: Wilson
mesh_terms: D000715:Anatomy / Q000193:education*; D003479:Curriculum; D004504:Education, Medical, Undergraduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D004522:Educational Status; D005793:Generalization, Psychological; D006653:Histology / Q000193:education*; D006801:Humans; D011582:Psychological Theory; D011594:Psychometrics; D015203:Reproducibility of Results; D012189:Retrospective Studies; D012577:Schools, Medical; D013337:Students, Medical / Q000523:psychology*; D011795:Surveys and Questionnaires; D013997:Time Factors; D014481:United States
keywords: anatomy assessment; curriculum integration; examination reliability; gross anatomy education; histology assessment; histology education; medical education