title: What Do You Think You Are Measuring? A Mixed-Methods Procedure for Assessing the Content Validity of Test Items and Theory-Based Scaling.
abstract: The valid measurement of latent constructs is crucial for psychological research. Here, we present a mixed-methods procedure for improving the precision of construct definitions, determining the content validity of items, evaluating the representativeness of items for the target construct, generating test items, and analyzing items on a theoretical basis. To illustrate the mixed-methods content-scaling-structure (CSS) procedure, we analyze the Adult Self-Transcendence Inventory, a self-report measure of wisdom (ASTI, Levenson et al., 2005). A content-validity analysis of the ASTI items was used as the basis of psychometric analyses using multidimensional item response models (N= 1215). We found that the new procedure produced important suggestions concerning five subdimensions of the ASTI that were not identifiable using exploratory methods. The study shows that the application of the suggested procedure leads to a deeper understanding of latent constructs. It also demonstrates the advantages of theory-based item analysis.
authors.affiliation: Department of Psychology, Alpen-Adria-Universität Klagenfurt Klagenfurt, Austria.
authors.forename: Ingrid
authors.initials: I
authors.lastname: Koller
authors.affiliation: Department of Human Development and Family Sciences, Oregon State University Corvallis, OR, USA.
authors.forename: Michael R
authors.initials: MR
authors.lastname: Levenson
authors.affiliation: Department of Psychology, Alpen-Adria-Universität Klagenfurt Klagenfurt, Austria.
authors.forename: Judith
authors.initials: J
authors.lastname: Glück
keywords: Adult Self-Transcendence Inventar (ASTI); CSS-procedure; content validity; item response models; mixed-methods; partial credit model; theory-based item analysis; wisdom