title: On the factor structure of standardized educational achievement tests.
abstract: This research analyzed the factor structure at both the item- and subtest-level of California's norm- and criterion-referenced standardized educational achievement tests (SEAT) used in that state's high-stakes educational accountability assessments. It was shown through full information factor analysis and multidimensional IRT models (e.g., TESTFACT and NOHARM) that, at the item-level, SEATs are invariably highly unidimensional (i.e., they appear to tap a unidimensional theta scale) even when items representing such diverse content areas such as English, science, mathematics, and history are analyzed simultaneously as a single measure. These item-level factors also accounted for a relatively small proportion (1/4 to 1/3) of the variance. It was also shown that, when these tests are analyzed using more reliable indicators such as subtests, a much richer factor structure emerged that accounted for a larger portion (about 2/3) of the total common variance. As expected, these factor structure configurations (and underlying dimensionality) were preserved across the item- and subtest-levels. However, the factors emerging from both the item- and subtest-level analyses were highly correlated and produced strong second-order and general factors. The meaning underlying these results was examined, along with their implications with respect to the assumptions underlying modern approaches to test calibration, scaling, and score interpretation.
authors.affiliation: California Department of Education, Sacramento, CA 95814-5901, USA. TGaffney@cde.ca.gov
authors.forename: Tim W
authors.initials: TW
authors.lastname: Gaffney
authors.forename: Robert
authors.initials: R
authors.lastname: Cudeck
authors.forename: Emilio
authors.initials: E
authors.lastname: Ferrer
authors.forename: Keith F
authors.initials: KF
authors.lastname: Widaman
mesh_terms: D000293:Adolescent; D002140:California; D002648:Child; D004521:Educational Measurement / Q000592:standards / Q000706:statistics & numerical data*; D006801:Humans; D015233:Models, Statistical