title: Construct validity of scores/measures from a developmental assessment in mathematics using classical and many-facet Rasch measurement.
abstract: Data from a developmental assessment comprised of 9 short answer mathematics tasks were validated using classical and three-faceted Rasch measurement methods. Field test data from a mixed age elementary school sample (N=280) were analyzed. Descriptive statistics on scores from the overall scale and two subdomains indicated improved performance with age. The data showed better fit with a two-factor model corresponding with the subdomain structure (Bentler's CFI=.94), than a one factor model (CFI=.87). The inter-factor correlation was.76. Convergent validity coefficients of scores with scaled scores of the Stanford AchievementTest mathematics battery ranged from.28 to.47; internal consistency reliability of the total and subdomain scores ranged from.87 to.89, respectively; and median inter-rater reliability was.75. On average, persons, tasks and raters showed acceptable fit with the three-facet Rasch model. Rasch logit difficulties of tasks suggested an ordered scale structure, although tasks tended to have high difficulty levels. The original and calibrated task ordering was consistent at the extreme ends of the scale. Gaps identified on the Rasch item map suggested a need for additional task construction. Conceptual and procedural differences in each technique are considered in deciding future improvements to the scale.
authors.affiliation: Dept. of Educational Measurement and Research, University of South Florida, Tampa, FL 33620, USA. banerji@tempest.coedu.usf.edu
authors.forename: M
authors.initials: M
authors.lastname: Banerji
mesh_terms: D001077:Aptitude Tests / Q000706:statistics & numerical data*; D002648:Child; D004521:Educational Measurement / Q000706:statistics & numerical data*; D005260:Female; D006801:Humans; D016014:Linear Models; D008297:Male; D008433:Mathematics; D011594:Psychometrics / Q000379:methods*; D015203:Reproducibility of Results