title: Psychometric testing of the Pecka Grading Rubric for evaluating higher-order thinking in distance learning.
abstract: This article describes development of the Pecka Grading Rubric (PGR) as a strategy to facilitate and evaluate students' higher-order thinking in discussion boards. The purpose of this study was to describe psychometric properties of the PGR. Rubric reliability was pilot tested on a discussion board assignment used by 15 senior student registered nurse anesthetist enrolled in an Advanced Principles of Anesthesia course. Interrater and intrarater reliabilities were tested using an interclass correlation coefficient (ICC) to evaluate absolute agreement of scoring. Raters gave each category a score, scores of the categories were summed, and a total score was calculated for the entire rubric. Interrater (ICC = 0.939, P < .001) and intrarater (ICC = 0.902 to 0.994, P < .001) reliabilities were excellent for total point scores. A content validity index was used to evaluate content validity. Raters evaluated content validity of each cell of the PGR. The content validity index (0.8-1.0) was acceptable. Known-group validity was evaluated by comparing graduate student registered nurse anesthetists (N = 7) with undergraduate senior nursing students (N = 13). Beginning evidence indicates a valid and reliable instrument that measures higher-order thinking in the student registered nurse anesthetist.
authors.forename: Shannon
authors.initials: S
authors.lastname: Pecka
authors.forename: Kendra
authors.initials: K
authors.lastname: Schmid
authors.forename: Bunny
authors.initials: B
authors.lastname: Pozehl
mesh_terms: D000328:Adult; D003657:Decision Making; D020375:Education, Distance / Q000458:organization & administration*; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D006801:Humans; D008297:Male; D009719:Nurse Anesthetists / Q000193:education*; D010865:Pilot Projects; D015397:Program Evaluation; D011594:Psychometrics; D015203:Reproducibility of Results; D013338:Students, Nursing; D013850:Thinking; D014481:United States