A an investigation of interrater reliability among athletic training accreditation site visitors.
Accreditation plays an important role in many allied health professions. The main purpose of this study was to determine interrater reliability among athletic training accreditation site visitors assigned to interpret and judge a college or university's educational program in accordance with published standards. Ten scenarios were sent electronically to 135 site visitors, of which 93 (68%) responded. Respondents rated whether hypothetical situations described in each scenario were compliant (i.e., meets standard) or noncompliant (i.e., does not meet standard). A comment section was included for site visitors to justify their responses and/or to clarify their selections. The results of this study showed that there was poor interrater reliability in the judgments made by the site visitors. Moreover, a majority of the respondents responded to two of the 10 scenarios incorrectly (scenarios 5 and 10). The average score was 6.77 +/- 1.57, or 68%. Site visitors who misinterpret standards or apply personal values pose serious threats to the accreditation process because they may judge educational programs compliant when in fact programs are noncompliant and vice versa. Furthermore, this study suggests that there may be a great deal of variability across academic programs in applying accreditation standards because of the low agreement among site visitors. Therefore, educational methods and training procedures for accreditation site visitors should be continually reexamined in an attempt to improve and ensure consistency in allied health professions.
Wimer
Jeffrey W
JW
Department of Wellness and Sport Sciences, Millersville University, P.O. Box 1002, 1 South George Street, Millersville, PA 17551-0302, USA. jeffrey.wimer@millersville.edu
D000068:Accreditation / Q000592:standards*; D000328:Adult; D005260:Female; D006801:Humans; D008297:Male; D015588:Observer Variation; D015203:Reproducibility of Results; D013177:Sports / Q000193:education*; D014495:Universities