Measuring the reliability of a computer-aided systematic observation instrument.
The inter- and intraobserver reliabilities of data collected by observers trained in the use of the coach analysis instrument (CAI) are reported. The CAI is part of the computerized coaching analysis system (CCAS). The CAI collects data related to the learning environment created by the coach in a team/sport practice situation. Both inter- and intraobserver reliabilities are reported for each of the instrument's seven dimensions as well as for the overall instrument. The reliability measures reported are total percent agreement (T) and Cohen's kappa (K). The results indicated that, although the overall coaching analysis instrument reached a T value of 90.8%, it failed to meet the 80% level for the K measure (K = 78.5%). A dimension by dimension examination of the data was undertaken to determine the reasons for these results and it was revealed that the problem was associated with three specific dimensions of the instrument.
Johnson
R B
RB
School of Physical Education and Recreation, University of British Columbia.
Franks
I M
IM
D000465:Algorithms; D003199:Computer Systems; D003194:Computer-Assisted Instruction / Q000379:methods*; D005069:Evaluation Studies as Topic; D005260:Female; D006801:Humans; D008297:Male; D010806:Physical Education and Training / Q000379:methods*; D014743:Videotape Recording / Q000379:methods*