A Tutorial on Reliability Testing in AAC Language Sample Transcription and Analysis.
Establishing reliability is an essential step in language sample transcription and analysis. This tutorial provides an illustration of replicable procedures for reliability testing during transcription and analysis of language samples generated by people who use augmentative and alternative communication (AAC) systems. Statistical measures used for testing agreement between raters coding categorical data are summarized. Detailed procedures for reliability testing in AAC language sample transcription and analysis are provided, beginning with the collection of raw language sample data. Procedures include guidelines for (a) establishing inter-judge agreement during the transcription process, and (b) using Cohen's kappa to establish inter-rater reliability during deeper analysis of transcribed utterances. All procedures are demonstrated in a case example using language samples from children who use AAC.
Department of Communication Science and Disorders, University of Pittsburgh , PA , USA.
Thomas
T
Kovacs
Katya
K
Hill
D002675:Child, Preschool; D003142:Communication; D003143:Communication Devices for People with Disabilities; D006801:Humans; D007802:Language; D008297:Male; D015588:Observer Variation; D015203:Reproducibility of Results; D013223:Statistics as Topic
Augmentative and alternative communication; Kappa; Language sample; Reliability; Utterance