Reliability of the PEDro scale for rating quality of randomized controlled trials.
OBJECTIVE
Assessment of the quality of randomized controlled trials (RCTs) is common practice in systematic reviews. However, the reliability of data obtained with most quality assessment scales has not been established. This report describes 2 studies designed to investigate the reliability of data obtained with the Physiotherapy Evidence Database (PEDro) scale developed to rate the quality of RCTs evaluating physical therapist interventions.


METHODS
In the first study, 11 raters independently rated 25 RCTs randomly selected from the PEDro database. In the second study, 2 raters rated 120 RCTs randomly selected from the PEDro database, and disagreements were resolved by a third rater; this generated a set of individual rater and consensus ratings. The process was repeated by independent raters to create a second set of individual and consensus ratings. Reliability of ratings of PEDro scale items was calculated using multirater kappas, and reliability of the total (summed) score was calculated using intraclass correlation coefficients (ICC [1,1]).


RESULTS
The kappa value for each of the 11 items ranged from.36 to.80 for individual assessors and from.50 to.79 for consensus ratings generated by groups of 2 or 3 raters. The ICC for the total score was.56 (95% confidence interval=.47-.65) for ratings by individuals, and the ICC for consensus ratings was.68 (95% confidence interval=.57-.76).


CONCLUSIONS
The reliability of ratings of PEDro scale items varied from "fair" to "substantial," and the reliability of the total PEDro score was "fair" to "good."
Maher
Christopher G
CG
School of Physiotherapy, Faculty of Health Sciences, The University of Sydney, PO Box 170, Lidcombe, New South Wales 1825, Australia. C.Maher@fhs.usyd.edu.au
Sherrington
Catherine
C
Herbert
Robert D
RD
Moseley
Anne M
AM
Elkins
Mark
M
D016206:Databases, Bibliographic; D019317:Evidence-Based Medicine; D006801:Humans; D026741:Physical Therapy Modalities; D011786:Quality Control; D016032:Randomized Controlled Trials as Topic / Q000592:standards*; D015203:Reproducibility of Results