Development of the AGREE II, part 1: performance, usefulness and areas for improvement.
BACKGROUND
We undertook research to improve the AGREE instrument, a tool used to evaluate guidelines. We tested a new seven-point scale, evaluated the usefulness of the original items in the instrument, investigated evidence to support shorter, tailored versions of the tool, and identified areas for improvement.


METHODS
We report on one component of a larger study that used a mixed design with four factors (user type, clinical topic, guideline and condition). For the analysis reported in this article, we asked participants to read a guideline and use the AGREE items to evaluate it based on a seven-point scale, to complete three outcome measures related to adoption of the guideline, and to provide feedback on the instrument's usefulness and how to improve it.


RESULTS
Guideline developers gave lower-quality ratings than did clinicians or policy-makers. Five of six domains were significant predictors of participants' outcome measures (p < 0.05). All domains and items were rated as useful by stakeholders (mean scores > 4.0) with no significant differences by user type (p > 0.05). Internal consistency ranged between 0.64 and 0.89. Inter-rater reliability was satisfactory. We received feedback on how to improve the instrument.


CONCLUSIONS
Quality ratings of the AGREE domains were significant predictors of outcome measures associated with guideline adoption: guideline endorsements, overall intentions to use guidelines, and overall quality of guidelines. All AGREE items were assessed as useful in determining whether a participant would use a guideline. No clusters of items were found more useful by some users than others. The measurement properties of the seven-point scale were promising. These data contributed to the refinements and release of the AGREE II.
Brouwers
Melissa C
MC
McMaster University, Program in Evidence-based Care, Cancer Care Ontario, ON, Canada. mbrouwer@mcmaster.ca
Kho
Michelle E
ME
Browman
George P
GP
Burgers
Jako S
JS
Cluzeau
Francoise
F
Feder
Gene
G
Fervers
Béatrice
B
Graham
Ian D
ID
Hanna
Steven E
SE
Makarski
Julie
J
D005069:Evaluation Studies as Topic; D006282:Health Personnel; D006801:Humans; D015588:Observer Variation; D017063:Outcome Assessment, Health Care; D017410:Practice Guidelines as Topic / Q000592:standards*; D011787:Quality of Health Care