Development of the AGREE II, part 2: assessment of validity of items and tools to support application.
BACKGROUND
We established a program of research to improve the development, reporting and evaluation of practice guidelines. We assessed the construct validity of the items and user's manual in the beta version of the AGREE II.


METHODS
We designed guideline excerpts reflecting high-and low-quality guideline content for 21 of the 23 items in the tool. We designed two study packages so that one low-quality and one high-quality version of each item were randomly assigned to each package. We randomly assigned 30 participants to one of the two packages. Participants reviewed and rated the guideline content according to the instructions of the user's manual and completed a survey assessing the manual.


RESULTS
In all cases, content designed to be of high quality was rated higher than low-quality content; in 18 of 21 cases, the differences were significant (p < 0.05). The manual was rated by participants as appropriate, easy to use, and helpful in differentiating guidelines of varying quality, with all scores above the mid-point of the seven-point scale. Considerable feedback was offered on how the items and manual of the beta-AGREE II could be improved.


CONCLUSIONS
The validity of the items was established and the user's manual was rated as highly useful by users. We used these results and those of our study presented in part 1 to modify the items and user's manual. We recommend AGREE II (available at www.agreetrust.org) as the revised standard for guideline development, reporting and evaluation.
McMaster University, Cancer Care Ontario, Hamilton, Ont, Canada. mbrouwer@mcmaster.ca
Melissa C
MC
Brouwers
Michelle E
ME
Kho
George P
GP
Browman
Jako S
JS
Burgers
Françoise
F
Cluzeau
Gene
G
Feder
Béatrice
B
Fervers
Ian D
ID
Graham
Steven E
SE
Hanna
Julie
J
Makarski
D005069:Evaluation Studies as Topic; D006282:Health Personnel; D006801:Humans; D008370:Manuals as Topic / Q000592:standards; D017410:Practice Guidelines as Topic / Q000592:standards*; D011785:Quality Assurance, Health Care / Q000379:methods; D015203:Reproducibility of Results