Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project.
BACKGROUND
International interest in clinical practice guidelines has never been greater but many published guidelines do not meet the basic quality requirements. There have been renewed calls for validated criteria to assess the quality of guidelines.


OBJECTIVE
To develop and validate an international instrument for assessing the quality of the process and reporting of clinical practice guideline development.


METHODS
The instrument was developed through a multi-staged process of item generation, selection and scaling, field testing, and refinement procedures. 100 guidelines selected from 11 participating countries were evaluated independently by 194 appraisers with the instrument. Following refinement the instrument was further field tested on three guidelines per country by a new set of 70 appraisers.


RESULTS
The final version of the instrument contained 23 items grouped into six quality domains with a 4 point Likert scale to score each item (scope and purpose, stakeholder involvement, rigour of development, clarity and presentation, applicability, editorial independence). 95% of appraisers found the instrument useful for assessing guidelines. Reliability was acceptable for most domains (Cronbach's alpha 0.64-0.88). Guidelines produced as part of an established guideline programme had significantly higher scores on editorial independence and, after the publication of a national policy, had significantly higher quality scores on rigour of development (p<0.005). Guidelines with technical documentation had higher scores on that domain (p<0.0001).


CONCLUSIONS
This is the first time an appraisal instrument for clinical practice guidelines has been developed and tested internationally. The instrument is sensitive to differences in important aspects of guidelines and can be used consistently and easily by a wide range of professionals from different backgrounds. The adoption of common standards should improve the consistency and quality of the reporting of guideline development worldwide and provide a framework to encourage international comparison of clinical practice guidelines.
D005060:Europe; D006302:Health Services Research / Q000379:methods; D006801:Humans; D017410:Practice Guidelines as Topic / Q000592:standards*; D016730:Program Development; D015397:Program Evaluation; D019984:Quality Indicators, Health Care