Design and testing of a tool for evaluating the quality of diabetes consumer-information Web sites.
BACKGROUND
Most existing tools for measuring the quality of Internet health information focus almost exclusively on structural criteria or other proxies for quality information rather than evaluating actual accuracy and comprehensiveness.


OBJECTIVE
This research sought to develop a new performance-measurement tool for evaluating the quality of Internet health information, test the validity and reliability of the tool, and assess the variability in diabetes Web site quality.


METHODS
An objective, systematic tool was developed to evaluate Internet diabetes information based on a quality-of-care measurement framework. The principal investigator developed an abstraction tool and trained an external reviewer on its use. The tool included 7 structural measures and 34 performance measures created by using evidence-based practice guidelines and experts' judgments of accuracy and comprehensiveness.


RESULTS
Substantial variation existed in all categories, with overall scores following a normal distribution and ranging from 15% to 95% (mean was 50% and median was 51%). Lin's concordance correlation coefficient to assess agreement between raters produced a rho of 0.761 (Pearson's r of 0.769), suggesting moderate to high agreement. The average agreement between raters for the performance measures was 0.80.


CONCLUSIONS
Diabetes Web site quality varies widely. Alpha testing of this new tool suggests that it could become a reliable and valid method for evaluating the quality of Internet health sites. Such an instrument could help lay people distinguish between beneficial and misleading information.
Seidman
Joshua J
JJ
Center for Information Therapy, Healthwise, Inc, 600 New Hampshire Avenue, NW, Washington, DC 20037, USA. jseidman@healthwise.org
Steinwachs
Donald
D
Rubin
Haya R
HR
D003920:Diabetes Mellitus; D005240:Feasibility Studies; D017408:Guidelines as Topic / Q000592:standards; D006801:Humans; D007255:Information Services / Q000592:standards / Q000706:statistics & numerical data; D020407:Internet / Q000592:standards / Q000706:statistics & numerical data; D010353:Patient Education as Topic / Q000379:methods / Q000592:standards / Q000706:statistics & numerical data; D015397:Program Evaluation / Q000379:methods / Q000706:statistics & numerical data; D011785:Quality Assurance, Health Care / Q000379:methods / Q000706:statistics & numerical data; D011786:Quality Control; D011787:Quality of Health Care / Q000592:standards / Q000706:statistics & numerical data