Assessing quality of case series studies: development and validation of an instrument by herbal medicine CAM researchers.
OBJECTIVE
When higher level of evidence is absent, findings of case series studies can be used to generate hypotheses for further research. However, to date, there has been no established instrument for determining their quality. This study was to develop and validate an instrument to assess the quality of case series studies on herbal medicines.


METHODS
A draft instrument was developed based on items generated by judges. Raters (including judges), consisting of herbal medicine researchers, herbal medicine practitioners, and other professionals, used the draft instrument to assess the quality of sample case series reports. The number of quality assessment items was then reduced as much as possible in the instrument while maintaining its reliability and validity. Content and construct validity, and inter-rater agreement of the instrument were used as the main outcome measures.


RESULTS
Seven (7) judges from different backgrounds independently generated 68 nonredundant items for inclusion in the initial draft instrument. The content validity of these items was assessed by all judges for consensus using a seven-point scale, and a 24-item second-draft instrument was derived. The wording and sequence of these items were further revised and then used by 20 raters (including seven judges) to assess the quality of 35 case series reports. A 13-item instrument was finally developed. All raters, including the judges, demonstrated a high level of consistency on quality of the reports by using this instrument, despite their different backgrounds.


CONCLUSIONS
High-level consistency can be achieved by using this instrument to assess the quality of case series studies. This instrument may facilitate the improvement of reporting quality of case series studies.
The RMIT Chinese Medicine Research Group, Division of Chinese Medicine, School of Health Sciences, RMIT University, Melbourne, Australia.
Angela Weihong
AW
Yang
Chun Guang
CG
Li
Cliff
C
Da Costa
Garry
G
Allan
John
J
Reece
Charlie Changli
CC
Xue
D035843:Biomedical Research / Q000379:methods*; D005069:Evaluation Studies as Topic; D029001:Herbal Medicine; D006801:Humans; D015588:Observer Variation; D008517:Phytotherapy; D015203:Reproducibility of Results; D012107:Research Design / Q000592:standards*