Effect of response format for clinical vignettes on reporting quality of physician practice.
BACKGROUND
Clinical vignettes have been used widely to compare quality of clinical care and to assess variation in practice, but the effect of different response formats has not been extensively evaluated. Our objective was to compare three clinical vignette-based survey response formats - open-ended questionnaire (A), closed-ended (multiple-choice) questionnaire with deceptive response items mixed with correct items (B), and closed-ended questionnaire with only correct items (C) - in rheumatologists' pre-treatment assessment for tumor-necrosis-factor (TNF) blocker therapy.


METHODS



METHODS
Prospective randomized study.


METHODS
Rheumatologists attending the 2004 French Society of Rheumatology meeting. Physicians were given a vignette describing the history of a fictitious woman with active rheumatoid arthritis, who was a candidate for therapy with TNF blocking agents, and then were randomized to receive questionnaire A, B, or C, each containing the same four questions but with different response formats, that asked about their pretreatment assessment.


METHODS
Long (recommended items) and short (mandatory items) checklists were developed for pretreatment assessment for TNF-blocker therapy, and scores were expressed on the basis of responses to questionnaires A, B, and C as the percentage of respondents correctly choosing explicit items on these checklists.


METHODS
Comparison of the selected items using pairwise Chi-square tests with Bonferonni correction for variables with statistically significant differences.


RESULTS
Data for all surveys distributed (114 As, 118 Bs, and 118 Cs) were complete and available for analysis. The percentage of questionnaire A, B, and C respondents for whom data was correctly complete for the short checklist was 50.4%, 84.0% and 95.0%, respectively, and was 0%, 5.0% and 5.9%, respectively, for the long version. As an example, 65.8%, 85.7% and 95.8% of the respondents of A, B, and C questionnaires, respectively, correctly identified the need for tuberculin skin test (p < 0.0001).


CONCLUSIONS
In evaluating clinical practice with use of a clinical vignette, a multiple-choice format rather than an open-ended format overestimates physician performance. The insertion of deceptive response items mixed with correct items in closed-ended (multiple-choice) questionnaire failed to avoid this overestimation.
Pham
Thao
T
Department of Rheumatology, CHU Conception, 13005 Marseille, France. thao.pham@ap-hm.fr
Roy
Carine
C
Mariette
Xavier
X
Lioté
Fréderic
F
Durieux
Pierre
P
Ravaud
Philippe
P
D000328:Adult; D000739:Anecdotes as Topic; D005260:Female; D005602:France; D006801:Humans; D008297:Male; D008875:Middle Aged; D010818:Practice Patterns, Physicians' / Q000592:standards*; D011446:Prospective Studies; D011787:Quality of Health Care; D012219:Rheumatology; D011795:Surveys and Questionnaires; D014409:Tumor Necrosis Factor-alpha
D014409:Tumor Necrosis Factor-alpha