title: Effect of response format for clinical vignettes on reporting quality of physician practice.
abstract: BACKGROUND
  Clinical vignettes have been used widely to compare quality of clinical care and to assess variation in practice, but the effect of different response formats has not been extensively evaluated. Our objective was to compare three clinical vignette-based survey response formats - open-ended questionnaire (A), closed-ended (multiple-choice) questionnaire with deceptive response items mixed with correct items (B), and closed-ended questionnaire with only correct items (C) - in rheumatologists' pre-treatment assessment for tumor-necrosis-factor (TNF) blocker therapy.
  
  
  METHODS
  
  
  
  METHODS
  Prospective randomized study.
  
  
  METHODS
  Rheumatologists attending the 2004 French Society of Rheumatology meeting. Physicians were given a vignette describing the history of a fictitious woman with active rheumatoid arthritis, who was a candidate for therapy with TNF blocking agents, and then were randomized to receive questionnaire A, B, or C, each containing the same four questions but with different response formats, that asked about their pretreatment assessment.
  
  
  METHODS
  Long (recommended items) and short (mandatory items) checklists were developed for pretreatment assessment for TNF-blocker therapy, and scores were expressed on the basis of responses to questionnaires A, B, and C as the percentage of respondents correctly choosing explicit items on these checklists.
  
  
  METHODS
  Comparison of the selected items using pairwise Chi-square tests with Bonferonni correction for variables with statistically significant differences.
  
  
  RESULTS
  Data for all surveys distributed (114 As, 118 Bs, and 118 Cs) were complete and available for analysis. The percentage of questionnaire A, B, and C respondents for whom data was correctly complete for the short checklist was 50.4%, 84.0% and 95.0%, respectively, and was 0%, 5.0% and 5.9%, respectively, for the long version. As an example, 65.8%, 85.7% and 95.8% of the respondents of A, B, and C questionnaires, respectively, correctly identified the need for tuberculin skin test (p < 0.0001).
  
  
  CONCLUSIONS
  In evaluating clinical practice with use of a clinical vignette, a multiple-choice format rather than an open-ended format overestimates physician performance. The insertion of deceptive response items mixed with correct items in closed-ended (multiple-choice) questionnaire failed to avoid this overestimation.
authors.affiliation: Department of Rheumatology, CHU Conception, 13005 Marseille, France. thao.pham@ap-hm.fr
authors.forename: Thao
authors.initials: T
authors.lastname: Pham
authors.forename: Carine
authors.initials: C
authors.lastname: Roy
authors.forename: Xavier
authors.initials: X
authors.lastname: Mariette
authors.forename: Fréderic
authors.initials: F
authors.lastname: Lioté
authors.forename: Pierre
authors.initials: P
authors.lastname: Durieux
authors.forename: Philippe
authors.initials: P
authors.lastname: Ravaud
mesh_terms: D000328:Adult; D000739:Anecdotes as Topic; D005260:Female; D005602:France; D006801:Humans; D008297:Male; D008875:Middle Aged; D010818:Practice Patterns, Physicians' / Q000592:standards*; D011446:Prospective Studies; D011787:Quality of Health Care; D012219:Rheumatology; D011795:Surveys and Questionnaires; D014409:Tumor Necrosis Factor-alpha
chemical_list: D014409:Tumor Necrosis Factor-alpha