Criterion validity of responses to patient vignettes: an analysis based on management of female urinary incontinence.
OBJECTIVE
Patient vignettes are often used for investigating the performance of physicians. The purpose of this study was to assess the criterion validity of vignettes.


METHODS
A random sample of incontinent women were interviewed, and vignettes depicting their cases were later presented to their own physicians. Questionnaires sent to the physicians included open questions about what drug treatment the doctor would give and a checklist (cueing items) for indicating other possible actions. Analysis compared actions taken with vignette patients and actual actions taken with the real patients.


RESULTS
Thirty-two general practitioners and 32 patients were included in the study. When cueing items were provided, the physicians claimed more actions with vignettes than were actually performed. Mean difference was 1.9 actions per case (95% confidence interval 1.3 to 2.4). No such difference was found for the open question. When cues were provided, the likelihood ratio for reporting a vignette action in comparison with actually performing the action with a similar real patient was 2.8 (2.2 to 3.5), and Kappa .39. When no cues were provided, the likelihood ratio was 5.1 (2.6 to 10), and Kappa .45. Total number of actions with vignettes and real patients correlated significantly (r = .65, P < .001).


CONCLUSIONS
Responses to vignettes with cueing items overestimate real performance. The agreement may be better when no cues are offered. The method may have acceptable validity for group comparisons.
Sandvik
H
H
Department of Public Health and Primary Health Care, University of Bergen, Norway.
D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D002983:Clinical Competence; D005194:Family Practice / Q000592:standards*; D005260:Female; D006801:Humans; D008485:Medical Audit / Q000379:methods*; D008875:Middle Aged; D009664:Norway; D017063:Outcome Assessment, Health Care; D015203:Reproducibility of Results; D012107:Research Design; D011795:Surveys and Questionnaires; D014549:Urinary Incontinence / Q000628:therapy*