How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients visits.
OBJECTIVE
This study was designed to determine the optimal nonobservational method of measuring the delivery of outpatient medical services.


METHODS
As part of a multimethod study of the content of primary care practice, research nurses directly observed consecutive patient visits to 138 practicing family physicians. Data on services delivered were collected using a direct observation checklist, medical record review, and patient exit questionnaires. For each medical service, the sensitivity, specificity, and Kappa statistic were calculated for medical record review and patient exit questionnaires compared with direct observation. Interrater reliability among eight research nurses was calculated using the Kappa statistic for a separate sample of videotaped visits and medical records.


RESULTS
Visits by 4,454 patients were observed. Exit questionnaires were returned by 74% of patients. Research nurse interrater reliabilities were generally high. The specificity of both the medical record and the patient exit questionnaire was high for most services. The sensitivity of the medical record was low for measuring health habit counseling and moderate for physical examination, laboratory testing, and immunization. The patient exit questionnaire showed moderate to high sensitivity for health habit counseling and immunization and variable sensitivity for physical examination and laboratory services.


CONCLUSIONS
The validity of the medical record and patient questionnaire for measuring delivery of different health services varied with the service. This report can be used to choose the optimal nonobservational method of measuring the delivery of specific ambulatory medical services for research and physician profiling and to interpret existing health services research studies using these common measures.
Stange
K C
KC
Department of Family Medicine, Case Western Reserve University, Cleveland, OH 44106, USA.
Zyzanski
S J
SJ
Smith
T F
TF
Kelly
R
R
Langa
D M
DM
Flocke
S A
SA
Jaén
C R
CR
D000328:Adult; D005194:Family Practice / Q000592:standards*; D005260:Female; D006302:Health Services Research / Q000379:methods* / Q000592:standards; D006801:Humans; D008297:Male; D008485:Medical Audit / Q000592:standards; D008499:Medical Records / Q000592:standards*; D008875:Middle Aged; D015588:Observer Variation; D009819:Office Visits; D009820:Ohio; D017060:Patient Satisfaction; D011320:Primary Health Care / Q000592:standards; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D011795:Surveys and Questionnaires / Q000592:standards*; D014743:Videotape Recording