Which data source in clinical performance assessment? A pilot study comparing self-recording with patient records and observation.
OBJECTIVE
A pilot study aimed to determine the extent to which each of three data sources could provide complete and reliable data for valid assessment of clinical performance.


METHODS
Clinical decisions taken in 168 consultations by seven family physicians were reviewed against guidelines for 15 clinical conditions. In total, 206 criteria were reviewed using three sources: medical records, observation in surgery, and structured self-recording by the physicians.


METHODS
Seven family practices in the Netherlands.


METHODS
Scores (%) of data recorded/total were obtained for each method. Kappa scores for the agreement between the three data sources were also obtained.


RESULTS
Medical record examination provided 40%, observation 72%, and physician self-recording 95% of the data required for the review against guidelines. Nine per cent of the clinical decisions could be reviewed when using medical records, 46% when using observation data, and 69% when using data from prospective self-recording. In particular, decisions in the area of patient education and diagnostic examinations could not be reviewed validly using medical records only. Kappa agreements between the data available from the three sources as well as between the review results appeared to be 0.79.


CONCLUSIONS
Medical records alone only supply sufficient information for the review of a very limited set of clinical decisions. Physician self-recording has significantly more potential for valid review of a broad range of clinical decisions. Furthermore, self-recording seems a reliable data collection method that deserves further research.
Spies
Teun H
TH
Center for Quality of Care Research and Department of Medical Informatics, University of Nijmegen, PO Box 1901, 6500 HB Nijmegen, The Netherlands. teun.spies@hetnet.nl
Mokkink
Henk G A
HG
De Vries Robbé
Pieter F
PF
Grol
Richard P T M
RP
D002983:Clinical Competence; D003625:Data Collection / Q000379:methods* / Q000592:standards*; D003657:Decision Making; D003920:Diabetes Mellitus / Q000175:diagnosis / Q000628:therapy; D005194:Family Practice / Q000592:standards*; D005260:Female; D006801:Humans; D006973:Hypertension / Q000175:diagnosis / Q000628:therapy; D008297:Male; D008499:Medical Records / Q000592:standards; D008875:Middle Aged; D009426:Netherlands; D010865:Pilot Projects; D017410:Practice Guidelines as Topic; D011787:Quality of Health Care