A new implicit review instrument for measuring quality of care delivered to pediatric patients in the emergency department.
BACKGROUND
There are few outcomes experienced by children receiving care in the Emergency Department (ED) that are amenable to measuring for the purposes of assessing of quality of care. The purpose of this study was to develop, test, and validate a new implicit review instrument that measures quality of care delivered to children in EDs.


METHODS
We developed a 7-point structured implicit review instrument that encompasses four aspects of care, including the physician's initial data gathering, integration of information and development of appropriate diagnoses; initial treatment plan and orders; and plan for disposition and follow-up. Two pediatric emergency medicine physicians applied the 5-item instrument to children presenting in the highest triage category to four rural EDs, and we assessed the reliability of the average summary scores (possible range of 5-35) across the two reviewers using standard measures. We also validated the instrument by comparing this mean summary score between those with and without medication errors (ascertained independently by two pharmacists) using a two-sample t-test.


RESULTS
We reviewed the medical records of 178 pediatric patients for the study. The mean and median summary score for this cohort of patients were 27.4 and 28.5, respectively. Internal consistency was high (Cronbach's alpha of 0.92 and 0.89). All items showed a significant (p < 0.005) positive correlation between reviewers using the Spearman rank correlation (range 0.24 to 0.39). Exact agreement on individual items between reviewers ranged from 70.2% to 85.4%. The Intra-class Correlation Coefficient for the mean of the total summary score across the two reviewers was 0.65. The validity of the instrument was supported by the finding of a higher score for children without medication errors compared to those with medication errors which trended toward significance (mean score = 28.5 vs. 26.0, p = 0.076).


CONCLUSIONS
The instrument we developed to measure quality of care provided to children in the ED has high internal consistency, fair to good inter-rater reliability and inter-rater correlation, and high content validity. The validity of the instrument is supported by the fact that the instrument's average summary score was lower in the presence of medication errors, which trended towards statistical significance.
Dharmar
Madan
M
Department of Pediatrics, University of California, Davis, USA. mdharmar@ucdavis.edu.
Marcin
James P
JP
Kuppermann
Nathan
N
Andrada
Emily R
ER
Cole
Stacey
S
Harvey
Danielle J
DJ
Romano
Patrick S
PS
N Engl J Med. 1985 Oct 31;313(18):1131-7
3930964
Med Care. 2000 Feb;38(2):152-61
10659689
JAMA. 1990 Oct 17;264(15):1969-73
2120476
Pediatrics. 2002 Oct;110(4):737-42
12359787
JAMA. 1997 Nov 19;278(19):1573-8
9370502
JAMA. 1992 May 6;267(17):2349-54
1564775
Ann Emerg Med. 2004 Apr;43(4):461-8
15039688
Pediatrics. 1987 May;79(5):718-22
3575028
Eval Health Prof. 1994 Mar;17(1):3-21
10132480
JAMA. 1990 Oct 17;264(15):1974-9
2214062
Acad Emerg Med. 2001 Feb;8(2):156-62
11157292
JAMA. 1988 Sep 23-30;260(12):1743-8
3045356
JAMA. 1991 Apr 17;265(15):1957-60
2008024
JAMA. 2001 Apr 25;285(16):2114-20
11311101
Int J Qual Health Care. 2001 Oct;13(5):357-65
11669563
Ann Intern Med. 1993 Apr 1;118(7):550-6
8442625
Med Care. 1979 Mar;17(3 Suppl):i-vi, 1-73
763009
JAMA. 1992 Feb 19;267(7):958-60
1734109
J Gen Intern Med. 1995 Apr;10(4):199-205
7790981
Pediatrics. 2002 Mar;109(3):505-8
11875148
Psychol Bull. 1979 Mar;86(2):420-8
18839484
Pediatrics. 2005 Feb;115(2):388-95
15687449
Acad Emerg Med. 2000 Jan;7(1):21-7
10894238
Emerg Med Serv. 2004 Nov;33(11):47-8, 50, 52 passim
15682750
Biometrics. 1977 Mar;33(1):159-74
843571
Lancet. 1989 Aug 12;2(8659):374-6
2569561
Pediatr Emerg Care. 1998 Apr;14(2):89-94
9583386
Pediatr Emerg Care. 1999 Feb;15(1):1-4
10069301
Med Care. 1999 Aug;37(8):798-808
10448722