Measuring implementation fidelity in a community-based parenting intervention.
BACKGROUND
Establishing the feasibility and validity of implementation fidelity monitoring strategies is an important methodological step in implementing evidence-based interventions on a large scale.


OBJECTIVE
The objective of the study was to examine the reliability and validity of the Fidelity Checklist, a measure designed to assess group leader adherence and competence delivering a parent training intervention (the Chicago Parent Program) in child care centers serving low-income families.


METHODS
The sample included 9 parent groups (12 group sessions each), 12 group leaders, and 103 parents. Independent raters reviewed 106 audiotaped parent group sessions and coded group leaders' fidelity on the Adherence and Competence Scales of the Fidelity Checklist. Group leaders completed self-report adherence checklists and a measure of parent engagement in the intervention. Parents completed measures of consumer satisfaction and child behavior.


RESULTS
High interrater agreement (Adherence Scale = 94%, Competence Scale = 85%) and adequate intraclass correlation coefficients (Adherence Scale = .69, Competence Scale = .91) were achieved for the Fidelity Checklist. Group leader adherence changed over time, but competence remained stable. Agreement between group leader self-report and independent ratings on the Adherence Scale was 85%; disagreements were more frequently due to positive bias in group leader self-report. Positive correlations were found between group leader adherence and parent attendance and engagement in the intervention and between group leader competence and parent satisfaction. Although child behavior problems improved, improvements were not related to fidelity.


CONCLUSIONS
The results suggest that the Fidelity Checklist is a feasible, reliable, and valid measure of group leader implementation fidelity in a group-based parenting intervention. Future research will be focused on testing the Fidelity Checklist with diverse and larger samples and generalizing to other group-based interventions using a similar intervention model.
Breitenstein
Susan M
SM
Rush University College of Nursing, Chicago, IL 60612, USA. susan_m_breitenstein@rush.edu
Fogg
Louis
L
Garvey
Christine
C
Hill
Carri
C
Resnick
Barbara
B
Gross
Deborah
D
D000704:Analysis of Variance; D001291:Attitude of Health Personnel; D057189:Checklist / Q000379:methods / Q000592:standards*; D002641:Chicago; D002653:Child Behavior Disorders / Q000517:prevention & control; D002656:Child Day Care Centers; D002675:Child, Preschool; D002983:Clinical Competence / Q000592:standards; D002985:Clinical Protocols; D004059:Diffusion of Innovation; D055317:Evidence-Based Practice; D005240:Feasibility Studies; D006266:Health Education / Q000592:standards; D006801:Humans; D007857:Leadership; D015403:Nursing Evaluation Research; D015588:Observer Variation; D010290:Parents / Q000193:education* / Q000523:psychology; D017060:Patient Satisfaction; D011203:Poverty; D015397:Program Evaluation / Q000379:methods*; D011594:Psychometrics
J Consult Clin Psychol. 2007 Dec;75(6):829-41
18085901
Child Care Health Dev. 2008 May;34(3):391-400
18410645
Implement Sci. 2007 Nov 30;2:40
18053122
Behav Ther. 2005;36(1):3-13
16718302
Psychol Assess. 2006 Sep;18(3):313-23
16953734
J Consult Clin Psychol. 1999 Oct;67(5):631-47
10535230
Clin Psychol Rev. 2007 Jul;27(6):739-49
17336435
J Abnorm Child Psychol. 2000 Feb;28(1):33-46
10772348
Health Educ Res. 2003 Apr;18(2):237-56
12729182
Prev Sci. 2006 Mar;7(1):57-74
16555144
Res Nurs Health. 2006 Jun;29(3):212-22
16676341
J Consult Clin Psychol. 2000 Jun;68(3):451-67
10883562
Prev Sci. 2008 Sep;9(3):215-29
18648936
Clin Child Fam Psychol Rev. 1999 Mar;2(1):37-53
11324096
Nurs Res. 2007 Jan-Feb;56(1):54-62
17179874
Prev Sci. 2004 Mar;5(1):47-53
15058912
Prev Sci. 2009 Mar;10(1):54-65
19067166
J Consult Clin Psychol. 2008 Aug;76(4):544-55
18665684
Ann Behav Med. 2005 Apr;29 Suppl:46-54
15921489
Am J Prev Med. 2001 Jan;20(1 Suppl):38-47
11146259