title: Treatment fidelity instrument to measure a brief opportunistic intervention for prenatal substance use.
abstract: OBJECTIVE
  To develop and psychometrically evaluate an instrument designed to measure the treatment fidelity associated with implementation of the I Am Concerned (IAC) brief opportunistic intervention by frontline, prenatal, primary care staff.
  
  
  METHODS
  A methodologic approach framed development of the IAC Treatment Fidelity Instrument in a six-phase protocol.
  
  
  METHODS
  A simulated prenatal clinic with standardized patients portraying substance-using pregnant women.
  
  
  METHODS
  Prenatal, primary care, frontline staff (N = 6), experienced in IAC implementation.
  
  
  METHODS
  Following development of the IAC treatment fidelity instrument, independent raters used the instrument to evaluate audio recordings (N = 49) of frontline staff implementing the IAC brief opportunistic intervention with standardized patients representing substance-using pregnant women.
  
  
  RESULTS
  Psychometric analysis provided evidence of content validity. Intraclass correlation coefficients calculated for inter-rater reliability were satisfactory for subscales (0.64) and (0.62) and ranged from -0.07 to 0.81 for individual items. Internal consistency alpha coefficients were satisfactory for the total scale (0.72) and lower than acceptable for adherence (0.54) and competence (0.56) subscales. Overall high rater percentage agreement and negatively skewed ratings distribution indicated reliability results were paradoxically low due to the base rate problem.
  
  
  CONCLUSIONS
  Results support revision and ongoing testing of the IAC treatment fidelity instrument. The impact on reliability statistics exerted by this study's skewed data distribution has implications for nursing research as low variance can be anticipated when measuring care provided to homogenous patient populations. It is important to recognize the resulting influence on inter-rater agreement to avoid making inaccurate interpretations about the reliability of an instrument's measurements.
authors.affiliation: Department of Nursing and Allied Health, Cuesta Community College, San Luis Obispo, CA 93403-8106, USA. atorrey@cuesta.edu
authors.forename: Antonia Rae
authors.initials: AR
authors.lastname: Torrey
mesh_terms: D000328:Adult; D001291:Attitude of Health Personnel; D002140:California; D002983:Clinical Competence; D005260:Female; D006801:Humans; D008962:Models, Theoretical; D015588:Observer Variation; D010348:Patient Care Team / Q000458:organization & administration*; D010818:Practice Patterns, Physicians'; D011247:Pregnancy; D011248:Pregnancy Complications / Q000175:diagnosis* / Q000523:psychology / Q000628:therapy; D011295:Prenatal Care / Q000379:methods* / Q000639:trends; D011320:Primary Health Care / Q000458:organization & administration; D011594:Psychometrics / Q000295:instrumentation; D058996:Quality Improvement; D012015:Reference Standards; D015203:Reproducibility of Results; D019966:Substance-Related Disorders / Q000175:diagnosis* / Q000523:psychology / Q000628:therapy