title: Evaluation of a collaborative program on smoking cessation: translating outcomes framework into practice.
abstract: BACKGROUND
  Although evaluating at multiple outcome levels has been proposed for continuing education activities and programs, it is a complex undertaking and is not done routinely, especially in collaborative, multicomponent programs. This article reports on strategies used and results obtained in an evaluation project that examined multiple outcomes of a US-based collaborative, multicomponent smoking cessation educational program for clinicians.
  
  
  METHODS
  Evaluation was organized conceptually around the 6 levels of an outcomes-based evaluation model and was conducted using registration data, postactivity evaluations, clinical vignettes with questions assessing knowledge and competence in participants and in a comparison group, a commitment to change approach, data from patient charts to assess clinician compliance on 8 performance measures, and tobacco cessation rates. Additional methods included a success case method study of 9 practices participating in performance improvement (PI) activities and assessment of partner collaboration using a written survey and interviews.
  
  
  RESULTS
  The program reached more than 43,000 clinicians who participated in a variety of activities. Participants indicated a high level of satisfaction with the program's educational activities and demonstrated higher scores than a comparison group on 6 of 7 competencies. The majority of participants who responded to commitment to change questions reported intended and implemented practice changes consistent with desired outcomes. Performance outcomes of 3 PI activities varied, with greater improvements observed in 1 activity (9.0% to 36.2% improvement across 8 measures). Lower performance outcomes, but a smoking quit rate of 46.8%, was observed in 2 other PI activities.
  
  
  CONCLUSIONS
  The program had an overall positive impact on the measured variables for clinicians and their patients. Use of 1 outcomes assessment framework acceptable to all members of the collaborative, common measures and evaluation techniques, and centralized data repositories contributed to the success of the program evaluation reported here and is recommended to others who are considering a collaborative program evaluation.
authors.affiliation: Office of Continuing Professional Development in Medicine and Public Health, University of Wisconsin-Madison, USA. mbshershneva@ocpd.wisc.edu
authors.forename: Marianna B
authors.initials: MB
authors.lastname: Shershneva
authors.forename: Christopher
authors.initials: C
authors.lastname: Larrison
authors.forename: Sheila
authors.initials: S
authors.lastname: Robertson
authors.forename: Mike
authors.initials: M
authors.lastname: Speight
mesh_terms: D000465:Algorithms; D001291:Attitude of Health Personnel; D019985:Benchmarking / Q000379:methods*; D002983:Clinical Competence / Q000592:standards*; D003299:Cooperative Behavior; D004502:Education, Medical, Continuing / Q000592:standards; D006282:Health Personnel / Q000145:classification / Q000523:psychology / Q000706:statistics & numerical data; D006801:Humans; D007400:Interprofessional Relations; D017146:Models, Organizational; D017063:Outcome Assessment, Health Care / Q000145:classification / Q000458:organization & administration* / Q000592:standards; D017410:Practice Guidelines as Topic / Q000592:standards; D015397:Program Evaluation; D058996:Quality Improvement / Q000592:standards; D016540:Smoking Cessation / Q000379:methods*; D011795:Surveys and Questionnaires; D014029:Tobacco Use Disorder / Q000517:prevention & control*; D014481:United States