A mandala of faculty development: using theory-based evaluation to explore contexts, mechanisms and outcomes.
Demonstrating the impact of faculty development, is an increasingly mandated and ever elusive goal. Questions have been raised about the adequacy of current approaches. Here, we integrate realist and theory-driven evaluation approaches, to evaluate an intensive longitudinal program. Our aim is to elucidate how faculty development can work to support a range of outcomes among individuals and sub-systems in the academic health sciences. We conducted retrospective framework analysis of qualitative focus group data gathered from 79 program participants (5 cohorts) over a 10-year period. Additionally, we conducted follow-up interviews with 15 alumni. We represent the interactive relationships among contexts, mechanisms, and outcomes as a "mandala" of faculty development. The mandala illustrates the relationship between the immediate program context, and the broader institutional context of academic health sciences, and identifies relevant change mechanisms. Four primary mechanisms were collaborative-reflection, self-reflection and self-regulation, relationship building, and pedagogical knowledge acquisition. Individual outcomes, including changed teaching practices, are described. Perhaps most interestingly, secondary mechanisms-psychological and structural empowerment-contributed to institutional outcomes through participants' engagement in change leadership in their local contexts. Our theoretically informed evaluation approach models how faculty development, situated in appropriate institutional contexts, can trigger mechanisms that yield a range of benefits for faculty and their institutions. The adopted methods hold potential as a way to demonstrate the often difficult-to-measure outcomes of educational programs, and allow for critical examination as to how and whether faculty development programs can accomplish their espoused goals.
Centre for Faculty Development, Faculty of Medicine, Li Ka Shing Knowledge Institute, University of Toronto at St. Michael's Hospital, 4th Floor, 209 Victoria Street, Toronto, ON, M5B 1W8, Canada. onyurab@smh.ca.
Betty
B
Onyura
Centre for Faculty Development, Faculty of Medicine, Li Ka Shing Knowledge Institute, University of Toronto at St. Michael's Hospital, 4th Floor, 209 Victoria Street, Toronto, ON, M5B 1W8, Canada.|Department of Speech-Language Pathology, University of Toronto, Toronto, ON, Canada.|Centre for Ambulatory Care Education, Women's College Hospital, Toronto, ON, Canada.|Rehabilitation Sciences Institute (RSI), University of Toronto, Toronto, ON, Canada.|The Wilson Centre, Faculty of Medicine, University of Toronto at University Health Network, Toronto, ON, Canada.
Stella L
SL
Ng
Centre for Faculty Development, Faculty of Medicine, Li Ka Shing Knowledge Institute, University of Toronto at St. Michael's Hospital, 4th Floor, 209 Victoria Street, Toronto, ON, M5B 1W8, Canada.|Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, ON, Canada.
Lindsay R
LR
Baker
Centre for Faculty Development, Faculty of Medicine, Li Ka Shing Knowledge Institute, University of Toronto at St. Michael's Hospital, 4th Floor, 209 Victoria Street, Toronto, ON, M5B 1W8, Canada.|Department of Psychiatry, Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
Susan
S
Lieff
Centre for Faculty Development, Faculty of Medicine, Li Ka Shing Knowledge Institute, University of Toronto at St. Michael's Hospital, 4th Floor, 209 Victoria Street, Toronto, ON, M5B 1W8, Canada.|Department of Radiation Oncology, Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
Barbara-Ann
BA
Millar
Centre for Faculty Development, Faculty of Medicine, Li Ka Shing Knowledge Institute, University of Toronto at St. Michael's Hospital, 4th Floor, 209 Victoria Street, Toronto, ON, M5B 1W8, Canada.|Department of Physical Therapy, Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
Brenda
B
Mori
D004501:Education, Medical / Q000458:organization & administration / Q000592:standards; D005180:Faculty, Medical / Q000458:organization & administration / Q000592:standards; D005260:Female; D017144:Focus Groups; D006801:Humans; D007407:Interviews as Topic; D008297:Male; D008962:Models, Theoretical; D012189:Retrospective Studies; D013192:Staff Development / Q000379:methods / Q000458:organization & administration
Faculty development; Program evaluation; Realist evaluation; Theory-based evaluation