title: A One-Day Dental Faculty Workshop in Writing Multiple-Choice Questions: An Impact Evaluation.
abstract: Long training workshops on the writing of exam questions have been shown to be effective; however, the effectiveness of short workshops needs to be demonstrated. The aim of this study was to evaluate the impact of a one-day, seven-hour faculty development workshop at the College of Dentistry, King Saud University, Saudi Arabia, on the quality of multiple-choice questions (MCQs). Kirkpatrick's four-level evaluation model was used. Participants' satisfaction (Kirkpatrick's Level 1) was evaluated with a post-workshop questionnaire. A quasi-experimental, randomized separate sample, pretest-posttest design was used to assess the learning effect (Kirkpatrick's Level 2). To evaluate transfer of learning to practice (Kirkpatrick's Level 3), MCQs created by ten faculty members as a result of the training were assessed. To assess Kirkpatrick's Level 4 regarding institutional change, interviews with three key leaders of the school were conducted, coded, and analyzed. A total of 72 course directors were invited to and attended some part of the workshop; all 52 who attended the entire workshop completed the satisfaction form; and 22 of the 36 participants in the experimental group completed the posttest. The results showed that all 52 participants were highly satisfied with the workshop, and significant positive changes were found in the faculty members' knowledge and the quality of their MCQs with effect sizes of 0.7 and 0.28, respectively. At the institutional level, the interviews demonstrated positive structural changes in the school's assessment system. Overall, this one-day item-writing faculty workshop resulted in positive changes at all four of Kirkpatrick's levels; these effects suggest that even a short training session can improve a dental school's assessment of its students.
authors.affiliation: Prof. AlFaris is Professor of Family Medicine and supervisor of King Saud University Chair for Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Naeem is Associate Professor and Head of Medical Education Department, Batterjee Medical College, Jeddah, Saudi Arabia; Prof. Irfan is Assistant Professor and Chair of Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Qureshi is Distinguished Professor of Family Medicine, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Saad is Assistant Professor, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Al Sadhan is Associate Professor and Head of Oral Medicine and Diagnostic Sciences, College of Dentistry, King Saud University, Riyadh, Saudi Arabia; Prof. Abdulghani is Associate Professor and Head of Assessment and Evaluation Centre, College of Medicine, King Saud University, Riyadh, Saudi Arabia; and Prof. Van der Vleuten is Professor of Education, Scientific Director of School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Department of Educational Development and Research, Maastricht University, Netherlands.
authors.forename: Eiad
authors.initials: E
authors.lastname: AlFaris
authors.affiliation: Prof. AlFaris is Professor of Family Medicine and supervisor of King Saud University Chair for Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Naeem is Associate Professor and Head of Medical Education Department, Batterjee Medical College, Jeddah, Saudi Arabia; Prof. Irfan is Assistant Professor and Chair of Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Qureshi is Distinguished Professor of Family Medicine, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Saad is Assistant Professor, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Al Sadhan is Associate Professor and Head of Oral Medicine and Diagnostic Sciences, College of Dentistry, King Saud University, Riyadh, Saudi Arabia; Prof. Abdulghani is Associate Professor and Head of Assessment and Evaluation Centre, College of Medicine, King Saud University, Riyadh, Saudi Arabia; and Prof. Van der Vleuten is Professor of Education, Scientific Director of School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Department of Educational Development and Research, Maastricht University, Netherlands.
authors.forename: Naghma
authors.initials: N
authors.lastname: Naeem
authors.affiliation: Prof. AlFaris is Professor of Family Medicine and supervisor of King Saud University Chair for Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Naeem is Associate Professor and Head of Medical Education Department, Batterjee Medical College, Jeddah, Saudi Arabia; Prof. Irfan is Assistant Professor and Chair of Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Qureshi is Distinguished Professor of Family Medicine, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Saad is Assistant Professor, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Al Sadhan is Associate Professor and Head of Oral Medicine and Diagnostic Sciences, College of Dentistry, King Saud University, Riyadh, Saudi Arabia; Prof. Abdulghani is Associate Professor and Head of Assessment and Evaluation Centre, College of Medicine, King Saud University, Riyadh, Saudi Arabia; and Prof. Van der Vleuten is Professor of Education, Scientific Director of School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Department of Educational Development and Research, Maastricht University, Netherlands. masoodfarhana1969@gmail.com.
authors.forename: Farhana
authors.initials: F
authors.lastname: Irfan
authors.affiliation: Prof. AlFaris is Professor of Family Medicine and supervisor of King Saud University Chair for Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Naeem is Associate Professor and Head of Medical Education Department, Batterjee Medical College, Jeddah, Saudi Arabia; Prof. Irfan is Assistant Professor and Chair of Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Qureshi is Distinguished Professor of Family Medicine, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Saad is Assistant Professor, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Al Sadhan is Associate Professor and Head of Oral Medicine and Diagnostic Sciences, College of Dentistry, King Saud University, Riyadh, Saudi Arabia; Prof. Abdulghani is Associate Professor and Head of Assessment and Evaluation Centre, College of Medicine, King Saud University, Riyadh, Saudi Arabia; and Prof. Van der Vleuten is Professor of Education, Scientific Director of School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Department of Educational Development and Research, Maastricht University, Netherlands.
authors.forename: Riaz
authors.initials: R
authors.lastname: Qureshi
authors.affiliation: Prof. AlFaris is Professor of Family Medicine and supervisor of King Saud University Chair for Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Naeem is Associate Professor and Head of Medical Education Department, Batterjee Medical College, Jeddah, Saudi Arabia; Prof. Irfan is Assistant Professor and Chair of Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Qureshi is Distinguished Professor of Family Medicine, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Saad is Assistant Professor, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Al Sadhan is Associate Professor and Head of Oral Medicine and Diagnostic Sciences, College of Dentistry, King Saud University, Riyadh, Saudi Arabia; Prof. Abdulghani is Associate Professor and Head of Assessment and Evaluation Centre, College of Medicine, King Saud University, Riyadh, Saudi Arabia; and Prof. Van der Vleuten is Professor of Education, Scientific Director of School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Department of Educational Development and Research, Maastricht University, Netherlands.
authors.forename: Hussain
authors.initials: H
authors.lastname: Saad
authors.affiliation: Prof. AlFaris is Professor of Family Medicine and supervisor of King Saud University Chair for Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Naeem is Associate Professor and Head of Medical Education Department, Batterjee Medical College, Jeddah, Saudi Arabia; Prof. Irfan is Assistant Professor and Chair of Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Qureshi is Distinguished Professor of Family Medicine, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Saad is Assistant Professor, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Al Sadhan is Associate Professor and Head of Oral Medicine and Diagnostic Sciences, College of Dentistry, King Saud University, Riyadh, Saudi Arabia; Prof. Abdulghani is Associate Professor and Head of Assessment and Evaluation Centre, College of Medicine, King Saud University, Riyadh, Saudi Arabia; and Prof. Van der Vleuten is Professor of Education, Scientific Director of School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Department of Educational Development and Research, Maastricht University, Netherlands.
authors.forename: Ra'ed
authors.initials: R
authors.lastname: Al Sadhan
authors.affiliation: Prof. AlFaris is Professor of Family Medicine and supervisor of King Saud University Chair for Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Naeem is Associate Professor and Head of Medical Education Department, Batterjee Medical College, Jeddah, Saudi Arabia; Prof. Irfan is Assistant Professor and Chair of Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Qureshi is Distinguished Professor of Family Medicine, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Saad is Assistant Professor, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Al Sadhan is Associate Professor and Head of Oral Medicine and Diagnostic Sciences, College of Dentistry, King Saud University, Riyadh, Saudi Arabia; Prof. Abdulghani is Associate Professor and Head of Assessment and Evaluation Centre, College of Medicine, King Saud University, Riyadh, Saudi Arabia; and Prof. Van der Vleuten is Professor of Education, Scientific Director of School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Department of Educational Development and Research, Maastricht University, Netherlands.
authors.forename: Hamza Mohammad
authors.initials: HM
authors.lastname: Abdulghani
authors.affiliation: Prof. AlFaris is Professor of Family Medicine and supervisor of King Saud University Chair for Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Naeem is Associate Professor and Head of Medical Education Department, Batterjee Medical College, Jeddah, Saudi Arabia; Prof. Irfan is Assistant Professor and Chair of Medical Education Research and Development, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Qureshi is Distinguished Professor of Family Medicine, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Saad is Assistant Professor, Department of Family and Community Medicine, College of Medicine, King Saud University, Riyadh, Saudi Arabia; Prof. Al Sadhan is Associate Professor and Head of Oral Medicine and Diagnostic Sciences, College of Dentistry, King Saud University, Riyadh, Saudi Arabia; Prof. Abdulghani is Associate Professor and Head of Assessment and Evaluation Centre, College of Medicine, King Saud University, Riyadh, Saudi Arabia; and Prof. Van der Vleuten is Professor of Education, Scientific Director of School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Department of Educational Development and Research, Maastricht University, Netherlands.
authors.forename: Cees
authors.initials: C
authors.lastname: Van der Vleuten
mesh_terms: D001291:Attitude of Health Personnel; D015331:Cohort Studies; D004521:Educational Measurement / Q000379:methods* / Q000592:standards; D005179:Faculty, Dental; D005246:Feedback; D005260:Female; D006801:Humans; D007858:Learning; D008297:Male; D010549:Personal Satisfaction; D036301:Qualitative Research; D011897:Random Allocation; D012529:Saudi Arabia; D013192:Staff Development; D013663:Teaching / Q000379:methods; D014956:Writing
keywords: Kirkpatrick’s framework; assessment; dental education; dental faculty; faculty development; multiple-choice questions