title: Reliability of a viva assessment of clinical reasoning in an Australian pre-professional osteopathy program assessed using generalizability theory.
abstract: Clinical reasoning is situation-dependent and case-specific; therefore, assessments incorporating different patient presentations are warranted. The present study aimed to determine the reliability of a multi-station case-based viva assessment of clinical reasoning in an Australian pre-registration osteopathy program using generalizability theory. Students (from years 4 and 5) and examiners were recruited from the osteopathy program at Southern Cross University, Lismore, Australia. The study took place on a single day in the student teaching clinic. Examiners were trained before the examination. Students were allocated to 1 of 3 rounds consisting of 5 10-minute stations in an objective structured clinical examination-style. Generalizability analysis was used to explore the reliability of the examination. Fifteen students and 5 faculty members participated in the study. The examination produced a generalizability coefficient of 0.53, with 18 stations required to achieve a generalizability coefficient of 0.80. The reliability estimations were acceptable and the psychometric findings related to the marking rubric and overall scores were acceptable; however, further work is required in examiner training and ensuring consistent case difficulty to improve the reliability of the examination.
authors.affiliation: School of Health and Human Sciences, Southern Cross University, Lismore, Australia.|College of Health and Biomedicine, Victoria University, Melbourne, Australia.|Institute of Sport, Exercise and Active Living, Victoria University, Melbourne, Australia.
authors.forename: Brett
authors.identifier: 0000-0002-8623-4558
authors.initials: B
authors.lastname: Vaughan
authors.affiliation: College of Health and Biomedicine, Victoria University, Melbourne, Australia.
authors.forename: Paul
authors.identifier: 0000-0003-0153-4832
authors.initials: P
authors.lastname: Orrock
authors.affiliation: School of Health and Human Sciences, Southern Cross University, Lismore, Australia.|College of Health and Biomedicine, Victoria University, Melbourne, Australia.
authors.forename: Sandra
authors.identifier: 0000-0002-0300-4088
authors.initials: S
authors.lastname: Grace
mesh_terms: D001315:Australia; D002983:Clinical Competence; D003430:Cross-Sectional Studies; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D010021:Osteopathic Medicine / Q000193:education*; D011594:Psychometrics; D015203:Reproducibility of Results; D013337:Students, Medical / Q000523:psychology*; D013850:Thinking
keywords: Australia; Osteopathic medicine; Physical examination; Psychometrics; Reproducibility of results