Assessment of 'borderline' clinical competence using generalisability and decision studies in dental skills training.
BACKGROUND
Generalisability coefficients are widely used as a measure of reliability in educational sciences. However, coefficients do not inform about how much individual and/or educational variance is needed for borderline competence. Probing depth measurements were used as an example to demonstrate the use of generalisability and decision studies in educational decision-making.


METHODS
All six probing depths of seven elements of the first quadrant were assessed in a random crossed four-way 'elements' (7), 'probing depth' (6) 'observers' (7-8) and 'repetitions' design for both students (8) and faculty staff (7). Borderline competence was assessed in four decision study designs.


RESULTS
In decision studies the generalisability coefficient varied from 0.31 to 0.88 for the first single and fourth repeated study design, respectively. The standard error of the measurement varied from 1.73 to 0.64 mm with a corresponding 95% confidence interval varying from 3 to 1 mm.


CONCLUSIONS
Repeated measurement is not the most efficient way to improve educational decision-making. What is new to this approach is that we used the 95% confidence interval of the standard error of measurement as a measure for 'borderline competence' expressed in the unity of the measurement instrument. Correctly performing the procedure of probing depth assessment is of clinical and educational importance in deciding whether further treatment is necessary and whether students fail or pass their exams.
Kropmans
T J B
TJ
Department of Educational Development and Quality Assurance, Faculty of Medical Sciences, Institute for Medical Education, University of Groningen, Groningen, The Netherlands. t.j.b.kropmans@med.rug.nl
Cosić
K
K
D000704:Analysis of Variance; D002983:Clinical Competence / Q000706:statistics & numerical data*; D016001:Confidence Intervals; D018592:Cross-Over Studies; D003657:Decision Making; D003661:Decision Support Techniques; D004497:Education, Dental / Q000706:statistics & numerical data*; D005179:Faculty, Dental; D006801:Humans; D008348:Manikins; D010514:Periodontal Pocket / Q000175:diagnosis; D010517:Periodontics / Q000193:education; D015203:Reproducibility of Results; D016037:Single-Blind Method; D013335:Students, Dental