Reliability of a 25-item low-stakes multiple-choice assessment of bronchoscopic knowledge.
BACKGROUND
A need for improved patient safety, quality of care, and accountability has prompted the development of competency-based educational processes. Assessment tools related to bronchoscopy training, however, have not yet been developed or validated.


OBJECTIVE
To determine whether 25 multiple-choice questions (MCQs) extracted from the free, Web-based Essential Bronchoscopist (EB) learning guide qualify in their original form as a preliminary pool of questions for a low-stakes assessment of bronchoscopic knowledge.


METHODS
Twenty-five randomly selected MCQs from among the top 70 question-answer sets of the EB were administered to 40 self-declared novice bronchoscopists (n = 13), experienced bronchoscopists (n = 21), and expert bronchoscopists (n = 6). A difficulty index and a discrimination index (DI) were calculated for each item. Internal consistency reliability was calculated using item-total correlation and Cronbach alpha. Content validity was determined by five independent experts. Ideal test items based on a difficulty index and item-total correlation were administered to a different group of 24 bronchoscopists to prospectively reassess internal consistency reliability.


RESULTS
The mean (+/- SD) score for the 40 participants was 16.47 +/- 3.72 (median score, 17; score range, 7 to 22). The mean difficulty index was 0.65 +/- 0.22, and the mean DI was 0.52 +/- 0.28. Item total-correlations ranged from - 0.01 to + 0.71. Test content was unanimously validated. The Cronbach alpha was 0.69. There was no significant correlation between scores and the number of bronchoscopies performed or self-declared expertise. Eleven ideal test MCQs were identified. The internal consistency of these items remained satisfactory (Cronbach alpha = 0.75) when assessed prospectively in a different cohort.


CONCLUSIONS
Reliable and valid MCQs were identified to initiate a preliminary pool of questions for a low-stakes assessment of bronchoscopic knowledge.
Quadrelli
Silvia
S
Instituto de Investigaciones Médicas Alfredo Lanari, Buenos Aires, Argentina.
Davoudi
Mohsen
M
Department of Pulmonary and Critical Care Medicine, University of California Irvine, UCI Medical Center, Orange, CA.
Galíndez
Fernando
F
Hospital María Ferrer, Buenos Aires, Argentina.
Colt
Henri G
HG
Department of Pulmonary and Critical Care Medicine, University of California Irvine, UCI Medical Center, Orange, CA. Electronic address: hcolt@uci.edu.
D001999:Bronchoscopy / Q000379:methods*; D002983:Clinical Competence; D004503:Education, Medical, Graduate; D004521:Educational Measurement / Q000592:standards*; D005260:Female; D006801:Humans; D008297:Male; D015272:Pulmonary Medicine / Q000193:education; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires / Q000592:standards*; D014481:United States