title: Developing a scoring rubric for resident research presentations: a pilot study.
abstract: BACKGROUND
  A requirement for all Accreditation Council for Graduate Medical Education (ACGME) approved residencies is the provision of "an opportunity for residents to participate in research." To comply with this requirement, most training programs encourage their residents to conduct research and to report their results. Few guidelines exist, however, for assessing the efficacy of the presentations. The goal of this pilot study was to develop a valid, one-page scoring rubric to be used during oral resident research presentations. Such a scoring rubric will facilitate acceptable agreement among faculty raters.
  
  
  METHODS
  Content validity was addressed by adhering to the Standards for Educational and Psychological Testing. A one-page, five-domain, behaviorally worded scoring rubric was developed. Inter-rater reliability was derived and three ACGME General Competencies were also addressed within the rubric.
  
  
  RESULTS
  The initial scoring rubric was tested with 11 resident oral presentations. The inter-rater reliability was 0.56 using Cronbach's alpha. The rubric was modified and the scale restricted to a 3-point scale. It was then tested with 17 additional presentations, which were independently rated by two general surgery faculty members. Cronbach's Alpha increased to 0.61.
  
  
  CONCLUSIONS
  An objective method to evaluate a resident's oral research presentation has been successfully piloted. This content valid rubric possesses good inter-rater reliability according to established guidelines. Clearly defined behaviors have been outlined within the rubric. Program directors will have psychometrically sound evidence for the ACGME. Future research will address generalizability and concurrent validity using other types of resident assessment data.
authors: Department of Surgery, Henry Ford Hospital, Detroit, Michigan 48202, USA. JMUSIAL1@hfhs.org
authors: Joseph L
authors: JL
authors: Musial
authors: Ilan S
authors: IS
authors: Rubinfeld
authors: Alton O
authors: AO
authors: Parker
authors: Craig A
authors: CA
authors: Reickert
authors: Sarah A
authors: SA
authors: Adams
authors: Sishir
authors: S
authors: Rao
authors: Alexander D
authors: AD
authors: Shepard
mesh_terms: D000068:Accreditation / Q000379:methods / Q000592:standards*; D035843:Biomedical Research / Q000592:standards*; D004503:Education, Medical, Graduate / Q000592:standards*; D004521:Educational Measurement / Q000379:methods / Q000592:standards*; D017408:Guidelines as Topic; D007396:Internship and Residency / Q000592:standards*; D010865:Pilot Projects; D015203:Reproducibility of Results