title: Validity and Feasibility of the Minicard Direct Observation Tool in 1 Training Program.
abstract: BACKGROUND
  Availability of reliable, valid, and feasible workplace-based assessment (WBA) tools is important to allow faculty to make important and complex judgments about resident competence. The Minicard is a WBA direct observation tool designed to provide formative feedback while supporting critical competency decisions.
  
  
  OBJECTIVE
  The purpose of this study was to collect validity and feasibility evidence for use of the Minicard for formative assessment of internal medicine residents.
  
  
  METHODS
  We conducted a retrospective cohort analysis of Minicard observations from 2005-2011 in 1 institution to obtain validity evidence, including content (settings, observation rates, independent raters); response process (rating distributions across the scale and ratings by month in the program); consequences (qualitative assessment of action plans); and feasibility (time to collect observations).
  
  
  RESULTS
  Eighty faculty observers recorded 3715 observations of 73 residents in the inpatient ward (43%), clinic (39%), intensive care (15%), and emergency department (3%) settings. Internal medicine residents averaged 28 (SD=8.4) observations per year from 9 (SD=4.1) independent observers. Minicards had an average of 5 (SD=5.1) discrete recorded observations per card. Rating distributions covered the entire rating scale, and increased significantly over the time in training. Half of the observations included action plans with action-oriented feedback, 11% had observational feedback, 9% had minimal feedback, and 30% had no recorded plan. Observations averaged 15.6 (SD=9.5) minutes.
  
  
  CONCLUSIONS
  Validity evidence for the Minicard direct observation tool demonstrates its ability to facilitate identification of "struggling" residents and provide feedback, supporting its use for the formative assessment of internal medicine residents.
authors.forename: Anthony A
authors.initials: AA
authors.lastname: Donato
authors.forename: Yoon Soo
authors.initials: YS
authors.lastname: Park
authors.forename: David L
authors.initials: DL
authors.lastname: George
authors.forename: Alan
authors.initials: A
authors.lastname: Schwartz
authors.forename: Rachel
authors.initials: R
authors.lastname: Yudkowsky
mesh_terms: D002983:Clinical Competence; D004521:Educational Measurement / Q000379:methods*; D005180:Faculty, Medical; D005246:Feedback; D006801:Humans; D007388:Internal Medicine / Q000193:education*; D007396:Internship and Residency / Q000379:methods*; D015588:Observer Variation; D015203:Reproducibility of Results; D012189:Retrospective Studies