title: Reconceptualizing variable rater assessments as both an educational and clinical care problem.
abstract: The public is calling for the U.S. health care and medical education system to be accountable for ensuring high-quality, safe, effective, patient-centered care. As medical education shifts to a competency-based training paradigm, clinician educators' assessment of and feedback to trainees about their developing clinical skills becomes paramount. However, there is substantial variability in the accuracy, reliability, and validity of the assessments faculty make when they directly observe trainees with patients. These difficulties have been treated primarily as a rater cognition problem focusing on the inability of the assessor to make reliable and valid assessments of the trainee.The authors' purpose is to reconceptualize the rater cognition problem as both an educational and clinical care problem. The variable quality of faculty assessments is not just a psychometric predicament but also an issue that has implications for decisions regarding trainee supervision and the delivery of quality patient care. The authors suggest that the frame of reference for rating performance during workplace-based assessments be the ability to provide safe, effective, patient-centered care. The authors developed the Accountable Assessment for Quality Care and Supervision equation to remind faculty that supervision is a dynamic, complex process essential for patients to receive high-quality care. This fundamental shift in how assessment is conceptualized requires new models of faculty development and emphasizes the essential and irreplaceable importance of the clinician educator in trainee assessment.
authors.affiliation: Dr. Kogan is associate professor, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania. Ms. Conforti is research associate for academic programs, American Board of Internal Medicine, Philadelphia, Pennsylvania. Dr. Iobst is vice president of academic affairs and clinical affairs and vice dean, Commonwealth Medical College, Scranton, Pennsylvania. During the preparation of this article, he was vice president of academic affairs, American Board of Internal Medicine, Philadelphia, Pennsylvania. Dr. Holmboe is senior vice president of milestones development and evaluation, Accreditation Council for Graduate Medical Education, Chicago, Illinois. During the preparation of this article, he was chief medical officer, American Board of Internal Medicine, Philadelphia, Pennsylvania.
authors.forename: Jennifer R
authors.initials: JR
authors.lastname: Kogan
authors.forename: Lisa N
authors.initials: LN
authors.lastname: Conforti
authors.forename: William F
authors.initials: WF
authors.lastname: Iobst
authors.forename: Eric S
authors.initials: ES
authors.lastname: Holmboe
mesh_terms: D002983:Clinical Competence; D015510:Clinical Medicine / Q000458:organization & administration*; D003695:Delivery of Health Care / Q000458:organization & administration; D004501:Education, Medical / Q000458:organization & administration*; D004521:Educational Measurement; D005180:Faculty, Medical / Q000458:organization & administration; D005260:Female; D006801:Humans; D008297:Male; D015588:Observer Variation; D018802:Patient-Centered Care / Q000458:organization & administration; D011785:Quality Assurance, Health Care; D014481:United States