Developing a valid evaluation for interpersonal and communication skills.
OBJECTIVE
The Accreditation Council for Graduate Medical Education requires that residency programs evaluate the acquisition of six general competencies, including Interpersonal and Communication Skills (ICS). To develop a 360-degree method for accomplishing this, a semantic-differentiation matrix for various communication traits for nurses to evaluate physician ICS was developed. The authors sought to determine whether this evaluation method could discriminate between more experienced medical communicators (faculty) and less experienced medical communicators (residents).


METHODS
A 98-item questionnaire measured several communication dimensions by using an eight-element semantic-differentiation scale. In addition, global assessment ratings assessed nursing perceptions of physician ICS skills. This process was repeated for various clinical scenarios.


RESULTS
There were 26 nurse evaluators, 19 emergency medicine (EM) residents (EM2 and EM3), and 30 EM faculty. Each physician received five independent evaluations (total, 245 evaluations). All EM residents (EM2 and EM3) were compared with the EM faculty. All eight items on the semantic-differentiation scale were compared. Likewise, the global assessment scores were compared. In every category, the faculty scored higher (Mann-Whitney U: p < 0.001).


CONCLUSIONS
An evaluation process integrating a semantic-differentiation matrix was applied to various clinical scenarios (as well as global assessment items) and demonstrated discriminatory results. Faculty physicians scored significantly higher than resident physicians. The ability to provide discriminatory results is a requisite in the development of a valid evaluation process. The described semantic-differentiation matrix and global assessment questions may be valid measurements of ICS.
Reisdorff
Earl J
EJ
Michigan State University Emergency Medicine Residency, Lansing, MI, USA. earl.reisdorff@irmc.org
Hughes
Mary Jo
MJ
Castaneda
Carlos
C
Carlson
Dale J
DJ
Donohue
William A
WA
Fediuk
Thomas A
TA
Hughes
William P
WP
D003142:Communication; D004635:Emergency Medicine / Q000193:education*; D005069:Evaluation Studies as Topic; D006801:Humans; D007396:Internship and Residency; D019555:Physician-Nurse Relations; D012660:Semantics; D011795:Surveys and Questionnaires