Exploring individual opinions of potential evaluators in a 360-degree assessment: four distinct viewpoints of a competent resident.
BACKGROUND
Despite the highly acclaimed psychometric features of a 360-degree assessment in the fields of economics, military, and education, there has been increased interest in developing 360-degree instruments to assess competencies in graduate medical education only in the past recent years. Most of the effort to date, however, has focused on developing instruments and testing their reliability and feasibility. Insufficient attention has gone into issues of construct validity and particularly understanding the underlying constructs on which the instruments are based as well as the phenomena that affect ratings.


OBJECTIVE
In preparation for developing a 360-degree assessment instrument, we explored variations in evaluators' opinion type of a competent resident and offer observation about evaluator's professional background and opinions.


METHODS
Evaluators from two residency programs ranked 36 opinion statements, using a relative-ranking model, based on their opinion of a competent resident. By-person factor analysis was used to structure opinion types.


RESULTS
Factor analysis of 156 responses identified four factors interpreted as four different opinion types of a competent resident: (a) altruistic, compassionate healer (n = 42 evaluators), (b) scientifically grounded clinician (n = 30), (c) holistic, humanistic clinician (n = 62), and (d) patient-focused, health manager (n = 31). Although 72% of nurses/respiratory therapist evaluators expressed type C, 28% expressed other types just as often. Only 14% of evaluator physicians expressed type D, and the remainders were evenly split among the other types.


CONCLUSIONS
Our evaluators in 360-degree system expressed four opinion types of a competent resident. The individual opinion and not professional background influences the characteristics an evaluator values in a competent resident. We propose that these values will have an impact on competency assessment and should be taken into account in a 360-degree assessment.
Thammasitboon
Satid
S
Department of Pediatrics, Robert C Byrd Health Sciences Center, Morgantown, West Virginia 26506-9214, USA. sthammasitboon@hsc.wvu.edu
Mariscalco
M Michele
MM
Yudkowsky
Rachel
R
Hetland
Monte D
MD
Noronha
Peter A
PA
Mrtek
Robert G
RG
D002983:Clinical Competence / Q000592:standards*; D004503:Education, Medical, Graduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D005163:Factor Analysis, Statistical; D006801:Humans; D007396:Internship and Residency; D010372:Pediatrics / Q000193:education*; D011594:Psychometrics