Simplifying the 360-degree peer evaluation in a physical medicine and rehabilitation residency program.
OBJECTIVE
A previous study determined that multiple questions on a 360-degree evaluation instrument were highly correlated, suggesting possible redundancy in what was being measured, and that some questions may be eliminated. The current study uses factor analysis and examines a larger data set to further explore this question.


METHODS
To evaluate the structure of the questionnaire, a factor analysis was performed on 3 yrs of data from a 19-item resident 360-degree evaluation. The number of factors was determined by examining the scree plot of eigenvalues for each item in the instrument, with a cutoff where the slope changes from rapid to slow decline. A reliability analysis was performed with the indicated number of factors, with deletion of each variable to evaluate its influence on overall reliability (Cronbach alpha).


RESULTS
There were 299 evaluations with complete responses to all 19 questions. The scree plot supported a single factor model. The reliability of the full, single factor survey was excellent (Cronbach α = 0.99). The three items with the highest loading on the factor were retained, which related to humanistic, moral/ethical, and professional responsibility behaviors. The reliability for these final three items remained excellent (Cronbach α = 0.96).


CONCLUSIONS
The factor analysis suggests that one overall opinion of the evaluated resident was informing the responses of the evaluator. Shortening the instrument to the three items responsible for the greatest influence on the survey does not result in a large decrease in reliability as measured by Cronbach alpha. It is possible that time limitations prevent residents from putting thought into the evaluation of their peers, which results in unidimensional responses. Shortening the instrument may improve evaluations and should be studied in the future.
MetroHealth Rehabilitation Institute of Ohio, Cleveland, OH 44109, USA.
John C
JC
Sorg
Richard D
RD
Wilson
Adam T
AT
Perzynski
Daniel
D
Tran
Mary M
MM
Vargo
D005163:Factor Analysis, Statistical; D005260:Female; D006801:Humans; D007396:Internship and Residency / Q000458:organization & administration*; D008297:Male; D009820:Ohio; D010380:Peer Review / Q000379:methods*; D010810:Physical and Rehabilitation Medicine / Q000193:education*; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires