Seeing the forest despite the trees. The benefit of exploratory data analysis to program evaluation research.
In the present article, it is argued that there is a benefit to applying techniques of exploratory data analysis (EDA) to program evaluation. To exemplify this, an evaluation of a rehabilitation program for people with rheumatoid arthritis is presented. The perceived health status of patients receiving intensive rehabilitation services from a major rehabilitation institute was compared with that of patients receiving customary office-based care over an 18-month period. The data were analyzed in a conventional way (analysis of variance) and then by way of EDA techniques (graphic display of medians and boxplots). The conventional analysis suggested that all patients improved over time and that intensive rehabilitation services provided no particular benefit or harm. The exploratory analysis showed that the distribution of the outcome variable was patently nonnormal, thus casting doubt on the validity of the conventional analysis. The EDA further showed that the rehabilitation group lagged behind the comparison group for a year, with a precipitious improvement at the 18-month period. This suggests that a selection factor was operating (i.e., those in the rehabilitation group could have been sicker) or that the patients in the rehabilitation group were made more aware of their condition by the intensive health services they received. The EDA provided an important insight.
Sinacore
J M
JM
Northwestern University Medical School, Chicago, IL 60611.
Chang
R W
RW
Falconer
J
J
D000704:Analysis of Variance; D001172:Arthritis, Rheumatoid / Q000534:rehabilitation*; D002641:Chicago; D003627:Data Interpretation, Statistical; D005260:Female; D006302:Health Services Research / Q000379:methods* / Q000706:statistics & numerical data; D006801:Humans; D008875:Middle Aged; D017063:Outcome Assessment, Health Care / Q000379:methods*; D015397:Program Evaluation / Q000379:methods*; D012047:Rehabilitation Centers / Q000592:standards*; D015983:Selection Bias; D012647:Self-Assessment