Modelling candidate effectiveness indicators for mental health services.
OBJECTIVE
Although Australia and the UK have both made efforts to systematize outcome measurement in mental health services, surprisingly little attention has been paid to how best to analyse routine outcome data in order to determine how services are performing.


METHODS
Outcome data collected in acute inpatient and ambulatory mental health services across Australia during the 2006-2007 financial year were used. three approaches to measuring effectiveness were explored: effect size (ES); the reliable change index (RCI); and standard error of measurement (SEM).


RESULTS
The most conservative results were produced by the RCI and the least conservative by the medium ES statistic and the SEM. By way of example, only 38.0% of inpatient admission-discharge periods of care showed significant improvement for adults when the RCI was used, whereas 67.4% and 72.9% did so when the medium ES and the SEM statistics were used, respectively.


CONCLUSIONS
In any routine outcome measurement exercise, the degree of effectiveness demonstrated by services will depend on the specific statistical indicator used to judge effectiveness. Routine outcome measurement has the potential to answer a range of crucial performance-related questions, but only if the same metric is used. Discussion of the appropriate statistical approach to take to facilitate cross-service, cross-area and even cross-national comparisons warrants attention.
School of Population Health, University of Queensland, Sumner Park BC, Qld, Australia. p.burgess@uq.edu.au
Philip
P
Burgess
Jane
J
Pirkis
Tim
T
Coombs
D000208:Acute Disease; D000293:Adolescent; D000328:Adult; D000368:Aged; D001315:Australia / Q000453:epidemiology; D002648:Child; D006760:Hospitalization / Q000706:statistics & numerical data; D006801:Humans; D001523:Mental Disorders / Q000453:epidemiology* / Q000534:rehabilitation / Q000628:therapy*; D008605:Mental Health Services / Q000706:statistics & numerical data*; D008875:Middle Aged; D017063:Outcome Assessment, Health Care / Q000706:statistics & numerical data*; D055815:Young Adult