The development, content validity and inter-rater reliability of the SMART-Goal Evaluation Method: A standardised method for evaluating clinical goals.
BACKGROUND
Goal setting is a complex skill. The use of formal goal writing procedures (including the use of the SMART goal model) has been advocated. However, a standardised method of writing and evaluating SMART goals is currently lacking. This study comprised of two phases. The aims of phase one was to (i) develop the SMART Goal Evaluation Method (SMART-GEM) based on a SMART goal model; and (ii) investigate the content validity of the SMART-GEM. The aim of phase two of the study was to test the inter-rater reliability of the SMART-GEM.


METHODS
Development of the SMART- GEM involved defining and constructing evaluation criteria suitable for auditing goal statements. A content validity assessment was conducted using an expert panel of Occupational Therapists (n = 10). Inter-rater reliability of the SMART-GEM was examined using a purposive sample of multiple raters (n = 24).


RESULTS
The SMART- GEM was rated as having good content validity (individual items CVI ranged from 0.90 to 1.00; total SMART- GEM CVI = 0.99, ρ = 0.05). Agreement between raters on individual items ranged from poor (κ = 0.254) to excellent (κ = 0.965) and agreement of overall grades was fair to good (κ = 0.582). Inter-rater agreement on total scores was found to be very good (ICC = 0.895, 95% CI = 0.743 to 0.986, ρ = 0.001) with excellent internal consistency (α = 0.995).


CONCLUSIONS
The SMART-GEM demonstrated good construct validity and very good inter-rater reliability on total score and shows promise as a standardised method to writing and evaluating clinical goals.
Research and Evaluation Service, Justice Health and Forensic Mental Health Network, Malabar, New South Wales, Australia.|Occupational Therapy Department, Faculty of Health Sciences, La Trobe University, Melbourne, Victoria, Australia.
Julia
J
Bowman
School of Medicine, University of Western Sydney, Campbelltown, New South Wales, Australia.
Lise
L
Mogensen
School of Science and Health, University of Western Sydney, Penrith South, New South Wales, Australia.
Elisabeth
E
Marsland
Occupational Therapy Department, Faculty of Health Sciences, La Trobe University, Melbourne, Victoria, Australia.|Occupational Therapy Department, Alfred Health, Melbourne, Australia.|John Walsh Centre of Rehabilitation Research, Northern Clinical School, Sydney Medical School, University of Sydney, St Leonards, New South Wales, Australia.
Natasha
N
Lannin
D006801:Humans; D015588:Observer Variation; D009788:Occupational Therapy / Q000379:methods* / Q000592:standards*; D017063:Outcome Assessment, Health Care / Q000379:methods* / Q000592:standards*; D010347:Patient Care Planning / Q000592:standards*; D015203:Reproducibility of Results
goal; occupational therapy; outcome assessment (health care); rehabilitation