Development of a content-valid standardized orthopedic assessment tool (SOAT).
BACKGROUND
Content validation of an instrument that measures student performance in OSCE-type practical examinations is a critical step in a tool's overall validity and reliability [Hopkins (1998), Educational and Psychological Measurement and Evaluation (8th ed.). Toronto: Allyn & Bacon].


OBJECTIVE
The purpose of the paper is to outline the process employed to create a content-valid Standardized Orthopedic Assessment Tool (SOAT). Orthopedic assessment skills are employed by athletic therapists, physiotherapists and physicians. All follow very similar diagnostic algorithms and that system was used to develop the original SOAT [Cyriax (1982). Textbook of Orthopaedic Medicine, (vol. 1) Bailliere Tindall].


METHODS
To validate the tool, the study followed procedures outlined by Violato, Salami, and Muiznieks (2002), Journal of Manipulative Physiological Therapeutics, 25, 111-115, and Butterwick, Paskevich, Vallevand and Lafave (2006), Journal of Allied Health: a modified Ebel procedure. An expert group of athletic therapists from across Canada were chosen to create the content valid tool. Representation from all accredited athletic therapy programs in Canada was sought. Experts participated in three stages of content validation: Stage one consisted of individuals grading tasks on difficulty (hard, medium, easy) and importance (essential, important, not important) for 8 body regions (cervical spine, lumbar spine, shoulder, elbow, wrist/hand/thumb, hip, knee and lower leg/foot/ankle) and three diagnoses for each body region (24 total). If 80% consensus was not achieved in the first stage, then in stage two, face to face discussion is meant to clarify positions and achieved consensus, if possible.


RESULTS
The history component, the observation component, scanning exams, clearing joints above and below the lesion site and active range of motion, passive range of motion and isometric resisted testing all yielded 80% consensus in the first two stages of development. A third stage was added to this process because a new model of measurement was conceptualized during the second stage due to a lack of consensus on two orthopedic assessment categories: special testing and palpation. The new model consisted of a ;;clinical reasoning" metric that tied each section of an orthopedic assessment (history; observation; scanning and clearing; examination, special testing; palpation; conclusion) together and would permit the expert rater to evaluate the student performance based on the student's rationale for tests chosen rather than the traditionally rigid checklists. At least 80% consensus was achieved with the new ;;clinical reasoning" metric and the originally contentious special testing and palpation categories.


CONCLUSIONS
The new SOAT that underwent content validation may be a hybrid between the traditional OSCE-type checklists and global scales that provide a more realistic and thus more valid depiction of orthopedic assessment clinical competence. Now that content validation has been established, the next steps include inter-rater reliability testing.
Lafave
Mark
M
Department of Physical Education and Recreation Studies, Mount Royal College, Calgary, AB, Canada. mlafave@mtroyal.ca
Katz
Larry
L
Butterwick
Dale
D
D000465:Algorithms; D002983:Clinical Competence; D004185:Disability Evaluation; D006801:Humans; D009985:Orthopedics / Q000592:standards*; D010808:Physical Examination / Q000592:standards*; D015203:Reproducibility of Results