Assessing observer agreement when describing and classifying functioning with the International Classification of Functioning, Disability and Health.
OBJECTIVE
The International Classification of Functioning, Disability and Health (ICF) is used increasingly to describe and classify functioning in medicine without being a psychometrically sound measure. All categories of the ICF are quantified using the same generic 0-4 scale. The objective of this study was to assess observer agreement when describing and classifying functioning with the ICF.


METHODS
A second-level category of the ICF, d430 lifting and carrying objects, was used as an example. To the qualifiers of this category, clinically meaningful definitions were assigned. Data were collected in a cross-sectional survey with repeated measurement. We report raw, specific and chance-corrected measures or agreement, a graphical method and the results of log-linear models for ordinal agreement.


METHODS
A convenience sample of patients requiring physical therapy in an acute hospital.


RESULTS
Twenty-five patients were assessed twice by 2 observers. Raw agreement was 0.52. Kappa was 0.36, indicating fair agreement. Different hierarchical log-linear models indicated that the strength of agreement was not homogeneous over all categories.


CONCLUSIONS
Observer agreement has to be evaluated when describing and classifying functioning using the ICF Qualifiers'scale. When assessing inter-observer reliability, the first step is to calculate a summary statistic. Modelling agreement yields valuable insight into the structure of a contingency table, which can lead to further improvement of the scale.
Grill
Eva
E
ICF Research Branch, WHO FIC Collaborating Center (DIMDI), Institute for Health and Rehabilitation Sciences, University of Munich, Germany.
Mansmann
Ulrich
U
Cieza
Alarcos
A
Stucki
Gerold
G
D003430:Cross-Sectional Studies; D004185:Disability Evaluation; D006801:Humans; D038801:International Classification of Diseases; D015588:Observer Variation; D017063:Outcome Assessment, Health Care; D026741:Physical Therapy Modalities; D012046:Rehabilitation / Q000145:classification*; D015203:Reproducibility of Results