title: Inter-observer reliability testing of pig welfare outcome measures proposed for inclusion within farm assurance schemes.
abstract: Consistency of assessment is essential to the farm assurance process. This study evaluated the inter-observer reliability of 31 farm assurance assessors, six veterinarians and four researchers for five pig welfare outcome measures proposed for inclusion into the UK pig farm assurance schemes. These were (1) tail lesions, (2) body lesions, (3) lameness, (4) pigs requiring hospitalisation and (5) oral behaviour. The following inter-observer reliability testing methods against a gold standard Trainer were used: a comparison of farm prevalence and the numbers of affected pigs in each pen identified by observers, Cohen's kappa (κ), Kendall's W, proportional agreement, sensitivity, and specificity. All measures achieved potentially high levels of inter-observer reliability and it was concluded that none should be excluded from farm assurance at this stage. However, across all the measures, 45% of observers did not record an overall farm prevalence 'close' to that of the gold standard Trainer. With the level of training and testing that took place in this study there would be a danger of significant bias occurring in a national assessment scheme. The data collected enabled some comparison of the methods used to assess inter-observer reliability. It is suggested that when the aim is to achieve agreement between observers on the overall farm prevalence the inter-observer reliability testing should focus on the closeness of the overall farm prevalence recorded by observers, but that other types of analysis may be helpful during training.
authors.affiliation: University of Bristol Veterinary School, Langford, Somerset BS40 5DU, UK. Electronic address: Siobhan.mullan@bris.ac.uk.
authors.forename: Siobhan
authors.initials: S
authors.lastname: Mullan
authors.affiliation: Newcastle University, School of Agriculture Food and Rural Development, Newcastle upon Tyne NE1 7RU, UK.
authors.forename: Sandra A
authors.initials: SA
authors.lastname: Edwards
authors.affiliation: University of Bristol Veterinary School, Langford, Somerset BS40 5DU, UK.
authors.forename: Andrew
authors.initials: A
authors.lastname: Butterworth
authors.affiliation: University of Bristol Veterinary School, Langford, Somerset BS40 5DU, UK.
authors.forename: Helen R
authors.initials: HR
authors.lastname: Whay
authors.affiliation: University of Bristol Veterinary School, Langford, Somerset BS40 5DU, UK.
authors.forename: David C J
authors.initials: DCJ
authors.lastname: Main
mesh_terms: D000822:Animal Husbandry / Q000592:standards*; D000827:Animal Welfare / Q000592:standards* / Q000706:statistics & numerical data; D000818:Animals; D015588:Observer Variation; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D013552:Swine; D013553:Swine Diseases / Q000175:diagnosis*; D006113:United Kingdom