Assessing inter-rater agreement of environmental audit data in a matched case-control study on bicycling injuries.
BACKGROUND
Environmental audit tools must be reliable in order to accurately estimate the association between built environmental characteristics and bicycling injury risk.


OBJECTIVE
To examine the inter-rater agreement of a built environment audit tool within a case-control study on the environmental determinants of bicycling injuries.


METHODS
Auditor pairs visited locations where bicycling injuries occurred and independently recorded location characteristics using the Systematic Pedestrian and Cyclist Environmental Scan (SPACES). Two case groups were defined: (1) where a bicyclist was struck by a motor-vehicle (MV) and (2) where the bicyclist's injuries required hospitalisation. The two corresponding control groups were (1) where non-MV bicycle-related injuries occurred and (2) where minor bicycle-related injuries occurred. Inter-rater reliability of each item on the tool was assessed using observed agreement and κ with 95% CI.


RESULTS
Ninety-seven locations were audited. Inter-observer agreement was generally high (≥95%); most items had a 1-2% difference in responses. Items with ≥5% differences between raters included path condition, slope and obstructions. For land use, path and roadway characteristics, κ ranged from 0.3 for presence of offices and cleanliness to 0.9 for schools and number of lanes; overall, 78% of items had at least substantial agreement (κ≥0.61). For bicyclists struck by a MV the proportion of items with substantial agreement was 60%, compared with 73% for non-MV related injuries. For hospitalisations and minor bicycle-related injuries, 76% of items had substantial agreement.


CONCLUSIONS
Agreement was substantial for most, but not all SPACES items. The SPACES provides reliable quantitative descriptions of built environmental characteristics at bicycling injury locations.
Departments of Paediatrics and Community Health Sciences, University of Calgary, , Calgary, Alberta, Canada.
Nicole T R
NT
Romanow
Amy B
AB
Couperthwaite
Gavin R
GR
McCormack
Alberto
A
Nettel-Aguirre
Brian H
BH
Rowe
Brent E
BE
Hagel
D000063:Accidents, Traffic / Q000706:statistics & numerical data*; D000293:Adolescent; D000328:Adult; D001642:Bicycling / Q000293:injuries*; D002170:Canada; D016022:Case-Control Studies; D002648:Child; D004779:Environment Design / Q000706:statistics & numerical data*; D005260:Female; D006801:Humans; D008297:Male; D015588:Observer Variation; D011634:Public Health / Q000379:methods; D055815:Young Adult