title: Inter-rater reliability of assessed prenatal maternal occupational exposures to solvents, polycyclic aromatic hydrocarbons, and heavy metals.
abstract: Because direct measurements of past occupational exposures are rarely available in population-based case-control studies, exposure assessment of job histories by multiple expert raters is frequently used; however, the subjective nature of this method makes measuring reliability an important quality control step. We evaluated inter-rater reliability of 7729 retrospective jobs reported in the National Birth Defects Prevention Study. Jobs were classified as exposed, unexposed, or exposure unknown by two independent industrial hygienists; exposed jobs were further evaluated for intensity, frequency, and routes. Exposure prevalence ranged from 0.1-9.8%. Inter-rater reliability for exposure (yes/no), assessed by kappa coefficients, was fair to good for cadmium (κ = 0.46), chlorinated solvents (κ = 0.59), cobalt (κ = 0.54), glycol ethers (κ = 0.50), nickel compounds (κ = 0.65), oil mists (κ = 0.63), and Stoddard Solvent (κ = 0.55); PAHs (κ = 0.24) and elemental nickel (κ = 0.37) had poor agreement. After a consensus conference resolved disagreements, an additional 4962 jobs were evaluated. Inter-rater reliability improved or stayed the same for cadmium (κ = 0.51), chlorinated solvents (κ = 0.81), oil mists (κ = 0.63), PAHs (κ = 0.52), and Stoddard solvent (κ = 0.92) in the second job set. Inter-rater reliability varied by exposure agent and prevalence, demonstrating the importance of measuring reliability in studies using a multiple expert rater method of exposure assessment.
authors.affiliation: Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Cincinnati, Ohio 45226, USA. CRocheleau@cdc.gov
authors.forename: Carissa M
authors.initials: CM
authors.lastname: Rocheleau
authors.forename: Christina C
authors.initials: CC
authors.lastname: Lawson
authors.forename: Martha A
authors.initials: MA
authors.lastname: Waters
authors.forename: Misty J
authors.initials: MJ
authors.lastname: Hein
authors.forename: Patricia A
authors.initials: PA
authors.lastname: Stewart
authors.forename: Adolfo
authors.initials: A
authors.lastname: Correa
authors.forename: Diana
authors.initials: D
authors.lastname: Echeverria
authors.forename: Jennita
authors.initials: J
authors.lastname: Reefhuis
mesh_terms: D002104:Cadmium; D016022:Case-Control Studies; D003035:Cobalt; D004651:Employment / Q000145:classification; D005260:Female; D006801:Humans; D006838:Hydrocarbons; D007587:Job Description; D018811:Maternal Exposure / Q000706:statistics & numerical data*; D019216:Metals, Heavy; D009532:Nickel; D015588:Observer Variation; D016273:Occupational Exposure / Q000706:statistics & numerical data*; D011084:Polycyclic Aromatic Hydrocarbons; D011247:Pregnancy; D015203:Reproducibility of Results; D012189:Retrospective Studies; D018570:Risk Assessment; D012997:Solvents
chemical_list: D006838:Hydrocarbons; D019216:Metals, Heavy; D011084:Polycyclic Aromatic Hydrocarbons; D012997:Solvents; D002104:Cadmium; D003035:Cobalt; D009532:Nickel; C012559:Stoddard solvent