title: Comparison of ordinal and nominal classification trees to predict ordinal expert-based occupational exposure estimates in a case-control study.
abstract: OBJECTIVE
  To evaluate occupational exposures in case-control studies, exposure assessors typically review each job individually to assign exposure estimates. This process lacks transparency and does not provide a mechanism for recreating the decision rules in other studies. In our previous work, nominal (unordered categorical) classification trees (CTs) generally successfully predicted expert-assessed ordinal exposure estimates (i.e. none, low, medium, high) derived from occupational questionnaire responses, but room for improvement remained. Our objective was to determine if using recently developed ordinal CTs would improve the performance of nominal trees in predicting ordinal occupational diesel exhaust exposure estimates in a case-control study.
  
  
  METHODS
  We used one nominal and four ordinal CT methods to predict expert-assessed probability, intensity, and frequency estimates of occupational diesel exhaust exposure (each categorized as none, low, medium, or high) derived from questionnaire responses for the 14983 jobs in the New England Bladder Cancer Study. To replicate the common use of a single tree, we applied each method to a single sample of 70% of the jobs, using 15% to test and 15% to validate each method. To characterize variability in performance, we conducted a resampling analysis that repeated the sample draws 100 times. We evaluated agreement between the tree predictions and expert estimates using Somers' d, which measures differences in terms of ordinal association between predicted and observed scores and can be interpreted similarly to a correlation coefficient.
  
  
  RESULTS
  From the resampling analysis, compared with the nominal tree, an ordinal CT method that used a quadratic misclassification function and controlled tree size based on total misclassification cost had a slightly better predictive performance that was statistically significant for the frequency metric (Somers' d: nominal tree = 0.61; ordinal tree = 0.63) and similar performance for the probability (nominal = 0.65; ordinal = 0.66) and intensity (nominal = 0.65; ordinal = 0.65) metrics. The best ordinal CT predicted fewer cases of large disagreement with the expert assessments (i.e. no exposure predicted for a job with high exposure and vice versa) compared with the nominal tree across all of the exposure metrics. For example, the percent of jobs with expert-assigned high intensity of exposure that the model predicted as no exposure was 29% for the nominal tree and 22% for the best ordinal tree.
  
  
  CONCLUSIONS
  The overall agreements were similar across CT models; however, the use of ordinal models reduced the magnitude of the discrepancy when disagreements occurred. As the best performing model can vary by situation, researchers should consider evaluating multiple CT methods to maximize the predictive performance within their data.
authors.affiliation: 1.Department of Biostatistics, School of Medicine, Virginia Commonwealth University, 830 East Main Street, Richmond, VA 23298, USA dcwheels@gmail.com.
authors.forename: David C
authors.initials: DC
authors.lastname: Wheeler
authors.affiliation: 1.Department of Biostatistics, School of Medicine, Virginia Commonwealth University, 830 East Main Street, Richmond, VA 23298, USA.
authors.forename: Kellie J
authors.initials: KJ
authors.lastname: Archer
authors.affiliation: 2.Drexel University, School of Public Health, Nesbitt Hall, 3215 Market Street, Philadelphia, PA 19104, USA.
authors.forename: Igor
authors.initials: I
authors.lastname: Burstyn
authors.affiliation: 3.Biostatistics Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, 9609 Medical Center Drive, MSC 9776, Bethesda, MD 20892, USA.
authors.forename: Kai
authors.initials: K
authors.lastname: Yu
authors.affiliation: 4.Stewart Exposure Assessments, LLC, 6045 27th Street North, Arlington, VA 22207, USA.
authors.forename: Patricia A
authors.initials: PA
authors.lastname: Stewart
authors.affiliation: 5.Occupational and Environmental Epidemiology Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, 9609 Medical Center Drive, MSC 9776, Bethesda, MD 20892, USA.
authors.forename: Joanne S
authors.initials: JS
authors.lastname: Colt
authors.affiliation: 5.Occupational and Environmental Epidemiology Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, 9609 Medical Center Drive, MSC 9776, Bethesda, MD 20892, USA.
authors.forename: Dalsu
authors.initials: D
authors.lastname: Baris
authors.affiliation: 6.Geisel School of Medicine at Dartmouth, 1 Medical Center Drive, 7927 Rubin Building, Lebanon NH 03756, USA.
authors.forename: Margaret R
authors.initials: MR
authors.lastname: Karagas
authors.affiliation: 7.Maine Cancer Registry, 286 Water Street, 4th Floor, 11 State House Station, Augusta, Maine 04333-0011, USA.
authors.forename: Molly
authors.initials: M
authors.lastname: Schwenn
authors.affiliation: 8.Vermont Cancer Registry, Vermont Department of Health, P.O. Box 70, Burlington, VT 05402-0070, USA.
authors.forename: Alison
authors.initials: A
authors.lastname: Johnson
authors.affiliation: 9.New Hampshire Department of Health and Human Services, 29 Hazen Drive, Concord, NH 03301, USA.
authors.forename: Karla
authors.initials: K
authors.lastname: Armenti
authors.affiliation: 5.Occupational and Environmental Epidemiology Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, 9609 Medical Center Drive, MSC 9776, Bethesda, MD 20892, USA.
authors.forename: Debra T
authors.initials: DT
authors.lastname: Silverman
authors.affiliation: 5.Occupational and Environmental Epidemiology Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, 9609 Medical Center Drive, MSC 9776, Bethesda, MD 20892, USA.
authors.forename: Melissa C
authors.initials: MC
authors.lastname: Friesen
mesh_terms: D000395:Air Pollutants, Occupational / Q000032:analysis; D000465:Algorithms; D016022:Case-Control Studies; D003661:Decision Support Techniques; D006801:Humans; D008297:Male; D015233:Models, Statistical; D016273:Occupational Exposure / Q000009:adverse effects / Q000032:analysis*; D012107:Research Design; D011795:Surveys and Questionnaires; D001335:Vehicle Emissions / Q000032:analysis
chemical_list: D000395:Air Pollutants, Occupational; D001335:Vehicle Emissions
keywords: classification; diesel exhaust; occupational exposure; ordinal data; statistical learning