Crowdsourcing data collection of the retail tobacco environment: case study comparing data from crowdsourced workers to trained data collectors.
OBJECTIVE
To assess whether crowdsourcing is a viable option for conducting surveillance of point of sale (POS) tobacco marketing practices.


METHODS
We posted jobs to an online crowdsourcing platform to audit 194 Florida licensed tobacco retailers over a 3-week period. During the same period, trained data collectors conducted audits at the same retail locations. Data were collected on cigarette advertising, cigarette promotions and product availability (electronic cigarettes, snus and dissolvables). We compared data collected by crowdsourced workers and trained staff and computed frequencies, percent agreement and inter-rater reliability. Photographs of e-cigarettes and exterior cigarette advertisements submitted by crowdsourced workers were used to validate responses.


RESULTS
Inter-rater reliability between crowdsourced and trained data collectors was moderate to high for coding exterior cigarette advertisements, product availability and some tobacco promotions, but poor to fair when coding presence of sales and interior cigarette advertisements. Photos submitted by crowdsourced workers confirmed e-cigarette availability that was missed by trained data collectors in three stores.


CONCLUSIONS
Crowdsourcing may be a promising form of data collection for some POS tobacco measures. Future studies should examine the cost-effectiveness of crowdsourcing compared with traditional trained data collectors and assess which POS measures are most amenable to crowdsourcing.
Public Health Policy Research Program, RTI International, Research Triangle Park, North Carolina, USA.
Annice E
AE
Kim
Community Health Promotion Research Program, RTI International, Research Triangle Park, North Carolina, USA.
Alicea J
AJ
Lieberman
Public Health Policy Research Program, RTI International, Research Triangle Park, North Carolina, USA.
Daniel
D
Dench
D003132:Commerce; D063045:Crowdsourcing / Q000379:methods*; D003625:Data Collection / Q000379:methods; D066300:Electronic Nicotine Delivery Systems; D005431:Florida; D006801:Humans; D008004:Licensure; D040541:Marketing; D015203:Reproducibility of Results; D012907:Smoking; D019460:Tobacco Industry; D062789:Tobacco Products; D014030:Tobacco, Smokeless
Advertising and Promotion; Environment; Surveillance and Monitoring