title: Forecasting Significant Societal Events Using The Embers Streaming Predictive Analytics System.
abstract: Developed under the Intelligence Advanced Research Project Activity Open Source Indicators program, Early Model Based Event Recognition using Surrogates (EMBERS) is a large-scale big data analytics system for forecasting significant societal events, such as civil unrest events on the basis of continuous, automated analysis of large volumes of publicly available data. It has been operational since November 2012 and delivers approximately 50 predictions each day for countries of Latin America. EMBERS is built on a streaming, scalable, loosely coupled, shared-nothing architecture using ZeroMQ as its messaging backbone and JSON as its wire data format. It is deployed on Amazon Web Services using an entirely automated deployment process. We describe the architecture of the system, some of the design tradeoffs encountered during development, and specifics of the machine learning models underlying EMBERS. We also present a detailed prospective evaluation of EMBERS in forecasting significant societal events in the past 2 years.
authors.affiliation: CACI Inc. , Lanham, Maryland.
authors.forename: Andy
authors.initials: A
authors.lastname: Doyle
authors.affiliation: CACI Inc. , Lanham, Maryland.
authors.forename: Graham
authors.initials: G
authors.lastname: Katz
authors.affiliation: CACI Inc. , Lanham, Maryland.
authors.forename: Kristen
authors.initials: K
authors.lastname: Summers
authors.affiliation: CACI Inc. , Lanham, Maryland.
authors.forename: Chris
authors.initials: C
authors.lastname: Ackermann
authors.affiliation: CACI Inc. , Lanham, Maryland.
authors.forename: Ilya
authors.initials: I
authors.lastname: Zavorin
authors.affiliation: CACI Inc. , Lanham, Maryland.
authors.forename: Zunsik
authors.initials: Z
authors.lastname: Lim
authors.affiliation: Virginia Tech , Arlington, Virginia.
authors.forename: Sathappan
authors.initials: S
authors.lastname: Muthiah
authors.affiliation: Virginia Tech , Arlington, Virginia.
authors.forename: Patrick
authors.initials: P
authors.lastname: Butler
authors.affiliation: Virginia Tech , Arlington, Virginia.
authors.forename: Nathan
authors.initials: N
authors.lastname: Self
authors.affiliation: Virginia Tech , Arlington, Virginia.
authors.forename: Liang
authors.initials: L
authors.lastname: Zhao
authors.affiliation: Virginia Tech , Arlington, Virginia.
authors.forename: Chang-Tien
authors.initials: CT
authors.lastname: Lu
authors.affiliation: Virginia Tech , Arlington, Virginia.
authors.forename: Rupinder Paul
authors.initials: RP
authors.lastname: Khandpur
authors.affiliation: BASIS Technology , Herndon, Virginia.
authors.forename: Youssef
authors.initials: Y
authors.lastname: Fayed
authors.affiliation: Virginia Tech , Arlington, Virginia.
authors.forename: Naren
authors.initials: N
authors.lastname: Ramakrishnan