title: HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition.
abstract: This paper describes novel event-based spatio-temporal features called time-surfaces and how they can be used to create a hierarchical event-based pattern recognition architecture. Unlike existing hierarchical architectures for pattern recognition, the presented model relies on a time oriented approach to extract spatio-temporal features from the asynchronously acquired dynamics of a visual scene. These dynamics are acquired using biologically inspired frameless asynchronous event-driven vision sensors. Similarly to cortical structures, subsequent layers in our hierarchy extract increasingly abstract features using increasingly large spatio-temporal windows. The central concept is to use the rich temporal information provided by events to create contexts in the form of time-surfaces which represent the recent temporal activity within a local spatial neighborhood. We demonstrate that this concept can robustly be used at all stages of an event-based hierarchical model. First layer feature units operate on groups of pixels, while subsequent layer feature units operate on the output of lower level feature units. We report results on a previously published 36 class character recognition task and a four class canonical dynamic card pip task, achieving near 100 percent accuracy on each. We introduce a new seven class moving face recognition task, achieving 79 percent accuracy.This paper describes novel event-based spatio-temporal features called time-surfaces and how they can be used to create a hierarchical event-based pattern recognition architecture. Unlike existing hierarchical architectures for pattern recognition, the presented model relies on a time oriented approach to extract spatio-temporal features from the asynchronously acquired dynamics of a visual scene. These dynamics are acquired using biologically inspired frameless asynchronous event-driven vision sensors. Similarly to cortical structures, subsequent layers in our hierarchy extract increasingly abstract features using increasingly large spatio-temporal windows. The central concept is to use the rich temporal information provided by events to create contexts in the form of time-surfaces which represent the recent temporal activity within a local spatial neighborhood. We demonstrate that this concept can robustly be used at all stages of an event-based hierarchical model. First layer feature units operate on groups of pixels, while subsequent layer feature units operate on the output of lower level feature units. We report results on a previously published 36 class character recognition task and a four class canonical dynamic card pip task, achieving near 100 percent accuracy on each. We introduce a new seven class moving face recognition task, achieving 79 percent accuracy.
authors.affiliation: Vision and Natural Computation Group, Institut National de la Santé et de la Recherche Médicale, Sorbonne Universités, Institut de la Vision, Université Paris 06, Paris, Paris, FranceFrance.
authors.forename: Xavier
authors.initials: X
authors.lastname: Lagorce
authors.affiliation: Singapore Institute for Neurotechnology (SINAPSE), National University of Singapore, Singapore.
authors.forename: Garrick
authors.initials: G
authors.lastname: Orchard
authors.affiliation: Vision and Natural Computation Group, Institut National de la Santé et de la Recherche Médicale, Sorbonne Universités, Institut de la Vision, Université Paris 06, Paris, Paris, FranceFrance.
authors.forename: Francesco
authors.initials: F
authors.lastname: Galluppi
authors.affiliation: Department of Electronic and Computer Engineering.
authors.forename: Bertram E
authors.initials: BE
authors.lastname: Shi
authors.affiliation: Vision and Natural Computation Group, Institut National de la Santé et de la Recherche Médicale, Sorbonne Universités, Institut de la Vision, Université Paris 06, Paris, Paris, FranceFrance.
authors.forename: Ryad B
authors.initials: RB
authors.lastname: Benosman
keywords: Biosensors; Cameras; Character recognition; Feature extraction; Object recognition; Visualization