Dual-memory neural networks for modeling cognitive activities of humans via wearable sensors.
Wearable devices, such as smart glasses and watches, allow for continuous recording of everyday life in a real world over an extended period of time or lifelong. This possibility helps better understand the cognitive behavior of humans in real life as well as build human-aware intelligent agents for practical purposes. However, modeling the human cognitive activity from wearable-sensor data stream is challenging because learning new information often results in loss of previously acquired information, causing a problem known as catastrophic forgetting. Here we propose a deep-learning neural network architecture that resolves the catastrophic forgetting problem. Based on the neurocognitive theory of the complementary learning systems of the neocortex and hippocampus, we introduce a dual memory architecture (DMA) that, on one hand, slowly acquires the structured knowledge representations and, on the other hand, rapidly learns the specifics of individual experiences. The DMA system learns continuously through incremental feature adaptation and weight transfer. We evaluate the performance on two real-life datasets, the CIFAR-10 image-stream dataset and the 46-day Lifelog dataset collected from Google Glass, showing that the proposed model outperforms other online learning methods.
School of Computer Science and Engineering, Seoul National University, Seoul 08826, South Korea.
Sang-Woo
SW
Lee
School of Computer Science and Engineering, Seoul National University, Seoul 08826, South Korea.
Chung-Yeon
CY
Lee
Interdisciplinary Program in Neuroscience, Seoul National University, Seoul 08826, South Korea.
Dong-Hyun
DH
Kwak
NAVER LABS, NAVER Corp., Bundang 13561, South Korea.
Jung-Woo
JW
Ha
NAVER LABS, NAVER Corp., Bundang 13561, South Korea.
Jeonghee
J
Kim
School of Computer Science and Engineering, Seoul National University, Seoul 08826, South Korea; Interdisciplinary Program in Neuroscience, Seoul National University, Seoul 08826, South Korea; Surromind Robotics, 1 Gwanak-ro Gwanak-gu, Seoul 08826, South Korea. Electronic address: btzhang@bi.snu.ac.kr.
Byoung-Tak
BT
Zhang
D001921:Brain / Q000502:physiology; D003071:Cognition; D006801:Humans; D008838:Microcomputers; D008959:Models, Neurological; D016571:Neural Networks, Computer
Complementary learning systems; Deep neural networks; Dual memory architecture; Hypernetworks; Lifelog dataset; Online learning