title: An information theoretic approach of designing sparse kernel adaptive filters.
abstract: This paper discusses an information theoretic approach of designing sparse kernel adaptive filters. To determine useful data to be learned and remove redundant ones, a subjective information measure called surprise is introduced. Surprise captures the amount of information a datum contains which is transferable to a learning system. Based on this concept, we propose a systematic sparsification scheme, which can drastically reduce the time and space complexity without harming the performance of kernel adaptive filters. Nonlinear regression, short term chaotic time-series prediction, and long term time-series forecasting examples are presented.
authors.affiliation: Forecasting Team, Amazon.Com, Seattle,WA 98104 USA. weifeng@amazon.com
authors.forename: Weifeng
authors.initials: W
authors.lastname: Liu
authors.forename: Il
authors.initials: I
authors.lastname: Park
authors.forename: José C
authors.initials: JC
authors.lastname: Principe
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D006801:Humans; D007257:Information Theory; D016018:Least-Squares Analysis; D017711:Nonlinear Dynamics; D013997:Time Factors