title: Local minima in hierarchical structures of complex-valued neural networks.
abstract: Most of local minima caused by the hierarchical structure can be resolved by extending the real-valued neural network to complex numbers. It was proved in 2000 that a critical point of the real-valued neural network with H-1 hidden neurons always gives many critical points of the real-valued neural network with H hidden neurons. These critical points consist of many lines in the parameter space which could be local minima or saddle points. Local minima cause plateaus which have a strong negative influence on learning. However, most of the critical points of complex-valued neural network are saddle points unlike those of the real-valued neural network. This is a prominent property of the complex-valued neural network.
authors: Mathematical Neuroinformatics Group, Human Technology Research Institute, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba Central 2, 1-1-1 Umezono, Tsukuba, Ibaraki, 305-8568, Japan. Electronic address: tohru-nitta@aist.go.jp.
authors: Tohru
authors: T
authors: Nitta
mesh_terms: D000465:Algorithms; D007858:Learning / Q000502:physiology*; D016571:Neural Networks, Computer; D009474:Neurons / Q000502:physiology*; D012815:Signal Processing, Computer-Assisted