Parameter Identifiability in Statistical Machine Learning: A Review.
This review examines the relevance of parameter identifiability for statistical models used in machine learning. In addition to defining main concepts, we address several issues of identifiability closely related to machine learning, showing the advantages and disadvantages of state-of-the-art research and demonstrating recent progress. First, we review criteria for determining the parameter structure of models from the literature. This has three related issues: parameter identifiability, parameter redundancy, and reparameterization. Second, we review the deep influence of identifiability on various aspects of machine learning from theoretical and application viewpoints. In addition to illustrating the utility and influence of identifiability, we emphasize the interplay among identifiability theory, machine learning, mathematical statistics, information theory, optimization theory, information geometry, Riemann geometry, symbolic computation, Bayesian inference, algebraic geometry, and others. Finally, we present a new perspective together with the associated challenges.
Chongqing Key Laboratory of Computational Intelligence, School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China ranzy@cqupt.edu.cn.
Zhi-Yong
ZY
Ran
NLPR & LIAMA, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China hubg@nlpr.ia.ac.cn.
Bao-Gang
BG
Hu