Neural network approximation of continuous functionals and continuous functions on compactifications.
This article characterizes the set of activation functions, bounded or unbounded, that allow feedforward network approximation of the continuous functions on the classic two-point compactification of R(1). The characterization fails when the set of targets are continuous functions on the classic compactifications of R(n), n>/=2. Nonpolynomial, analytic activation functions with input-to-hidden weights in very limited sets allow approximation of continuous function over compact sets in R(n), while even sigmoidal activation functions with weights in limited sets cannot approximate continuous functions on compactifications. The abstract structure foregrounded by compactification leads directly to possibility results for multi-layer networks and possibility results for neural networks in infinite dimensional settings.
Stinchcombe
M B.
MB
Department of Economics, The University of Texas at Austin, Austin, TX, USA