title: Fractal analysis illuminates the form of connectionist structural gradualness.
abstract: We examine two connectionist networks-a fractal learning neural network (FLNN) and a Simple Recurrent Network (SRN)-that are trained to process center-embedded symbol sequences. Previous work provides evidence that connectionist networks trained on infinite-state languages tend to form fractal encodings. Most such work focuses on simple counting recursion cases (e.g., anbn), which are not comparable to the complex recursive patterns seen in natural language syntax. Here, we consider exponential state growth cases (including mirror recursion), describe a new training scheme that seems to facilitate learning, and note that the connectionist learning of these cases has a continuous metamorphosis property that looks very different from what is achievable with symbolic encodings. We identify a property-ragged progressive generalization-which helps make this difference clearer. We suggest two conclusions. First, the fractal analysis of these more complex learning cases reveals the possibility of comparing connectionist networks and symbolic models of grammatical structure in a principled way-this helps remove the black box character of connectionist networks and indicates how the theory they support is different from symbolic approaches. Second, the findings indicate the value of future, linked mathematical and empirical work on these models-something that is more possible now than it was 10 years ago.
authors.affiliation: Department of Psychology and Program in Cognitive Science, University of Connecticut, Storrs, CT 06269, USA. whitney.tabor@uconn.edu
authors.forename: Whitney
authors.initials: W
authors.lastname: Tabor
authors.forename: Pyeong Whan
authors.initials: PW
authors.lastname: Cho
authors.forename: Emily
authors.initials: E
authors.lastname: Szkudlarek
mesh_terms: D000465:Algorithms; D003198:Computer Simulation; D017709:Fractals; D005793:Generalization, Psychological; D006801:Humans; D007802:Language; D008962:Models, Theoretical; D016571:Neural Networks, Computer
keywords: Dynamical systems; Fractal grammars; Generalization; Neural (connectionist) networks; Recursion; Self-organization; Simple Recurrent Network (SRN); Symbolic models