Broken symmetries in a location-invariant word recognition network.
We studied the feedforward network proposed by Dandurand et al. (2010), which maps location-specific letter inputs to location-invariant word outputs, probing the hidden layer to determine the nature of the code. Hidden patterns for words were densely distributed, and K-means clustering on single letter patterns produced evidence that the network had formed semi-location-invariant letter representations during training. The possible confound with superseding bigram representations was ruled out, and linear regressions showed that any word pattern was well approximated by a linear combination of its constituent letter patterns. Emulating this code using overlapping holographic representations (Plate, 1995) uncovered a surprisingly acute and useful correspondence with the network, stemming from a broken symmetry in the connection weight matrix and related to the group-invariance theorem (Minsky & Papert, 1969). These results also explain how the network can reproduce relative and transposition priming effects found in humans.
Hannagan
Thomas
T
Dandurand
Frédéric
F
Grainger
Jonathan
J
D000465:Algorithms; D001185:Artificial Intelligence; D006801:Humans; D007802:Language; D016014:Linear Models; D016571:Neural Networks, Computer; D010363:Pattern Recognition, Automated / Q000592:standards*; D011932:Reading; D014705:Verbal Behavior / Q000502:physiology; D014795:Visual Pathways / Q000502:physiology