Qualitatively different feedback effects in recurrent models of spoken word recognition

We explored the possibility that different loci of feedback in Elman networks (which have a limited memory 'context' based on previous hidden unit states) and Jordan networks (where context is based on previous output states) produce qualitatively different behaviors. These different kinds of context could theoretically yield qualitatively different outcomes, although we expected outcomes to be similar. We trained Elman networks and Jordan networks to activate the current word based on a sequence of phonemes. With a moderately sized lexicon (1000 words), training outcomes were highly similar, with high accuracy and patterns of activation and competition over time that resemble those found in studies with human participants. Both networks struggled to correctly handle embedded words and such pairs were the source of most errors. However, contrary to our expectations, there were also salient differences. Jordan networks required substantially smaller learning rates for successful learning, yet learned much faster than Elman networks. Most dramatically, Elman lexical activations were roughly related to the conditional probabilities of different words over time given phonemic inputs, though activations tended to decline at word offsets, while Jordan lexical activations approximated conditional probabilities very closely from early in training, and sustained target word activations even at the final input step. Tests for top-down influence using ambiguous and nonword inputs suggest that both models show high bottom-up priority but also that lexical knowledge strongly influences their interpretation of ambiguous inputs. We discuss the computational pressures that drive the observed differences and similarities between architectures.