Less-is-More in LSTM Language Models: A Task-Conditional Capacity-by-Input Interaction in Subject–Verb Agreement

Children learn language remarkably well despite limited memory and attention. The Less-is-More hypothesis (Newport, 1988, 1990) proposes that these very limits help, by forcing learners to break language into reusable parts. A stronger version, that less capacity is simply better, has weak support; a later version (Newport, 2016) holds that limited capacity helps only when it works together with the order in which language is encountered. This thesis tests that idea in an LSTM neural network trained on English and evaluated on subject–verb agreement. Giving the network less memory hurt its accuracy on hard sentences, where a distracting noun sits between the subject and the verb, but presenting training sentences from simple to complex removed that cost. Reduced memory alone was not an advantage; it became harmless only when paired with well-ordered input. The benefit was clearest where memory is taxed, though modest at this scale.