Models of human learning should capture the multimodal complexity and communicative goals of the natural learning environment

Children do not learn language from language alone. Instead, children learn from social interactions with multidimensional communicative cues that occur dynamically across timescales. A wealth of research using in-lab experiments and brief audio recordings has made progress in explaining early cognitive and communicative development, but these approaches are limited in their ability to capture the rich diversity of children’s early experience. Large language models represent a powerful approach for understanding how language can be learned from massive amounts of textual (and in some cases visual) data, but they have near-zero access to the actual, lived complexities of children’s everyday input. We assert the need for more descriptive research that densely samples the natural dynamics of children’s everyday communicative environments in order to grasp the long-standing mystery of how young children learn, including their language development. With the right multimodal data and a greater focus on active participation in a social environment, researchers will be able to go beyond large language models to build developmentally grounded efficient communication models that truly take into account the dimensionality of children’s diverse perceptual and social environments.