Dendritic computation bridges neuroscience and foundation models

A prevailing view holds that although neuroscience historically inspired AI, its role in the recent evolution of AI, driven by the scaling of foundation models, is minor. We challenge this perspective by establishing a biological analogy between simplified dendritic neuron models and the attention mechanisms central to AI foundation models. We show that AI attention mechanisms bear direct mappings to internal neuronal complexities arising from dendritic organization, rather than to the mere inter-neuronal connections of simple point neurons. This re-conceptualization not only offers a neuroscience-informed perspective for understanding core AI methodologies beyond their formulation as abstract mathematical operations, but also paves the way for neuro-inspired foundation models that leverage neuroscientific insights to sustainably advance modern AI beyond scaling laws. As an example of this paradigm, we introduce a dendritic-inspired complexity framework, and design a hybrid-attention foundation model with dendritic-inspired sparse computation, which enhances memory capacity while reducing computational demands. Our model supports a 50M context window-equivalent to roughly 40 volumes of Shakespeare-demonstrating superior long-term memory capabilities. This work pioneers a paradigm that harnesses intra-neuronal dynamics to drive sustained advances in modern AI, while also offering a potential route to understanding dendritic computational mechanisms. How insights from neuroscience can inform future AI foundation models is an active area of exploration. Here, the authors show that AI attention mechanisms resemble dendritic computations within individual neurons rather than interactions between neurons, inspiring a dendritic-inspired model with improved long-term memory and reduced computational cost.