Why Language Models Work: A Philosophical Investigation

"For all its runaway success, nobody knows exactly how - or why - it works." This is the MIT Technology Review (2024), summarising the state of knowledge about deep learning. Boaz Barak, a computer scientist at Harvard on secondment to OpenAI's superalignment team, compares the current situation to "physics at the beginning of the 20th century - we have a lot of experimental results that we don't completely understand, and often when you do an experiment it surprises you." Anthropic, the company that built Claude, states in its own research publications that "we don't understand how models do most of the things they do." Hattie Zhou, an AI researcher at the University of Montreal, describes arriving in the field and asking her teachers why these systems work: "My assumption was that scientists know what they're doing. But I was told there weren't good answers." The engineers who build these systems know how they work - the architecture is well documented, the mathematics is precise, the training procedures are reproducible. What they do not know is why a system trained to predict the next token in a sequence produces outputs that exhibit what appears to be comprehension, reasoning, creativity, and contextual sensitivity across every domain of human discourse. The "why" question is treated as an engineering puzzle: more research, more interpretability tools, more scaling laws. It has never been treated as what it is - a question about the structure of language itself. If the models work, the reason must lie in something about language that makes it amenable to statistical approximation by systems that possess no understanding. What is that something? This paper conducts a philosophical investigation into the question. It examines six major frameworks in the philosophy of language - Saussure, Derrida, Wittgenstein, Quine, Austin, Davidson - and tests each against the empirical fact of the models' success. Each framework proves illuminating. Each proves insufficient. The investigation does not announce its conclusion in advance. The reader is invited to follow the inquiry and discover where it leads.