A Survey of Large Language Models - Foundations and Future Directions

This survey presents a comprehensive overview of large language models (LLMs), with a particular focus on the evolution and foundational role of attention mechanisms—tracing their origins from psychological and algorithmic precursors to the modern self-attention frameworks that underpin today’s Transformer architectures. We explore the structural and conceptual continuity between attention, feature selection, and attribution methods.Building on this foundation, the survey reviews the development of LLMs across key dimensions: architectural design (e.g., encoder-only, decoder-only, encoder-decoder hybrids), scaling laws and emergent capabilities, training paradigms (including instruction tuning and reinforcement learning from human feedback), and evaluation methodologies. We synthesize recent advances in alignment, robustness, and multimodal integration, and highlight emerging directions such as object-centric world modeling and structured reasoning architectures. By connecting historical foundations with current innovations and future challenges, this work offers a unified perspective on the trajectory of large language models and the evolving role of attention as a tool for modeling relevance, structure, and cognition in artificial intelligence.