Architectural Evolution of Large Language Models: A Systematic Survey on Attention Mechanisms, Efficiency, Adaptation, and Emerging Paradigms

This survey paper reviews the architectural evolution of Large Language Models (LLMs), including Transformer architectures, efficient attention mechanisms, LoRA, Retrieval-Augmented Generation (RAG), Mixture-of-Experts (MoE), multimodal LLMs, inference optimization, and emerging paradigms in modern generative AI systems.