From Attention to Inference - A Technical Study of Large Language Models

This paper presents a technical overview of the Transformer architecture and its role in modern Natural Language Processing (NLP). It examines the core components of the paradigm including self-attention mechanisms, tokenization, positional encoding, model families (encoder-only, decoder-only, and encoder–decoder), pretraining objectives, fine-tuning, and inference processes. System-level considerations such as KV caching, Time to First Token, throughput, and VRAM usage are also discussed. The paper is intended as a structured technical reference for practitioners and students working in machine learning and NLP.