From Perceptrons to Transformers: A Review of Neural Network and Deep Learning Architectures, Training Methods and Applications

Abstract—This paper reviews how artificial neural networks developed from single perceptrons into the deep architectures that now dominate machine learning. It follows the progression through multi-layer networks, convolutional networks for images, gated recurrent networks for sequences, and transformers built on self-attention, and it contrasts these families in two comparison tables and a historical timeline. A small experiment on MNIST and CIFAR-10 shows how an MLP, a CNN and an LSTM compare under identical training conditions. The review also explains how supervised, self-supervised and reinforcement learning, together with regularization and optimization methods, make very deep models trainable. Uses in medical imaging, protein-structure prediction, autonomous systems, language technology and financial analysis are summarized. The paper closes with open problems: energy cost, interpretability, adversarial vulnerability, catastrophic forgetting, privacy and bias. It is intended as a compact, source-backed starting point for students entering the field. Keywords—neural networks, deep learning, convolutional neural networks, recurrent neural networks, transformers, natural language processing, review