Scalability and Load Balancing in Distributed Systems: Trade-offs, OSI Layers, and Architectural Decision Frameworks

This article presents a rigorous, practitioner-oriented reference on two foundational pillars of distributed system design: application scalability and load balancing. The work covers the full trade-off space between vertical scaling (Scale Up) and horizontal scaling (Scale Out), analysing cost implications, failure-mode characteristics, and the architectural conditions under which each approach is appropriate. The article systematically addresses load balancing across all seven layers of the OSI reference model, detailing when each layer is relevant to load balancing decisions, which tools operate at each layer, and the constraints that layer selection imposes on future architectural choices. Special attention is given to the distinction between Layer 4 (Transport) and Layer 7 (Application) load balancing, including the irreversible nature of that selection as an architectural constraint rather than a performance optimisation. Six load balancing algorithms are analysed comparatively — Round Robin, Weighted Round Robin, Least Connections, IP Hash, Least Response Time, and Generic Hash — across six operational dimensions: implementation complexity, throughput, session affinity, adaptive behavior, cloud adoption, and stateful tracking requirements. A structured decision matrix and a binary decision flowchart are provided to guide algorithm selection based on workload characteristics. The article concludes with a mapping of major cloud-managed load balancers (AWS ALB/NLB, Google Cloud HTTPS LB / TCP LB, Azure Application Gateway / Load Balancer) and open-source software solutions (NGINX, HAProxy, Traefik) to their respective OSI layers, with selection heuristics for cloud-native, Kubernetes, on-premises, and hybrid deployment environments. Five IEEE-grade figures are included to support visual comprehension and academic use.