Designing highly resilient AI fabrics: Networking architectures for large-scale model training

The fast development of big AI models, mostly big language models (LLMs), has caused new challenges to networking infrastructure as never seen before. As training expands towards hundreds and even thousands of GPUs in distributed systems, the resilience, efficiency and performance of AI fabrics become paramount to long-run throughput and reliability. This paper discusses some of the architecture design concepts and new technologies creating resilient AI fabrics to build large-scale model training. We discuss the use of high-bandwidth interconnects like RoCEv2 and 800G / 1.6T Ethernet, look at topology-aware routing schemes and evaluate how network-level fault-tolerance mechanisms can be made resilient. With the help of case studies and benchmarking, we point out both the good and bad practice of existing AI training networks. Our results give some advice to future-proof design of AI networking architectures to scale to model complexity in next generation models.