Adaptive Networking and Resource Allocation for Distributed Data Parallel Training of Generative AI in Constrained Environments

As AI models and datasets grow larger, the demand for training resources has increased, leaving smaller companies with limited servers and no powerful GPUs at risk of falling behind. This study examines how small-scale environments can use Distributed Data Parallel (DDP) to participate in AI advancements, even with resource constraints. By deploying Docker containers across CPU-based servers connected via Ethernet, the experiment simulates low-resource conditions. Key network metrics are monitored, and a recommendation system is proposed for optimal node selection. The results show that distributed training is feasible in constrained environments, enabling smaller organizations to remain competitive in AI development.