A Distributed Training Framework for Cross-Domain Heterogeneous Clusters

Through the management and scheduling of cross-domain heterogeneous intelligent computing clusters, the sharing and coordination of computing resources can be achieved. However, it contains computing resources with different technological ecosystems and is difficult to manage, orchestrate and schedule efficiently. This paper focuses on task-level distributed training within cross-domain heterogeneous intelligent computing clusters, proposing a cross-domain heterogeneous distributed training framework that achieves unified resource management across domains and effectively allocates and schedules heterogeneous intelligent computing resources located in different geographic regions. Built upon the Parameter Server(PS) architecture, the framework addresses the ecological discrepancies caused by resource heterogeneity and coordinates heterogeneous intelligent computing hardware to accomplish a single computational task. A multi-layered scheduling architecture is adopted to tackle the issue of geographic isolation, enabling cross-domain distributed computing. Experimental validation demonstrates the effectiveness of the proposed framework, providing a reliable solution for collaborative scheduling in large-scale clusters across different geographical areas.