CPU and GPU Servers for AI Infrastructure Migration: A Workload-Based Comparison and Deployment Framework

This paper presents a workload-based framework for migrating CPU-based data centers to infrastructure that combines CPU and GPU servers for AI workloads. It draws on a focused literature review and compares two published single-node GPT-J text summarization results from the MLPerf Inference v5.0 Datacenter Closed division. At the same benchmark quality threshold, the selected GPU-accelerated system achieved 6.59 times the throughput of the selected CPU-only system in the Server scenario and 3.40 times its throughput in the Offline scenario. These ratios apply to the submitted hardware and software configurations. Energy use and cost require separate measurements. The proposed pilot compares optimized configurations against the same workload, output-quality threshold, and latency targets. It measures energy and cost per request that meets those targets, including idle time where relevant. The framework also specifies checks for rack power, cooling, and recovery before GPU capacity is expanded. It gives data center operators a method for deciding which workloads to retain on CPUs, which to move to GPUs, and what to verify before scaling.