Federated carbon intelligence for sustainable AI: Real-time optimization across heterogeneous hardware fleets

Abstract As AI infrastructure expands globally, managing the sustainability of large-scale inference workloads across diverse hardware fleets has become a critical challenge. While prior frameworks such as EcoServe and Google’s carbon-intelligent computing have addressed carbon-aware scheduling, they lack integration with real-time hardware health telemetry and adaptive degradation modeling. FCI bridges this gap by combining State-of-Health metrics, dynamic grid carbon data, and reinforcement learning-based orchestration to achieve lifecycle-optimized sustainability. Here, we propose a federated carbon intelligence (FCI) framework that unifies telemetry-informed degradation modeling (SoH-AI), real-time grid carbon monitoring, and workload-specific inference profiling to dynamically route AI jobs across platforms such as NVIDIA A100/H100, Google TPUv5i, and Cerebras WSE-2. Leveraging graph-based modeling and reinforcement learning agents, our approach balances emissions, hardware longevity, and SLA constraints. In modeled scenarios, our scheduler reduced cumulative CO₂ emissions by up to 45% (37 ± 8%) over a three-year simulated period compared to static allocation, representing the upper range of performance achievable under realistic workload and grid-mix assumptions. This work introduces a new paradigm for lifecycle-aware, emissions-adaptive AI inference scheduling—laying the foundation for climate-aligned, self-optimizing AI infrastructure. Graphic Abstract