Architectures for AI

Deep Neural Networks (DNNs), especially those powering Generative AI models, are revolutionizing computing and driving unprecedented investments in infrastructure. While DNNs are more challenging than traditional workloads in their immense demand for compute and memory bandwidth, they also provide new opportunities for optimization due to their high-level tensor dataflow construction and tolerance for approximation. Additionally, the DNN workload landscape is highly diverse, spanning distinctions like training vs. inference, datacenter vs. client deployment, and prompt vs. token processing. This variability has spurred the adoption of a wide array of computing architectures, including GPUs, custom ASICs, SoC-integrated accelerators, CPUs, and FPGAs. In this talk, I will share insights from deploying DNN solutions across several of these architectures, discuss some current work, and highlight challenges and opportunities for future architecture development.