VeriRAG: Design AI-Specific CPU Co-processor with RAG-Enhanced LLMs

In recent years, many industrial CPUs have integrated AI-specific co-processors as "CPU for AI" solution. However, the implementation details of co-processors are diverse. In this work, we present VeriRAG, an LLM-assisted agile design methodology for AI-specific CPU co-processor, which includes a summary-template RAG-enhanced LLM flow for agile RTL generation. We first present a general RISC-V-based AI-specific co-processor architecture template to support both Matrix and Nonlinear computations with detailed instruction extensions. Furthermore, we develop a series of AI-specific co-processors based on VeriRAG for different CPUs, i.e., high-end Xuantie C910 and low-power CVA6 CPU. It is observed that the AI coprocessors can be effectively and properly designed by VeriRAG within a short design cycle, and co-processors gain notable performance boost for AI tasks.