HEX-SIM: Evaluating Multi-modal Large Language Models on Multi-chiplet NPUs

Deep learning models are scaling in both parameters and modalities. Multi-modal large language models are increasingly used in robotic applications, driving the need for large-scale deep learning accelerators. Multi-chiplet heterogeneous neural network accelerators are an effective solution for today’s multi-modal large language models. Different types of chiplets provide diverse functionalities, enabling large data storage, high on-chip bandwidth, and significant computing capability. While single-core or multi-core NPU accelerators can be validated through simulation, there is still a lack of software-level cycle-accurate simulators for multi-chiplet NPUs.In this work, we propose Hex-sim, a configurable multi-chiplet deep learning accelerator simulator. Hex-sim offers designers various macro architectures and system parameters to better evaluate accelerator designs. We conduct extensive simulation experiments using Hex-sim, demonstrating the effects of parallelism, bandwidth, buffer size, and the number of computing engines on inference latency. These insights can significantly aid users in optimizing their designs. Our project code is open-sourced and available at https://github.com/jimrelief/HEX-SIM.