Low latency transformer inference on FPGAs for physics applications with hls4ml
Abstract This study presents an efficient implementation of transformer architectures in Field-Programmable Gate Arrays (FPGAs) using hls4ml . We demonstrate the strategy for implementing the multi head attention, softmax, and normalization layer and evaluate three distinct models. Their deployment on VU13P FPGA chip achieved latency less than 2 μs, demonstrating the potential for real-time applications. hls4ml 's compatibility with any TensorFlow-built transformer model further enhances the scalability and applicability of this work.
Authors
- Scott Hauck (ORCID: https://orcid.org/0000-0001-9516-0311)
- Z. Jiang (ORCID: https://orcid.org/0000-0002-9422-9396)
- Philip Harris (ORCID: https://orcid.org/0000-0001-8189-3741)
- Vladimir Lončar (ORCID: https://orcid.org/0000-0003-3651-0232)
- Ekaterina Govorkova
- Eric Moreno (ORCID: https://orcid.org/0000-0001-5666-3637)
- Yi-Hui Chen (ORCID: https://orcid.org/0000-0003-1146-4073)
- E. E. Khoda (ORCID: https://orcid.org/0000-0002-8340-9455)
- S.‐C. Hsu (ORCID: https://orcid.org/0000-0001-6214-8500)
- Dennis Yin
Institutions
- University of Washington (US)
- University of Washington Applied Physics Laboratory (US)
- Massachusetts Institute of Technology (US)
Publication Details
- Journal
- Journal of Instrumentation
- Published
- 2025-04-01
- DOI
- https://doi.org/10.1088/1748-0221/20/04/p04014
- Citations
- 2
- Primary Topic
- Particle Detector Development and Performance
- Type
- article
- Field-Weighted Citation Impact
- 3.74