Breakthrough Edge AI Inference Performance Using Northpole in 3U Vpx Form Factor
We present preliminary results demonstrating AI (artificial intelligence) inference using the IBM AIU NorthPole Chip [1], [2] incorporated into a compact, rugged 3U VPX form factor module (NP-VPX) [3]. NP-VPX allows NorthPole to be used in edge applications with stringent cooling requirements, high-speed switch fabrics, and rugged environments. NP-VPX processes 965 frames per second (fps) with a Yolo-v4 network with <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$640 \times 640$</tex> pixel images at 73.5 W at full-precision accuracy, achieving 13.2 frames / J (fps/W). NP-VPX processes over 40,300 fps with a ResNet-50 network with <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$224 \times 224$</tex> pixel images at 65.9 W at full-precision accuracy, achieving 611 frames <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$/ \mathrm{J}$</tex>.
Authors
- Filipp Akopyan (ORCID: https://orcid.org/0009-0001-5312-1311)
- Guillaume Garreau (ORCID: https://orcid.org/0000-0002-4115-9232)
- Myron Flickner
- Michael E. Criscolo
- Takanori Ueda (ORCID: https://orcid.org/0000-0003-1637-4640)
- Rafael Vieira de Sousa (ORCID: https://orcid.org/0000-0001-5885-5368)
- Rathinakumar Appuswamy
- Michael DeBole (ORCID: https://orcid.org/0000-0002-3068-9430)
- Peter J. Carlson
- E. G. Colgan (ORCID: https://orcid.org/0000-0001-6971-9839)
- Brian Taba (ORCID: https://orcid.org/0009-0008-7831-1139)
- W. P. Risk (ORCID: https://orcid.org/0009-0004-3211-0738)
- Megumi Ito (ORCID: https://orcid.org/0009-0002-3627-5308)
- Andrew S. Cassidy (ORCID: https://orcid.org/0000-0001-7305-4198)
- Deepika Bablani (ORCID: https://orcid.org/0000-0002-8987-8201)
- Ignacio Terrizzano (ORCID: https://orcid.org/0000-0001-6179-2601)
- Steven K. Esser (ORCID: https://orcid.org/0009-0008-4089-0577)
- Jeffrey L. McKinstry
- Nathaniel J. McClatchey
- Dharmendra S. Modha (ORCID: https://orcid.org/0009-0005-6714-8127)
- Yutaka Nakamura (ORCID: https://orcid.org/0000-0002-6921-1266)
- Pallab Datta (ORCID: https://orcid.org/0000-0002-0515-9729)
- Carlos Ortega Otero
- Jeffrey A. Kusnitz
- Heinz Baier
- Jun Sawada
- Jennifer L. Klamo (ORCID: https://orcid.org/0009-0001-9042-0476)
- John V. Arthur
- Neil McGlohon (ORCID: https://orcid.org/0000-0001-8304-4911)
- Rajamohan Gandhasri (ORCID: https://orcid.org/0009-0003-5068-1924)
- Jay Sivagnaname
- Alexander Andreopoulos (ORCID: https://orcid.org/0009-0001-2408-5171)
- Arnon Amir (ORCID: https://orcid.org/0009-0000-6814-165X)
- Tapan K. Nayak (ORCID: https://orcid.org/0000-0001-8941-8961)
- P. Mann
- Daniel F. Smith
- Kai Schleupen
Publication Details
- Published
- 2024-09-23
- DOI
- https://doi.org/10.1109/hpec62836.2024.10938474
- Citations
- 2
- Primary Topic
- Optical Systems and Laser Technology
- Type
- article
- Field-Weighted Citation Impact
- 0.42