Speed and Conversational Large Language Models: Not All Is About Tokens per Second
The speed of open-weights large language models (LLMs) and its dependency on the task at hand, when run on GPUs, is studied to present a comparative analysis of the speed of the most popular open LLMs.
Authors
- Fabrizio Lombardi (ORCID: https://orcid.org/0000-0003-3152-3245)
- Javier Conde (ORCID: https://orcid.org/0000-0002-5304-0626)
- Pedro Reviriego (ORCID: https://orcid.org/0000-0003-2540-5234)
- Zhen Gao (ORCID: https://orcid.org/0000-0001-9887-1418)
- Shanshan Liu (ORCID: https://orcid.org/0000-0001-6226-2880)
- Miguel Ángel Pesquera González
Institutions
- Northeastern University (US)
- University of Electronic Science and Technology of China (CN)
- Tianjin University (CN)
- Universidad Politécnica de Madrid (ES)
Publication Details
- Journal
- Computer
- Published
- 2024-08-01
- DOI
- https://doi.org/10.1109/mc.2024.3399384
- Citations
- 7
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 2.43
Funders
- Agencia Estatal de Investigación