Efficient Methods for Natural Language Processing: A Survey
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows. Such resources include data, time, storage, or energy, all of which are naturally limited and unevenly distributed. This motivates research into efficient methods that require fewer resources to achieve similar results. This survey synthesizes and relates current methods and findings in efficient NLP. We aim to provide both guidance for conducting NLP under limited resources, and point towards promising research directions for developing more efficient methods.
Authors
- Michael Hassid
- Roy Schwartz (ORCID: https://orcid.org/0000-0003-3487-5713)
- Jesse Dodge
- Noam Slonim (ORCID: https://orcid.org/0000-0001-5171-8264)
- Leon Derczynski (ORCID: https://orcid.org/0000-0002-8656-3431)
- Iryna Gurevych (ORCID: https://orcid.org/0000-0003-2187-7621)
- Manuel R. Ciosici (ORCID: https://orcid.org/0000-0001-7768-738X)
- Pedro H. Martins (ORCID: https://orcid.org/0000-0002-1656-990X)
- Colin Raffel
- Emma Strubell (ORCID: https://orcid.org/0000-0003-2798-0726)
- André F. T. Martins (ORCID: https://orcid.org/0000-0001-8282-625X)
- Jessica Zosa Forde
- Peter Milder (ORCID: https://orcid.org/0000-0003-1146-3011)
- Edwin Simpson (ORCID: https://orcid.org/0000-0002-6447-1552)
- Kenneth Heafield (ORCID: https://orcid.org/0000-0002-6344-9927)
- Qingqing Cao (ORCID: https://orcid.org/0000-0002-9306-0306)
- Betty van Aken
- Tianchu Ji
- Sara Hooker (ORCID: https://orcid.org/0000-0002-0190-6459)
- Ji-Ung Lee (ORCID: https://orcid.org/0000-0002-8428-2003)
- Niranjan Balasubramanian (ORCID: https://orcid.org/0000-0003-4187-9368)
- Marcos Treviso
Institutions
- University of North Carolina at Chapel Hill (US)
Publication Details
- Journal
- UNC Libraries
- Published
- 2025-03-19
- DOI
- https://doi.org/10.17615/cs07-kr85
- Primary Topic
- Natural Language Processing Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Novo Nordisk
- Novo Nordisk Fonden
- HORIZON EUROPE Framework Programme