LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks
Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Hanna, Alexander Koller, Andre Martins, Philipp Mondorf, Vera Neplenbroek, Sandro Pezzelle, Barbara Plank, David Schlangen, Alessandro Suglia, Aditya K Surikuchi, Ece Takmaz, Alberto Testoni. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2025.
Authors
- Sandro Pezzelle
- Desmond Elliott (ORCID: https://orcid.org/0000-0003-3112-7904)
- Alessandro Suglia (ORCID: https://orcid.org/0000-0002-3177-5197)
- Mario Giulianelli (ORCID: https://orcid.org/0009-0004-1281-9686)
- Alberto Testoni
- Raffaella Bernardi (ORCID: https://orcid.org/0000-0002-3423-1208)
- Albert Gatt (ORCID: https://orcid.org/0000-0001-6388-8244)
- David Schlangen (ORCID: https://orcid.org/0000-0002-2686-6887)
- Esam Ghaleb (ORCID: https://orcid.org/0000-0002-0603-9817)
- Ece Takmaz
- Aditya K Surikuchi
- André L. Martins (ORCID: https://orcid.org/0000-0002-4794-0644)
- Alexander Koller (ORCID: https://orcid.org/0000-0002-5317-6689)
- Barbara Plank (ORCID: https://orcid.org/0000-0002-4394-1965)
- Anna Bavaresco (ORCID: https://orcid.org/0009-0004-3854-0616)
- Leonardo Bertolazzi
- Philipp Mondorf
- Vera Neplenbroek
- Michael Hanna (ORCID: https://orcid.org/0009-0006-5574-3137)
- Raquel Fernández (ORCID: https://orcid.org/0009-0004-5807-8860)
Publication Details
- Published
- 2025-01-01
- DOI
- https://doi.org/10.18653/v1/2025.acl-short.20
- Citations
- 16
- Primary Topic
- Artificial Intelligence in Law
- Type
- article
- Field-Weighted Citation Impact
- 136.35
Funders
- Villum Fonden
- Eidgenössische Technische Hochschule Zürich
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
- Mathematisches Forschungsinstitut Oberwolfach