Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
Authors
- Rui Men (ORCID: https://orcid.org/0000-0002-4429-3461)
- Z. Y. Wang (ORCID: https://orcid.org/0009-0008-3278-319X)
- Dayiheng Liu (ORCID: https://orcid.org/0000-0002-8755-8941)
- Bo Zheng (ORCID: https://orcid.org/0000-0002-4037-6315)
- Ivan Titov (ORCID: https://orcid.org/0000-0002-2583-1893)
- Junyang Lin (ORCID: https://orcid.org/0000-0001-9931-383X)
- Jingren Zhou (ORCID: https://orcid.org/0000-0002-7159-901X)
- Zeyu Huang (ORCID: https://orcid.org/0000-0002-8117-3231)
- Zihan Qiu
- Kaiyue Wen
Publication Details
- Published
- 2025-01-01
- DOI
- https://doi.org/10.18653/v1/2025.acl-long.249
- Citations
- 5
- Primary Topic
- Simulation Techniques and Applications
- Type
- article
- Field-Weighted Citation Impact
- 10.53