A Systematic Evaluation of Large Code Models in API Suggestion: When, Which, and How
API suggestion is a critical task in modern software development, assisting programmers by predicting and recommending third-party APIs based on the current context. Recent advancements in large code models (LCMs) have shown promise in the API suggestion task. However, they mainly focus on suggesting which APIs to use, ignoring that programmers may demand more assistance while using APIs in practice including when to use the suggested APIs and how to use the APIs. To mitigate the gap, we conduct a systematic evaluation of LCMs for the API suggestion task in the paper.
Authors
- Shuzheng Gao (ORCID: https://orcid.org/0000-0002-8102-480X)
- Chaozheng Wang (ORCID: https://orcid.org/0000-0002-3935-7328)
- Chun Yong Chong (ORCID: https://orcid.org/0000-0003-1164-0049)
- Cuiyun Gao (ORCID: https://orcid.org/0000-0003-4774-2434)
- Michael R. Lyu (ORCID: https://orcid.org/0000-0002-3666-5798)
- Shan Gao (ORCID: https://orcid.org/0009-0006-2695-2968)
- Wenxuan Wang (ORCID: https://orcid.org/0000-0002-9803-8204)
Institutions
- Chinese University of Hong Kong (HK)
- Harbin Institute of Technology (CN)
- Huawei Technologies (China) (CN)
Publication Details
- Published
- 2024-10-18
- DOI
- https://doi.org/10.1145/3691620.3695004
- Citations
- 4
- Primary Topic
- Web Data Mining and Analysis
- Type
- article
- Field-Weighted Citation Impact
- 3.12
Funders
- National Natural Science Foundation of China
- Natural Science Foundation of Guangdong Province