Assessing Code Synthesis In LLMS: CodeGeex and Codeium

Large Language Models (LLMs) have revolutionized software development in recent years, enabling the generation of high-quality source code across various domains. As the landscape of LLMs continues to evolve, developers are faced with the daunting task of selecting the most suitable model for their specific needs, with limited research providing practical guidance on this front. To address this challenge, this paper presents a comprehensive evaluation of two prominent LLMs, CodeGeeX and Codeium, on a diverse set of 25 programming tasks in Python and Java. The assessment encompasses both functional and non-functional aspects of the synthesized code, including performance, reliability, and maintainability, as well as developer usability and readability. The results show that while both models produced effective code, Codeium outperformed CodeGeeX in terms of code quality, clarity, and overall usability, with human evaluators confirm these findings through manual reviews. This study provides valuable insights for developer seeking to harness the potential of LLMs in their software development workflows and high-lights the importance of evaluation and comparison of these models to ensure optimal results.