A comparative analysis of CodeBERT and CodeLlama models: Architecture, functionality and application in software coding tasks

The relevance of the research was conditioned by the need to compare the large language models CodeBERT and CodeLlama, which were actively used for automating code generation and analysis with the aim of improving the efficiency and quality of software. The aim of the study was a comprehensive juxtaposition of the architectural and functional characteristics of the selected language models CodeBERT and CodeLlama. Interpretative, comparative, systemic and structural-categorical analyses were used to study the architectures, tasks, and relevance of the models. A comprehensive comparative analysis of the CodeBERT and CodeLlama models was carried out according to key parameters: model architecture (the RoBERTa encoder architecture in CodeBERT versus the Llama 2 decoder architecture in CodeLlama), the scale and sources of training data, the range of supported tasks, performance on benchmark datasets, advantages and limitations, typical areas of application, and conditions of accessibility and licensing. The results showed that the difference in architecture and training data significantly affected the effectiveness of the models in different types of tasks, and also determined the practical capabilities and limitations. Particular attention was paid to the issues of implementing the models in practical scenarios, taking into account hardware resources and licensing policy. The results showed that CodeLlama required significantly greater computational resources for effective operation, whereas CodeBERT was easier to implement on standard equipment. It was also established that the licensing conditions of CodeLlama were more restrictive, which could complicate its use in commercial projects, in contrast to CodeBERT with an open licence. It was concluded that these models performed predominantly complementary functions: CodeBERT was an effective tool for code-understanding tasks, whereas CodeLlama demonstrated high results in generation tasks. The conclusions outlined the challenges and prospects for the development of next-generation models with multitasking and multimodality. Practical value – assistance to developers and researchers in choosing the optimal tool, taking into account technical and licensing aspects