Enhancing Neural Code Representation With Additional Context

Automated program comprehension underpins many software engineering tasks, from code summarisation to clone detection. While recent deep learning models have achieved strong results, they typically rely on source code alone and overlook other information that developers routinely use to understand programs, such as version history or structural relationships. This gap limits the ability of current models to capture the broader context in which code evolves and operates. To address this, we conduct an empirical study of how enriching code representations with these additional contextual signals affects the performance of neural models on key program comprehension tasks. We evaluate four downstream software engineering tasks, code clone detection, code classification, code summarisation, and vulnerability detection, using SeSaMe (1,679 semantically similar Java methods from 11 open-source projects), CodeSearchNet (2,117 projects; 63,259 methods paired with natural-language documentation), and Vul4J (178 vulnerability-inducing methods). To ensure a robust comparison, we fine-tune five foundational code models (tree-based and pre-trained Transformer architectures), namely CodeBERT, GraphCodeBERT, CodeT5, PLBART, and ASTNN, under both code-only and context-augmented settings. We further expand the evaluation to seven code-specialised large language models (CodeLlama 7B/13B/34B, Qwen2.5-Coder 1.5B/7B/14B, and DeepSeek-Coder 6.7B), comparing prompt-based context injection against fine-tuned use of context. Experimental results show that context improves classification performance for all five foundational models and, after fine-tuning, for most of the seven LLMs. For summarisation, the effect depends on the model family: version history improves all four foundational models on every metric, whereas context injected into the prompt of a decoder-only LLM helps two of the seven models (Qwen2.5-Coder 7B and 14B, via call-graph context) and lowers both lexical and semantic metrics for the other five. In particular, encoding version history never hurts and usually improves clone detection across the five foundational models (up to +15.92% F1 for CodeT5) and summarisation (e.g., GraphCodeBERT +5.56% METEOR; CodeT5 +3.05% BLEU-4). By contrast, call-graph context delivers model- and task-dependent gains. Combining multiple types of context (version history, call graphs, and method age) can further enhance improvements, reaching up to +21.48% macro-F1 in code classification and +15.04% F1 in clone detection for specific models under concatenation-based aggregation. Finally, a human evaluation with two annotators on 100 Java snippets (rank-ordering with ties evaluation) showed that context-augmented summaries were ranked significantly higher than summaries from models that rely on source code only for Accuracy and Content Adequacy (Wilcoxon \(p\leq 0.026\) ; Cliff's \(\lvert\delta\rvert\) up to 0.55), while improvements in Conciseness were context-dependent; inter-rater agreement was high (Kendall's \(\tau_{b}\) ). Our study highlights the potential benefit of utilising rich and diverse context to enhance the code comprehension capability of neural-based models. An exploratory evaluation on vulnerability detection provides initial evidence that context-enriched representations transfer to security-oriented tasks. It also opens new research directions for optimising contextual encoding strategies in software engineering, for instance automated bug localisation, where call-graph context helps trace fault propagation paths [68].