STALL + : Boosting LLM-based Repository-level Code Completion with Static Analysis

Repository-level code completion refers to the challenging completion scenario involving complicated contexts from multiple files in a repository. To date, researchers have proposed two technical categories to enhance LLM-based repository-level code completion: retrieval-augmented generation (RAG) and static analysis integration. This work performs the first study on static analysis integration in LLM-based repository-level code completion by investigating integration strategies across different phases. We first implement a framework STALL + , which supports extensible, customizable integration of multiple static analysis strategies into the full pipeline of LLM-based repository-level code completion. Based on STALL + , we conduct extensive experiments including different LLMs on the latest repository-level code completion benchmark CrossCodeEval. Our findings show that integrating file-level dependencies in the prompting phase performs best, while post-processing phase integration performs worst. Additionally, we observe different improvements between dynamic and static languages, i.e., the best combination is prompting-phase with decoding-phase integration for Java, and prompting-phase with post-processing-phase integration for Python due to limitations in statically analyzing dynamic languages. We also find complementarity between RAG and static analysis integration, and that prompting-phase integration is more cost-effective than RAG.