Oracle-Guided Vulnerability Diversity and Exploit Synthesis of Smart Contracts Using LLMs

Many smart contracts are prone to exploits, which has given rise to analysis tools that try to detect and fix vulnerabilities. Such analysis tools are often trained and evaluated on limited data sets, which has the following drawbacks: 1. The ground truth is often based on the verdict of related tools rather than an actual verification result; 2. Data sets focus on low-level vulnerabilities like reentrancy and overflow; 3. Data sets lack concrete exploit examples. To address these shortcomings, we introduce XploGen, which uses a model-based oracle specification of the business logic of the smart contracts to synthesize valid exploits using LLMs. Our experiments, involving 104 synthesized vulnerability-exploit pairs, demonstrated a 57% success rate in exploiting targeted aspects of the contract. They achieved exploit efficiency with an average of only 3.5 transactions per exploit, highlighting the effectiveness of our methodology.