Optimizing Decompiler Output by Eliminating Redundant Data Flow in Self-Recursive Inlining

Decompilation, which aims to lift a binary to a high-level language such as C, is one of the most common approaches software security analysts use for analyzing binary code. Recovering decompiled code with high readability is essential, as humans must understand the code's functionality correctly. However, some compilation optimization strategies will introduce obfuscation into the binary code, thereby reducing the readability of decompiled code. Among them, the function inlining related optimization strategies combine functions, causing the original function's code volume and complexity to multiply. Especially with self-recursive inlining optimization, it transforms initially simple functions into ones with significantly increased code volume and complex logic, greatly hindering the understanding of security engineers. In this paper, we present Erase, the first approach to reverse the self-recursive inlining optimization technique. We compare Erase with state-of-the-art decompilers Ghidra and Hex-Rays to evaluate ERASE's improvement for the functions affected by self-recursive inlining. Experimental results show that Erase's output is 78.4% and 88.9% more compact (fewer lines of code) than Ghidra and Hex-Rays, respectively. Moreover, reverse engineers spend 88.5% less time analyzing ERASE's output than analyzing Ghidra and 90.4% less time than analyzing Hex-Rays, and the accuracy of analyzing Erase's output is 2.75 times higher than both Ghidra and Hex-Rays.