acceptodds
Under review as a conference paper at ICLR 2027

LLMs for Binary Decompilation: Can Large Language Models Reconstruct Source Code from Compiled Binaries?

Abstract

Software decompilation in which experts reconstruct the original source code from a compiled binary file is essential for software security analysis. Given the exponential growth in coding abilities of Large Language Models (LLMs), there is growing interest in using them for decompilation. This growth has been fueled by advancements in reinforcement learning (RL) and test-time scaling. However, whether these strategies can help LLMs in reconstructing source-like code from compiled binaries remains under-investigated. To explore this question, we first present DecompBench, a dataset of 660k paired decompiled and source functions from over 2k C/C++ projects in the Gentoo package tree. We use the training split of this dataset for supervised fine-tuning, followed by RL with a reward based on control-flow graph edit distance (GED). On held-out test packages, the RL-trained model achieves better structural reconstruction quality than supervised fine-tuning. Further analysis on test-time scaling shows that sampling from SFT achieves lower oracle minimum GED at larger sampling budgets. Under a fixed candidate budget, we compare single-checkpoint and cross-checkpoint sampling using a learned selector. Cross-checkpoint sampling produces better reconstructions relative to single-checkpoint sampling, but learned selection captures only part of the improvement available in the candidate pools. We observe that better individual reconstructions do not necessarily yield better test-time scaling, and highlight the importance of evaluating candidate generation and selection separately. We release our code, data, and trained models to support future research.

Primary area:
datasets and benchmarks
Keywords:
llms for binary decompilation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_UG6LOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigated whether LLM can be finetuned on decompiled code. The authors first prepared a dataset, DECOMPBENCH, consisting 660k source/decompiled code from Gentoo package tree (~2k projects). Then, fine-tune the LLMs LLM with GRPO using a decompilation-quality reward. The results show that

  1. Cross checkpoint gives better reconstruction, but not all samples are gaining improvements
  2. Better individual reconstructions do not necessarily yield better test scaling, and highlight the importance of evaluating candidate generation and selection separately.
Critical strengths

The authors introduced a well-documented dataset, and in this work authors group also by package version to mitigate data leakage between training and evaluation splits. This makes it a valuable data resource for the decompilation.

GED serves as an effective optimization metric for training LLMs on decompilation tasks, minimizing GED indicates generated code better aligned with original source.

Critical weaknesses

Whether LLMs can reconstruct source code from compiled binaries is already recognized feasible and is an active research area. Finetuning LLMs for decompilation has been studied extensively since LLM4Decompile, and other works such as Idioms and SK2Decompile have also explored this direction.

The dataset contribution is also limited. Several datasets and benchmarks already exist for this task, including DecompileBench, Decompile-Bench, and the proposed dataset does not appear novel enough to hold as core contribution.

Reported performance gains are not surprising. Because GRPO explicitly rewards lower GED, observing lower GED after training is the expected outcome, and by itself this provides limited evidence of a novel research contribution. Although the evaluation includes packages unseen during training, gains on those packages are likewise not a significant finding, since the model is still being measured on the same metric it was trained to optimize. In addition, recent LLM decompilers work are less relying on single metrics to measure the output quality including R2I, re-compile, and re-exec over tests, while this paper doesn't measure on other metrics

Ref:

Zou, M., Cai, H., Wu, H., Basque, Z.L., Khan, A., Celik, Z.B., Tian, D., Bianchi, A., Wang, R., & Xu, D. (2025). D-LiFT: Improving LLM-based Decompiler Backend via Code Quality-driven Fine-tuning. ArXiv, abs/2506.10125.

Tan, H., Li, W., Tian, X., Wang, S., Liu, J., Li, J., & Zhang, Y. (2025). SK2Decompile: LLM-based Two-Phase Binary Decompilation from Skeleton to Skin. ArXiv, abs/2509.22114.

Dramko, L., Le Goues, C., & Schwartz, E.J. (2025). Idioms: Neural Decompilation With Joint Code and Type Definition Prediction.

Questions for authors

It would be better to explain how this approach differs from D-lift and include it as a baseline, or discuss the difference/improvements.

Common evaluation includes the re-compile and re-exec as metrics to ensure that LLM will not change semantics of the decompiled program in drastic way pursuing readability or rewards. Is there any measure over this aspect?

There are established readability metrics such as R2I to measure the general decompiled output readability, providing this would help justify the approach’s effectiveness rather than single GED measurement.

Reviewer_jT95OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the use of LLMs for binary decompilation. It introduces DECOMPBENCH, a dataset of over 660K paired decompiled and source functions, and investigates supervised fine-tuning, reinforcement learning using control-flow graph edit distance (GED) as a reward, and test-time scaling. The results show that RL improves single-sample structural reconstruction, while SFT provides more diverse candidates under repeated sampling.

Critical strengths

The paper addresses an interesting problem, and the large-scale dataset is a useful contribution. I particularly appreciate the analysis of how RL affects test-time scaling. The finding that better single-sample performance does not necessarily translate into better results with multiple samples is interesting. The paper also makes a useful distinction between candidate generation and selection.

Critical weaknesses

My main concern is the evaluation setting. The binaries are compiled with -O0 and retain debug information, which is available to the decompiler. This is considerably easier than decompiling optimized or stripped binaries, which are common in real-world reverse engineering. It is unclear how well the findings generalize to these more challenging settings.

Another concern is the reliance on CFG edit distance as the main evaluation metric. A lower GED does not necessarily mean that the reconstructed program is functionally correct. For example, two programs may have identical control-flow graphs but perform entirely different computations. The paper acknowledges this limitation, but does not evaluate functional equivalence or compilation correctness. The reported exact source match rates are also relatively low.

Finally, the benefits of cross-checkpoint sampling appear modest in practice. Table 1 shows that it provides almost no improvement in valid-only mean GED over GRPO sampling, with the improvement in failure-penalized GED mainly coming from fewer scoring failures. This makes the practical benefit of the additional inference cost less convincing.

Questions for authors

How well does the approach generalize to optimized binaries (-O2 or -O3) without debug information?

Can the authors evaluate functional correctness or semantic equivalence, rather than relying solely on CFG similarity?

How does the approach compare with existing learning-based decompilation methods, such as LLM4Decompile and SK2Decompile?

Given the limited improvement from cross-checkpoint sampling, is the additional inference cost justified?

Reviewer_8tFvOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces DECOMPBENCH, a dataset of 660,559 paired decompiled and source functions from 2,032 Gentoo packages, for studying debug-assisted decompilation refinement. It combines supervised fine-tuning with GRPO using a control-flow graph edit distance (GED) reward, improving structural similarity over SFT on held-out packages. The paper also investigates test-time scaling, showing that cross-checkpoint sampling with a learned selector outperforms single-checkpoint strategies on failure-penalized mean GED. Evaluation on Ghidra and angr sailr reveals mixed transfer results, highlighting limitations in cross-decompiler generalization.

Critical strengths

The paper introduces a dataset containing approximately 660,000 paired decompiled and source functions.

It empirically evaluates supervised fine-tuning and reinforcement learning for decompilation refinement.

Critical weaknesses

The paper’s scientific contribution is unclear beyond dataset construction and empirical experimentation. It largely follows a conventional workflow: construct a benchmark, evaluate several training and sampling strategies, and optimize against a chosen metric. While this constitutes a useful engineering effort, the paper does not clearly articulate a novel methodological contribution or yield sufficiently generalizable insights. In its current form, it reads more like a project report than a research paper with scientific contributions.

Questions for authors

Choice of evaluation metric. Why is control-flow graph edit distance selected as the primary evaluation metric and training reward? What advantages does it offer over alternative metrics, and how well does it reflect meaningful improvements in decompilation quality? Since structural similarity does not establish functional equivalence, why not complement it with metrics assessing compilation success and functional correctness?

Need for a new benchmark. What specific limitations of existing benchmarks, such as DecompileBench and decompile-bench, motivate constructing another dataset? The paper should clarify the distinct research need addressed by DECOMPBENCH, particularly given its focus on debug-assisted refinement of binaries compiled at -O0.

Value of specialized model training. Given the strong decompilation capabilities of current Claude and GPT models, what benefits does training specialized models provide? Comparisons with these models would help establish whether the proposed approach offers advantages in reconstruction quality, inference cost, deployment requirements, or another practically relevant dimension.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.