- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a reverse engineering pipeline that decompiles neural-network binaries in unstripped and stripped setup. The model in this work is XGBoost model trained on assembly features with a LoRA-tuned Qwen2.5-3B-Instruct model that outputs structured chain-of-thought rationale. Authors made a benchmark of 400 compiled Binaries across 3 compilers and different architectures. The evaluation in the paper shows that this model can outperform the frontier model in identifying operators and hyperparameter estimation and topology reconstruction.
Critical strengths
The paper introduces an evaluation dataset consisting of 400 compiled binaries. This diversity provides an evaluation baseline rather than for evaluating a single case. Also, The evaluation setup is systematic, including different optimization and model families.
This work also considers the obfuscation case, and the training incorporates the part obfuscated with NeuroShield.
The work attempts to use LLMs with finetune approach to address a valuable question on neural network decompilation.
Critical weaknesses
The margin of improvement compared to the base Qwen implementation is low. Shown in Table 5 and Table 2a, Qwen reaches 78.1% accuracy under obfuscation, whereas the full framework achieves 79.7%, marking a ~2 percentage point increase on both cases. The ablation study does not work in favor of the approach this work proposes, and the proposed finetune and approach is not well justified for its performance gain.
The baseline models in Table 1 are not using sota frontier models, and the reference to Opus lacks version details required for reproducibility. In addition, a comparison of prior neural network decompilers is not presented.
The system does not function as a full neural-network decompiler. As shown in Appendix B, its outputs are operator labels, architecture families, and topology graphs. Prior work such as DnD and BTD provide stronger capabilities, recovering complete model specifications and evaluating executable reconstructions within their supported frameworks.
Questions for authors
Model used in this work (Qwen2.5) and other evaluated model are relatively outdated, dating from 2 years to 1 years old, and models listed does not label their generation. As frontier models such as GPT6 and Fable are advancing rapidly, is it possible to show that either this approach also holds for the recent models, or update the evaluation of more recent frontier models?
The ablation experiment shows a small improvement compared to vanilla Qwen2.5, can you elaborate more on the motivation?
7 comments
Thank you for the clarifications. The explanation that the Qwen results refer to the fine-tuned reasoner reduces my concern about the small reported improvement. However, my concern about the age of the evaluated models and main motivation remains unresolved. A model-agnostic design does not establish that the reported advantages persist with newer models. I understand the cost constraints, but a limited comparison against a recent strong baseline would help address this.
I will raise my score from 1 to 2 given the clarification of W3, but W1/W2 are not convincing enough in my view.
We thank the reviewer again for their time.
R1/W1: Motivation for the finetuning
- Frontier models given the same prompt, including the same prior, score 8.7 - 19.5 points below NeuroProbe under obfuscation (Table R1); off-the-shelf LLMs often override a correct prior with wrong guesses. NeuroProbe's training closes most of that gap with a 3B model.
- Auditable predictions. Every prediction carries a structured rationale (evidence, constraints, hypothesis, self-check), and our verifier checks each cited value against the prompt.
Why Qwen2.5-3B. (i) It was pretrained on a large mix of code and mathematical data, so instruction mnemonics, register names and numeric feature ratios are in-distribution. (ii) Its open weights permit research fine-tuning.
R1/W2: Comparison against current frontier models
We agree that comparing against current frontier models gives a more up-to-date picture of NeuroProbe's performance, so we have evaluated GPT-6 Astra, Claude Fable 5.2, and Claude Opus 5.5 alongside the baselines in the submission.
**Evaluation protocol : ** All models are evaluated on the identical held-out kernels. Each uses the same four-block prompt (FUNCTION, ASM/BYTES, PRIOR, CONTEXT), the same 28-label operator vocabulary, and greedy decoding. The comparison is therefore controlled for input information and output space.
Table R1. Per-kernel Operator F1 (%).
| Model | Clean compilation | Obfuscated |
|---|---|---|
| NeuroProbe | 99.7 | 79.7 |
| GPT-6 Astra | 89.8 | 71.0 |
| Claude Fable 5.2 | 88.8 | 69.5 |
| Claude Opus 5.5 | 88.5 | 69.3 |
| Claude Opus 4 (prior baseline) | 86.5 | 67.1 |
| GPT-4o | 79.6 | 60.2 |
| Hopefully this clarifies point 2. Please let us know if any questions remain. |
We thank the reviewer for a sharp review.
W1 / Q2: The margin over Qwen
The "Qwen" column in Tables 2a and 5 is not a plain Qwen model. It is the fine-tuned reasoner trained with NeuroProbe's own components (name-blind prior, leak-free context, SCA-CoT, Prior Dropout, SFT and GRPO), scored before fusion, so these tables isolate only the effect of fusion. The "Zero-Shot" label in Table 5 is wrong, and we include this correction in the revised version. The framework's gain over unadapted models is in Table 1: under obfuscation NeuroProbe beats the best zero-shot frontier model by 12.6 Operator F1 and 30.9 Exact points.
On clean binaries, fusion cuts the error from 2.5% to 0.3% (97.5 to 99.7 Operator F1, an 88% relative reduction) and beats both components (prior alone 99.2). Under obfuscation it raises Operator F1 from 78.1 to 79.7, and per-kernel family accuracy is 95.8% vs. 95.0% for the prior alone.
The prior alone reaches 81.0 Operator F1 under obfuscation, which we consider the paper's main security finding: stripping identifiers does not protect a compiled model. With every symbol name removed, it recovers 83.7% Operator F1 (80.5% Exact) and 94.3% family accuracy (52.9% majority baseline) on the full test set, and 76.6% on TensorRT plans with no disassembly. We include this result first in the revised version. The LLM's role on top is auditability: each prediction carries a rationale whose evidence is checked against the prompt.
W2 / Q1: Frontier models and version details
The Opus model in Table 1 is Claude Opus 4. On model choice, the very newest frontier models (e.g., Fable) are expensive to run at the scale of our evaluation, so we evaluated on lighter models: Claude Opus, GPT-4o, DeepSeek and Llama 4 Maverick. The pipeline is model-agnostic (prompt format, SFT targets and reward do not depend on Qwen2.5, and the comparison models are only queried zero-shot), so the conclusion does not hinge on a particular model, and newer models can be substituted directly.
W3: Not a full decompiler
We respectfully disagree that prior work provides stronger capabilities in the settings we target. DnD and BTD are strong within a narrow scope: unobfuscated CPU executables of supported CNN operators. Even there, coverage is partial: DnD supports 59 of 70 ONNX operators and cannot represent recurrent structures, and BTD reports correctly recovering 59 of 65 executables and notes that it does not guarantee correct output. Outside that scope they do not apply: both are CPU-only, so GPU engine plans such as TensorRT are out of reach, neither reports transformer results, and in our Table 3b BTD fails on all three transformer models. Both also assume unobfuscated binaries. Other work on recovering DNN structure from executables (e.g., Libsteal, Neuroscope) and general-purpose LLM binary-analysis tools such as ReCopilot, LLM4Decompile and ReSym address a different task (function names, types and source-level code, not DNN operators), so they are not directly comparable. NeuroDeX, the closest LLM-based work, covers CNNs only and does not evaluate obfuscated binaries.
NeuroProbe works across exactly the regimes where these tools do not: TensorRT plans with no disassembly, Transformer architectures, and binaries with stripped identifiers. From further prompting it can produce much detailed information and full model recovery without pretraining.
We thank the reviewer again; their point has made the paper's contribution considerably clearer. We are happy to answer any further questions.