- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a benchmark for black-box LLM extraction under text-only API access. It is organized around four "lifecycle" stages: query acquisition, victim interaction, surrogate training and evaluation.
Critical strengths
-
A real problem and potentially useful infrastructure. Extraction attacks and defenses are currently evaluated under incompatible victims, budgets, data and metrics. The paper reimplements 18 methods with a shared query pool, shared victim transcripts, held-out evaluation and released run manifests. That engineering effort could serve as a starting point for a more relevant benchmark.
-
A sound methodological stance on defenses. The authors do not collapse heterogeneous defenses into a single score. Each defense family gets its own objective-specific metric, paired with the utility of the resulting surrogate. Adaptive rewriting is evaluated after surrogate training rather than only on the rewritten text. Both choices address real weaknesses of prior evaluations.
Critical weaknesses
The threat model targets last-generation chat models, not the reasoning-model distillation that defines LLM extraction risk in 2026. Both victims are non-reasoning instruct models (Llama-3.3-70B-Instruct, Qwen2.5-72B-Instruct). Generation is greedy with at most 1536 new tokens, so long chain-of-thought traces cannot fit. Today, extraction in practice means distilling reasoning traces from frontier reasoning models.
As a result, it is unclear whether any headline finding transfers to the setting that practitioners and defenders currently care about. A benchmark paper's contribution is to serve as a reference point, so this limits significance more than any single implementation issue.
Questions for authors
-
Have you run any part of the benchmark with a reasoning victim (an open-weight model emitting long CoT), and with CoT-hidden or CoT-summarized API variants? Do the anti-distillation and provenance conclusions hold there?
-
Trace Rewriting and Antidistillation Sampling were proposed for reasoning traces. What justifies evaluating them on short chat responses?
5 comments
Thank you for your detailed response. The additional QwQ experiments partially address my concern, and I appreciate the effort.
My main remaining concern is generalization to frontier models on agentic tasks. Evaluating the extraction method with frontier models such as GLM-5.3 and DeepSeek-v4-flash on agentic benchmarks such as Terminal-Bench would show whether the approach holds beyond the current model family and in realistic, multi-step settings. I think this evidence would substantially strengthen the paper and would be the key factor in raising my assessment.
I understand that such experiments are resource-intensive and may not be feasible within the revision period. Even so, I cannot raise my score based on the current version. I would be glad to reconsider if the authors can provide these results, or at least a limited-scale pilot on a subset of Terminal-Bench tasks, with a clear discussion of the remaining limitations.
Follow-up to Reviewer VDBk
Thank you for your follow-up and for suggesting a limited-scale pilot on Terminal-Bench. We have completed an additional experiment with DeepSeek-V4-Flash as the victim and report the results below.
Extraction from a frontier victim on agentic tasks
Answer
We evaluated SeqKD, SODA, and LoRD with DeepSeek-V4-Flash-0731 as the victim and Qwen3.5-9B as the initial surrogate on 20 fixed Terminal-Bench 2.1 tasks. This extends the evaluation to a new victim–surrogate pair and multi-step terminal interaction. All three attacks reuse victim trajectories from 96 independent synthetic training exercises, covering file processing, code repair, algorithm implementation, and numerical tasks. The victim solved all 96 exercises, yielding 493 assistant turns. SODA starts from the SeqKD checkpoint, while LoRD starts from the initial surrogate.
Evidence
All five models use Terminus 2 through Harbor, with the same evaluation tasks and limits. Each task receives one attempt, with temperature 0, a 16384-token context, up to 20 interaction rounds, and at most 2048 generated tokens per round. Success is determined by each task's verifier.
| Model | Tasks passed | Success rate | Success-set overlap with victim |
|---|---|---|---|
| Victim (DeepSeek-V4-Flash) | 7/20 | 35% | — |
| Initial surrogate (Qwen3.5-9B) | 4/20 | 20% | 4/7 |
| SeqKD | 1/20 | 5% | 1/7 |
| SODA | 3/20 | 15% | 3/7 |
| LoRD | 0/20 | 0% | 0/7 |
Success-set overlap is the Jaccard similarity between the model's and victim's sets of passed tasks. Here, every task passed by a surrogate was also passed by the victim, so the union contains seven tasks.
Task performance and agreement. None of the attacks improves aggregate success or success-set overlap over the initial surrogate. SeqKD loses three previously passed tasks and gains none. SODA newly solves cobol-modernization, which the victim solves and the initial surrogate fails, but loses two previously passed tasks. LoRD training causes output degeneration into repeated newlines, preventing valid agent actions. Its 20 tasks end in 15 timeouts and five context-limit failures.
These results show that training on successful victim trajectories does not ensure improved multi-step task completion under this setup. SODA transfers one additional successful outcome while also losing existing successes.
Limitations. This pilot covers one model pair and 20 tasks with one attempt each. The victim and initial surrogate exhaust the context on 11 and 14 tasks, respectively, so the results reflect the specified interaction limits. The training exercises are independently constructed and simpler than the evaluation tasks. The experiment provides evidence about extraction under this configuration; broader task coverage, repeated attempts, and larger interaction budgets would be needed to assess generality.
Manuscript changes
We added Appendix E.2 with the experimental setup, aggregate and per-task results, success-set overlap, and limitations (Tables 10 and 11).
Response to Reviewer VDBk
Thank you for raising the question of whether our conclusions extend to reasoning models.
W1 / Q1. Extraction from a reasoning victim
Answer
We evaluated six response defenses against SeqKD using QwQ-32B as the victim and Qwen2.5-7B as the initial surrogate. We trained on full visible responses to the same 1000 queries (B=1000) in each condition. We increased the generation limit from 1536 to 8192 tokens for QwQ-32B, and 26 of the 1000 clean responses reached the new limit.
Evidence
Anti-distillation. Clean SeqKD raises macro-average accuracy from 58.04% to 58.95%. ADS yields the largest accuracy drop relative to clean (0.45 percentage points) and reduces final-answer BERTScore F1 from 0.0395 to 0.0051. Under ADS, 915 of the 1000 victim training responses reach the generation limit, and none contain a final answer. DOGe and Trace Rewriting each lower accuracy by at most 0.16 points and BERTScore F1 by at most 0.0022. The clean surrogate emits no explicit reasoning block on the evaluation prompts, so these comparisons measure changes in task performance and final-answer fidelity.
Provenance. We probe each protected surrogate and its matched clean control with the same 1000 extraction queries.
ADFP stays close to zero and Radioactivity yields a modest decrease in p-value. GINSEW shows the largest relative change on its own scale, with a protected score about 420 times its clean score.
Manuscript changes
We placed the QwQ response-defense evaluation in Appendix E.1, which reports the evaluation setup, full surrogate results, and paired provenance scores.
Q2. Why evaluate ADS and Trace Rewriting on chat responses?
Answer
ADS adjusts the next-token distribution during decoding, and our Trace Rewriting implementation rewrites the full response (Appendix B.3). Both can operate on ordinary chat responses without an explicit reasoning trace. Such responses are relevant to extraction because an API may expose only the final answer.
Evidence
In our ordinary chat-response experiments, ADS lowers surrogate accuracy by 3.51 percentage points for QEDKS and 3.07 for LoRD, and lowers BERTScore F1 for both. Trace Rewriting lowers accuracy for SeqKD, QEDKS, and SODA and raises it for Model Leeching, LoRD, and GAD (Fig. 2 and Table 13). These results show that the effect of each defense on surrogate performance varies across attacks in this setting. In the QwQ-32B experiment with visible reasoning traces, both defenses change SeqKD accuracy by less than half a point. ADS produces the largest drop in final-answer fidelity, and Trace Rewriting shortens the average training response from 1891 to 1365 tokens with little change in fidelity (Appendix E.1).
Manuscript changes
We have clarified in §3.2 that the ordinary chat-response experiments test whether ADS and Trace Rewriting also suppress extraction in that setting. Appendix E.1 reports the results for both defenses in the visible-trace setting.