- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates whether recovered system prompts can reproduce the behavior of the original LLM application. It proposes a paired diagnostic framework evaluating textual recovery, behavioral gains over blank and random controls, and sensitivity to source-rule modifications. Using a fixed four-query extraction procedure, the authors evaluate 100 controlled prompts and 80 public tests across seven Qwen3.5 and Gemma 4 checkpoints. Additional experiments examine rule deletion and replacement, candidate selection, and scoring sensitivity. Results show that recovered prompts achieve a 14.04% behavioral gain over controls but remain 23.43% below the original prompts. Moreover, improved textual recovery does not consistently translate into better behavioral reuse, and rule-level interventions yield mixed results. The paper highlights the importance of evaluating prompt extraction through behavioral comparisons and rule-specific interventions rather than textual similarity alone.
Critical strengths
- Important distinction between textual recovery and behavioral reuse. The paper highlights that recovering system-prompt text does not necessarily reproduce application-specific behaviors, providing a more practically relevant perspective on prompt extraction risks.
- Systematic framework for evaluating behavioral reuse. The paired diagnostic framework uses blank and random controls to isolate recovery-induced behavioral gains, while source-prompt references quantify remaining gaps. Same-checkpoint comparisons further reduce confounding factors.
- Fine-grained evaluation through rule-level interventions. By deleting or replacing individual source rules and comparing targeted with unchanged-rule probes, the framework examines whether recovered prompts preserve specific behavioral effects beyond aggregate similarity.
- Comprehensive experimental analysis and robustness checks. The evaluation covers multiple model families and sizes, reports confidence intervals, and examines candidate selection, scoring sensitivity, and response-level traces. The authors also explicitly acknowledge statistically uncertain findings.
Critical weaknesses
- Limited methodological novelty. The proposed framework mainly integrates existing evaluation techniques, including textual similarity metrics, control-based behavioral comparisons, and rule-level interventions. While their combination is useful, the paper does not clearly establish what fundamentally new methodological insights it offers beyond applying these techniques to prompt recovery.
- Insufficient evaluation against stronger prompt extraction methods. The experiments rely on four fixed, nonadaptive extraction queries and simple candidate selection strategies. Without comparisons against more advanced methods such as PLeak and PRSA under comparable budgets, it remains unclear whether the observed behavioral gaps reflect inherent limitations of prompt recovery or the limited effectiveness of the chosen extraction procedure.
- Limited validity of behavioral reuse measurements. The evaluation primarily relies on predefined rules and narrow, often heuristic scoring criteria. In particular, each controlled prompt contains only three rules, with a single probe per rule. Such evaluations may not adequately capture semantic equivalence or more complex application behaviors, raising concerns about whether the reported gains accurately reflect meaningful behavioral reuse.
- Insufficient evidence from rule-level interventions. The rule deletion and replacement experiments are limited to two checkpoints, and only two of the four primary comparisons show statistically significant positive effects after multiple-comparison correction. Furthermore, the unchanged-probe-adjusted replacement effects remain inconclusive. These mixed results weaken the evidence that recovered prompts consistently preserve the behavioral effects of individual source rules.
Questions for authors
- What are the key methodological contributions of the proposed framework beyond combining existing evaluation techniques?
- How would the findings change when using stronger prompt extraction methods, such as PLeak or PRSA, under comparable query budgets?
- To what extent can the current rule-based probes and heuristic scoring criteria reliably capture meaningful behavioral reuse beyond simple instruction-following tasks?
- Given the limited checkpoint coverage and mixed statistical results, what additional evidence supports the effectiveness of rule-level interventions in assessing functional recovery?
3 comments
Thank you for the author's detailed response and additional experiments. The paper provides a large number of experimental results and valuable observations, but it has not fully elucidated which research findings can be generalized to different experimental conditions, and how to use these comprehensive findings to guide future prompt recovery research.
Thank you for the follow-up and for asking us to clarify the framework’s practical use.
To apply it to a new checkpoint, extractor, or workflow, we suggest evaluating the source, recovered, blank, and length-matched random prompts on the same checkpoint and held-out tasks within each setting. Matching query budgets and output caps makes extractor comparisons interpretable, while fixing tools and initial state serves the same purpose for workflow comparisons. The gain over controls and the remaining gap to source performance then show how much useful behavior the recovered prompt retains.
For query-efficient recovery, researchers can tune candidate selection and stopping rules on development data, then evaluate the fixed strategies on held-out tasks across several low query budgets. This lets them report the smallest tested budget that meets a prespecified behavioral criterion, such as adherence to a critical rule, alongside the number of generated tokens and the attacker’s computational cost.
Thank you again for your feedback, and please let us know if we can clarify anything further.
Thank you. We added method comparisons, qualified semantic scoring and broader rule interventions.
1. Methodological contribution. We credit prior functional evaluation in output2prompt and PRSA. Our contribution joins three paired references: recovered-minus-control measures added tested utility, source-minus-recovered measures remaining behavior and source edits followed through extraction and redeployment test instruction responsiveness. Unchanged-rule probes track accompanying changes, and direct-source responses provide a reference.
G+Dsrc=E[T−B] explains why positive reuse and a source gap coexist. Interventions describe the edited recovery process and rule-specific interpretation requires comparable nonspecific edit effects across focal and unchanged probes.
2. Changing the extractor. We compare PRSA-local with official pretrained output2prompt and disclosure-first on five targets, matching four target acquisition calls with 256-token caps. Controlled differences are PRSA-local minus comparator, in percentage points (pp):
Intervals use 10,000 paired prompt-cluster resamples and are pointwise. On public prompts, PRSA-minus-disclosure ranges from -22.41 to +2.73 pp and only Gemma E4B's CI excludes zero. All five public comparisons with output2prompt include zero. Qwen27's comparator subsets differ. Appendix S details the separate 1,024-token condition, adaptation and offline/attacker costs.
Conclusion: the supported changes in both directions on Gemma E4B show that extractor effects depend on the comparator. Positive control gains and source gaps coexist for PRSA-local on common controlled subsets. Thus, joint diagnosis extends across tested methods and the four-query findings describe specific methods/budgets.
3. Meaningful behavior and semantic scoring. E3 adds paraphrases and context/condition checks. Local τ2-bench retail/airline workflows hold tools/state fixed: 1,928/2,160 primary test endpoints are valid and saved-output recovery adds 177 separate sensitivity endpoints (Appendix V). This extends coverage to task execution and tool-protocol failures. Rule adherence and task completion are distinct outcomes.
Qwen27/Gemma31 scored the same 720 Mistral answers from 20 controlled and 20 public prompts in a blinded evaluation. Both passed frozen 90%-accuracy/99%-validity anchor gates. Each returned 720 valid outputs: numeric scores 684/669, undetermined 36/51. On 669 jointly scored answers, Gemma31-minus-Qwen27 is +4.97 pp, mean absolute difference 6.79 pp, exact agreement 616/669. Binary agreement is 829/885 (93.67%), Cohen's κ=0.864 and 51/936 annotations are unresolved.
This adds semantic evidence: most decidable judgments agree, while mean scores depend on the evaluator. Appendix T keeps the exploratory 7,953-rating post-outcome 75/99 analysis separate.
4. Rule-intervention evidence. Six targets and 100 prompts yield 7,200 extraction and 57,600 behavioral outputs. Original-probe deletion specificity is 10-21.5 pp and adjusted replacement is 10.5-27 pp. Six of twelve secondary comparisons survive Holm correction, extending detectable edit responses beyond the original two checkpoints.
Qualified Qwen27 single-judge scoring covers 20 prompts on Gemma E4B/Gemma12/Mistral24: 5,155/5,760 ratings returned, 5,040 numeric, 115 undetermined, 605 missing. Complete recovered-path pairs yield:
Gemma E4B has only 1/3 pairs (descriptive and Appendix U also gives direct-source references). Pointwise CIs use 10,000 prompt-cluster resamples missingness is not zero-filled. Mistral's positive adjusted replacement supports an edit response through recovery beyond one probe wording. Deletion remains uncertain on a different paired subset. This identifies supported and unresolved rule responses and independent recomputation verified both supplements.