acceptodds
Withdrawn from ICLR 2027

Security-Constrained Prompt Optimization Under Partial Observation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

This paper develops a fixed-budget best-prompt identification algorithm for selecting the highest quality prompt of a given task for large language models (LLMs), under the constraints of various quality and security measures. Specifically, the selected prompt needs to satisfy a security constraint against attacks such as indirect prompt injection, which inserts malicious instructions in documents to redirect a language model away from its intended task. The inclusion of the security constraint fundamentally changes prompt evaluation, as an attacked response does not estimate clean-task performance, and a clean response provides no evidence of security. Thus each pull reveals only part of the objective vector. The resulting security-induced partial observation differs from conventional best-arm identification, where each pull reveals all objectives. We formulate this setting as a fixed-budget best-feasible prompt identification problem, and solve it by using a new Guaranteed-Floor Constrained Successive Rejects (GF-CSR) algorithm. GF-CSR adopts a reward-independent two-phase schedule that front-loads attacked pulls until surviving prompts obtain a prescribed security-evidence floor, then shifts sampling toward clean objectives. Under bounded independent rewards and explicit separation conditions, we prove that GF-CSR identifies the unique best feasible prompt with high probability while accounting for bias from incomplete cycles through the attack suite. Experiment results on real-world datasets with indirect prompt injection attacks demonstrate that GF-CSR outperforms baseline algorithms such as partial observation CSR (PO-CSR) and the Uniform algorithm.

Primary area:
reinforcement learning
Keywords:
Prompt optimization, best arm identification, indirect prompt injection attacks

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ZKvjOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies fixed-budget prompt selection under security constraints. The selected prompt should satisfy a security constraint against attacks such as indirect prompt injection to select the highest-quality prompt for a given task. The paper proposes the GF-CSR algorithm, which combines successive elimination with a deterministic allocation schedule that prioritizes security observations early before returning to a lower attack-evaluation rate. The paper provides a sufficient condition for identifying the best feasible prompt under the security constraints. Comprehensive evaluations are performed on two summarization datasets to show that GF-CSR outperforms baseline algorithms such as partial observation CSR (PO-CSR) and the Uniform algorithm.

Critical strengths
  1. The paper targets one important problem. The paper identifies a useful and practical gap between security and task quality. An attacked response fails to estimate clean-task performance, while a clean response provides no evidence of security.

  2. The proposed methodology is well formulated and straightforward to implement. The analysis explicitly accounts for the bias introduced by incomplete attack cycles. The paper includes a theory section that is at least aligned with the algorithmic design.

Critical weaknesses
  1. The evaluated tasks on abstractive news summarization are rather outdated. The ROUGE-Lsum F1 is also not a representative utility metric for existing LLMs. I would like to see results on more challenging and interesting tasks.

  2. The experiments do not isolate the contribution of the two-phase allocation schedule. The main contribution is the front-loaded security-evaluation policy, rather than the successive-rejection backbone. However, the paper does not compare it with a constant-rate policy matched for the total number of attacked evaluations, or examine sensitivities.

  3. A lot of setups and details are missing. For example, Theorem 1 requires the feasibility and separation conditions in Equations (21)–(22) to hold in every round. But the paper does not give an explicit budget requirement or explain how to choose T0, ρfloor, and ρbase to satisfy them. Also, the selection of key experimental parameters is insufficiently explained and compared.

  4. The empirical evaluation provides limited evidence about when the allocation strategy is effective. Section 7 leaves unclear whether GF-CSR’s advantage extends beyond this particular task and evaluation setup. Since the proposed contribution is an allocation strategy, the key question is whether its gains persist when the relative difficulty of estimating task quality and security changes. Experiments on varying tasks, target models, or attack suites would be helpful.

Questions for authors
  1. Under what conditions does GF-CSR outperform PO-CSR? In particular, do the gains persist when the relative difficulty of estimating task quality and security changes?

  2. How does GF-CSR compare with a constant-rate policy matched for the total number of attacked evaluations?

  3. For the reported experimental settings, are the conditions in Equations (21)–(22) satisfied? If not, what budget would be needed for the guarantee to become informative?

  4. How were the allocation parameters and feasibility thresholds selected? Are they fixed independently of the evaluation reward tables? Does the relative performance of GF-CSR and PO-CSR remain stable under alternative settings, particularly on XSum?

Reviewer_e2qLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies security-constrained prompt selection under a fixed evaluation budget, where clean and attacked evaluations reveal different objectives. The authors propose GF-CSR, a two-phase successive-rejection algorithm that prioritizes security evaluation before shifting toward task quality estimation. They provide a finite-sample identification guarantee under separation conditions and evaluate the method on CNN/DailyMail and XSum. Experiments show that GF-CSR generally outperforms PO-CSR and Uniform sampling, with additional held-out evaluations supporting the reported improvements.

Critical strengths
  1. Meaningful problem formulation. Making attack resistance a hard feasibility condition rather than a compensatory reward is well motivated: a prompt that refuses to do the task should not earn its way into selection merely by resisting injections. The clean/attacked feedback distinction also exposes a genuine evaluation-budget allocation issue (Sections 3–4).
  2. Careful experimental measurement for the stated setting. The method combines deterministic observation-type counts with an explicit correction for incomplete attack cycles (Lemma 1 and Section 6). The experimental design includes matched simulation seeds, disjoint confirmation documents, paired method comparisons, and a documented compliance-judge audit (Section 7; Appendix C.3). These are useful checks, even though they do not resolve the concerns below.
Critical weaknesses
  1. The main theorem does not certify the budgets where empirical improvements are reported. Theorem 1 requires its separation certificates (21)–(22) in every elimination round. For the principal experiment, K=30, B/K=20 (B=600), T0=8, and rho_floor=0.75. Using the paper's own successive-rejects schedule (Appendix A.1), the first round has N1=6 pulls per prompt: only q1=4 attacked and u1=2 clean observations. The security target is not reached until t*=ceil(8/0.75)=11 pulls, so the first elimination can happen before the proposed floor is attained. More importantly, at delta=0.05, the clean confidence radius in (20) at round 1 is approximately 1.70 (the security radius is approximately 1.54). All rewards lie in [0,1], and the clean brevity feasibility margin with tau_2=0.50 is at most 0.50, so condition (21) cannot hold. The same difficulty arises across the B/K=10–25 budgets shown in Figure 1. This does not invalidate the conditional proof, but makes the principal recovery theorem vacuous for the reported operating regime, substantially weakening its explanatory and practical significance.
  2. The empirical comparison does not isolate the claimed mechanism and tests a narrow security regime. GF-CSR uses a 0.75 attacked-pull rate before its switch, whereas PO-CSR and Uniform use Bernoulli attack sampling at 0.30 throughout (Section 7.1; Appendix A). Thus the improvement may reflect a different early clean/security allocation rather than the deterministic evidence floor per se. There is no deterministic constant-rate CSR control, randomized two-phase control, matched total attacked-pull allocation, or sensitivity study for T0/rho_floor/rho_base. The held-out GF-CSR versus PO-CSR paired interval on XSum also crosses zero (Figure 2), so that comparison is not statistically resolved. Finally, all security tests use six fixed, benign, non-adaptive appended instructions, two summarization datasets, and a single target model, making broad prompt-injection robustness claims difficult to infer (Section 7; Appendix C.1).
Questions for authors
  1. Can the authors quantify the smallest budget at which conditions (21)–(22) are simultaneously non-vacuous for their measured gaps and thresholds? In particular, how should the claimed “guaranteed floor” be understood when the first elimination at B/K=20 occurs at N1=6 < t*=11, and can a finite-budget analysis explain the observed gains at B/K=10–25?
  2. Can the authors compare GF-CSR against (i) deterministic interleaving at a constant, tuned attack rate, (ii) a randomized two-phase policy using the identical expected attack allocation, and (iii) a PO-CSR baseline matched for the total or round-wise number of attacked pulls? Please report how conclusions vary with T0, rho_floor, and rho_base. These controls would determine whether the “floor” itself drives the gain.
  3. Do the selected prompts remain feasible when evaluated against attack instructions held out from the six-attack suite, or against attacks inserted at other locations in the untrusted source rather than appended to it? Please clarify whether the advertised security benefit is intended to extend beyond the six fixed, prompt-agnostic instructions and the one target model used here.
Reviewer_GMeeOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies security-constrained prompt optimization under partial observation. The goal of this paper is to select the best prompt that satisfies both task quality and security requirements within a fixed evaluation budget. The paper proposes GF-CSR, a two-phase sampling algorithm that first prioritizes security evaluations and then allocates more budget to clean task evaluations. The paper provides theoretical guarantees for the proposed algorithm and evaluates it on two summarization datasets, showing improvements over existing baseline methods.

Critical strengths
  1. The paper introduces an interesting formulation of security-constrained prompt optimization under partial observation.

  2. The proposed GF-CSR algorithm provides a simple two-phase sampling strategy to allocate the evaluation budget between clean and attacked inputs.

  3. The paper provides theoretical guarantees for the proposed algorithm and demonstrates improvements over baseline methods on two summarization datasets.

Critical weaknesses
  1. The evaluation is limited to a single task scenario. The paper only evaluates news summarization on CNN/DailyMail and XSum using one target LLM and a fixed set of six attacks. It remains unclear whether the proposed algorithm can achieve similar improvements on other tasks, models, or more challenging prompt injection attacks.

  2. The practical benefit of the proposed prompt selection method is unclear. The paper shows that GF-CSR achieves higher strict rewards than the baselines, but this metric assigns zero reward whenever the selected prompt violates a constraint. It is therefore unclear how much improvement comes from selecting better prompts rather than simply avoiding infeasible ones. A more direct comparison of the selected prompts' task quality and security performance would be helpful.

  3. The evaluation of computational cost is incomplete. Although the paper compares different algorithms under the same sampling budget, it does not report actual inference time, GPU hours, or the additional cost of using an LLM judge for security evaluation. Given that the proposed method requires repeated clean and attacked evaluations across multiple candidate prompts, a more detailed cost analysis would help assess its practical value.

  4. The security evaluation does not consider task utility under attack. The security reward only measures whether the model follows the injected instructions, without checking whether it still completes the original task. As a result, a prompt that refuses to answer when an attack is detected may receive a high security score even though it fails to perform the user's task. This could lead to selecting overly conservative prompts.

  5. The paper does not evaluate the selected prompts against unseen or adaptive attacks. The security score is calculated using only six fixed prompt injection attacks, and the attacker does not adapt to the candidate prompts. As a result, the selected prompt may perform well against these specific attacks but remain vulnerable to other attack strategies. It is unclear whether the selected prompts can maintain their security performance when attackers use new attacks or adapt their attacks to the selected prompt.

  6. The paper lacks an ablation study on the key hyperparameters of GF-CSR. The authors set the attack sampling rates to 0.75 and 0.30 for the two phases, with a security observation target of 8, but do not explain why these values were chosen. It is unclear how sensitive the results are to these settings or whether other configurations could achieve better performance. An ablation study with different sampling rates and security targets would help justify the proposed design.

Questions for authors

Please refer to the weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.