acceptodds
Withdrawn from ICLR 2027

Safe Alone, Unsafe Together: Compositional Curvature Safety Alignment for Prompt Robustness

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Safety-aligned language models are typically evaluated against isolated prompt transformations or sampled attack compositions. We identify a distinct failure mode in which transformations that preserve appropriate behavior individually interact to cross the model's safety boundary. We formalize compositional prompt safety over a lattice of policy-preserving factors and introduce adverse safety curvature, the negative part of the mixed discrete derivative of a sequence-level appropriate-response margin. We show that factor-pair-specific curvature bounds yield a lower bound on every composed margin and a certified composition order on an exhaustively verified finite lattice. Guided by this result, we propose , which discovers violated base-margin, singleton, curvature, and utility conditions and applies minimum-change updates in an editable parameter subspace. During alignment, evaluates curvature contexts of size at most three, requiring lattice nodes only through order five; orders six through eight are reserved exclusively for higher-order extrapolation. Across three open-weight model families and eight prompt factors, of harmful requests that are safe under every singleton transformation fail at one or more test-only orders. Adverse curvature predicts these failures with an AUROC of . Relative to the strongest curvature-hinge baseline, reduces ASR across orders six through eight from to and SCFR from to , while changing benign helpfulness by only points. Pair-specific certification covers of test prompts through order three, and the robustness gains persist under unseen factor realizations, held-out factor families, factor-order permutations, and adaptive post-repair attacks. These results establish adverse safety curvature as a measurable and controllable source of compositional alignment failure.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language model safety, compositional robustness, jailbreak defense, safety alignment, adverse safety curvature, certified robustness, prompt transformations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kWyzOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes C2SAC^2SA, a violation-guided minimum-change algorithm that directly controls the contextual curvature and improves strict higher-order robustness without obtaining safety through broad refusal. Experiment resutls show that C2SAC^2SA reduces ASR across orders six through eight from 22.0% to 18.0% and SCFR from 9.5% to 7.5%, while changing benign helpfulness by only 0.3 points.

Critical strengths
  1. The problem is real and motivated. Solving compositional attacks is important and challenging.
  2. The curvature-based framing of compositional attack offers an interesting theoretical perspective in understanding the mechanism of compositional attack.
Critical weaknesses
  1. The paper is poorly written. Specifically, this paper contains
    1. A lot of undefined variables/terminologies. For example, the meaning of x∅x_{\emptyset} in line 148, C\mathcal{C} in Line 149, LME in line 157. It also does not provide a detailed explanation of why L(x)⊆2[K]\mathcal{L}(x)\subseteq 2^{[K]} in line 152, why ∣S∣≥2|S|\geq2 in line 166 etc. Making it extremely hard to understand the theory in Section 3 and hard to understand the results in the later sections. I also don't understand what the meaning of "policy-preserving" in line 145, since policy can refer to both "language model as the policy model π\pi" and "the safety constraints that the model should follow.
    2. The threat model is never stated, and the theoretical contribution reads hollow. Computing the bound requires margins at all 2K2^K nodes and may be hard for large KK.
  2. The reported numbers seem weird. I don't understand why most of the numbers in Table 1, Table 3, Table 8, and Table 9 end with .0 and .5. Most importantly, in Table 8 and Table 9 (in the supplementary material), for almost every method, the gap between Llama, Qwen, and Gemma is the same. See the table below

Table 8: SCFR 6:8 (%)

MethodLlamaQwenGemmaLlama→QwenQwen→Gemma
Base model18.521.525.1+3.0+3.6
Single-factor tuning16.018.822.2+2.8+3.4
Pairwise training13.516.018.5+2.5+2.5
Random composition11.514.016.5+2.5+2.5
Hard-negative training10.513.015.5+2.5+2.5
ASCOT-inspired9.512.014.5+2.5+2.5
Margin hinge10.513.015.5+2.5+2.5
Active margin projection8.511.013.5+2.5+2.5
Curvature hinge7.09.512.0+2.5+2.5
C2SA5.07.510.0+2.5+2.5

Table 9: ASR 6:8 (%)

MethodLlamaQwenGemmaLlama→QwenQwen→Gemma
Base model37.040.544.3+3.5+3.8
Single-factor tuning34.538.041.5+3.5+3.5
Pairwise training30.534.037.5+3.5+3.5
Random composition27.531.034.5+3.5+3.5
Hard-negative training25.529.032.5+3.5+3.5
ASCOT-inspired24.027.531.0+3.5+3.5
Margin hinge25.028.532.0+3.5+3.5
Active margin projection22.025.529.0+3.5+3.5
Curvature hinge18.522.025.5+3.5+3.5
C2SA14.518.021.5+3.5+3.5
Given this, I feel not confident in trusting the numbers reported in the paper and think the paper should be clearly rejected.
Questions for authors

I have listed my questions in the weaknesses section.

Reviewer_sJtBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies singleton-safe, composition-unsafe failure mode in LLMs. Specifically, a model may react safely against one harmful attack but cannot show the same robustness level when the attacks become compositional. The paper therefore formalizes the safe behavior as a set function over a controlled lattice of policy-preserving prompt factors. Then, authors also introduce a compositional curvature safety alignment method that identifies and optimizes for the most violable factor/adverse curvature at each iteration. The experiments show C2SAC^2SA to be affective in reducing ASR and SCFR while maintaining helpfulness levels.

Critical strengths
  • The paper researched on an important problem that being robust to one prompt transformation does not naturally imply being robust to their compositions.
  • The experimental results and ablations are relatively comprehensive and demonstrated the effectiveness of C2SAC^2SA.
Critical weaknesses
  • In the related work, multiple defenses are mentioned and cited, including ASCot, but it is not very clear to me how are they used in the experiments. For example, Table 1 covers Single-factor tuning, Pairwise training, Random composition, Hard-negative training, ASCpT-inspired, but does it mean authors used these literature's exact algorithms/codes to produce these results? if not, why and what changes have been done compared to their individual algorithms.
  • Similar to above, there are reporting of H4RM3L- and CHIMERA-style attacks
  • The paper only mentioned that they use about the same compute for the baselines, but did not mention how much more compute does it need compared to normal fine-tuning. Specifically, I think it would be more meaningful to compare the performance vs cost curve to make this point more explicit. Otherwise, the baselines could have converged early and the extra cost spent in this paper will be misleading.
Questions for authors
  • What is the definitions of Fθ,xF_{\theta, x}? why is xx not in kijθ(S)k_{ij}^\theta(S), i.e. Eq 4
  • What does this note mean in line 199-201? "Because K = 8, the certificate is verified exactly after checkpoint selection and serves as a structural decomposition rather than an evaluation shortcut. It certifies the anchor-based margin, not every sampled completion; all out-of-lattice results remain empirical."
  • What does Figure 1, first block, bottom, mean? i.e. from {1,..., K} to the empty set
  • I might have missed it, but where is δi\delta_i in eq 10 defined?
  • Same as above, where is UϕU_\phi, Uϕ0U_{\phi_0}, η\eta defined?
  • What and why does this mean "During alignment, curvature contexts satisfy |S|→3, requiring nodes through order five."
  • Will thinking mode help defend the composition, i.e. lower SCFR/ASR
Reviewer_z8rGOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies safety failures that arise when individually tolerated prompt transformations are combined. It introduces adverse safety curvature to measure interaction-induced degradation of an appropriate-response margin and derives a pair-specific compositional bound. The proposed C²SA method repairs violated margin, curvature, and utility constraints through minimum-change parameter updates. Experiments show improved robustness on composition orders excluded from training, with further gains on unseen transformations and adaptive attacks. Ablations and compute-matched comparisons support the contribution of contextual curvature constraints.

Critical strengths
  1. The paper connects its diagnosis to the training objective. Adverse curvature measures interaction effects, appears in the compositional bound, and guides alignment. This gives the method a coherent rationale.

  2. The ablations address several plausible explanations for the gains. The active margin projection control and compute-matched comparisons help separate the benefit of curvature constraints from active selection, optimization machinery, and additional computation.

Critical weaknesses
  1. The certificate guarantees an anchor-based margin, not safe generation. Its safety meaning depends on how well the chosen anchors represent possible model outputs.

  2. Certification still requires exhaustive lattice evaluation. The bound explains interactions, but offers limited practical advantage over checking each composition directly.

Questions for authors
  1. Would a simpler regularizer on composition-induced margin degradation achieve similar gains? This would help establish the benefit of the curvature formulation beyond the existing margin-based controls.

  2. Does using pair-specific tolerances during alignment improve robustness over the shared tolerance? The current comparison establishes tighter certification, but not whether this refinement improves the learned model.

  3. Do the margin and certification improvements persist with independently constructed response anchors that were not used during optimization?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.