acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Safety Alignment

Abstract

Recent work has shown that a model's input word embeddings can serve as effective control variables for steering its behavior toward outputs that satisfy desired properties. However, this has only been demonstrated for pretrained text-completion models on the relatively simple objective of reducing surface-level profanity in short continuations. A natural and practically important question is how well input embeddings can control aligned models, which produce an imbalanced bimodal refuse-or-comply output distribution rather than the smooth distribution characteristic of open-ended generation. We explore this in the context of safety, showing that input word embeddings can be optimized in a sub-lexical manner to minimize the semantic harmfulness of aligned model responses. Our approach uses zeroth-order gradient estimation of a black-box text-moderation API with respect to the input embeddings, and then applies gradient descent on these embeddings to minimize the harmfulness of the generated text. Experiments show that the proposed method neutralizes every safety-flagged response on standard safety benchmarks for four of the five evaluated models, and reduces the number of flagged responses on the fifth.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
zeroth-order optimization, safety, moderation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_moVJOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Test-Time Safety Alignment (TSA), which reduces unsafe LLM outputs by iteratively optimizing prompt embeddings at inference time. TSA uses zeroth-order optimization to minimize scores from a black-box moderation model without updating the LLM parameters. Experiments across multiple models and harmful-prompt benchmarks show substantial reductions in flagged responses, including improvements measured by held-out safety evaluators.

Critical strengths
  1. Novel test-time formulation. TSA controls model safety by optimizing continuous prompt embeddings without retraining or modifying model parameters.

  2. Broad empirical evaluation. The method is tested across multiple model families and safety benchmarks, with additional evaluation using safety judges not used during optimization.

Critical weaknesses
  1. Incomplete and computationally uneven comparison.
  • The baseline evaluation is incomplete across model families: AdaSteer and RESTA are evaluated only on Llama 3.1-8B, while TSA is evaluated on all five models, and runtime is reported only for TSA.
  • A compute-matched Best-of-N baseline is missing (Huang et al., 2025; Jinnai et al., 2025). Specifically, the model could generate the same number of responses as TSA and return the response with the lowest moderation score. Without this comparison, it is unclear whether TSA benefits from its embedding optimization or simply from a larger inference budget.
  • Finally, the paper lacks an overview figure showing the iterative generation, moderation, and embedding-update loop, making both the method and its computational overhead difficult to understand.

  1. The aggregate evaluation does not clearly isolate TSA’s actual contribution. Prompts whose initial moderation score is below the threshold receive no TSA update, but they are included in the final metrics. The authors should separately report the success rate, score reduction, and residual flagged count among prompts that actually trigger optimization.

[1] Huang, A., Block, A., Liu, Q., Jiang, N., Krishnamurthy, A., and Foster, D. J. Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment. ICML, 2025.

[2] Jinnai, Y., Morimura, T., Ariu, K., and Abe, K. Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignment. NAACL, 2025.

Questions for authors

Q1. Can the authors report throughput under concurrent workloads and compare TSA with the baselines under matched computational budgets? An overview figure showing the generation, moderation, and embedding-update loop would also clarify the method and its computational cost.

Q2. TSA searches for embedding updates that reduce the OpenAI Moderation score, and Table 1 evaluates the resulting responses using scores and flags from the same moderation model. Thus, Table 1 alone cannot distinguish genuine improvements in safety from improvements specific to that moderation model. Although the HarmBench and Llama Guard results alleviate this concern, can the authors provide further evidence that TSA improves safety beyond the moderation model used during optimization?

Q3. Can the authors provide systematic human evaluation to verify that TSA produces genuinely safer responses, rather than responses that merely receive lower scores from automatic safety evaluators?

Q4. Can the authors evaluate whether TSA preserves helpfulness, relevance, coherence, and task completion on both harmful and benign prompts? Lower safety scores alone do not demonstrate that response utility is preserved.

Q5. Why does TSA leave 59 responses flagged on Gemma 3 while reducing the flagged count to zero on the other four models? Do these failures arise from the fixed optimization budget, model-specific embedding properties, or limitations of the moderation objective?

5 comments
AuthorsOpenReview ↗

Rebuttal — Part 1/3

Thank you for your careful review.

We address each weakness and question below with new experiments. Unless noted otherwise, all new experiments use the adversarial-harmful split of WildJailbreak. Comparisons are paired by prompt over the prompts that enter TSA's optimization loop, meaning prompts whose first response scores at least 0.1, with Holm correction across models.


[Weakness] "AdaSteer and RESTA are evaluated only on Llama 3.1-8B … runtime is reported only for TSA"

We ran RESTA on Llama, Qwen, Phi and Gemma. We swept its noise scale σ\sigma per model, and Llama keeps its paper's 0.08. The table gives the mean moderation score as undefended / RESTA / TSA.

WildJailbreak, adv. harmfulHarmBench
Llama (σ=0.08\sigma = 0.08)0.106 / 0.098 / 0.0120.055 / 0.076 / 0.009
Qwen (σ=0.12\sigma = 0.12)0.097 / 0.046 / 0.0230.053 / 0.032 / 0.025
Phi (σ=0.12\sigma = 0.12)0.081 / 0.074 / 0.0190.050 / 0.058 / 0.020
Gemma (σ=0.05\sigma = 0.05)0.137 / 0.142 / 0.0650.102 / 0.102 / 0.062

TSA's mean score is significantly lower than RESTA's in every cell.

However, AdaSteer is different. RESTA has a single noise scale to sweep, while AdaSteer fits steering layers and coefficient maps to each model. Among all the models tested, only Llama 3.1-8B has a published AdaSteer configuration, so a port would be our own unvalidated calibration rather than the published method.

We also measured the runtime of every baseline. The table gives the mean time per prompt on WildJailbreak adversarial-harmful prompts.

SmoothLLMRESTAAdaSteerTSA
Llama3.0 s2.7 s3.3 s3.6 s
Qwen6.3 s5.4 s—14.6 s
Phi2.2 s3.2 s—5.2 s
Gemma0.85 s4.2 s—20.1 s

TSA's time is end to end over a uniform prompt mix, where a prompt below the threshold costs one generation and one moderation call. TSA spends its extra time only on prompts that enter its loop, and on every model this buys a mean moderation score on these prompts considerably lower than SmoothLLM, RESTA or AdaSteer reaches (Table 1 and the RESTA table above).


[Weakness] "A compute-matched Best-of-N baseline is missing"

We ran the compute-matched Best-of-N you suggested. For every prompt, it samples as many completions as TSA generated on that prompt (early stopping included) and keeps the one with the lowest moderation score. Generations and oracle calls are therefore matched prompt by prompt. We ran it at T=0.1T = 0.1 and at T=1.0T = 1.0, where a deliberately high temperature is used for diverse samples.

TSA lower, T=0.1T = 0.1Flagged TSA / BoN, T=0.1T = 0.1TSA lower, T=1.0T = 1.0Flagged TSA / BoN, T=1.0T = 1.0
Llama692 of 6970 / 91635 of 6970 / 11
Qwen496 of 6470 / 55337 of 6470 / 15
Phi421 of 5330 / 31328 of 5330 / 7
Gemma446 of 84559 / 86105 of 84559 / 10

On Llama, Qwen and Phi at both temperatures, and on Gemma at T=0.1T = 0.1, TSA has a significantly lower mean moderation score and leaves significantly fewer prompts flagged. With the budget matched, resampling does not reach TSA's mean score on these models, so TSA's results are not explained by its larger budget alone.

We will cite both papers [1, 2] you pointed to when the revised paper discusses Best-of-N.

On Gemma, Best-of-N at T=1.0T = 1.0 leaves fewer prompts flagged than TSA. Gemma is the model on which TSA converges most slowly (Table 1), and every prompt TSA leaves flagged there ran out of iterations before its score fell below 0.1. Extra budget helps the two methods differently. Best-of-N draws every sample from the same distribution, so each added sample lowers its expected best score by less than the previous one did, and no number of samples can take its best score below the lowest score that distribution can produce. TSA changes the input after every iteration, so its samples do not all come from one distribution, and we expect it to overtake Best-of-N on Gemma as the budget grows.

Following Reviewer HAsA's suggestion, we also ran two other baselines with oracle access:

  1. RESTA lets each of its 10 noisy copies generate a full response and keeps the lowest-scoring one.
  2. The safety-prompt baseline regenerates a flagged response under a fixed safety system prompt.

The table counts the prompts each method leaves flagged.

TSARESTA + oracleSafety prompt
Llama0097$^*$
Qwen0532$^*$
Phi0247$^*$
Gemma59204$^*$121$^*$

∗^*Significantly more than TSA.

With the same oracle, neither baseline leaves fewer prompts flagged than TSA on any model, and TSA leaves significantly fewer than the safety prompt on every model and than RESTA on Gemma.

AuthorsOpenReview ↗

Rebuttal — Part 2/3

[Weakness & Q1] "An overview figure showing the generation, moderation, and embedding-update loop"

Thank you for the suggestion. We will add an overview figure to Section 4 showing the generation, moderation and embedding-update loop, annotated with its cost. Each iteration costs N+1N + 1 generations and oracle calls, with the NN perturbed ones issued as one batch, and a prompt already below the threshold costs one of each.


[Weakness] "separately report the success rate, score reduction, and residual flagged count among prompts that actually trigger optimization"

We now report these metrics separately. The table covers Table 1's runs on the prompts whose first response scores at least 0.1, which are the only ones TSA modifies (WildJailbreak adversarial-harmful).

Mean score, undefended →\to TSAEnds below 0.1Ends unflaggedResidual flagged
Llama0.240 →\to 0.003100%100%0
Qwen0.240 →\to 0.027100%100%0
Phi0.223 →\to 0.018100%100%0
Gemma0.276 →\to 0.12065%93%59

[Q1] "report throughput under concurrent workloads and compare TSA with the baselines under matched computational budgets"

We measured end-to-end throughput, oracle calls included, for Llama 3.1-8B on the two A6000s of Appendix B. The engine served adversarial-harmful prompts at increasing concurrency, with three runs per level.

Concurrent requests12481216
Throughput (prompts/s)0.280.490.811.211.361.44
Mean latency (s)3.64.14.96.58.610.7

The plateau beyond 12 concurrent requests comes from the serving engine's 32 sequence slots filling up, while GPU utilization falls. With one request in flight, the oracle takes 11% of wall-clock time. Under load, its latency overlaps with other requests' generation, and removing the oracle would raise throughput by at most 4–11%. The API rate limit was never approached (peak 148 of 2,000 requests per minute).

In short, TSA's throughput rises almost fivefold with concurrency before the engine's slots fill, and the oracle is not the bottleneck. The matched-budget comparison is Best-of-N above.


[Q2 & Q3] "further evidence that TSA improves safety beyond the moderation model used during optimization" and "systematic human evaluation"

We ran a blinded human audit that uses no automatic evaluator. Two annotators compared the undefended and TSA responses on 200 pairs on which TSA acted. The 157 adversarial-harmful pairs are those where TSA lowered the moderation score, taken from the four models in proportion, so the audit asks whether the oracle's reductions are visible to a human. The other 43 are all the adversarial-benign prompts, across the four models, that enter the loop (i.e., initial score larger than 0.1). The annotators never saw which system produced which response, and the A/B order was random. Each pair received three kinds of answers:

  1. Which response is safer? (A, B, or tie)
  2. Which better addresses the prompt's legitimate intent? (A, B, tie, or not applicable)
  3. Does either response contain actionable harmful content? (yes or no for each)

Their agreement was κ=0.76\kappa = 0.76 on safety and κ=0.66\kappa = 0.66 on legitimate intent (Cohen's κ\kappa, agreement beyond chance, with 1 for perfect agreement). Disagreements were settled jointly, and the table counts the agreed labels. The fourth column ignores ties and reports how often TSA's response was the safer one among the remaining pairs. Its pp-value is an exact binomial test of that rate against 50%, the value expected if neither system were safer.

PairsTSA saferUndefended saferTSA safer, share of non-tied pairsEqually safe
All 2001552686% (p<0.001p < 0.001)19
157 harmful1251987% (p<0.001p < 0.001)13
43 benign30781% (p<0.001p < 0.001)6

Each annotator's labels alone give 85% and 89%. Actionable harmful content appeared in 44 undefended and 3 TSA responses, a significant difference. The two held-out judges in Section 5.1 and Appendix E, which TSA never optimizes against, point the same way.

Human readers who never saw a moderation score therefore judge TSA's responses genuinely safer, so its gains are not specific to the moderation model it optimizes.

AuthorsOpenReview ↗

Rebuttal — Part 3/3

[Q4] "whether TSA preserves helpfulness, relevance, coherence, and task completion on both harmful and benign prompts"

We ran a blinded LLM judge (Claude Opus 5.5 on extra-high effort, three runs per item), which rated all 840 benign responses (the four models' answers to the 210 adversarial-benign prompts) on a scale of 1 to 5 on helpfulness, relevance, coherence and task completion. The table gives each model's mean rating as TSA / undefended.

CoherenceRelevanceHelpfulnessTask completion
Llama3.84 / 3.914.02 / 4.193.08 / 3.172.18 / 2.20
Qwen3.90 / 3.924.44 / 4.442.85 / 2.852.11 / 2.11
Phi3.79 / 3.874.28 / 4.312.96 / 2.942.13 / 2.12
Gemma3.81 / 3.824.06 / 4.042.75 / 2.762.05 / 2.06

Task completion is low for both systems because many adversarial-benign prompts have no task to complete. No difference from the undefended model survives Holm correction on any model, and every Hodges–Lehmann estimate of the difference is within 0.17. Put plainly, the judge finds no significant drop in any of the four qualities over the benign split.

TSA leaves responses below the threshold unchanged, so any cost is confined to the prompts where it acts. There, the human audit judged 27 of 42 applicable benign responses worse at serving the legitimate intent and 15 equal or better, and among the judge's ratings only Llama's drop is significant. Although these prompts are benign, TSA's first response to them already scores at least 0.1, and TSA gives up some relevance and helpfulness to lower that score. It corresponds to only the 3.2% of all adversarial-benign model–prompt pairs.

On harmful prompts we do not score task completion, because completing the request is the failure TSA prevents and many such requests cannot be helped without declining. On coherence and responsiveness, Qwen's responses are surprisingly much more coherent than the undefended model's (4.30 against 3.97 out of 5) while engaging the request less directly. Gemma stays near the undefended level on both, with no difference surviving Holm correction. Llama's responses mostly, and Phi's often, stop addressing the prompt to prevent the model from engaging with the malicious request.

In the revision, these measurements replace the informal reading in Section 5.1.


[Q5] "Why does TSA leave 59 responses flagged on Gemma 3"

TSA reduces Gemma's flagged responses from 278 to 59, against 225 for SmoothLLM for comparison (Table 1). On the 845 Gemma prompts of the comparisons above, it leaves fewer flagged than every baseline compared there except Best-of-N at T=1.0T = 1.0 (discussed in the Best-of-N response). Taking the three causes you name in turn:

  1. The fixed iteration budget: All 59 are prompts on which TSA used its entire iteration budget without its score falling below 0.1. On the other three models in our table of prompts that enter the loop, every such prompt falls below 0.1 within the budget, so the 59 are where Gemma's slower convergence (Table 1) meets the budget. We consider this the explanation for the 59.
  2. Model-specific embedding properties: Appendix F tests the two explanations that the tested models confound, embedding dimension and base alignment. Qwen3-0.6B has a smaller embedding dimension than Gemma (d=1024d = 1024 against 1152) and starts worse (470 flagged against 278), yet TSA clears every flag in about one iteration. TSA also clears all 527 flags of an abliterated Qwen3-14B. What slows Gemma's convergence is not its embedding dimension or its base alignment, and Appendix F does not single out one property.
  3. The moderation objective: It still carries signal on Gemma. Following Reviewer HAsA's suggestion, we replaced TSA's estimated direction with a random direction of the same norm, run under the same iteration cap. On Gemma, the random direction scores significantly worse and leaves more prompts flagged, 121 against 59 on the same 845 prompts. On the prompts TSA acts on (same table), the mean score falls from 0.276 to 0.120.

The revised paper will include all of these results.

[1] Huang et al., "Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment," ICML, 2025.

[2] Jinnai et al., "Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignment," NAACL, 2025.

Reviewer_ZVVmOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Test-Time Safety Alignment (TSA), a test-time aligment method against LLM jailbreak attacks. TSA optimizes input token embeddings by zeroth-order gradient estimation with Gaussian smoothing, which doesn’t need to update the parameters of LLMs and perform gradient back propagation.

Evaluations on five instruction-tuned models spanning 1B to 20B parameters over WildJailbreak and HarmBench show strong jailbreak mitigation, while preserving normal responses for benign inputs.

Critical strengths

The strengths of the paper include:

  • The paper proposes a novel zeroth-order gradient optimization method without modifying the parameters of LLM, saving large resources of fine-tuning.
  • The proposed test-time safety alignment is a promising direction of safety alignment of LLMs.
  • Strong empirical performance on jailbreak benchmarks and avoids over-refusal on benign prompts.
Critical weaknesses

The weaknesses of the paper include:

  • Only applicable to open-weight models since embedding read/write access is required, limiting its application scenarios.
  • High inference overhead from multiple forward passes, increasing the inference cost..
Questions for authors

Could you provide the derivation procedure of Equation 2 as well as its related background knowledge? at least in the Appendix, to help readers better understanding the core methodology.

1 comment

Thank you for the positive feedback and question.


[Question] Derivation of Eq. (2) and its background

Eq. (2) is the Gaussian-smoothing identity of [1]. We give the derivation here and will add it, together with the background below, as a new appendix section referenced from Section 4.

Let p(U)=(2π)−Td/2exp⁡(−∥U∥F2/2)p(U) = (2\pi)^{-Td/2} \exp(-\Vert U \Vert_F^2 / 2) be the density of U∈RT×dU \in \mathbb{R}^{T \times d} with i.i.d. N(0,1)\mathcal{N}(0, 1) entries, so that Φμ(X)=EU[Φ(X+μU)]=∫Φ(X+μU)p(U) dU\Phi_\mu(X) = \mathbb{E}_U[\Phi(X + \mu U)] = \int \Phi(X + \mu U) p(U) \thinspace dU. The substitution Z=X+μUZ = X + \mu U, with dU=μ−TddZdU = \mu^{-Td} dZ, moves every occurrence of XX out of Φ\Phi and into the Gaussian density:

Φμ(X)=μ−Td∫Φ(Z) p(Z−Xμ)dZ.\Phi_\mu(X) = \mu^{-Td} \int \Phi(Z) \thinspace p\left(\frac{Z - X}{\mu}\right) dZ.

This is the step that frees the identity from any smoothness of Φ\Phi, since the integrand now depends on XX only through the Gaussian factor. Differentiating under the integral, with ∇Xp=p ∇Xlog⁡p\nabla_X p = p \thinspace \nabla_X \log p and ∇X[log⁡p((Z−X)/μ)]=(Z−X)/μ2\nabla_X \big[ \log p\big((Z - X)/\mu\big) \big] = (Z - X)/\mu^2, gives

∇Φμ(X)=μ−Td∫Φ(Z) p(Z−Xμ)Z−Xμ2 dZ=EU[Φ(X+μU)μU],\nabla \Phi_\mu(X) = \mu^{-Td} \int \Phi(Z) \thinspace p\left(\frac{Z - X}{\mu}\right) \frac{Z - X}{\mu^2} \thinspace dZ = \mathbb{E}_U\left[ \frac{\Phi(X + \mu U)}{\mu} U \right],

where the second equality undoes the substitution, using (Z−X)/μ2=U/μ(Z - X)/\mu^2 = U/\mu. Dominated convergence licenses the interchange, since Φ\Phi is measurable and bounded; Φ\Phi is bounded because hh maps into [0,1][0, 1] by Eq. (1).

Subtracting Φ(X)U/μ\Phi(X) U / \mu inside the expectation, which has mean zero because EU[U]=0\mathbb{E}_U[U] = 0 and Φ(X)\Phi(X) does not depend on UU, leaves the expectation unchanged and yields Eq. (2). The subtracted term is a control variate that removes the component of each sample proportional to the base-point value. For a Lipschitz Φ\Phi, it keeps the variance of the Monte Carlo average gkg_k bounded as μ→0\mu \to 0 [1], whereas without it the variance grows as μ−2\mu^{-2} whenever Φ(X)≠0\Phi(X) \neq 0.

Appendix A of [2] gives a complete proof of Eq. (2) along these lines, under a growth condition that every bounded Φ\Phi satisfies.

Eq. (2) is thus the Gaussian score-function identity with a zero-mean control variate, and its right-hand side needs only values of Φ\Phi. This is what lets Φ=h∘f\Phi = h \circ f be optimized through forward passes and oracle queries, with no gradient of either ff or hh.

Section 4 states the identity under the Lipschitz hypothesis of [1]; the appendix will note that the weaker conditions above suffice, so the identity applies to a Φ\Phi built from discrete autoregressive sampling. The quantitative results in Section 4 keep their assumptions, namely Lipschitz continuity of Φ\Phi for the O(μTd)\mathcal{O}(\mu\sqrt{Td}) smoothing error and, in addition, Lipschitz smoothness for the bound on ∥∇Φμ−∇Φ∥\Vert \nabla\Phi_\mu - \nabla\Phi \Vert.


[Weakness] Only applicable to open-weight models

We agree, and state the condition in Sections 1 and 6. The requirement is lighter than that of decoding-level and activation-steering defenses, which need hidden states or output logits (Section 2); TSA touches only the input embedding layer.


[Weakness] High inference overhead

Section 6 records this as a limitation. The timings in Table 1 are averages over the prompts that enter the optimization loop, and only prompts whose score reaches the 0.1 early-stopping threshold do, so the amortized cost over a mixed workload is lower than those figures suggest (Section 5.1).


We thank the reviewer again for recommending acceptance.


[1] Nesterov & Spokoiny, "Random Gradient-Free Minimization of Convex Functions," FoCM, 2017.

[2] Kalogerias & Powell, "Zeroth-Order Stochastic Compositional Algorithms for Risk-Aware Learning," SIAM J. Optim., 2022.

Reviewer_HAsAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose Test-Time Safety Alignment (TSA), a training-free defense for aligned open-weight LLMs. For each prompt, it optimizes the input word embeddings using zeroth-order gradient estimates of a black-box oracle, the OpenAI Moderation API, with the goal of minimizing the worst-category harm score of the generated response. The method builds on prior test-time detoxification work and extends it from base models to safety-aligned chat models facing adversarial jailbreaks. Across five models (1B–20B) on WildJailbreak and HarmBench, TSA removes all moderation-flagged responses for four models, typically within one or two steps, and reduces flags on Gemma 3-1B. The gains partially transfer to two held-out judges, and benign prompts are largely unaffected. The optimized embeddings still decode to the original tokens, which the authors call a "sub-lexical" intervention. They argue this reveals the embedding layer as an underused control surface.

Critical strengths

Clean problem framing. The question of whether embedding-level control transfers from smooth base-model generation to bimodal refuse-or-comply aligned models is reasonable and practically relevant.

Minimal assumptions. It needs no training, calibration data, auxiliary trained model, intent classifier, or hidden-state access. Transfer to two judges never optimized against.

Transferability. The direction of the effect holds in all 16 base-to-TSA comparisons on the harmful splits.

Critical weaknesses
  • The algorithm is identical to TIDE (2026), and the contribution is an application study. The novelty is an empirical transfer claim.

  • None of the three reported baselines gets oracle access, so the comparison is slightly unfair. Can RESTA be incorporated with Oracle choosing the output as a separate baseline? Other simpler baselines can be tested: Best-of-N rejection sampling under the same oracle budget, Regenerate with a safety system prompt when flagged.

  • The paper shows N=1 is often sufficient and Llama converges in one step, and says “even a crude descent direction suffices.” A random-direction ablation with the same step norm is essential. Without it, TSA may just be stochastic search with a selector. Benign prompts under 0.1 never enter the loop (only 1/210 does for Llama), so “leaves safe responses intact” is true by construction. The real test is benign-but-sensitive prompts that trip the moderator: medical, security education, fiction with violence, self-harm support. Evaluating on benchmarks like XSTest or OR-Bench could strengthen the paper.

  • Not enough human evaluation is performed. Helpfulness rests on “the authors read the responses,” with no quantitative metric and no inter-annotator protocol.

  • TIDE (2026) shows that a higher temperature affects the sample size. Can you try increasing the temperature to observe how it increases k?

Reference:

TIDE (2026): Test-Time Detoxification without Training or Learning Anything

Questions for authors

See Weaknesses

4 comments
AuthorsOpenReview ↗

Rebuttal — Part 1/3

Thank you for the detailed review. We ran every experiment you suggested, and the results follow.

Unless noted otherwise, all new experiments use the adversarial-harmful split of WildJailbreak. Comparisons are paired by prompt over the prompts that enter TSA's optimization loop, that is, prompts whose first response scores at least 0.1, with Holm correction across models.


"None of the three reported baselines gets oracle access"

We ran the three oracle-access baselines you suggested. Each uses the same moderation oracle as TSA.

  1. Best-of-N: For each prompt, it samples as many responses as TSA generated on that prompt (e.g., fewer when TSA stopped early) and returns the one with the lowest moderation score. It therefore matches TSA's generation and oracle-call counts prompt by prompt. We ran it at T=0.1T = 0.1 and at T=1.0T = 1.0, where a deliberately high temperature is used for diverse samples.
  2. RESTA + oracle access: RESTA normally merges its noisy copies with a token-level vote, so it never produces separate responses to choose from. We let each of its 10 noisy copies generate a full response, and the oracle returns the lowest-scoring one. We tuned its σ\sigma parameter per model.
  3. Safety-prompt regeneration: Whenever the response is flagged, the model regenerates it under a fixed safety system prompt.

The table counts the prompts that remain flagged after each method, out of the 533 to 845 prompts per model that enter TSA's loop.

UndefendedTSABoN (T=0.1T = 0.1)BoN (T=1.0T = 1.0)RESTA + oracleSafety prompt
Llama218091$^*$11$^*$097$^*$
Qwen165055$^*$15$^*$532$^*$
Phi126031$^*$7$^*$247$^*$
Gemma2785986$^*$10∗^*204$^*$121$^*$

∗^*Significantly different from TSA (McNemar, Holm p≤0.02p \le 0.02).

With the same oracle, TSA leaves significantly fewer prompts flagged than at least two of the baselines on every model, and only Best-of-N at T=1.0T = 1.0 on Gemma leaves significantly fewer than TSA.

Gemma, where Best-of-N at T=1.0T = 1.0 does better, has the slowest convergence (Table 1), and every prompt TSA leaves flagged there used its full iteration budget. The two methods use additional budget differently. Best-of-N keeps sampling from one fixed distribution, so each extra sample improves its expected best score by less than the one before, and its best score is bounded by the lowest score that distribution can ever produce. TSA moves the input at every iteration and samples from a new distribution each time, so this bound does not apply to it. We therefore expect TSA to overtake Best-of-N on Gemma as the iteration budget grows.


"A random-direction ablation with the same step norm is essential"

We ran the suggested ablation, replacing the zeroth-order estimate with a random Gaussian direction. The ablation keeps TSA's step norm, cosine projection, early stopping, best-iterate return, and iteration cap.

Mean moderation score (lower is safer); TSA / randomPaired tt-test, Holm pp
Llama0.003 / 0.097<0.001< 0.001
Qwen0.027 / 0.142<0.001< 0.001
Phi0.018 / 0.119<0.001< 0.001
Gemma0.120 / 0.260<0.001< 0.001

The difference is significant on every model, including Llama, where TSA usually converges in one step, so the direction of a single step already matters. On Llama, Qwen and Phi, neither method leaves any prompt flagged, so the difference appears only in the scores. On Gemma, the random direction also leaves significantly more prompts flagged, 121 against 59. The estimated direction therefore carries signal, so TSA is not just stochastic search with a selector.

AuthorsOpenReview ↗

Rebuttal — Part 2/3

"The real test is benign-but-sensitive prompts that trip the moderator" and "XSTest"

We evaluated all four models on the 250 safe prompts of XSTest [1]. A blinded LLM judge (Claude Opus 5.5 on extra-high effort) labelled each output with XSTest's compliance and refusal categories. Below the 0.1 threshold, TSA does not modify the response.

Prompts tripping the moderatorExplicit refusals among on-topic outputs (undefended → TSA)Outputs not addressing the prompt
Llama28%6.9% → 9.7%23%
Qwen26%4.0% → 10.1%10%
Phi25%8.1% → 9.0%12%
Gemma30%9.1% → 10.0%0%

The prompts are benign, but the undefended model's own responses to them sometimes score above 0.1 (actionable threshold), and TSA lowers these scores as it would for any response. On Llama, Qwen and Phi, the larger cost is in relevance, since some outputs stop addressing the prompt as part of defense (the judge counts these as refusals).

The cost follows the oracle's (score-wise) false positives on benign content, so a better-calibrated oracle would reduce it. Improving the oracle is outside the scope of this work, and TSA works with any oracle (Appendix E).

We will state in the paper that "already-safe" means below the oracle's threshold, and discuss this trade-off.


"Not enough human evaluation … no quantitative metric and no inter-annotator protocol"

We added both, through a blinded human audit and an LLM-judge evaluation. The LLM judge evaluated all 840 benign responses (210 per model across 4 models).

Human audit: Two annotators compared the undefended and TSA responses on 200 pairs where TSA changed the response:

  • 157 adversarial-harmful pairs drawn from the four models in proportion, to test whether the oracle's reductions reflect real safety gains to a human reader;
  • all 43 adversarial-benign prompts that enter TSA's loop across the four models.

Neither annotator knew which response came from which system, and the A/B order was random. For each pair, they answered four questions:

  1. Which response is safer? (A, B, or tie)
  2. Which response better addresses the prompt's legitimate intent? (A, B, tie, or not applicable)
  3. Does response A contain actionable harmful content? (yes or no)
  4. Does response B contain actionable harmful content? (yes or no)

The annotators agreed with κ=0.76\kappa = 0.76 on which response is safer and κ=0.66\kappa = 0.66 on legitimate intent (Cohen's κ\kappa measures agreement beyond chance, where 1 is perfect agreement). They resolved disagreements together, and the table uses the agreed labels. Its fourth column leaves out ties and gives the percentage of the remaining pairs in which TSA's response was judged safer. The pp-value is an exact binomial test against 50%, the rate expected if neither system were safer.

PairsTSA saferUndefended saferTSA safer, % of non-tied pairsEqually safe
All 2001552686% (p<0.001p < 0.001)19
157 harmful1251987% (p<0.001p < 0.001)13
43 benign30781% (p<0.001p < 0.001)6

Before resolving disagreements, each annotator's own labels give 85% and 89%. Actionable harmful content appeared in 44 undefended responses and 3 TSA responses, all on harmful prompts (exact McNemar p<0.001p < 0.001).

On the benign prompts, TSA was judged worse at serving the legitimate intent in 27 of 42 applicable pairs, equal in 7 and better in 8. TSA leaves every other benign response unchanged, so these 27 are 3.2% of all 840 adversarial-benign model–prompt pairs.

LLM judge: A blinded judge (Claude Opus 5.5 on extra-high effort; three runs per item) rated helpfulness, relevance, coherence and task completion on a 1-5 scale on all 840 adversarial-benign responses. No difference from the undefended model survives Holm correction on any model, and every Hodges–Lehmann estimate of the difference is within 0.17. In other words, the judge detects no significant loss in helpfulness, relevance, coherence or task completion across the benign split. On the benign prompts where TSA acts, the ratings drop for Llama and Phi and less for Qwen, while Gemma holds. This matches the trade-off seen on XSTest. The undefended responses on these prompts already score at least 0.1, and TSA gives up some relevance and helpfulness to bring that score below the threshold.

In short, human readers confirm that TSA's lower scores correspond to safer responses, and the loss in helpfulness affects only 3.2% of benign responses.

In the revision, these measurements replace the informal reading in Section 5.1.

AuthorsOpenReview ↗

Rebuttal — Part 3/3

"Can you try increasing the temperature to observe how it increases k?"

We swept the temperature on Llama and Gemma, the fastest- and slowest-converging models. The table gives the average number of iterations on the full adversarial-harmful split. In the last column, each score TSA computes is averaged over M′=8M' = 8 sampled responses instead of taken from one. This is variance reduction, since a single response at T=1.0T = 1.0 gives a noisy score, and TIDE uses it in the same setting [2].

T=0.1T = 0.1T=0.3T = 0.3T=0.7T = 0.7T=1.0T = 1.0T=1.0T = 1.0; M′=8M' = 8
Llama1.001.011.021.071.01
Gemma5.705.475.605.277.00

TSA remains effective at T=1.0T = 1.0. Llama converges on every prompt with no flagged response, and Gemma's mean score and flagged count are both lower than at T=0.1T = 0.1. Averaging speeds convergence on Llama, consistent with TIDE's observation [2]. It is slow on Gemma because the loop then stops only when the average of eight responses falls below 0.1, not when a single response happens to.


"The algorithm is identical to TIDE (2026), and the contribution is an application study"

The optimizer is TIDE's [2], as Section 2 states. Our contribution is showing that it transfers to aligned models under adversarial prompts. Section 1 explains why it might not have, since these models give refuse-or-comply outputs and harm is defined semantically. One result has no counterpart in base models, the only setting TIDE evaluates. Even on an abliterated Qwen3-14B, whose refusal behavior has been removed, TSA reduces flagged responses from 527 to 0 (Appendix F).


All new results will be added to the revised paper.

[1] Röttger et al., "XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models," NAACL, 2024.

[2] Saglam & Kalogerias, "Test-Time Detoxification without Training or Learning Anything," ICML, 2026.

Reviewer_HAsAOpenReview ↗

Thank you for the detailed rebuttal

Thank you for the detailed rebuttal and the new experiments. The random-direction ablation, the baselines with oracle access, XSTest and the human audit address most of my concerns about soundness.

However, my concern about novelty remains. The optimizer and update rule are taken directly from TIDE, and the main change is the objective. Showing that an existing method works on aligned models is a useful empirical result, but I don’t think it is sufficient of a methodological contribution for ICLR. I will keep my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal summary and revised PDF

We thank the reviewers for their feedback. They found our proposed method (TSA) novel (moVJ) and promising (ZVVm), its assumptions minimal (HAsA), its evaluation broad (moVJ) and its results strong (ZVVm).

The table summarizes our rebuttal.

ConcernRaised byWhat we didResult
Missing oracle-access and compute-matched baselinesHAsA, moVJRan Best-of-N, RESTA with oracle selection, and regeneration of flagged responses under a safety promptTSA's gains are not explained by compute or oracle access alone, since it leaves significantly fewer prompts flagged in 12 of 16 comparisons.
RESTA and AdaSteer evaluated only on LlamamoVJRan RESTA on four models; of these, only Llama has a published AdaSteer configurationTSA significantly outperforms RESTA on every model, so its advantage is not specific to Llama.
Missing random-direction ablationHAsAReplaced TSA's estimated direction with a random oneTSA is more than random search, since its estimated direction significantly outperforms a random one on every model.
No results restricted to prompts TSA optimizes; 59 left flagged on GemmamoVJReported results on these prompts alone and analyzed the 59TSA succeeds on nearly every prompt it optimizes, clearing all flags on Llama, Qwen and Phi and 93% on Gemma, where the remaining 59 ran out of optimization budget.
Insufficient human evaluation; gains may be specific to the oracleHAsA, moVJRan a blinded human audit of 200 undefended/TSA response pairsHuman annotators confirm the gains are not specific to the oracle, judging TSA's response safer in 86% of non-tied pairs and finding actionable harm in 3 TSA vs. 44 undefended responses.
No quantitative evidence that helpfulness is preservedHAsA, moVJRated all benign responses on four quality measures with a blinded LLM judgeTSA largely preserves helpfulness, with no significant drop in any measure on any model and only 3.2% of benign responses judged less helpful by humans.
Untested on benign-but-sensitive prompts that trip the oracleHAsAEvaluated TSA on a new benchmark (XSTest)TSA's cost on safe prompts, in refusals and relevance, arises only on those the oracle misjudges as risky (25–30%).
High inference overhead; runtime reported only for TSAmoVJ, ZVVmMeasured every baseline's runtime and TSA's throughput under concurrent requests on LlamaTSA is slower than the baselines on average, but pays its optimization cost only on the prompts that need it, and its throughput rises about fivefold under concurrency.
Effect of temperature on iteration count untestedHAsASwept the temperature from 0.1 to 1.0 on Llama and GemmaTSA is robust to temperature, staying effective up to 1.0 with little change in its iteration count.
Novelty, given that the optimizer is TIDE's [1]HAsAClarified that the contribution is showing TIDE's optimizer transfers from base models to aligned models under adversarial promptsThe transfer is a nontrivial new result, since aligned models refuse or comply and harm is semantic.

We have updated the paper with the revised PDF to include all new results and an overview figure of TSA's loop and its cost (moVJ). Changes are colored by reviewer:

  • Red: moVJ
  • Blue: HAsA
  • Green: ZVVm
  • Purple: changes requested by both moVJ and HAsA
  • Dark gold: other edits

[1] Saglam & Kalogerias, "Test-Time Detoxification without Training or Learning Anything," ICML, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.