- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes Test-Time Safety Alignment (TSA), which reduces unsafe LLM outputs by iteratively optimizing prompt embeddings at inference time. TSA uses zeroth-order optimization to minimize scores from a black-box moderation model without updating the LLM parameters. Experiments across multiple models and harmful-prompt benchmarks show substantial reductions in flagged responses, including improvements measured by held-out safety evaluators.
Critical strengths
-
Novel test-time formulation. TSA controls model safety by optimizing continuous prompt embeddings without retraining or modifying model parameters.
-
Broad empirical evaluation. The method is tested across multiple model families and safety benchmarks, with additional evaluation using safety judges not used during optimization.
Critical weaknesses
- Incomplete and computationally uneven comparison.
- The baseline evaluation is incomplete across model families: AdaSteer and RESTA are evaluated only on Llama 3.1-8B, while TSA is evaluated on all five models, and runtime is reported only for TSA.
- A compute-matched Best-of-N baseline is missing (Huang et al., 2025; Jinnai et al., 2025). Specifically, the model could generate the same number of responses as TSA and return the response with the lowest moderation score. Without this comparison, it is unclear whether TSA benefits from its embedding optimization or simply from a larger inference budget.
- Finally, the paper lacks an overview figure showing the iterative generation, moderation, and embedding-update loop, making both the method and its computational overhead difficult to understand.
- The aggregate evaluation does not clearly isolate TSA’s actual contribution. Prompts whose initial moderation score is below the threshold receive no TSA update, but they are included in the final metrics. The authors should separately report the success rate, score reduction, and residual flagged count among prompts that actually trigger optimization.
[1] Huang, A., Block, A., Liu, Q., Jiang, N., Krishnamurthy, A., and Foster, D. J. Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment. ICML, 2025.
[2] Jinnai, Y., Morimura, T., Ariu, K., and Abe, K. Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignment. NAACL, 2025.
Questions for authors
Q1. Can the authors report throughput under concurrent workloads and compare TSA with the baselines under matched computational budgets? An overview figure showing the generation, moderation, and embedding-update loop would also clarify the method and its computational cost.
Q2. TSA searches for embedding updates that reduce the OpenAI Moderation score, and Table 1 evaluates the resulting responses using scores and flags from the same moderation model. Thus, Table 1 alone cannot distinguish genuine improvements in safety from improvements specific to that moderation model. Although the HarmBench and Llama Guard results alleviate this concern, can the authors provide further evidence that TSA improves safety beyond the moderation model used during optimization?
Q3. Can the authors provide systematic human evaluation to verify that TSA produces genuinely safer responses, rather than responses that merely receive lower scores from automatic safety evaluators?
Q4. Can the authors evaluate whether TSA preserves helpfulness, relevance, coherence, and task completion on both harmful and benign prompts? Lower safety scores alone do not demonstrate that response utility is preserved.
Q5. Why does TSA leave 59 responses flagged on Gemma 3 while reducing the flagged count to zero on the other four models? Do these failures arise from the fixed optimization budget, model-specific embedding properties, or limitations of the moderation objective?
5 comments
Rebuttal — Part 2/3
[Weakness & Q1] "An overview figure showing the generation, moderation, and embedding-update loop"
Thank you for the suggestion. We will add an overview figure to Section 4 showing the generation, moderation and embedding-update loop, annotated with its cost. Each iteration costs generations and oracle calls, with the perturbed ones issued as one batch, and a prompt already below the threshold costs one of each.
[Weakness] "separately report the success rate, score reduction, and residual flagged count among prompts that actually trigger optimization"
We now report these metrics separately. The table covers Table 1's runs on the prompts whose first response scores at least 0.1, which are the only ones TSA modifies (WildJailbreak adversarial-harmful).
| Mean score, undefended TSA | Ends below 0.1 | Ends unflagged | Residual flagged | |
|---|---|---|---|---|
| Llama | 0.240 0.003 | 100% | 100% | 0 |
| Qwen | 0.240 0.027 | 100% | 100% | 0 |
| Phi | 0.223 0.018 | 100% | 100% | 0 |
| Gemma | 0.276 0.120 | 65% | 93% | 59 |
[Q1] "report throughput under concurrent workloads and compare TSA with the baselines under matched computational budgets"
We measured end-to-end throughput, oracle calls included, for Llama 3.1-8B on the two A6000s of Appendix B. The engine served adversarial-harmful prompts at increasing concurrency, with three runs per level.
| Concurrent requests | 1 | 2 | 4 | 8 | 12 | 16 |
|---|---|---|---|---|---|---|
| Throughput (prompts/s) | 0.28 | 0.49 | 0.81 | 1.21 | 1.36 | 1.44 |
| Mean latency (s) | 3.6 | 4.1 | 4.9 | 6.5 | 8.6 | 10.7 |
The plateau beyond 12 concurrent requests comes from the serving engine's 32 sequence slots filling up, while GPU utilization falls. With one request in flight, the oracle takes 11% of wall-clock time. Under load, its latency overlaps with other requests' generation, and removing the oracle would raise throughput by at most 4–11%. The API rate limit was never approached (peak 148 of 2,000 requests per minute).
In short, TSA's throughput rises almost fivefold with concurrency before the engine's slots fill, and the oracle is not the bottleneck. The matched-budget comparison is Best-of-N above.
[Q2 & Q3] "further evidence that TSA improves safety beyond the moderation model used during optimization" and "systematic human evaluation"
We ran a blinded human audit that uses no automatic evaluator. Two annotators compared the undefended and TSA responses on 200 pairs on which TSA acted. The 157 adversarial-harmful pairs are those where TSA lowered the moderation score, taken from the four models in proportion, so the audit asks whether the oracle's reductions are visible to a human. The other 43 are all the adversarial-benign prompts, across the four models, that enter the loop (i.e., initial score larger than 0.1). The annotators never saw which system produced which response, and the A/B order was random. Each pair received three kinds of answers:
- Which response is safer? (
A,B, ortie) - Which better addresses the prompt's legitimate intent? (
A,B,tie, ornot applicable) - Does either response contain actionable harmful content? (
yesornofor each)
Their agreement was on safety and on legitimate intent (Cohen's , agreement beyond chance, with 1 for perfect agreement). Disagreements were settled jointly, and the table counts the agreed labels. The fourth column ignores ties and reports how often TSA's response was the safer one among the remaining pairs. Its -value is an exact binomial test of that rate against 50%, the value expected if neither system were safer.
| Pairs | TSA safer | Undefended safer | TSA safer, share of non-tied pairs | Equally safe |
|---|---|---|---|---|
| All 200 | 155 | 26 | 86% () | 19 |
| 157 harmful | 125 | 19 | 87% () | 13 |
| 43 benign | 30 | 7 | 81% () | 6 |
Each annotator's labels alone give 85% and 89%. Actionable harmful content appeared in 44 undefended and 3 TSA responses, a significant difference. The two held-out judges in Section 5.1 and Appendix E, which TSA never optimizes against, point the same way.
Human readers who never saw a moderation score therefore judge TSA's responses genuinely safer, so its gains are not specific to the moderation model it optimizes.
Rebuttal — Part 3/3
[Q4] "whether TSA preserves helpfulness, relevance, coherence, and task completion on both harmful and benign prompts"
We ran a blinded LLM judge (Claude Opus 5.5 on extra-high effort, three runs per item), which rated all 840 benign responses (the four models' answers to the 210 adversarial-benign prompts) on a scale of 1 to 5 on helpfulness, relevance, coherence and task completion. The table gives each model's mean rating as TSA / undefended.
| Coherence | Relevance | Helpfulness | Task completion | |
|---|---|---|---|---|
| Llama | 3.84 / 3.91 | 4.02 / 4.19 | 3.08 / 3.17 | 2.18 / 2.20 |
| Qwen | 3.90 / 3.92 | 4.44 / 4.44 | 2.85 / 2.85 | 2.11 / 2.11 |
| Phi | 3.79 / 3.87 | 4.28 / 4.31 | 2.96 / 2.94 | 2.13 / 2.12 |
| Gemma | 3.81 / 3.82 | 4.06 / 4.04 | 2.75 / 2.76 | 2.05 / 2.06 |
Task completion is low for both systems because many adversarial-benign prompts have no task to complete. No difference from the undefended model survives Holm correction on any model, and every Hodges–Lehmann estimate of the difference is within 0.17. Put plainly, the judge finds no significant drop in any of the four qualities over the benign split.
TSA leaves responses below the threshold unchanged, so any cost is confined to the prompts where it acts. There, the human audit judged 27 of 42 applicable benign responses worse at serving the legitimate intent and 15 equal or better, and among the judge's ratings only Llama's drop is significant. Although these prompts are benign, TSA's first response to them already scores at least 0.1, and TSA gives up some relevance and helpfulness to lower that score. It corresponds to only the 3.2% of all adversarial-benign model–prompt pairs.
On harmful prompts we do not score task completion, because completing the request is the failure TSA prevents and many such requests cannot be helped without declining. On coherence and responsiveness, Qwen's responses are surprisingly much more coherent than the undefended model's (4.30 against 3.97 out of 5) while engaging the request less directly. Gemma stays near the undefended level on both, with no difference surviving Holm correction. Llama's responses mostly, and Phi's often, stop addressing the prompt to prevent the model from engaging with the malicious request.
In the revision, these measurements replace the informal reading in Section 5.1.
[Q5] "Why does TSA leave 59 responses flagged on Gemma 3"
TSA reduces Gemma's flagged responses from 278 to 59, against 225 for SmoothLLM for comparison (Table 1). On the 845 Gemma prompts of the comparisons above, it leaves fewer flagged than every baseline compared there except Best-of-N at (discussed in the Best-of-N response). Taking the three causes you name in turn:
- The fixed iteration budget: All 59 are prompts on which TSA used its entire iteration budget without its score falling below 0.1. On the other three models in our table of prompts that enter the loop, every such prompt falls below 0.1 within the budget, so the 59 are where Gemma's slower convergence (Table 1) meets the budget. We consider this the explanation for the 59.
- Model-specific embedding properties: Appendix F tests the two explanations that the tested models confound, embedding dimension and base alignment. Qwen3-0.6B has a smaller embedding dimension than Gemma ( against 1152) and starts worse (470 flagged against 278), yet TSA clears every flag in about one iteration. TSA also clears all 527 flags of an abliterated Qwen3-14B. What slows Gemma's convergence is not its embedding dimension or its base alignment, and Appendix F does not single out one property.
- The moderation objective: It still carries signal on Gemma. Following Reviewer HAsA's suggestion, we replaced TSA's estimated direction with a random direction of the same norm, run under the same iteration cap. On Gemma, the random direction scores significantly worse and leaves more prompts flagged, 121 against 59 on the same 845 prompts. On the prompts TSA acts on (same table), the mean score falls from 0.276 to 0.120.
The revised paper will include all of these results.
Rebuttal — Part 1/3
Thank you for your careful review.
We address each weakness and question below with new experiments. Unless noted otherwise, all new experiments use the adversarial-harmful split of WildJailbreak. Comparisons are paired by prompt over the prompts that enter TSA's optimization loop, meaning prompts whose first response scores at least 0.1, with Holm correction across models.
[Weakness] "AdaSteer and RESTA are evaluated only on Llama 3.1-8B … runtime is reported only for TSA"We ran RESTA on Llama, Qwen, Phi and Gemma. We swept its noise scale σ per model, and Llama keeps its paper's 0.08. The table gives the mean moderation score as
undefended / RESTA / TSA.TSA's mean score is significantly lower than RESTA's in every cell.
However, AdaSteer is different. RESTA has a single noise scale to sweep, while AdaSteer fits steering layers and coefficient maps to each model. Among all the models tested, only Llama 3.1-8B has a published AdaSteer configuration, so a port would be our own unvalidated calibration rather than the published method.
We also measured the runtime of every baseline. The table gives the mean time per prompt on WildJailbreak adversarial-harmful prompts.
TSA's time is end to end over a uniform prompt mix, where a prompt below the threshold costs one generation and one moderation call. TSA spends its extra time only on prompts that enter its loop, and on every model this buys a mean moderation score on these prompts considerably lower than SmoothLLM, RESTA or AdaSteer reaches (Table 1 and the RESTA table above).
[Weakness] "A compute-matched Best-of-N baseline is missing"We ran the compute-matched Best-of-N you suggested. For every prompt, it samples as many completions as TSA generated on that prompt (early stopping included) and keeps the one with the lowest moderation score. Generations and oracle calls are therefore matched prompt by prompt. We ran it at T=0.1 and at T=1.0, where a deliberately high temperature is used for diverse samples.
On Llama, Qwen and Phi at both temperatures, and on Gemma at T=0.1, TSA has a significantly lower mean moderation score and leaves significantly fewer prompts flagged. With the budget matched, resampling does not reach TSA's mean score on these models, so TSA's results are not explained by its larger budget alone.
We will cite both papers [1, 2] you pointed to when the revised paper discusses Best-of-N.
On Gemma, Best-of-N at T=1.0 leaves fewer prompts flagged than TSA. Gemma is the model on which TSA converges most slowly (Table 1), and every prompt TSA leaves flagged there ran out of iterations before its score fell below 0.1. Extra budget helps the two methods differently. Best-of-N draws every sample from the same distribution, so each added sample lowers its expected best score by less than the previous one did, and no number of samples can take its best score below the lowest score that distribution can produce. TSA changes the input after every iteration, so its samples do not all come from one distribution, and we expect it to overtake Best-of-N on Gemma as the budget grows.
Following Reviewer HAsA's suggestion, we also ran two other baselines with oracle access:
The table counts the prompts each method leaves flagged.
∗Significantly more than TSA.
With the same oracle, neither baseline leaves fewer prompts flagged than TSA on any model, and TSA leaves significantly fewer than the safety prompt on every model and than RESTA on Gemma.