acceptodds
Desk rejected at ICLR 2027

TTS-RiskArena: Unsafe at Scale? Evaluating and Mitigating Risk in Test-Time Scaling

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Test-time scaling (TTS) improves the capability of large language models by allocating additional inference-time computation to explore multiple reasoning trajectories. However, its safety implications remain largely unexplored. A model that appears aligned under standard decoding may behave differently when given larger search budgets, more samples, or iterative refinement, raising a fundamental question: Does TTS preserve safety, or can it amplify unsafe behavior? This work introduces TTS-RiskArena, a benchmark to systematically evaluate safety under test-time scaling. TTS-RiskArena spans seven safety-critical domains, including health, finance, coding, child, scientific discovery, travel, and biochemistry, and enables controlled evaluation across diverse TTS methods and inference budgets. The benchmark further organizes prompts across multiple abstraction levels, making it possible to analyze how prompt specificity interacts with safety under increasingly exploratory inference. Further, we argue that TTS is fundamentally a search over latent reasoning space, with the decoding process verbalizing the trajectories it reaches. Consequently, the chance of producing a safe response is largely determined by whether early inference makes safe regions of that space more reachable. Based on this intuition, we propose a contrastive decoding method that guides test-time exploration toward safer regions of the output space. Experiments show that the proposed method consistently improves safety while retaining the benefits of TTS.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Safety, Test time scaling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_j3gaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces TTS-RiskArena, a benchmark for evaluating how test-time scaling affects LLM safety. It contains 42,000 prompts across seven safety-critical domains and evaluates three TTS strategies. The authors show that additional test-time computation can increase unsafe response rates, particularly under more concrete and procedural prompts. They further separate generation risk from selection risk and propose a contrastive decoding method, Safe Search-Space Steering, to guide TTS toward safer trajectories without additional training.

Critical strengths
  1. The paper studies an important and timely safety problem. Test-time scaling is increasingly used to improve model reasoning, so understanding how safety changes with larger inference budgets is practically relevant.
  2. Treating safety as a trajectory over inference budget, rather than as a fixed property of the model, is a useful perspective.
Critical weaknesses
  1. The benchmark construction and seed-selection procedure are not sufficiently specified. The benchmark is largely built by selecting and adapting examples from existing safety benchmarks, but the paper does not clearly report how many examples come from each source, how they are distributed across domains, how many samples remain after each filtering stage, or the exact criteria used to select "representative" seed queries. Since the benchmark itself is a central contribution, these missing details limit reproducibility.
  2. LLM-as-a-judge is still a debatable evaluation choice, and some of the cited benchmarks also discuss its limitations. This paper relies heavily on automated LLM judging despite covering difficult, domain-specific safety problems such as health, scientific discovery, biochemistry, finance, and child safety. Human or domain-expert evaluation on at least a subset would make the conclusions more convincing.
  3. The model coverage is limited and relatively outdated, with no closed-source frontier LLMs evaluated. Broader testing on more recent models would strengthen the generality of the conclusions.
Questions for authors

Could the authors provide more detailed benchmark-construction statistics? In particular, how many seed examples come from each source benchmark and domain, how many remain after each filtering stage, which reference model and thresholds are used for safety-salience filtering, and how the final 700 malicious / 300 benign examples per domain are sampled?

Minor presentation issue: the current appendix is poorly organized and quite difficult to follow. Better ordering and clearer section structure would improve readability.

1 comment
AuthorsOpenReview ↗

Response to Reviewer j3ga

Thank you for recognizing the practical importance of our work and the value of tracing how a model’s safety changes as that budget grows. We respond to your questions and concerns below.

W1 and Q1. How were seeds selected, filtered, and sampled?

We drew seeds from benchmarks including HarmBench, JailbreakBench, and SpecBench (§3.1). We selected queries relevant to each domain’s safety criteria that could be adopted or lightly adapted for single-turn use. Appendix A describes exact and embedding-based deduplication, format checks, repair or removal of rewrites at the wrong abstraction level, and primary-domain assignment. The final safety-salience check used a reference model at a fixed budget and retained prompts with a safety-classifier signal or a human concern flag.

Figure 1 shows random sampling for balance, with a final quota of 700 malicious and 300 benign tasks in each domain (§3.4). The submission gives these final counts by domain and intent. It does not tabulate seeds by source and domain or specify per-filter counts, the reference checkpoint, or the numerical cutoffs for deduplication and safety salience.

W2. Human or domain-expert validation

We agree that expert validation would be helpful, and would be particularly informative for borderline responses in health and biochemistry. We compared Llama-Guard-3-8B with human judgments on 150 responses to malicious prompts. The human criterion was whether the answer materially helped carry out the requested action. The judge agreed on 133/150 (88.7%, Cohen’s κ = 0.773). It flagged 10 of 80 human-rated safe answers (12.5% false positives) and missed 7 of 70 human-rated unsafe answers (10.0% false negatives).

W3. Model coverage and applicability to closed models

We evaluated two sizes each from Qwen2.5, Llama 3.1, and Gemma 2, spanning 7B to 70B parameters. Running the search gives us the candidate pools, and open weights give us the intermediate-layer scores used by the steering rule. For a closed model with adjustable reasoning effort, final-answer safety and benign utility are measurable at each supported setting. Exposure and promotion additionally require access to the candidates considered.

Minor comment. Appendix organization

Thank you for the helpful suggestion! We will place the task examples next to the construction protocol, group the safety and utility trajectories in Figures 4 through 7 together, and move the core steering rule into §5 with the derivation in Appendix G.

Reviewer_Nc7YOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how test-time scaling (TTS) affects the safety of LLMs. The authors introduce TTS-RiskArena, a benchmark covering seven safety-critical domains, with both malicious and benign tasks at different abstraction levels. They evaluate Best-of-N, MCTS, and Self-Consistency under different inference budgets and examine how safety changes as the budget increases. An interesting aspect of the evaluation is the distinction between unsafe exposure during generation and unsafe promotion during candidate selection. The paper also proposes a contrastive-decoding-based method to steer the search process toward safer outputs without additional training or external reward models.

Critical strengths
  1. Safety under test-time scaling is underexplored, and the paper makes a convincing case that safety should be evaluated as a function of inference budget and search mechanism rather than under a single fixed decoding configuration. This is consequential because deployed systems increasingly use sampling, search, and reranking.

  2. TTS-RiskArena’s combination of seven safety-critical domains, malicious/benign splits, multi-level abstraction, and safety-trajectory metrics has the potential to support controlled comparisons. The decomposition into unsafe exposure vs. unsafe promotion rate is a particularly useful lens for understanding where TTS risk comes from.

Critical weaknesses
  1. Insufficient validation of the proposed mitigation method. The paper claims that safety-guided contrastive decoding improves safety while preserving utility, but provides no direct quantitative results, comparisons with safety baselines, or ablation studies.

  2. Insufficient benchmark validation and reproducibility. The relationship between the reported 42000 prompts and 7000 underlying tasks is not clearly explained. The paper also lacks sufficient validation of LLM-based safety judgments, including human agreement and error rates.

Questions for authors
  1. Could the authors provide quantitative results for the proposed safety-guided contrastive decoding method, including comparisons with safety baselines and ablation studies, to substantiate its effectiveness?

  2. Could the authors clarify how the 42,000 prompts were constructed from the 7000 underlying tasks and provide evidence of the reliability of LLM-based safety judgments, such as human agreement and error rates?

1 comment
AuthorsOpenReview ↗

Response to Reviewer Nc7Y

Thank you for highlighting our comprehensive design and for recognizing the importance of ivestigating how risk changes with the inference budget. We respond to your questions and concerns below.

W1 and Q1. What evidence supports Safe Search-Space Steering?

We evaluated Llama-3.1-8B-Instruct at budget 32 across all seven domains, using Llama-Guard-3-8B to score both conditions. Each cell shows the result with steering from off to on. Final ASR is the fraction of final answers judged unsafe. Safe Utility measures helpfulness among final answers judged safe.

Search methodFinal ASRUnsafe exposureSafe Utility
Best-of-N18.9% → 11.2%33.2% → 19.8%90.9% → 89.6%
Self-Consistency18.9% → 11.1%33.3% → 19.9%91.0% → 90.5%
Reward-guided beam search17.0% → 15.3%26.6% → 24.0%96.0% → 95.8%

Under Best-of-N, both final ASR and unsafe exposure fell in every domain. For beam search, the comparison includes only prompts for which both runs reached budget 32.

Safety baselines. On 5,400 malicious prompt instances from health, biochemistry, and code, final ASR at budget 32 was 32.1% with reward-only selection, 21.8% with steering, 19.4% with an independent HarmBench classifier filtering completed candidates, and 20.8% with a lexical filter. Both filters act after generation. Steering also reduced the fraction of pools containing an unsafe answer from 51.6% to 36.0%.

Component ablations. On a fixed subset from those three domains, replacing a broad safety lexicon with one that excludes ambiguous tokens lowered final ASR from 30.0% to 21.8%. Applying guidance only in the first four decoding steps yielded 21.0%. The safety term accounted for most of the improvement on this subset. The contribution of layer contrast varied across prompts.

W2 and Q2(a). How do 7,000 tasks yield 42,000 prompts?

Each of the seven domains contributes 700 malicious and 300 benign tasks, giving 4,900 malicious and 2,100 benign underlying tasks (§3.4). For each task, we count the original prompt, an adversarial counterpart, and four rewrites ranging from general intent to a procedural request (§3.1, Fig. 1). Thus Table 1 counts 7,000 × 6 = 42,000 prompt instances. The six forms let us compare how wording changes safety while holding the underlying objective fixed.

W2 and Q2(b). How reliable are the output judgments?

We compared Llama-Guard-3-8B with human judgments on 150 responses to malicious prompts. The human criterion was whether the answer materially helped carry out the requested action. The judge agreed on 133/150 (88.7%, Cohen’s κ = 0.773). It flagged 10 of 80 human-rated safe answers (12.5% false positives) and missed 7 of 70 human-rated unsafe answers (10.0% false negatives). On the three-domain subset above, rescoring the same generations with the HarmBench classifier also showed a reduction under steering, from 80.0% to 66.0% final ASR.

Reviewer_G5mNOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper benchmarks the safety of parallel CoT + majority vote, i.e. the test time scaling before deepseek R1 comes out. The benchmark is composed of questions from lots of different domains. The paper also suggests an approach for improving the safety, however it is not presented in the main text. Overall I don't find many interesting and exciting things to learn from the paper.

Critical strengths
  • The problem studied is somewhat important in 2024;

  • The benchmark considered covered a lot of domains

  • The authors considered 6 (very old) models.

Critical weaknesses
  • The template is incorrect: The page margin is wrong.

  • The "test time scaling" definition studied here is out of date.

  • There is no solution provided to improve the safety of parallel CoT in the main text.

Questions for authors

Can the authors explain why they submitted the manuscript to iclr 2027 and what's the value of the benchmark when parallel CoT / MCTS is no longer a focus of practitioners?

1 comment

Dear Reviewer G5mN,

Hope you are doing fine.


First of all, we applied the official template for this submission. I humbly ask for a pointer to the wrong margin size of our submission.

W2 and Q1. Why is this benchmark relevant after DeepSeek-R1?

If the same model can answer a request at low or high reasoning effort, which setting does its reported safety score describe? Developers already make this choice through OpenAI’s reasoning-effort parameter [1]. Anthropic’s Research system scales the number of parallel agents and tool calls with the complexity of the question [2]. In scientific discovery, Google’s Co-Scientist spends additional test-time compute generating, critiquing, and ranking hypotheses. Figure 2 of its 2026 Nature paper tracks how hypothesis quality changes as that computation increases [3]. TTS-RiskArena asks how safety changes along the same axis. For a scientist reading the top-ranked proposal, it matters whether the system considered an unsafe idea and discarded it or recommended it despite having a safe alternative.

As for the suggested historical cutoff, majority voting has a column of its own in Table 2 of the DeepSeek-R1 report [4]. In our benchmark, it accounts for one of three conditions. Best-of-N ranks completed answers, and MCTS chooses which branches to expand. Running all three on six open checkpoints lets us inspect the candidate pools and measure both unsafe exposure during generation and unsafe promotion when a safe alternative is available (§4.4).

W3. Placement of the safety intervention

The main text introduces the intervention on p. 8 in §5, titled Enhancing TTS Safety via Search-Space Steering. The method adjusts token scores during candidate generation and applies its strongest safety guidance early in decoding. Appendix G gives the derivation. We will bring the central decoding rule into §5 for ease of reference.

[1] OpenAI. Reasoning models, section on reasoning effort.

[2] Anthropic (2025). How we built our multi-agent research system.

[3] Gottweis et al. (2026). Accelerating scientific discovery with Co-Scientist, Fig. 2. Nature.

[4] DeepSeek-AI (2025). DeepSeek-R1 technical report, Table 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.