acceptodds
Under review as a conference paper at ICLR 2027

Robust Distributed Test-Time Scaling

Abstract

Test-time scaling (TTS) improves the reasoning performance of large language models by allocating more computation at inference time. At scale, the independent generations underlying TTS can naturally be distributed across multiple workers and aggregated into a final prediction. This paradigm, however, inherits a fundamental challenge from distributed computing: tolerating faulty or compromised workers. To address this challenge, we develop proof-of-thought consensus, a robust aggregation framework for distributed TTS. The key insight is that producing a chain-of-thought (CoT) trace that remains statistically consistent with the model’s decoding distribution is inherently costly for an adversary. We then introduce an efficient CoT verification mechanism that turns this asymmetry into a practical filter by detecting subtle distributional inconsistencies in manipulated CoT traces before aggregation. We complement the design with theoretical and adversarial analyses: we prove high-probability acceptance guarantees for honest submissions, and formalize three strong attack strategies for stress-testing. Experiments show that, even with up to 30% malicious participants, we limit accuracy degradation to 6.7% on average relative to an oracle baseline that aggregates only the clean samples, while incurring only a 1% additional inference-time overhead.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Collaborative Inference, Robust Ensembles, Federated Ensembles

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XvhAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a defense method against tolerating faulty in distributed test-time scaling paradigm. The key idea is to use an oddity statistic to identify unreliable CoTs. Experiments on several reasoning models and benchmarks demonstrate the efficacy of the proposed method.

Critical strengths
  1. The threat is well motivated and practically relevant.
  2. The proposed method is reasonable and easy to follow.
  3. Evaluation covers several attack scenarios, including prompt substitution, post-generation trace manipulation, and model substitution.
Critical weaknesses
  1. The proposed oddity metric averages token-level statistics over the entire sequence. Consequently, an attack affecting only a small number of token positions may be diluted.
  2. The framework assumes that all honest workers use the same model and decoding configurations. This limits applicability to heterogeneous test-time scaling scenarios, since correct heterogeneous models may be identified as attackers.
  3. The proposed metric is similar to existing likelihood-based statistics, and using this statistics to detect distributional inconsistencies is intuitive. The novelty appears limited.
Questions for authors

Please refer to weaknesses.

2 comments

Thank you for your valuable comments. We address all the reviewer’s questions below.

Weakness 1. An attack affecting only a small number of token positions

The original submission already evaluates attacks that modify only a small number of token positions through trace-side injection (Section 5.1 and Appendix B.3). Our experimental results show that PoT can detect these attacks by combining oddity with answer-token certainty (Section 4.4). We explain how below.

In trace-side injection, the attacker generates a valid CoT trace and replaces its final answer with a target incorrect answer. Since honest answer tokens typically have perplexity close to 1 (see Section 4.4), this replacement can increase answer-token perplexity and trigger PoT’s answer-token certainty check, whose signal is not diluted over the entire trace. To evade this check, the most effective approach among those tested replaces all occurrences of the original answer in the trace with the target answer. This lowers answer-token perplexity but can introduce locally unlikely tokens and increase oddity (Figure 10), creating a trade-off that enables the two checks to provide complementary defenses. As shown in Table 2, 31.17% of trace-side injection candidates pass the oddity check alone, and 14.83% pass the answer-token check alone. Only 6.00% pass both checks, demonstrating the benefit of combining these complementary signals.

Weakness 2. Applicability to heterogeneous test-time scaling

While the original submission considers a common model to preserve equivalence with conventional local TTS, PoT can also support heterogeneous models. The verifier assigns each worker a reference model, so it knows which model to use when verifying each returned trace. Using the corresponding model’s configuration, the verifier filters each trace separately and then pools the retained answers across models for a single vote.

To demonstrate this, we conducted an additional experiment on AIME25 using 10 Qwen3-4B workers and 10 DeepSeek-R1-8B workers per question. We used the same three attacks described in Section 5 and Appendix B of the original submission. For each attack, we corrupted three submissions from each model, yielding a pool with 30% malicious submissions, and evaluated all methods on the same subsets of n=15 submissions drawn from this pool.

Across the three attacks, PoT retained 89.0% of honest Qwen traces and 85.2% of honest DeepSeek traces, preserving the large majority of honest contributions from both models. As shown below, PoT improves accuracy over Self-Consistency by 13.8–24.6 percentage points and achieves the highest accuracy among the non-oracle methods under every attack. This is consistent with the trend observed in our homogeneous-model experiments and demonstrates that PoT’s robustness benefits extend to heterogeneous TTS.

The following table reports accuracy (%) on AIME25 for n=15 submissions drawn from the heterogeneous worker pool with 30% malicious submissions.

MethodPrompt-side SteeringModel-side SubstitutionTrace-side Injection
Self-Consistency41.8%41.9%41.9%
Self-Certainty28.7%25.1%29.9%
LLM-as-a-Judge39.5%43.0%44.7%
Median-based Robust36.7%36.7%36.8%
Proof-of-Thought (Ours)66.4%66.1%55.7%
Oracle70.9%70.8%70.7%

Weakness 3. Novelty of PoT

While oddity is derived from token probabilities, PoT does not simply identify low-likelihood traces as malicious. The experiments in Sections 3.5 and 5 show that perplexity and Self-Certainty (Kang et al., 2025) fail to reliably distinguish honest from malicious submissions in distributed TTS. These results demonstrate that straightforward likelihood-based scoring and aggregation do not provide adequate protection against the evaluated attacks.

To further evaluate PoT against existing likelihood-based statistics, we added the method of Miao & Fang (2025) [1] as a new baseline. Their method compares the submitted token’s logit with the highest logit at each position and uses the lowest-confidence window. We evaluated this baseline on AIME25 with Qwen3-4B and 30% malicious participants. PoT achieves higher average accuracy across all three attack surfaces. Specifically, under trace-side injection, PoT achieves 61.14% average accuracy, compared with 49.20% for Miao & Fang (2025), an improvement of 11.94 percentage points. These results support our contribution as a verification-and-aggregation framework designed to address different forms of adversarial manipulation, rather than the use of token probabilities alone.

[1] Qijun Miao and Zhixuan Fang. 2025. User-side Model Consistency Monitoring for Open Source Large Language Models Inference Services. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics.

Dear Reviewer,

Thank you once more for your insightful and constructive comments and for the time you have dedicated to reviewing our paper. As the discussion phase comes to an end, kindly let us know if you have any further questions or concerns.

Best regards,

Authors

Reviewer_nR7EOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies the robustness of distributed test-time scaling (TTS) when some workers are malicious. It proposes proof-of-thought (PoT) consensus, which requires workers to return chain-of-thought (CoT) traces and filters submissions before answer aggregation. The main verification statistic, oddity, is based on rectified centered surprisal under the reference model. The system also uses an answer-token certainty check and similarity-based duplicate filtering. The paper provides concentration results for honest traces and a conditional consensus-recovery analysis, and evaluates the approach across four reasoning models, four benchmarks, and three attack surfaces.

Critical strengths
  • Relevant problem setting. Robust aggregation is a meaningful concern once TTS sampling is distributed across untrusted workers, and the paper motivates this setting clearly.
  • Lightweight verification mechanism. Oddity can be computed from a teacher-forced prefill rather than additional autoregressive generation. The answer-token check and anti-copying step address manipulation modes that the oddity score alone may miss.
  • Broad empirical evaluation. The experiments cover multiple model sizes, reasoning benchmarks, malicious-worker ratios, and three attack surfaces. Under the evaluated attacks, PoT is consistently more robust than the aggregation baselines.
Critical weaknesses
  • The security claim is stronger than what the verifier directly establishes. PoT tests whether a submitted trace is statistically consistent with the reference model; it does not show that the worker faithfully executed the prescribed inference procedure. In particular, a malicious worker can search among plausible traces and selectively return one supporting an incorrect answer. The paper should more clearly characterize robustness to this type of strategic sample selection and delimit what is actually being proved.
  • The theoretical recovery guarantee is conditional on an assumed attack-success bound. Appendix C assumes an upper bound q_A on the conditional probability that each malicious submission passes verification, but this bound is not derived from the proposed threat model. The empirical rates in Table 2 are useful measurements, but they do not establish such a bound for an adaptive adversary. In addition, the completeness analysis is based on the oddity test, whereas the implemented verifier also includes answer-token filtering and anti-copying; the relationship between the full system and the stated recovery guarantee is therefore incomplete.
  • The white-box attack evaluation could be stronger. The three attacks are verifier-aware, but they are still specific hand-designed strategies. Given that the adversary knows the verification rule and thresholds, it would be useful to evaluate an attack that explicitly searches or optimizes for a low verification score subject to producing a targeted wrong answer under the same compute budget.
  • The contribution of individual verification components is not isolated. The final system combines oddity, answer-token certainty, and anti-copying, but the evaluation does not provide a clear component-level ablation. This makes it difficult to determine which mechanism is responsible for the robustness gains and how much each refinement contributes.
  • The reported overhead does not yet establish end-to-end distributed efficiency. Figure 8 compares the verification time of a trace with its generation time. However, the design uses a verifier that processes submissions from multiple workers. Throughput, batching behavior, GPU demand, and latency as the number of submitted traces grows are not evaluated, so the approximately 1% per-trace ratio should not be interpreted directly as system-level overhead.
Questions for authors
  • What precise security guarantee does PoT provide, particularly against an adversary that repeatedly samples or searches for an incorrect but statistically plausible trace?
  • Can qAq_A in Appendix C be derived under an explicit attack model? If not, could the theorem be stated more clearly as a conditional result?
  • How do the additional answer-token and anti-copying steps affect the theoretical analysis, and can they be incorporated into the stated guarantee?
  • How robust is PoT against an optimization-based adaptive attack designed to evade the verifier?
  • Can a component-level ablation isolate the contribution of each part of the verification procedure?
  • How do verifier throughput and latency scale with workload size and the number of workers?
4 comments

Thank you for your valuable comments. We address all the reviewer’s questions below.

Weakness 1 & Question 1: Repeated sampling and selective submission

For this attack to succeed, the adversary faces two challenges: obtaining a trace that passes verification within its online compute budget and knowing which answer is correct. We analyze below how these limit the effectiveness of repeated sampling and selective submission, and also evaluate this attack in a new experiment.

Budget for searching. Our threat model explicitly allows a malicious worker to search for a trace supporting the target answer and selectively submit it (Section 5.1 and Appendix B). As illustrated in Figure 9, each malicious worker has the same online compute budget as an honest worker. After seeing the query, the attacker can allocate this budget to generating traces, modifying existing traces, and evaluating candidates. However, each fresh sample requires autoregressive generation, so repeated sampling quickly consumes the available budget. Selective submission is therefore allowed, but the search needed to obtain a suitable submission must fit within this budget.

Knowledge of the target answer. During actual execution, the attacker does not necessarily know which answers are correct and which are incorrect. This means the attacker cannot necessarily identify and selectively submit a trace that supports an incorrect answer.

To assess this attack, we conducted an additional experiment on AIME25 using Qwen3-4B, with 30% malicious participants. After receiving the original query, each malicious worker repeatedly samples fresh traces using the reference model, searching for a candidate that supports the targeted incorrect answer and passes verification. The search continues until the online compute budget is exhausted, using the same budget as the three original attack types. As shown below, PoT remains close to the honest-only Oracle under this attack.

PoT@n35791215
Repeated sampling63.3%66.0%68.0%69.3%70.5%71.2%
Honest-only Oracle65.6%68.4%70.3%72.1%73.1%73.4%

Weakness 2 & Question 2: Conditional recovery guarantee

We agree with the reviewer that the upper bound qAq_A is not derived from the threat model. It is used only as input for Theorems C.3 and C.5 to obtain a probability bound on PoT failure, given this bound on the per-attempt attack success rate. For clarity, we will revise the relevant paragraph in Appendix C.2 to state it as a named assumption. We will also make the two subsequent theorems explicitly conditional on it.

A general, informative upper bound qAq_A is not available. This is because qAq_A depends on the exact attack the adversary implements, and our threat model constrains the adversary’s online compute budget but allows it to choose how to generate and select candidates. As an extreme, the adversary could behave honestly, thus raising the probability of passing the checks. As explained in the text, we use empirical measurements of qAq_A (available in Table 2) for each attack to interpret the theorem.

Weakness 2 & Question 3: Completeness analysis

Answer-token check. This is already part of our guarantee on the attack side. The upper bound qAq_A, as well as the values from Table 2, concerns the joint pass rate, i.e., passing both the oddity and answer-token checks. This is consistent with the definitions in Section 4.4.

For the honest side, we can directly extend Theorem C.5 with a union bound by replacing δH\delta_H with δH+δans\delta_H + \delta_{ans}, where δans\delta_{ans} is calibrated with the values from Table 1.

Anti-copying. This can only remove submissions, so it does not affect the bound on the attack side. Its role is to make the conditional assumption discussed in response 2 above plausible. Without it, one accepted trace could be copied across nodes, raising the conditional pass probability for subsequent copies from a bound of qAq_A to 1. On the honest side, it can only drop a vote if two independently sampled honest traces with the same answer exceed a cosine similarity of 0.97. This never happened in our experiments.

Weakness 3 & Question 4: White-box adaptive attacks

The trace-side injection attack in the submission (Section 5.1 and Appendix B.3) already instantiates the constrained, verifier-aware search described in this question. Its objective is to produce a target incorrect answer while simultaneously passing the oddity and answer-token perplexity checks within the online compute budget.

Specifically, the attacker first generates a trace for the original query and replaces its final answer with the target answer. Rather than submitting this edited trace immediately, it then modifies it and evaluates both verification scores after each modification, continuing the search until both checks pass or the budget is exhausted. The attacker knows the complete verification rule and thresholds, and the budget includes generation, modification, and verifier evaluations.

Figure 10 illustrates this search: replacing answer occurrences throughout the trace reduces answer-token perplexity but can increase oddity, so the attacker must find a candidate satisfying both constraints. Thus, although the editing operations are predefined, the attack is not a one-shot perturbation. It searches for a successful submission using feedback from the verification scheme.

Weakness 4 & Question 5: Component-level ablation

Table 2 already reports separate and joint pass rates for oddity and answer-token certainty. For each attack, we evaluate both checks on the same set of malicious candidates. The individual pass rates show how many candidates each check would accept on its own, and the joint rate shows how many pass both. Oddity alone lets 0.00% of prompt-side steering candidates and 0.33% of model-side substitution candidates pass. For trace-side injection, 31.17% pass oddity alone and 14.83% pass the answer-token check alone, while only 6.00% pass both. The answer-token check therefore provides its largest additional benefit against trace-side injection, where changing a few answer tokens can leave the sequence-level oddity low.

Moreover, the anti-copying criterion is a hard requirement on the entire submission pool. Before submission, workers check their candidate traces against this criterion and submit only traces that satisfy it. As a result, the verifier’s anti-copying check removes no submitted traces in our experiment.

Weakness 5 & Question 6: End-to-end distributed efficiency

Figure 8 already measures verification overhead in an end-to-end distributed setup, where multiple workers send their traces to a verifier that processes submissions in batches using vLLM. We benchmarked the verifier under a higher-load setting with a maximum batch size of 64. In many of our evaluated TTS settings, accuracy gains taper off at around 15 traces, so we consider 1–15 traces per query a practical operating range. The reported verification-to-decoding time ratio of 1% was measured under this heavier batched workload.

Dear Reviewer,

Thank you once more for your insightful and constructive comments and for the time you have dedicated to reviewing our paper. As the discussion phase comes to an end, kindly let us know if you have any further questions or concerns.

Best regards,

Authors

Thank you for the additional information. However, I still find the evaluation insufficient to justify raising my score, so I will maintain my original rating.

Reviewer_CWQ6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper considers the setting where TTS is done decentrally for the sake of speed or spreading compute while making sure (statistically) that the individual nodes are actually sampled from the reference model. Their contribution is the oddity metric, that takes a CoT trace and final output and computes, under the given "reference" model (that is supposed to be used), how "odd" it is to see the trace (and final result). For this they basically look at the (centered) surprisal of each token.

Their experiments on math and coding benchmarks include adversarial nodes that - alter prompt for generation, modify a trace to induce a different answer or just completely replace the model that is sampled from. It outperforms simple baselines.

Critical strengths

Writing is clear and understandable. Benchmarks the method was tried on are good enough for me.

Critical weaknesses

There are some related methods that try to verify whether a CoT or just sequence was sampled using a certain model. I'm lacking them in the baseline comparison (e.g. Intellect-2 afaik tries to do a very similar thing, though not claiming robustness against arbitrary attacks - at least for some of the attacks they would have been a more worthy baseline than the ones presented in the current paper).

The paper also mentions more costly but thorough verifiers from the cryptographic side - and stresses that their approach is computationally much more efficient - it would have been good to see tradeoffs in this regard in the experimental section. Similarly the work Miao & Fang 25 or Gao 25 should not be completely dismissed, as these methods can definitely also be used for the same purpose. I don't expect that those experiments would change the message of the paper too much, but nonetheless it's crucial to have more reasonable baselines than just non-robust aggregation methods and median-based aggregation ...

Questions for authors

See weaknesses. Please run those experiments and report the results. If done properly, I'd be fine to raise the score to acceptance.

2 comments

Thank you for highlighting these methods. To address the reviewer’s concerns, we have extended our evaluation with INTELLECT-2 and Miao & Fang (2025), and benchmarked the cryptographic cost of DeepProve. The results show that PoT achieves higher average accuracy than both baselines across all three attack surfaces, while the DeepProve benchmark demonstrates the substantially higher cost of cryptographic verification. We also clarify why Gao et al. (2025) cannot serve as a direct baseline for our problem setting.

INTELLECT-2. INTELLECT-2 uses TOPLOC to check whether the workers perform the inference faithfully (§2.3). As shown in the following table, INTELLECT-2 achieves substantially lower accuracy than PoT across all three attack surfaces. This is because, under our threat model, an attacker can first construct a malicious trace and then run a prefill pass using the reference model to produce matching fingerprints. The malicious submission can therefore pass the check, since it verifies activation consistency but does not establish that the submitted tokens were sampled from the given decoding distribution.

Miao & Fang (2025). Miao & Fang (2025) compute sequence confidence following Eqs. (1)–(4) in their paper. As shown in the following table, this method achieves lower average accuracy than PoT across all three attack surfaces. The largest gap is under trace-side injection, where most of the original trace remains unchanged and only selected tokens are manipulated. This result highlights the limitations of its confidence-based screening when faced with targeted trace manipulation, and shows the benefit of our PoT’s design.

The following table reports accuracy (%) on AIME25 using Qwen3-4B, with 30% malicious participants across three attack surfaces under the informed setting.

AttackMethod@1@3@5@7@9@11@13@15Average
Trace-side InjectionProof-of-Thought (Ours)53.1157.4060.6261.9363.2363.7164.3764.7561.14
Miao & Fang (2025)43.1645.7348.6449.8650.4351.5751.8452.3549.20
INTELLECT-244.5548.2550.4952.6253.2154.0854.8255.2451.66
Model-side SubstitutionProof-of-Thought (Ours)61.7663.0766.0167.7469.5970.4371.0571.3767.63
Miao & Fang (2025)60.1562.2964.5366.3467.8268.9669.9070.5866.32
INTELLECT-242.7045.5947.5848.5048.5148.9349.8450.0047.71
Prompt-side SteeringProof-of-Thought (Ours)62.2363.8166.7769.0470.2271.2271.9472.4168.45
Miao & Fang (2025)61.9363.9066.5568.4269.7470.8371.5471.9868.11
INTELLECT-245.2748.7951.5552.3753.1653.5653.7654.0351.56

DeepProve. DeepProve is an open-source system for end-to-end verification of full LLM inference using zero-knowledge proofs and was accepted at CCS 2026. Since the implementation does not support the models studied in our work, we benchmark it on its supported Llama 2 7B configuration, which is comparable in parameter count to DeepSeek-R1-8B. We use the same hardware configuration as our main experiments, with the results shown below.

Prompt tokensGenerated tokensProof-generation timeProof sizeProof-verification time
8822 min 9 s64.61 MiB37.28 s

On AIME25, our PoT verifier took 2.71 s per trace on average for DeepSeek-R1-8B outputs averaging about 14,100 generated tokens. DeepProve’s verifier took about 14 times as long on the 16-token sequence.

In addition, we agree that Gao et al. (2025) is relevant to our work because their method tests whether a worker’s output distribution differs from that of the reference model. However, their method is not directly applicable to our current setting. It requires multiple outputs from a worker to assess that worker, whereas our verifier receives only one CoT trace per worker.

Dear Reviewer,

Thank you once more for your insightful and constructive comments and for the time you have dedicated to reviewing our paper. As the discussion phase comes to an end, kindly let us know if you have any further questions or concerns.

Best regards, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.