acceptodds
Under review as a conference paper at ICLR 2027

Equilibrium of Thoughts: Reasoning as Iterative Stabilization Under Adversarial Probing

Abstract

Reasoning frameworks from Chain-of-Thought to Tree-of-Thoughts are constructive: they build an answer step by step, so a single early error cascades and is rarely caught. These methods find an answer but cannot certify it. We propose Equilibrium of Thoughts (EoT), which reframes reasoning as stabilization under adversarial probing: generate an answer, attack it for the single strongest objection, revise if the attack lands, and repeat until no valid objection remains. The surviving answer is the reasoning equilibrium, grounded in fixed-point, argumentation, and game theory. Our central finding concerns who should attack. A model cannot find the flaws it just made, so self-critique is unreliable. The bottleneck is the critic. Holding the reasoner fixed and varying only the adversary, we show a strong, decorrelated cross-model critic is the active ingredient: it rescues a weak reasoner ( on AIME for a 32B model) and lets critic capability substitute for reasoner capability, while a same-strength same-family critic does not, which isolates the contribution of decorrelation from that of raw strength. Recast as cross-model adversarial verification, this prompt-only recipe helps across math, science, code, and multi-hop QA, including benchmarks where multi-agent debate degrades accuracy. Finally, EoT wraps an initializer instead of replacing it: composed on Tree-of-Thoughts with the same critic, it beats debate on every hard benchmark (AIME-2024 vs. ). Adversarial verification thus complements constructive reasoning as a primitive that amplifies it: the stronger the initializer, the higher EoT reaches.

Primary area:
foundation or frontier models, including LLMs
Keywords:
test-time scaling, adversarial verification, self-correction, multi-agent debate, LLM reasoning, fixed-point theory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GzLcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Equilibrium of Thoughts, which iterates between a critic's concrete objection and a reasoner's revision, stopping on critic concession or a round cap. Experiments on eight benchmarks vary critics and initializers, reporting gains from cross-family critique and compatibility with Tree-of-Thoughts.

Critical strengths
  1. Experiments cover eight benchmarks, multiple reasoners, and several critic and initializer choices, with useful ablations and paired significance tests.
  2. The fixed-reasoner critic sweeps suggest that external feedback can substantially help difficult reasoning tasks. The prompt-only method also composes with existing search strategies.
Critical weaknesses
  1. The proposed method exhibits limited novelty, and the attribution of its contributions remains incomplete. External critique followed by revision has clear precedents, including critique-model supervision at test time (Xi et al., 2024, arXiv:2411.16579). Model-family difference is used as a proxy for decorrelation, but error correlations are not measured, and critic capability is not cleanly controlled.
  2. Critic concession does not certify correctness. The claimed Nash-equilibrium, preferred-extension, and Tarski fixed-point equivalences lack the necessary game, argumentation, and order-theoretic conditions. Moreover, Appendix D uses a marginal objection-soundness probability as though it were conditioned on an incorrect answer in the accuracy-drift derivation.
  3. The compute comparison is not fully fair. Despite the claim of matched budgets, Table 10 reports about 251K tokens for EoT (ToT, Opus) versus 60K for debate. The pooled AIME improvement over ToT alone is not statistically significant (Table 14, p=0.293). Cost, latency, and critic strength should be controlled before claiming superiority over debate.
  4. Algorithm 1 presents a single model for both attack and revision and returns the current answer on oscillation, while the text specifies cross-model attack and majority-vote cycle resolution. Table 3's nominal token budgets also differ markedly from measured usage in Table 10.
Questions for authors
  1. Can the authors provide valid conditions for the three equilibrium interpretations and correct the probability conditioning in Appendix D? How is a critic's 'no objection' distinguished from a missed error?
  2. Can the authors measure reasoner–critic error overlap and compare capability-matched same-family and cross-family critics, as well as a generic external-critique baseline?
  3. Can the authors compare methods at matched measured tokens, monetary cost, and latency, preferably with the same strong critic, and reconcile Algorithm 1 and the budget tables with the implementation?

I am willing to raise my score if you solve the above issues.

1 comment
AuthorsOpenReview ↗

Gentle reminder: author–reviewer discussion period ending in a few days

Dear Reviewer GzLc,

We wanted to send a brief reminder that the author–reviewer discussion period concludes in a few days.

In our response, we re-ran the comparisons at a 30K per-call cap on 290 competition-math problems and reported measured tokens, list-price cost and sequential depth for every method; measured each critic's standalone accuracy and error overlap with the reasoner, including a capability-matched pair of critics; reported the ToT result as a composition with its cost; rewrote Algorithm 1 to match the implementation exactly; scoped the game-theoretic, argumentation and fixed-point interpretations; made the conditioning in Proposition 1 explicit; separated critic concession from the other stopping conditions; and added Xi et al. (2024) to the related work.

If these updates address your concerns, we would be glad to hear it, and we would be grateful if you would consider reassessing your rating accordingly. If there are any remaining questions or concerns, we would be happy to address them during the remaining discussion period.

Thank you again for the time and effort you put into reviewing our work.

Best regards,

Authors

Reviewer_diEyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

the authors propose Equilibrium of Thoughts (EoT) which is an iterative reasoning framework where a critic detects objections to an answer generated by the reasoner and the reasoner revises it until there are no more objections. We interpret EoT through equilibrium theory and show improvements on several reasoning benchmarks using cross-model critics.

Critical strengths

The paper offers an interesting view of how to improve LLM reasoning by adversarial critique instead of multiple independent solutions. Experimental evaluation is done on several reasoning benchmarks and model families (e.g., AIME, GPQA), with significant improvements on hard tasks (AIME) and GPQA. Different critics, initializers and convergence behavior are also analyzed which provides some empirical insights about iterative reasoning.

Critical weaknesses
  1. The central claim that error decorrelation is what drives the improvements is not very well supported. The experiments compare critics from different model families without measuring their error correlations directly or controlling for critic capability (e.g. Claude Opus is the strongest reasoner). In fact, Appendix C.3 shows a same-family critic can outperform some cross-family ones on AIME. So it remains unclear whether the gains are due to error decorrelation or stronger critic capabilities.
  2. The theoretical justification of the reasoning equilibrium is not convincing. The paper links attack–revise to Nash equilibrium, argumentation theory and fixed-point theory but the conditions required for these interpretations are not rigorously established for the actual LLM procedure (e.g., in Appendix D convergence results assume contractivity rather than show that EoT satisfies it). A critic's failure to identify an objection does not necessarily imply that the answer is correct. So the claimed verification interpretation is questionable.
Questions for authors
  1. Can the authors evaluate the reverse setting (Claude Opus as reasoner and Qwen as critic)? Along with standalone critic accuracy and direct measurements of error correlation this would separate out the effects of critic capability and error decorrelation.

  2. Can the authors please explain how monotonicity and contractivity are needed for the actual attack–revise operator? What guarantees does the proposed equilibrium give besides that a critic cannot find any further objection?

Reviewer_LiUuOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether language models can produce more accurate answers by having another model check and help revise their work. The proposed method, Equilibrium of Thoughts (EoT), first lets a model produce a complete solution. A second model then looks for a specific error in that solution and sends its feedback to the original model, which revises its answer. This process repeats until the checking model finds no further error, the answers begin to cycle, or a preset number of rounds is reached. The authors call a solution accepted by the checking model a “reasoning equilibrium.” The method uses prompting and does not require additional training. The paper evaluates this approach on eight benchmarks covering mathematics, science questions, code generation and factual question answering. Its main reported finding is that a strong checking model from a different model family often helps more than asking the original model to check itself. The authors also combine EoT with methods that search for a better initial solution, and report further accuracy gains on some difficult benchmarks.

Critical strengths
  1. The method is straightforward. The paper provides prompts and a clear description of the critique-and-revision loop. It can be applied to existing models without additional training.

  2. The experiments cover several tasks and model combinations. They include cases where critique improves answers and cases where it makes them worse. The reported token costs and iteration curves also help assess the trade-offs.

Critical weaknesses
  1. W1. Limited novelty beyond existing critique-and-refinement methods. EoT repeatedly generates an answer, obtains model feedback, and revises the answer. Self-Refine [1] already uses this process at inference time without additional training, while CAI [2] uses model-generated critiques and revisions to construct safety-training data. EoT adds a cross-model critic, a prompt requesting a specific objection, and a stopping rule based on the absence of further errors. However, the paper does not establish what these choices add beyond ordinary feedback-driven revision using the same models. Gains from introducing a stronger critic alone do not demonstrate a sufficiently distinct methodological contribution.

  2. W2. The feedback can give the reasoner the answer. The critic can provide more than error identification. In Figure 1, it supplies both the corrective operation and the value to subtract. The verification prompt in Appendix A explicitly asks it to “state your answer and the specific step where the proposed reasoning goes wrong,” then passes this feedback to the reasoner. A successful revision may therefore amount to following a better solution supplied by the critic. Without separating error-only feedback from feedback containing answers or repair steps, the results do not establish how much improvement comes from the proposed verification mechanism.

  3. W3. The gains may come from a stronger teacher model. Introducing a stronger critic also introduces a more capable source of knowledge and reasoning guidance. The improvement may therefore reflect the original model benefiting from a stronger teacher, rather than the proposed EoT procedure. Keeping the reasoner fixed does not remove this confound. The current comparisons do not establish how much EoT adds beyond ordinary guidance from the same strong model.

  4. W4. Repeated critique has limited supporting evidence. Figure 5 shows that most AIME gains arrive within two rounds and largely plateau by rounds 4–5. This experiment uses self-critique, leaving the additional benefit of multiple rounds over one round of strong external feedback unresolved in the main cross-model setting.

  5. W5. The method incurs substantial inference costs. Each additional critique–revision round requires calls to both the critic and the reasoner. Table 10 reports 43K tokens per problem for CoT-initialized EoT, compared with 3.6K for plain CoT, nearly a twelvefold increase. The highest-performing ToT-initialized version uses 251K tokens, largely due to its search-based initialization. Repeated calls to a strong external critic also add to the cost of the overall system, while the sequential interaction introduces latency. These overheads limit the method's practicality, especially given the unclear gains from later rounds.

References

[1] Aman Madaan et al. Self-Refine: Iterative Refinement with Self-Feedback. 2023. arXiv:2303.17651.

[2] Yuntao Bai et al. Constitutional AI: Harmlessness from AI Feedback (CAI). 2022. arXiv:2212.08073.

Questions for authors
  1. What does EoT add beyond existing critique-and-refinement methods such as Self-Refine [1], when both use the same reasoner and critic?
  2. How often does critic feedback contain an answer or key solution steps, and how much improvement remains when feedback only identifies errors?
  3. What evidence shows that the gains come from EoT rather than access to a stronger teacher? How does ordinary guidance from the same teacher compare?
  4. With the same strong critic, how much additional accuracy do multiple rounds provide over a single round of feedback and revision?
  5. How does EoT compare with directly using the strong teacher at a similar total cost, including token usage and latency?
1 comment
AuthorsOpenReview ↗

Gentle reminder: author–reviewer discussion period ending in a few days

Dear Reviewer LiUu,

We wanted to send a brief reminder that the author–reviewer discussion period concludes in a few days.

In our response, we clarified what EoT adds beyond Self-Refine and showed that, with the same reasoner as its own critic, EoT improves over Self-Refine by 3.4 points [+0.7, +6.6]; classified every objection that led to a revision to measure how much information the critic provides, and ran an error-only critic prompt; showed that critics weaker than the reasoner still improve it by 2.4–5.9 points, while upfront guidance from the strong model barely helps; reported accuracy after each round; and compared EoT with using the strong model directly, with full token and cost accounting.

If these updates address your concerns, we would be glad to hear it, and we would be grateful if you would consider reassessing your rating accordingly. If there are any remaining questions or concerns, we would be happy to address them during the remaining discussion period.

Thank you again for the time and effort you put into reviewing our work.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.