acceptodds
Under review as a conference paper at ICLR 2027

Emergent Communication Topologies from Local Reasoning Refinements in Multi-Agent Systems

16 views

Abstract

Effective collaboration in Large Language Model (LLM)-based multi-agent systems (MAS) requires communication that adapts to how agents can improve one another's evolving reasoning. However, existing approaches based on task or query information or overall agent assessments do not directly identify which reasoning states can provide useful corrections to a particular peer. We introduce refinement-specific communication and instantiate it in LoRIT, a training-free framework that organizes communication around directional, recipient-specific opportunities for reasoning improvement. Agents assess how their current reasoning states can correct or complete one another, and these local relations induce a global communication structure that accommodates mutual refinement and evolves with reasoning. This couples reasoning updates with topology self-adaptation without imposing a global agent ranking or optimizing a separate graph-level objective. Experiments across diverse reasoning and coding tasks demonstrate broad improvements over single-agent and multi-agent baselines with both weak and strong LLM backbones. With stronger refinement assessment, LoRIT enables groups of smaller models to achieve collective performance comparable to or exceeding that of larger single-agent models. These results highlight the potential of refinement-specific communication to turn complementary information in imperfect individual solutions into stronger collective reasoning.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Large language models, multi-agent systems, collaborative reasoning, refinement-specific communication, communication topology、

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PmtrOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LoRIT, a training-free multi-agent reasoning framework that dynamically constructs communication topologies based on refinement relationships between agents. Specifically, multiple agents independently solve a given problem and evaluate whether each agent's reasoning can improve that of another agent. Based on these directional refinement relationships, the communication topology is dynamically constructed, and each agent iteratively updates its reasoning state by incorporating useful information from connected agents.

Critical strengths
  1. Refinement-specific communication: Instead of sharing reasoning among all agents indiscriminately, LoRIT identifies directional, recipient-specific refinement relationships, allowing agents to selectively incorporate useful information from others.
  2. Efficient topology construction: By leveraging Strongly Connected Components (SCCs) and Transitive Reduction (TR), LoRIT constructs a communication topology that preserves refinement reachability while removing redundant component-level connections.
  3. Dynamic reasoning–topology co-evolution: LoRIT dynamically reconstructs the communication topology as agents update their reasoning states, enabling adaptive collaboration based on evolving refinement relationships.
  4. Comprehensive empirical evaluation: LoRIT demonstrates consistent improvements across diverse reasoning and coding benchmarks, multiple backbone scales, and heterogeneous agent configurations. The ablation studies further validate the contributions of its topology construction components.
Critical weaknesses
  1. High Inference Cost: Although LoRIT achieves the highest accuracy across the evaluated benchmarks, Figures 7~9 show that its token consumption is substantially higher than that of several baselines, including G-Designer, TopoDIM, and MacNet. Its token cost is generally comparable to that of SelfOrg, rather than consistently lower than that of other multi-agent methods. Moreover, the fact that LoRIT lies on the Pareto frontier provides limited evidence of computational efficiency, as the highest-accuracy method is necessarily non-dominated regardless of its computational cost. These results raise concerns about whether the accuracy improvements justify the additional inference overhead.
  2. Limited Guarantees for Information Preservation under Transitive Reduction: Although SCC-based condensation and transitive reduction preserve graph reachability, they do not guarantee the preservation of reasoning information. For example, even if A → B → C exists, the information received by C through B may differ from that received directly from A, as intermediate agents may omit or distort useful reasoning. This concern is particularly relevant given the limited number of reasoning rounds (T=2) and dynamic topology re-induction, which may further restrict indirect information propagation. While the authors acknowledge this theoretical limitation and demonstrate the empirical benefits of transitive reduction in Appendix E.2, the extent to which removing transitive edges leads to information loss remains unclear. A more targeted analysis of information preservation along indirect refinement paths would strengthen the justification for this design choice.
Questions for authors
  1. Impact of Agent Role Diversity: The proposed framework uses eight agents with diverse roles, including General Assistant, Mathematician, Economist, Psychologist, Programmer, Lawyer and Historian, Biologist, and Physicist and Chemist. However, the relevance of certain roles, such as Lawyer and Historian, Biologist, and Physicist and Chemist, to mathematical reasoning and coding tasks is unclear. What motivates the use of such heterogeneous roles? Would a team consisting entirely of task-specific agents (e.g., eight Mathematician agents for mathematical reasoning) achieve comparable or better performance? An ablation study on agent role composition would help clarify the contribution of role diversity.
  2. Reasoning Diversity and Premature Consensus: The authors mention in the Introduction that premature consensus in multi-agent systems may suppress alternative solutions and propagate incorrect reasoning. However, it remains unclear whether LoRIT effectively mitigates this issue. Does LoRIT exhibit similar convergence behavior during iterative reasoning updates? An analysis of reasoning diversity, answer agreement, and incorrect consensus across collaboration rounds would help clarify whether refinement-specific communication preserves diverse reasoning trajectories.
  3. Comparison with Multi-Agent Debate: Although the paper compares LoRIT with several multi-agent baselines, it does not include direct comparisons with representative MAD methods, such as [1] and [2]. Comparisons in terms of accuracy, token consumption, and end-to-end inference latency under comparable computational budgets would strengthen the empirical evaluation.
    [1] Improving Factuality and Reasoning in Language Models through Multiagent Debate, ICML 2024
    [2] Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate, EMNLP 2024
  4. Sensitivity to the Number of Reasoning Rounds: The framework sets the maximum number of reasoning updates to (T=2), with early stopping based on topology stability. However, the paper does not report the average number of rounds actually performed or the frequency of early stopping. Could the authors provide these statistics and clarify the motivation for choosing (T=2)? Additionally, an ablation study varying the maximum number of rounds would help assess the impact on reasoning accuracy, inference cost, and convergence behavior.
  5. Understanding Performance Gains with Weakly Connected Refinement Graphs: Table 3 shows that Qwen3-0.6B produces relatively sparse refinement graphs, with low refinement density and nearly eight SCCs on average. Nevertheless, Table 2 reports substantial performance improvements over single-agent reasoning. What primarily explains these gains when the identified refinement connectivity is limited?
8 comments
AuthorsOpenReview ↗

Response to Reviewer Pmtr's questions

Thank you so much for recognizing our novelty and the breadth of our evaluation. We especially appreciate your insightful comments. We added the following suggested ablations and analyses to address your concerns and enhance our manuscript. The corresponding revisions are highlighted in red in the revised manuscript.

Q1. Impact of agent role diversity

Revision location: Appendix E.2, p. 26.

The default role pool is intended to provide varied initial reasoning perspectives across all our mathematical (e.g. GSM8K, GSM-Hard), scientific (e.g., GPQA), knowledge (e.g., MMLU), and coding benchmarks. Using a common role-prompt pool also standardizes the prompts available to the compared multi-agent methods. Here we further compare three eight-agent configurations:

  • Mixed-domain: the default eight-role pool.
  • Generalist: eight agents sharing the General Assistant prompt.
  • Mathematical specialists: Mathematical Modeler, Algebraist, Arithmetic Specialist, Geometry Specialist, Probability and Combinatorics Specialist, Units and Proportions Specialist, Mathematical Logician, and Mathematical Verifier.

We use each role pool across all four benchmarks. Within each backbone, all other settings remain fixed; initial responses are generated separately.

Table R6: Role-composition ablation of LoRIT.

BackboneRole compositionGSM8KGSM-HardAIMEGPQA
Qwen3-1.7BMixed-domain (default)83.1164.3118.8943.94
Qwen3-1.7BGeneralist82.4962.4718.8942.42
Qwen3-1.7BMathematical specialists85.7567.2822.2240.91
Qwen3-8BMixed-domain (default)94.4772.3031.1155.56
Qwen3-8BGeneralist92.1571.5230.0053.54
Qwen3-8BMathematical specialists95.4173.9834.4452.02
Qwen3-30B-A3BMixed-domain (default)97.8079.7436.6760.10
Qwen3-30B-A3BGeneralist97.1278.8937.7858.59
Qwen3-30B-A3BMathematical specialists98.3081.0241.1156.57

Mathematical specialists improve accuracy in all nine mathematical settings, gaining 0.50–2.64 percentage points on GSM8K, 1.28–2.97 on GSM-Hard, and 3.33–4.44 on AIME over Mixed-domain. However, the same mathematical pool reduces GPQA accuracy by 3.03–3.54 points. Role specialization benefits task-aligned benchmarks, while the mixed-domain pool offers broader applicability, achieving the highest GPQA accuracy at every backbone scale.

Q2. Reasoning diversity and premature consensus

To address this concern, we examine both answer convergence and the frequency with which agents converge to a shared incorrect answer. We analyze eight-agent LoRIT runs on GSM-Hard and GPQA with Qwen3-1.7B, 8B, and 30B-A3B using the reasoning backbone as the refinement judge. We inspect all eight states at initialization (t=0t=0) and after each update (t=1,2t=1,2), including states excluded from final readout. For early-stopped problems, we carry the last states forward so all rounds cover the same questions.

All metrics use the same saved states within this diagnostic batch:

  • Distinct answers: mean number of distinct normalized answers among eight agents.
  • Agreement: mean fraction of agents sharing the most frequent answer.
  • Wrong unanimity: percentage of all problems where all eight agents give the same incorrect answer.

Table R7: Answer diversity and consensus across refinement rounds. Each entry lists t=0/1/2t=0/1/2.

BackboneDatasetDistinct answersAgreement (%)Wrong unanimity (%)
Qwen3-1.7BGSM-Hard2.91 / 1.92 / 1.6360.0 / 78.9 / 85.217.5 / 10.5 / 2.7
Qwen3-8BGSM-Hard2.10 / 1.32 / 1.2576.7 / 92.5 / 96.123.1 / 17.0 / 5.2
Qwen3-30B-A3BGSM-Hard1.58 / 1.28 / 1.1386.9 / 93.4 / 96.921.6 / 16.3 / 9.6
Qwen3-1.7BGPQA2.39 / 1.79 / 1.7366.2 / 79.4 / 81.629.3 / 25.8 / 8.1
Qwen3-8BGPQA2.11 / 1.52 / 1.3273.4 / 86.4 / 93.833.8 / 23.2 / 10.1
Qwen3-30B-A3BGPQA1.83 / 1.40 / 1.2578.5 / 80.5 / 95.330.8 / 19.2 / 10.1

Table R7 shows that LoRIT exhibits answer-level convergence, accompanied by reduced incorrect unanimity. Distinct answers decrease from 1.58–2.91 initially to 1.13–1.73 finally, while agreement rises from 60.0–86.9% to 81.6–96.9%. Wrong unanimity falls from 17.5–33.8% to 2.7–10.1%, a reduction of 12.0–23.7 percentage points. Meanwhile, some answer-level diversity remains: the final mean number of distinct answers exceeds one in every setting, reaching 1.63 on GSM-Hard and 1.73 on GPQA with Qwen3-1.7B. Thus, convergence is accompanied by fewer shared incorrect answers and residual disagreement. To examine the underlying reasoning process, the following case study traces a useful alternative interpretation in initially incorrect states through recipient revision and final readout.

AuthorsOpenReview ↗

Response to Reviewer Pmtr's concerns in weakness part

W1. Response to the concern on inference cost

Thank you for raising this concern. We agree that LoRIT's higher accuracy is accompanied by larger token consumption than several baselines. We see substantial scope for reducing this overhead through engineering optimization while retaining the original refinement-based coordination framework. Specifically, we introduce lazy evaluation of pairwise refinement judgments, which reduces assessment calls once the execution topology is fully determined.

Exact lazy evaluation of refinement judgments. Both the original and optimized implementations use eight agents and retain the same refinement criterion, communication rules, and readout procedure. For a fixed reasoning-state snapshot, let PP denote the relations already judged positive and UU the unqueried relations. We construct

\qquad \mathcal{D}^{+}=(\mathcal{V},P\cup U),$$ which treat all unqueried relations as negative and positive, respectively. Let $\Phi(\mathcal{D})$ denote the original LoRIT execution-graph construction: retain the original intra-SCC edges and expand each edge of the transitively reduced condensation graph into all-to-all inter-component connections. Lazy evaluation stops when $$\boxed{\Phi(\mathcal{D}^{-})=\Phi(\mathcal{D}^{+})},$$ where equality means identical directed edge sets. At this point, the remaining judgments cannot change the execution graph. Checking this condition requires only graph operations. **Proposition (Exactness of lazy evaluation).** Every completion $\mathcal{D}=(\mathcal{V},E)$ satisfying $P\subseteq E\subseteq P\cup U$ induces the same execution graph if and only if $\Phi(\mathcal{D}^{-})=\Phi(\mathcal{D}^{+})$. **Proof.** Necessity follows because $\mathcal{D}^{-}$ and $\mathcal{D}^{+}$ are both valid completions. For sufficiency, suppose $\Phi(\mathcal{D}^{-})=\Phi(\mathcal{D}^{+})$. The execution-graph construction preserves reachability, so $\mathcal{D}^{-}$ and $\mathcal{D}^{+}$ have identical reachability relations. Every completion satisfies $$\operatorname{Reach}(\mathcal{D}^{-}) \subseteq \operatorname{Reach}(\mathcal{D}) \subseteq \operatorname{Reach}(\mathcal{D}^{+}).$$ Thus, all completions have the same reachability relation, SCC partition, and quotient partial order. Their component-level cover edges and corresponding all-to-all expansions are therefore identical. Moreover, no unqueried edge can lie within one of these SCCs. Such an edge would be retained in $\Phi(\mathcal{D}^{+})$ but absent from $\Phi(\mathcal{D}^{-})$, contradicting their equality. The original intra-SCC edges are therefore also fixed. Both parts of the execution graph are consequently identical for every completion. **This criterion preserves each agent's incoming peer set and the source-component readout pool.** With shared initialization, fixed corresponding judgment outcomes, and identical state ordering and random draws for corresponding generation calls, the lazy and exhaustive implementations follow the same reasoning trajectory and terminate at the same round.

W1. Response to the concern on inference cost — Experimental results

We evaluate the optimized implementation with Qwen3-8B on GSM-Hard and GPQA. The optimized version is independently rerun with fresh reasoning samples. Table R12 reports accuracy and total token consumption across all inference stages.

Table R12: Accuracy and inference cost with Qwen3-8B on GSM-Hard and GPQA. T/A denotes Tokens (K) divided by accuracy expressed as a percentage; lower is better.

MethodGSM-Hard Acc.GSM-Hard Tokens (K)GSM-Hard T/AGPQA Acc.GPQA Tokens (K)GPQA T/A
G-Designer65.4710.20.15645.9614.80.322
TopoDIM59.599.70.16339.3917.60.447
MacNet66.0717.90.27143.9428.40.646
SelfOrg66.8126.70.40050.5142.20.835
LoRIT (N=8N=8, exhaustive judgments)72.3025.30.35055.5641.90.754
LoRIT (N=8N=8, lazy judgments)71.8619.10.26656.0630.50.544

The results show that lazy judgments reduce total token consumption by 24.5% on GSM-Hard and 27.2% on GPQA, with accuracy remaining within 0.50 percentage points of the exhaustive implementation. The optimized version achieves 71.86% and 56.06% accuracy, respectively, exceeding all four compared baselines. Its T/A values of 0.266 and 0.544 lie between those of G-Designer/TopoDIM and MacNet/SelfOrg, placing it in the intermediate range of the baseline token-to-accuracy ratios. Compared with SelfOrg, it improves accuracy by 5.05 and 5.55 percentage points while using 28.5% and 27.7% fewer tokens. These results demonstrate that implementation optimization can substantially reduce inference overhead while retaining LoRIT's accuracy advantage.

Core contribution and further optimization.

More broadly, LoRIT's central contribution is a self-organizing framework in which communication is induced by recipient-specific opportunities for reasoning refinement. Local judgments identify how one agent's reasoning can improve another's, organize cyclic refinement within components and directed communication between components, and adapt the topology as reasoning states evolve. The lazy implementation demonstrates that this refinement-based coordination can be realized with substantially lower token consumption while maintaining high accuracy. We will continue to optimize refinement assessment and inference execution to further improve LoRIT's accuracy–cost trade-off.

Reviewer_S1oLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LoRIT, a training-free framework that adapts multi-agent communication to directional refinement opportunities between current reasoning states. It constructs a communication topology through SCC condensation and transitive reduction, updates agents’ responses, and rebuilds the topology after each round. Experiments across reasoning and coding benchmarks report improvements over single-agent and multi-agent baselines.

Critical strengths
  1. Recipient-specific refinement is a sensible basis for communication. It captures mutual assistance that a single ranking of agents cannot express, including useful contributions from otherwise incorrect solutions.

  2. The evaluation covers multiple tasks and model scales, with generally consistent gains.

Critical weaknesses
  1. The experiments do not isolate the benefit of LoRIT’s communication mechanism. A key baseline is missing: the same eight role-conditioned agents solving independently, followed by synthesis using the same readout LLM.

  2. The evidence for complementary reasoning remains incomplete. The pairwise update study shows that incorrect sources can be useful, but does not establish whether these improvements survive final candidate selection or contribute to the system’s gains.

Questions for authors
  1. Can you compare LoRIT against direct synthesis of the same eight initial responses, using the same readout model and prompt? Please also clarify the Self-Consistency configuration and include a comparison at a matched inference budget.

  2. What are the average number of refinement rounds and the early-stopping rate? Among final candidates, how many received peer information, and how many changed from incorrect to correct? A concrete example tracing such a correction through final readout would help.

2 comments
AuthorsOpenReview ↗

Response to Reviewer S1oL

Thanks so much for recognizing our novelty and the value of recipient-specific refinement, and for suggesting controls for independent synthesis and inference costs. We added the suggested comparisons to address your concerns and further enhance our manuscript. The added analyses and tables are highlighted in red in the revised manuscript.

Q1. Direct synthesis comparison

Revision location: Appendix E.3, p. 27.

We compare three conditions on GSM-Hard and GPQA with Qwen3-1.7B, Qwen3-8B, and Qwen3-30B-A3B. All reuse the same eight saved role-conditioned initial responses per problem:

  • Initial synthesis: directly synthesizes the initial responses without communication.
  • LoRIT + all-state synthesis: synthesizes all eight final states after refinement.
  • Full LoRIT: retains its native source-component selection and readout.

The first two conditions share the readout model, neutral synthesis prompt, decoding settings, and input/output limits. Their comparison holds initialization and readout fixed while changing whether refinement occurs before synthesis.

Table R3: Direct synthesis versus refinement-based collaboration. Accuracy (%); all conditions share the same initial responses.

BackboneMethodGSM-HardGPQA
Qwen3-1.7BInitial synthesis54.5836.36
Qwen3-1.7BLoRIT + all-state synthesis62.0940.40
Qwen3-1.7BFull LoRIT64.3143.94
Qwen3-8BInitial synthesis64.5547.47
Qwen3-8BLoRIT + all-state synthesis68.8750.51
Qwen3-8BFull LoRIT72.3055.56
Qwen3-30B-A3BInitial synthesis73.8055.56
Qwen3-30B-A3BLoRIT + all-state synthesis77.5858.08
Qwen3-30B-A3BFull LoRIT79.7460.10

Refinement followed by all-state synthesis improves accuracy in all six settings, by 3.78–7.51 percentage points on GSM-Hard and 2.52–4.04 points on GPQA. These results support updating reasoning states before synthesis under a shared readout procedure. Full LoRIT further improves accuracy by 2.02–5.05 points, supporting the additional value of its native candidate-selection and readout procedure.

Q2. Self-Consistency configuration and budget control

Revision locations: Appendix C.1, pp. 18–19; Appendix E.3, p. 27.

Our standard Self-Consistency (SC; Wang et al., 2022) baseline samples 50 independent reasoning trajectories per problem using the same backbone, the General Assistant prompt, temperature 0.5, and a maximum generation length of 8,192 tokens per trajectory. It selects the most frequent extracted answer, without communication or LLM-based synthesis. The initial-synthesis control above additionally matches LoRIT's role-conditioned initialization and LLM-based readout.

We agree that controlling inference cost is important. For budget-matched SC, we adjust the sampling budget separately for each backbone and benchmark to approximately match LoRIT's average total token consumption. Sampling stops between completed responses, and the answers are aggregated by majority voting. Token accounting includes input and output tokens across all inference stages, including refinement assessment and final readout where applicable.

Table R4: Comparison with budget-matched Self-Consistency. Accuracy (%) and average total tokens per problem (K).

BackboneMethodGSM-Hard Acc.Tokens (K)GPQA Acc.Tokens (K)
Qwen3-1.7BBudget-matched SC53.5821.833.3333.2
Qwen3-1.7BFull LoRIT64.3121.743.9433.4
Qwen3-8BBudget-matched SC66.0925.347.4742.1
Qwen3-8BFull LoRIT72.3025.355.5641.9
Qwen3-30B-A3BBudget-matched SC74.1933.554.0443.7
Qwen3-30B-A3BFull LoRIT79.7433.460.1043.5

Average token costs differ by at most approximately 0.6%, relative to LoRIT's cost. LoRIT achieves higher accuracy in all six settings, with gains of 5.55–10.73 percentage points on GSM-Hard and 6.06–10.61 points on GPQA. This demonstrates the full framework's advantage over independent sampling and voting at comparable token cost.

AuthorsOpenReview ↗

Follow-up on Our Response

Thanks so much again for your careful reading and constructive comments. As the rebuttal period draws to a close, we would like to briefly follow up on the additional analyses addressing your questions:

  • For your Q1, we added a direct-synthesis comparison using the same initial responses and neutral readout. Refinement improves accuracy in all six settings, supporting its contribution beyond initial-response synthesis.
  • For your Q2, we clarified the Self-Consistency configuration and added budget-matched comparisons. LoRIT achieves higher accuracy in all six settings, with average token costs differing by at most approximately 0.6%.
  • For your Q3, we analyzed final-candidate trajectories and added a case study with controlled replays. Corrections outnumber regressions in every setting, and the case traces useful peer information into corrections retained through final readout.

These additions appear in Appendices C.1, E.3, and D.2, and Section 4.1, with the quantitative results in Tables R3–R5 of our responses.

We sincerely hope these additions address your concerns and clarify the value of refinement-based collaboration. If any questions remain, we would be very happy to provide further clarification. And if your concerns have been addressed, we would greatly appreciate an updated assessment reflecting the additional evidence.

Thanks again for your time and helpful feedback.

Best,

The authors

Reviewer_3aCzOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose LoRIT, a training-free framework that organizes multi-agent communication around directional opportunities to improve a recipient’s current reasoning. Given a question QQ, each agent independently produces a state Si(t)=(ai(t),ri(t))S_i^{(t)}=(a_i^{(t)},r_i^{(t)}), containing its answer and reasoning trace. A prompted LLM evaluates whether the source state contains information that could correct an error, supply a missing step, or resolve an inconsistency in the recipient state. The directional refinement judgments is then defined by:

Rt(i,j)=1![Si(t)≻tSj(t)],(i,j)∈Eref(t)  ⟺  Rt(i,j)=1,R_t(i,j)=\mathbf{1}\\!\left[S_i^{(t)}\succ_t S_j^{(t)}\right], \qquad (i,j)\in E_{\mathrm{ref}}^{(t)} \iff R_t(i,j)=1,

with Rt(i,i)=0R_t(i,i)=0 and D(t)=(V,Eref(t))D^{(t)}=(V,E_{\mathrm{ref}}^{(t)}). While each direction is evaluated separately, the resulting graph records the judge’s assessments of how each source could improve each recipient.

LoRIT derives the communication structure from reachability in D(t)D^{(t)}. The relation

i⪯tj  ⟺  i=jori⇝j in D(t)i\preceq_t j \iff i=j\\ \text{or}\\ i\leadsto j \text{ in }D^{(t)}

is a preorder, and mutual reachability, i∼tj  ⟺  i⪯tj∧j⪯tii\sim_t j\iff i\preceq_t j\land j\preceq_t i, partitions the graph into strongly connected components. Writing qt(i)=[i]tq_t(i)=[i]_t for the component containing agent ii, the authors construct

G(t)=Hasse⁡(V/∼t,⪯tq)=TR⁡!(Condensation⁡(D(t))).G^{(t)} = \operatorname{Hasse}(V/{\sim_t},\preceq_t^q) = \operatorname{TR}\\!\left(\operatorname{Condensation}(D^{(t)})\right).

Here, TR⁡\operatorname{TR} denotes transitive reduction. The resulting component graph retains the cover relations of the quotient partial order and is the unique minimum-edge graph preserving reachability on that fixed component set. Communication is executed through a separate agent-level graph H(t)H^{(t)}, whose edges are

EH(t)=(i,j)∈Eref(t):qt(i)=qt(j)∪(i,j):(qt(i),qt(j))∈E(G(t)).E_H^{(t)} = \\{(i,j)\in E_{\mathrm{ref}}^{(t)}:q_t(i)=q_t(j)\\} \cup \\{(i,j):(q_t(i),q_t(j))\in E(G^{(t)})\\}.

LoRIT retains the original refinement edges within each SCC. Across each retained component edge, every agent in the source component sends its state to every agent in the recipient component. The construction preserves reachability:

Reach⁡(G(t))=Reach⁡!(Condensation⁡(D(t))),Reach⁡(H(t))=Reach⁡(D(t)).\operatorname{Reach}(G^{(t)}) = \operatorname{Reach}\\!\left(\operatorname{Condensation}(D^{(t)})\right), \qquad \operatorname{Reach}(H^{(t)}) = \operatorname{Reach}(D^{(t)}).

Agents update synchronously using their own states and incoming neighbors’ states from the same snapshot:

Sj(t+1)=F!(Sj(t),Si(t):i∈NH(t)−(j)),S_j^{(t+1)} = F\\!\left( S_j^{(t)}, \\{S_i^{(t)}:i\in N^-_{H^{(t)}}(j)\\} \right),

where FF is instantiated by the backbone LLM conditioned on QQ. The revised states are then reassessed to reconstruct the topology. The default configuration uses eight role-conditioned agents and two updates after initialization. Algo. 1 terminates early when consecutive execution graphs have identical edge sets as their pre-defined stopping rule.

At the final round TT, LoRIT selects source components of the most recently induced graph:

M(T)=C∈V/∼T:deg⁡G(T)−(C)=0,a^=ρ!(Si(T):qT(i)∈M(T)).\mathcal{M}^{(T)} = \\{C\in V/{\sim_T}:\deg^-_{G^{(T)}}(C)=0\\}, \qquad \widehat a = \rho\\!\left( \\{S_i^{(T)}:q_T(i)\in\mathcal{M}^{(T)}\\} \right).

LoRIT collects the reasoning states of agents in these source components. The backbone-based readout ρ\rho combines the selected states to produce the final answer. When the selection contains a single state, LoRIT returns that state’s answer directly.

The evaluation spans mathematical, scientific, knowledge, and coding tasks. In the main comparison, LoRIT reports average benchmark scores of 57.62, 71.04, and 74.74 with Qwen3-1.7B, Qwen3-8B, and Qwen3-30B-A3B, exceeding SelfOrg, the strongest baseline by average, by 4.85, 5.60, and 3.63 points. Additional analyses examine judgment validity, topology construction, heterogeneous teams, agent count, and inference cost. The stronger-judge experiments use GPT-5-mini for refinement evaluation, with the smaller backbone performing the reasoning updates and final readout.

Critical strengths
  • LoRIT selects communication links by assessing how a source’s reasoning could improve a particular recipient’s current answer. The proposed method groups mutually reachable agents into SCCs, removes redundant component-level edges through transitive reduction, and specifies the messages exchanged during execution. SelfOrg also constructs response-dependent communication graphs; LoRIT contributes a recipient-specific refinement criterion and an SCC-based construction for organizing these relations. SelfOrg
Critical weaknesses
  • The final readout excludes all states outside the source SCCs, and it seems the benefit of this selection rule is insufficiently established. A refinement relation identifies information that could improve a recipient’s reasoning. A recipient with a correct answer could therefore be excluded because another agent supplies a missing step or useful correction. Appendix D.1 also shows that useful refinements can originate from sources with incorrect final answers. Figure 6(b) shows that the selection favors stronger backbones, providing some support for the rule. A comparison using the same final states and readout model is needed to establish how restricting the readout to source SCCs affects final-answer accuracy.
Questions for authors
  • Have you compared the source-component selection in Eqs. (11)–(12) with using all agents’ final states for answer synthesis? Such a comparison, using the same final states and readout model, would help establish whether the selection improves final-answer accuracy. Please clarify how the singleton case and input-token budget would be handled if possible.
5 comments
AuthorsOpenReview ↗

Response to Reviewer 3aCz

Thank you so much for your careful reading, recognition of the novelty of our method, and constructive suggestions. We have strengthened the paper by incorporating your suggested ablations and analyses. All corresponding additions and revisions are highlighted in red in the revised manuscript. We indicate the location of each addition below.

Q1. Validation of source-component readout

Revision location: Appendix E.6, Table 17, p. 29.

To address this concern, we conducted a controlled readout ablation that holds the final agent states fixed and directly evaluates the effect of candidate selection.

Controlled readout comparison. For each problem, we freeze the eight final agent states and the final refinement graph from a single LoRIT run and evaluate four readout variants:

  • Native: the original LoRIT readout, using its original synthesis prompt and directly returning the answer when exactly one candidate is selected.
  • Source: synthesis over all agent states belonging to source SCCs, defined as components with zero indegree in the final component graph.
  • All-state: synthesis over all eight final agent states.
  • Random: synthesis over a random subset with the same number of candidates as the source pool for that problem.

Source, All-state, and Random share the same readout model, neutral synthesis prompt, and decoding settings. The neutral prompt removes the original description of the candidates as “refinement-maximal” and the statement that no other state has been judged able to refine them. These descriptions are specific to source-component selection and do not generally apply to the other candidate pools. Removing them allows us to compare candidate selection under a common synthesis instruction. All three variants invoke synthesis even when only one candidate is provided. They also share the same input-context limit and maximum output length.

Table R1: Final-answer accuracy (%) under controlled readout variants with Qwen3 backbones:

BackboneDatasetNativeSourceAll-stateRandom
0.6BGSM-Hard37.2339.4333.8731.25
0.6BGPQA26.2628.2820.2020.71
1.7BGSM-Hard64.3164.8862.0961.11
1.7BGPQA43.9442.9340.4039.39
8BGSM-Hard72.3071.9768.8768.05
8BGPQA55.5656.0650.5152.02

As shown in Table R1. Source exceeds All-state in all six settings by 2.53–8.08 percentage points. Because both use the same underlying final states and synthesis procedure, this comparison directly supports the benefit of source-component selection. Source also exceeds Random by 3.54–8.18 points, showing that topology-informed selection outperforms random subsets of the same cardinality. The deployed Native policy also exceeds All-state in every setting, by 2.22–6.06 points. Native is not uniformly better than Source. Their comparison changes both prompt wording and singleton handling and therefore does not isolate either factor. The Source–All-state comparison provides the controlled evidence for candidate selection itself.

Q2. Correct-answer retention and singleton handling

Revision location: Appendix E.6, Table 18, p. 30.

We measure correct-answer retention as the fraction of problems whose source pool retains at least one correct answer, among those with a correct answer in the full final-state pool.

Retention ranges from 92.2% to 98.4%. (Table R2) On the remaining 1.6–7.8% of these problems, selection removes all correct candidates. Across all evaluated settings, Source achieves higher final synthesis accuracy than All-state. A singleton denotes a source pool containing exactly one agent state. On the same singleton problems, direct return matches or exceeds synthesis accuracy in five of six settings. These results support direct return as the default singleton readout, saving an additional synthesis call.

Table R2: Readout diagnostics: Retention is conditional on the full final-state pool containing at least one correct answer. Singleton accuracies compare direct return and synthesis on the same problems with exactly one source candidate. Token values report Source/All-state mean readout input plus output tokens per problem, in thousands, averaged over all evaluation problems.

BackboneDatasetRetention (%)Singleton accuracy (%): Direct / SynthesisReadout tokens (K): Source / All-state
0.6BGSM-Hard98.047.82 / 39.460.89 / 1.21
0.6BGPQA92.234.29 / 28.571.16 / 1.49
1.7BGSM-Hard97.171.84 / 72.991.05 / 1.50
1.7BGPQA94.839.58 / 37.501.68 / 2.14
8BGSM-Hard94.372.73 / 71.092.05 / 3.32
8BGPQA98.460.87 / 60.873.53 / 5.31

Readout cost. The synthesis variants share the same input and output limits, while actual token consumption varies with the candidate set and generated output. Source reduces average readout token consumption by 21.5–38.3% relative to All-state while achieving higher accuracy. These savings concern the final readout stage only and are measured without singleton bypass, since both variants always invoke synthesis.

Together, these results support source-component selection as an effective empirical readout policy: it preserves most correct-answer coverage and improves final synthesis accuracy while reducing readout cost.

AuthorsOpenReview ↗

Follow-up on Our Response

Thanks so much again for your positive rating and constructive comments. As the rebuttal period draws to a close, we would like to briefly follow up on the additional analyses addressing your concerns about source-component readout:

  • For your Q1, we conducted controlled readout comparisons. Source outperforms All-state and size-matched Random in all six settings, with 2.53–8.08 points higher accuracy and 21.5–38.3% fewer readout tokens than All-state.
  • For your Q2, we evaluated correct-answer retention and singleton handling. Source retains a correct answer in 92.2–98.4% of problems where the full final-state pool contains one; direct return matches or exceeds synthesis in five of six singleton settings.

These results are reported in Appendix E.6, pp. 29–30, and Tables R1–R2 in our responses.

We sincerely hope these additional experiments address your concerns and clarify the empirical basis for our readout policy. If any questions remain, we would be very happy to provide further clarification. And if your concerns have been addressed, we would greatly appreciate an updated assessment reflecting the additional evidence.

Thanks again for your time and helpful feedback.

Best,

The authors

Comments on the paper

7 comments

Q3. Refinement trajectory statistics and final-candidate survival

Revision location: Appendix D.2, p. 21.

We analyze running logs and final candidates across the same three backbone scales and two benchmarks, using eight agents and the reasoning backbone as the refinement judge. We count completed reasoning updates rather than topology-induction or readout calls. Early stopping occurs when the execution-edge set remains unchanged before the two-update limit. This criterion measures structural stability, rather than convergence of the reasoning content. We report the mean number of completed updates and the proportion of evaluated problems that terminate early under this criterion. Failures or timeouts do not qualify as structural early stopping.

Corrections retained in final candidates. For each problem qq, let Cq\mathcal{C}_q be the final source-selected candidate set used by the native readout, including singleton cases. Received records whether a candidate actually received peer information in at least one update input. W→C records an initially wrong answer that becomes correct in the final state, and C→W records the reverse transition. All three percentages use the pooled final-candidate count, ∑q∣Cq∣\sum_q |\mathcal{C}_q|, as their denominator, counting each selected candidate once.

Table R5: Execution and final-candidate statistics for default LoRIT, using the reasoning backbone as the refinement judge.

BackboneDatasetMean updatesEarly stop (%)Received (%)W→C (%)C→W (%)
Qwen3-1.7BGSM-Hard1.584269.8213.311.49
Qwen3-1.7BGPQA1.346679.5814.421.75
Qwen3-8BGSM-Hard1.772367.399.251.21
Qwen3-8BGPQA1.534772.9011.252.79
Qwen3-30B-A3BGSM-Hard1.356556.156.571.82
Qwen3-30B-A3BGPQA1.257568.559.254.78

Table R5 shows that the mean number of updates ranges from 1.25 to 1.77, with 23–75% of problems stopping early. Among final candidates, 56.15–79.58% received peer information. This is compatible with source-component selection: agents can receive information before becoming final source candidates, and agents within a source SCC can communicate with one another.

Across the six settings, 6.57–14.42% of final candidates changed from incorrect to correct, while 1.21–4.78% changed from correct to incorrect. Corrections outnumber regressions in every setting, yielding a net accuracy improvement of 4.47–12.67 percentage points among the selected candidates relative to those same agents' initial answers. These results show that reasoning improvements survive candidate selection and remain available to the final readout.

Q3-Case study: A correction retained through final readout

Revision location: Section 4.1, pp. 9–10.

Here we show an example tracing. The following GSM-Hard trace connects useful observations in imperfect source states with recipient corrections retained through final readout. It uses eight Qwen3-8B agents with the backbone itself as the refinement judge. No initial state explicitly reports the correct numerical result, but two states flag a key unresolved assumption about the price increase. Repeated one-step replays from the same saved recipient states compare self-revision, revision with the logged peer inputs, and revision with those two source states removed.

Case study: A useful correction from an imperfect state

Setting. Eight role-conditioned Qwen3-8B agents; Qwen3-8B judge; agent temperature 0.50.5; two updates.

Task. Carrots initially cost P0=1,604,814P_0=1{,}604{,}814. The price increases by 5%5\% of the original price each year. Find the price after three years, rounded to the nearest integer.


1. An incorrect answer can contain a useful warning.

Agents A0A_0 and A3A_3 use compound growth but also question that interpretation. Although A3A_3 reports 1,857,7731{,}857{,}773, its initial reasoning states:

“which implies a simple annual increase rather than compound interest. However, the calculation assumes compounding”

Recipients A4A_4 and A5A_5 also use compound growth and report incorrect numerical answers.

2. Recipients resolve the assumption and revise their answers.

Both recipients receive A0A_0 and A3A_3 among their logged incoming states. During the first update, they replace compounding with a fixed annual increase:

=1,845,536.1⟶1,845,536.=1{,}845{,}536.1 \longrightarrow\boxed{1{,}845{,}536}.

Both retain this result after the second update.

RecipientInitialUpdate 1Update 2
A4A_41,776,1511,845,5361,845,536
A5A_51,859,0031,845,5361,845,536

3. The corrections remain available to final readout.

The final graph has one source SCC containing all eight agents, so both corrected recipients remain candidates. Six parsed answer fields contain 1,845,5361{,}845{,}536, and final synthesis returns {"answer": "1845536"}. No agent is excluded in this example.

4. Replays test the contribution of peer information.

Each condition restarts the first update from the same saved recipient state, using independent fresh samples. Self-revision omits all peer inputs; the last condition removes only the states of A0A_0 and A3A_3 from the logged peer set.

RecipientSelf-revisionLogged peersPeers without A0,A3A_0,A_3
A4A_40/4024/400/32
A5A_50/408/400/32

Entries are correct outputs / fresh samples; the original logged update is excluded.


Interpretation. The replays show higher recipient correction rates when the peer states containing the unresolved assumption are included. In the logged trajectory, both corrected recipients remain in the final candidate pool, and final synthesis returns the correct answer.

Case study: A useful correction from an imperfect state

Setting. Eight role-conditioned Qwen3-8B agents; Qwen3-8B judge; agent temperature 0.50.5; two updates

Task. Carrots initially cost P0=1,604,814P_0=1{,}604{,}814. The price increases by 5%5\% of the original price each year. Find the price after three years, rounded to the nearest integer.


1. An incorrect answer can contain a useful warning.

Agents A0A_0 and A3A_3 use compound growth but also question that interpretation. Although A3A_3 reports 1,857,7731{,}857{,}773, its initial reasoning states:

“which implies a simple annual increase rather than compound interest. However, the calculation assumes compounding”

Recipients A4A_4 and A5A_5 also use compound growth and report incorrect numerical answers.

2. Recipients resolve the assumption and revise their answers.

Both recipients receive A0A_0 and A3A_3 among their logged incoming states. During the first update, they replace compounding with a fixed annual increase:

=1,845,536.1⟶1,845,536.=1{,}845{,}536.1 \longrightarrow\boxed{1{,}845{,}536}.

Both retain this result after the second update.

RecipientInitialUpdate 1Update 2
A4A_41,776,1511,845,5361,845,536
A5A_51,859,0031,845,5361,845,536

3. The corrections remain available to final readout.

The final graph has one source SCC containing all eight agents, so both corrected recipients remain candidates. Six parsed answer fields contain 1,845,5361{,}845{,}536, and final synthesis returns {"answer": "1845536"}. No agent is excluded in this example.

4. Replays test the contribution of peer information.

Each condition restarts the first update from the same saved recipient state, using independent fresh samples. Self-revision omits all peer inputs; the last condition removes only the states of A0A_0 and A3A_3 from the logged peer set.

RecipientSelf-revisionLogged peersPeers without A0,A3A_0,A_3
A4A_40/4024/400/32
A5A_50/408/400/32

Entries are correct outputs / fresh samples; the original logged update is excluded.


Interpretation. The recipients revise their growth assumption and reach the correct answer after receiving peer states that contain a useful alternative interpretation. The replays show higher correction rates when those states are included. Both corrected recipients remain in the final candidate pool, and final synthesis returns the correct answer. Together with the roundwise statistics, this example illustrates convergence accompanied by substantive reasoning revision, connecting useful differences in peer reasoning with corrections retained through readout.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.