- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DyTopo. Multiple worker agents first generate messages and then describe what they need and what they can provide. The system uses these descriptions to build a communication graph and sends selected messages into the next round. A manager updates the shared goal and decides when to stop.
Critical strengths
- DyTopo obtains the best result in all 16 model-task settings in Table 1. This suggests that the method is not effective only in one isolated case.
- The random-routing, low-similarity, and fixed top-k comparisons support the claim that message selection matters. Under a fixed five-round budget, DyTopo also reports lower token usage than several baselines.
Critical weaknesses
-
The most important fixed-topology comparisons are missing. The paper does not compare against LLM Debate, nor does it include a fixed Solver-Verifier structure for the mathematical tasks. The paper lacks a same-framework control that freezes the semantic graph constructed in the first round. Therefore, it is unclear whether the gains come from semantic routing or from rebuilding the graph at every round.
-
The comparison with direct dynamic methods is missing, and generalization is difficult to assess. The paper does not compare against methods such as DyLAN. The mathematical experiments use only 70-problem subsets of MATH-500 and Omni-MATH, while APPS and Omni-MATH are also used for threshold selection. Results with four models and threshold transfer provide some evidence, but they are not enough to establish stable performance on independent distributions.
Questions for authors
- Under the same model, roles, message budget, and total token budget, could you compare DyTopo with LLM Debate, a fixed Solver-Verifier structure for mathematics, and a version that freezes the semantic graph after the first round? These comparisons are needed to determine whether dynamic rewiring provides an independent gain.
- Could you include a directly relevant dynamic collaboration baseline such as DyLAN? Please report accuracy, number of model calls, and token usage, and explain how the costs of team selection and early stopping are handled fairly.
- Could you select the roles, prompts, and thresholds on an independent development set, then evaluate on a complete mathematical benchmark or a sufficiently large held-out test set that was not used for selection? Please report paired confidence intervals. The current 70-problem subsets and partial evaluation-set reuse make generalization difficult to assess.
General Response to Area Chair and All Reviewers
We sincerely thank the Area Chair and all reviewers for their time and constructive feedback. We are encouraged by your recognition of our intuitive, training-free need–offer semantic routing (
PJnv,2pQR,9bdD), top performance across all 16 settings (2pQR,9bdD,eD44), substantial token/runtime savings (2pQR,9bdD,eD44), routing ablations (9bdD,eD44), and clear presentation (9bdD,eD44).We briefly recap our core contributions: DyTopo pioneers within-task structural self-improvement by (1) formulating reasoning as a coupled update of agent state and topology G(t); (2) proposing training-free query–key matching with verification-gated halting; and (3) ranking 1st in all 16 settings (+5.2 pts mean gain; paired bootstrap +6.31 pts, p<0.01), while cutting tokens by 35%–56% and latency by up to 53%.
Summary of New Experiments & Responses to Core Concerns
Below we summarize how our new experiments address each core concern (see individual replies for full tables):
1. Isolating dynamic rewiring from Frozen Round-1 topology & Manager (
9bdDW1/Q1,2pQRW3/Q1,eD44W1/Q1,PJnvW1). On Qwen3-8B: (i) Frozen Round-1 Graph (G(t)≡G(0)) drops accuracy from 89.63% to 42.68% on HumanEval ($-46.95pts)and51.43-$30.00 pts), as Round-1 memories Hi(0)=∅ lack draft solutions and miss late-stage verification edges (Developer<->Tester); (ii) Static Manager + Dynamic Routing reaches 78.05% on HumanEval (+35.37 pts over Frozen Round-1 and +31.10 pts overSingle-turn[46.95%]), while dynamic Manager subgoals (89.63%/51.43%) synergize on multi-step tasks. With halting disabled (5 rounds), DyTopo still cuts tokens by 35% (MiMo) and 20%–31% (Qwen: 8,594 vs. 10,720–12,371).2. Comparisons with LLM Debate, Fixed Solver–Verifier, and DyLAN (
eD44W1/W2/Q1/Q2,2pQRW4/Q3). On Qwen3-8B (fairly counting all peer-rating and Manager costs): DyTopo (89.63% HumanEval, 75.71% MATH-500, 4,736 tokens, 15.3s) beats 5-roundLLM Debate(68.29%/54.29%, 17,934 tokens) by +21.34/+21.42 pts with 73.6% fewer tokens, matched-budgetLLM Debate($\sim$8,500 tokens: 59.76%/48.57%) by +29.26/+27.14 pts,Fixed Solver–Verifier(Single-turn: 46.95%/57.14%; 5-roundAgentScope: 80.00%/72.86%, 10,720 tokens), andDyLAN(77.44%/68.57%, 11,378 tokens, 35.4s) by +12.19/+7.14 pts with 58.4% fewer tokens and 56.8% lower latency.3. Disjoint Dev/Test threshold validation (
9bdDQ2,eD44W2/Q3). We only coarsely selected τedge on one dataset per domain to avoid extreme channel density/sparsity. A strict 50% Dev → 50% Test Split yields +7.64 to +11.43 pts gains (21.36% vs. 13.72% on APPS Llama3; 37.14% vs. 25.71% on Omni-MATH Llama3; 48.57% vs. 40.00% on Omni-MATH Qwen3), alongside 100% held-out HumanEval (+5.75 pts, p=0.003) and MATH-500 (+7.58 pts, p=0.001).4. Scalability, noise robustness, and generality (
PJnvW2/W4/Q1–Q3,2pQRW2/W4/Q2/Q3). (i) Scaling N: Adding math agents into HumanEval (N=5,6,7, Qwen3) shows DyTopo isolates irrelevant agents (89.63%, 89.63%, 89.02%) at 1.8–2.3 ms/round (<0.03% overhead). (ii) Noise: Under Gaussian score noise and 10% descriptor corruption, DyTopo reaches 88.41%/42.86% and 84.15%/40.00%, still beating unperturbed MasRouter (80.98%/38.72%). (iii) Generality: DyTopo beatsAgentScopeby +4.49 pts on MMLU-Pro (67.24% vs. 62.75%), and by +7.31/+4.29 pts (64.02% vs. 56.71%; 50.00% vs. 45.71%) in a mixedQwen3+Llama3pool.With only about three days left in the discussion period, we warmly welcome any further questions or suggestions for prompt discussion!