acceptodds
Under review as a conference paper at ICLR 2027

Learning Topology Preferences for Adaptive Communication Topology Generation in Multi-Agent Systems

Abstract

Large language model (LLM)-based multi-agent systems (MAS) have demonstrated strong capabilities in solving complex tasks, where communication topology plays a critical role in determining performance. However, most methods leverage execution feedback at the whole-graph level, leaving the fine-grained structural decisions underlying topology generation insufficiently explored. In particular, the relative preferences revealed by comparative topology executions are rarely captured as explicit guidance for generation. To this end, we propose Topology Preference-guided Adaptive Generation (TopoPAG), a framework that transforms execution-derived topology preferences into task-specific structural guidance for autoregressive MAS topology generation. Specifically, TopoPAG learns preferences from the comparative utilities of candidate topologies and embeds them into a compact continuous structural space using graph descriptors. The predicted preference provides an initial topology representation beyond discrete template selection. During generation, this representation is adaptively refined based on the partially constructed graph to guide subsequent role and communication-link predictions. Furthermore, the learned preference weights graph generation, allowing all evaluated topologies to contribute according to their relative utility. Experiments on six reasoning and coding benchmarks demonstrate that TopoPAG consistently produces more effective and communication-efficient topologies. Our code is available at https://anonymous.4open.science/r/TopoPAG-7853.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Multi-Agent Systems, Communication Topology Design, Topology Preference Learning, Large Language Models, Graph Generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Z2d3OpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TopoPAG, which learns task-specific preferences from the execution outcomes of candidate communication topologies and uses them to guide topology generation in LLM-based multi-agent systems. The method constructs a continuous topology representation from the learned preferences and updates it during generation to guide the prediction of agent roles and communication edges. It also uses preference-weighted training to train the topology generator. Experiments on six benchmarks show improvements over the compared methods.

Critical strengths

1.The paper addresses an interesting problem. Existing methods mainly use execution feedback to evaluate complete graphs, whereas this work uses the relative performance of different topologies to guide graph generation step by step.

2.The experimental results are consistent. TopoPAG outperforms the compared methods on all six benchmarks. The ablation studies also show the contributions of preference learning and adaptive updates. In addition, evaluations with different LLM backbones provide further evidence of the method's effectiveness.

Critical weaknesses

1.The need for the added complexity is not sufficiently established. TopoPAG improves average performance over ARG-Designer by 2.24 percentage points, with smaller gains on some benchmarks. Although the paper includes ablation studies, it lacks a direct comparison with a simpler topology selection method. As a result, it remains unclear whether the proposed generation mechanism is necessary.

2.The experimental evaluation has some gaps. The paper does not report statistical uncertainty for the main results or examine the stability of the learned topology preferences. In addition, the efficiency analysis mainly focuses on prompt-token usage and does not fully account for offline topology evaluation, training costs, or inference latency.

Questions for authors

1.How does TopoPAG compare with a simpler method that directly predicts and selects the most suitable candidate topology using the same training data? Could this method achieve similar results without continuous topology representations and adaptive generation?

2.Some improvements are relatively small, particularly on GSM8K (+0.51 percentage points) and MMLU (+1.31 percentage points). Could the authors report results across multiple runs, together with standard deviations or confidence intervals?

3.What is the total computational cost of TopoPAG, including the evaluation of 18 candidate topologies, training, and inference? How does it compare with ARG-Designer and OFA-MAS when all these costs are taken into account?

4.Are candidate topologies evaluated multiple times for each training task? Given the stochastic nature of LLM outputs, how stable are the resulting topology preferences across repeated executions?

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer Z2d3

We thank the reviewer for recognizing the novelty of our approach and its consistent empirical improvements. We have strengthened the evidence for the proposed mechanism and addressed the concerns regarding statistical reliability, computational costs, and preference stability. All corresponding updates are included in the revised manuscript and highlighted in blue.

Q1. Comparison with Simpler Alternatives

We thank the reviewer for raising this important concern. To isolate the benefits of the proposed generation mechanism, we evaluated two simpler alternatives: TopoPAG-Top1, which directly executes the highest-preference anchor, and TopoPAG-NoState, which retains the same preference-weighted training data but removes the PCA-based topology state. Under comparable execution settings, TopoPAG consistently outperforms both alternatives across all six benchmarks, achieving 93.03% average accuracy versus 90.67% and 89.89%, respectively. These improvements (+2.36 and +3.14 percentage points) provide direct evidence that continuous preference-guided generation offers benefits beyond simpler topology selection and generation. These comparisons have been incorporated into Table 1 and the discussion in Section 4.2, with implementation details provided in Appendix D.

MethodMMLUGSM8KAQuAMultiArithSVAMPHumanEvalAverage
TopoPAG-NoState85.1191.0284.5696.2992.5089.8789.89
TopoPAG-Top186.3792.2184.4097.0293.8490.1590.67
TopoPAG88.2493.8487.3899.4696.1593.1193.03

Q2. Statistical Reliability

We thank the reviewer for emphasizing the importance of statistical reliability. We show 10 repeated runs across all six benchmarks and additionally computed 95% paired bootstrap confidence intervals using 10,000 query-level resamples against the strongest baseline on each benchmark. As shown below, TopoPAG maintains consistent performance across repeated runs, with particularly low variability on GSM8K (93.84 ± 0.15%). Moreover, the paired confidence intervals have positive lower bounds on five of the six benchmarks, providing further support for the reported performance gains. The results are presented in the new Statistical Reliability analysis and table in Appendix E.

BenchmarkMean ± Std (%)Paired Gain (pp)95% Paired CI (pp)
MMLU88.24 ± 1.31+1.31[0.00, 3.27]
GSM8K93.84 ± 0.15+0.51[0.08, 0.99]
AQuA87.38 ± 0.81+3.74[1.18, 6.69]
MultiArith99.46 ± 0.19+1.33[0.33, 2.50]
SVAMP96.15 ± 0.12+1.82[0.90, 2.80]
HumanEval93.11 ± 0.65+3.00[0.61, 6.10]

Mean ± Std is computed across 10 repeated inference runs. Confidence intervals are obtained from 10,000 paired bootstrap resamples of matched query-level outcomes.

Q3. Comprehensive Computational Cost

We thank the reviewer for emphasizing the need for a comprehensive cost evaluation. We quantified the offline evaluation of 18 candidate topologies, model training, and end-to-end inference, comparing TopoPAG with ARG-Designer and OFA-MAS on MMLU. As shown below, TopoPAG incurs higher offline token consumption (2.93M), but achieves the lowest training cost (0.29 GPU-h) and competitive inference latency (12.04 s/query), close to ARG-Designer (11.20 s/query) and substantially lower than OFA-MAS (33.10 s/query). These results provide a more complete picture of the computational trade-offs underlying TopoPAG's performance improvements. We have added the comprehensive cost comparison and corresponding discussion in Section 4.2, with measurement details provided in Appendix E.

MethodCollection (M) ↓Inference (Tokens/Q) ↓Total Tokens (M) ↓Training (GPU-h) ↓Latency (s/Q) ↓
ARG-Designer2.552,5612.9420.3611.20
OFA-MAS2.624,6083.3250.3333.10
TopoPAG2.933,2153.4220.2912.04
AuthorsOpenReview ↗

Author Response to Reviewer Z2d3

Q4. Topology Preference Stability

We thank the reviewer for highlighting the impact of stochastic LLM executions on preference reliability. In the original data collection, each query–anchor pair is evaluated once. To assess stability, we repeated all 18 anchor executions three times on 18 sampled development queries (3 per benchmark), totaling 972 executions with the original reward function.

We measure Pairwise Agreement (consistency of anchor-pair reward orderings across runs), Coverage (proportion of comparisons retained after excluding ties), and Top-1 Set Jaccard (overlap of highest-reward anchor sets across runs). The respective macro averages of 97.56%, 96.82%, and 94.44% indicate that both relative preferences and top-performing anchors remain largely consistent across repeated executions, supporting the reliability of the rollout-derived preference supervision. We have included the preference stability results and analysis in the new Section 4.5 (Topology Preference Analysis), with detailed metric definitions and formulas provided in Appendix E.

BenchmarkQueriesPairwise AgreementCoverageTop-1 Set Jaccard
MMLU391.68%97.24%88.89%
GSM8K3100.00%96.73%100.00%
AQuA3100.00%96.73%100.00%
MultiArith398.05%96.73%100.00%
SVAMP3100.00%96.73%100.00%
HumanEval395.65%96.73%77.78%
Macro Average1897.56%96.82%94.44%

Thank you for your detailed responses and additional experiments. I appreciate the authors' efforts to address my concerns.

However, I still have substantial reservations about the paper's core contribution. The proposed method appears to mainly combine existing techniques, without introducing a clearly novel learning principle, architecture, or generalizable theoretical insight. Moreover, the performance improvements over strong baselines are relatively modest, while the method consumes more inference tokens than ARG-Designer. Without a comparison under matched token budgets, it remains unclear whether the gains come from the proposed mechanism or simply from additional computation. Overall, the contribution appears to be primarily engineering-oriented, and concerns remain regarding the attribution of the experimental gains and the broader scientific significance of the work. Therefore, I am still not fully convinced of the significance of the proposed approach beyond combining existing methods.

Nevertheless, I acknowledge the additional experimental evidence and the authors' efforts to address my concerns. But the novelty of this paper is insufficient, I keep score at 2. The current computational cost analysis is limited to MMLU. I would like to see cost evaluations on additional benchmarks before considering raising my score.

AuthorsOpenReview ↗

Author Response to Reviewer Z2d3

We sincerely thank the reviewer for their continued engagement and thoughtful feedback, and apologize for our delayed response. We hope that our additional clarifications and experimental results will help address the remaining concerns.

The distinction of TopoPAG lies in how topology design itself is conceptualized. Existing approaches primarily view topology design as finding, selecting, or optimizing effective structures. In contrast, TopoPAG takes a step further by treating topology design as a process of learning reusable, task-specific structural preferences from past execution experience and applying them to construct new topologies. Specifically, it uses the comparative outcomes of candidate topologies to learn these preferences, embeds them into a continuous topology space, and progressively updates the resulting structural guidance during graph generation. In this way, TopoPAG connects past execution experience with future topology construction, allowing learned preferences to guide the generation of new topologies. This shifts topology design from learning which structures perform well to learning how structural preferences can guide the construction of new topologies, and more broadly, from identifying the best existing topologies to learning how effective topologies can be constructed.

In addition, we have extended the cost evaluation from MMLU to all six benchmarks. The complete results are presented below.

Computational cost comparison across six benchmarks.

DatasetMethodCollection (M) ↓Inference (Tokens/Q) ↓Total Tokens (M) ↓Training (GPU-h) ↓Latency (s/Q) ↓
MMLUARG-Designer2.552,5612.940.3611.20
MMLUOFA-MAS2.624,6083.330.3333.10
MMLUTopoPAG2.933,2153.420.2912.04
GSM8KARG-Designer1.954,8948.410.3212.55
GSM8KOFA-MAS1.804,9738.360.2722.60
GSM8KTopoPAG2.224,1467.690.3411.02
AQuAARG-Designer2.234,3213.330.3223.10
AQuAOFA-MAS1.824,3052.910.2643.14
AQuATopoPAG2.645,0193.920.3522.49
MultiArithARG-Designer1.734,4184.380.3310.81
MultiArithOFA-MAS1.544,9124.490.3122.21
MultiArithTopoPAG2.093,9954.490.308.35
SVAMPARG-Designer1.434,5806.010.3711.24
SVAMPOFA-MAS1.394,3405.730.3219.60
SVAMPTopoPAG1.623,9915.610.337.69
HumanEvalARG-Designer2.492,5332.910.379.75
HumanEvalOFA-MAS3.034,8983.830.3326.20
HumanEvalTopoPAG2.732,5563.150.3010.26
AverageARG-Designer2.063,8844.660.3413.11
AverageOFA-MAS2.034,6734.770.3027.81
AverageTopoPAG2.373,8214.710.3211.98

The results show that TopoPAG incurs moderately higher offline collection costs (2.37M vs. 2.06M tokens for ARG-Designer). However, its average inference token consumption is comparable to ARG-Designer (3,821 vs. 3,884 tokens/query) and lower than OFA-MAS (4,673 tokens/query). Consequently, the overall token consumption also remains comparable (4.71M for TopoPAG, 4.66M for ARG-Designer, and 4.77M for OFA-MAS). TopoPAG also achieves lower average inference latency than both baselines (11.98 vs. 13.11 and 27.81 s/query, respectively). We have added these results to Section 4.2 and Appendix E of the updated paper.

Reviewer_1tnnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TopoPAG for task-specific communication topology generation in LLM-based multi-agent systems. It learns preferences from candidate topology executions, maps them into a continuous structural space, and updates the resulting state during autoregressive role and edge generation.

Critical strengths
  1. The method explicitly converts comparative execution feedback into structural preferences that guide both role and edge generation. Once trained, it generates task-specific topologies without executing candidate anchors at inference time.
  2. The reported improvements over strong baselines across six benchmarks are encouraging. Additional backbone evaluations and component ablations provide further empirical support for the framework.
Critical weaknesses
  1. The source of the performance gains is insufficiently isolated. The preference predictor already scores the 18 anchors, but the paper does not show the performance of simply executing the top-ranked one. The ablations support the proposed components within the generative framework, but do not show whether generation improves over direct selection using the same rollout data and comparable settings. Without this control, it is difficult to distinguish the benefits of rollout-based supervision and training from those of the proposed structural representation and adaptive generation.
  2. Offline costs are not quantified. Collecting supervision requires executing all 18 anchors for each training query, yet data-collection and model-training costs are not reported. The efficiency analysis focuses on inference prompt tokens, leaving the overall cost–benefit trade-off insufficiently assessed.
Questions for authors
  1. How does TopoPAG compare with direct Top-1 anchor selection using the same rollout data and comparable training and execution settings?
  2. Please quantify the offline data-collection and model-training costs, including LLM usage and training time, and compare them with the main baselines.
  3. Could you provide a quantitative comparison of topology preferences across task types, beyond the illustration in Figure 1 and the two examples in Figure 5? For example, a heatmap of average predicted preferences on held-out queries would be helpful. I would consider raising my score if the authors can show a clear benefit over direct Top-1 anchor selection using the same rollout data and comparable settings, and clarify the offline costs.
3 comments
AuthorsOpenReview ↗

Author Response to Reviewer 1tnn

We appreciate the reviewer's recognition of our approach and constructive suggestions for strengthening its empirical evaluation. In response, we have provided additional evidence on the benefits of topology generation, computational efficiency, and task-specific preferences. These results and clarifications are now incorporated into the revised manuscript, with changes highlighted in blue.

Q1. Top-1 Comparison

We thank the reviewer for this important suggestion. We introduced a TopoPAG-Top1 (Direct Top-1 selection) baseline that selects and executes the highest-ranked anchor using the same rollout-derived preference predictor, candidate anchors, and agent execution settings as TopoPAG. As shown below, TopoPAG consistently outperforms TopoPAG-Top1 across all six benchmarks, achieving 93.03% vs. 90.67% macro-average accuracy (+2.36 percentage points). These results demonstrate that the gains extend beyond preference-based anchor selection, supporting the additional value of adaptive topology generation. We have included these comparisons in Table 1 and the analysis in Section 4.2, with the shared experimental settings clarified in Appendix D.

MethodMMLUGSM8KAQuAMultiArithSVAMPHumanEvalAverage
TopoPAG-Top186.3792.2184.4097.0293.8490.1590.67
TopoPAG88.2493.8487.3899.4696.1593.1193.03

Q2. Comprehensive Computational Cost

We thank the reviewer for highlighting the importance of a comprehensive cost comparison. We quantified offline data collection, model training, and inference costs against ARG-Designer and OFA-MAS under the same MMLU evaluation setting. As shown below, TopoPAG requires more offline collection tokens (2.93M), but achieves the lowest training cost (0.29 GPU-h) and competitive inference latency (12.04 s/query). Compared with ARG-Designer, its training cost is reduced by 19.4%, with only 0.84 s additional inference latency. These results clarify the computational trade-off associated with preference-guided topology generation. We have included the comprehensive cost comparison in the new table and discussion in Section 4.2, with cost measurement details provided in Appendix E.

MethodCollection (M) ↓Inference (Tokens/Q) ↓Total Tokens (M) ↓Training (GPU-h) ↓Latency (s/Q) ↓
ARG-Designer2.552,5612.9420.3611.20
OFA-MAS2.624,6083.3250.3333.10
TopoPAG2.933,2153.4220.2912.04

Q3. Cross-Task Preferences

We thank the reviewer for this helpful suggestion. We analyzed the average predicted preferences over 18 candidate anchors across six held-out sets. As shown in the heatmap, 2–4-agent topologies generally receive higher preference mass (55.8–64.8%), with the strongest tendency observed on MMLU (64.8%). Within this shared tendency, task-dependent variations remain evident: MMLU favors S3 (0.105), AQuA favors FC4 (0.072), and HumanEval favors S4 (0.077). These findings illustrate both common structural preferences and task-specific differences, supporting the effectiveness of task-conditioned preference learning. We have incorporated the heatmap (Figure 5 in the updated paper) and corresponding analysis into the new Section 4.5 (Topology Preference Analysis), with evaluation details provided in Appendix E.

AuthorsOpenReview ↗

Author Response to Reviewer 1tnn

Just a gentle follow-up on our response above. We'd appreciate any further thoughts you may have when you have a chance.

AuthorsOpenReview ↗

Author Response to Reviewer 1tnn

In addition, we have extended the cost evaluation from MMLU to all six benchmarks. These results are added to Section 4.2 and Appendix E of the updated paper. The complete results are presented below.

Computational cost comparison across six benchmarks.

DatasetMethodCollection (M) ↓Inference (Tokens/Q) ↓Total Tokens (M) ↓Training (GPU-h) ↓Latency (s/Q) ↓
MMLUARG-Designer2.552,5612.940.3611.20
MMLUOFA-MAS2.624,6083.330.3333.10
MMLUTopoPAG2.933,2153.420.2912.04
GSM8KARG-Designer1.954,8948.410.3212.55
GSM8KOFA-MAS1.804,9738.360.2722.60
GSM8KTopoPAG2.224,1467.690.3411.02
AQuAARG-Designer2.234,3213.330.3223.10
AQuAOFA-MAS1.824,3052.910.2643.14
AQuATopoPAG2.645,0193.920.3522.49
MultiArithARG-Designer1.734,4184.380.3310.81
MultiArithOFA-MAS1.544,9124.490.3122.21
MultiArithTopoPAG2.093,9954.490.308.35
SVAMPARG-Designer1.434,5806.010.3711.24
SVAMPOFA-MAS1.394,3405.730.3219.60
SVAMPTopoPAG1.623,9915.610.337.69
HumanEvalARG-Designer2.492,5332.910.379.75
HumanEvalOFA-MAS3.034,8983.830.3326.20
HumanEvalTopoPAG2.732,5563.150.3010.26
AverageARG-Designer2.063,8844.660.3413.11
AverageOFA-MAS2.034,6734.770.3027.81
AverageTopoPAG2.373,8214.710.3211.98

The results show that TopoPAG incurs moderately higher offline collection costs (2.37M vs. 2.06M tokens for ARG-Designer). However, its average inference token consumption is comparable to ARG-Designer (3,821 vs. 3,884 tokens/query) and lower than OFA-MAS (4,673 tokens/query). Consequently, the overall token consumption also remains comparable (4.71M for TopoPAG, 4.66M for ARG-Designer, and 4.77M for OFA-MAS). TopoPAG also achieves lower average inference latency than both baselines (11.98 vs. 13.11 and 27.81 s/query, respectively).

Reviewer_KYbtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper learns which communication structures suit a task, represents those preferences in a continuous topology space, and uses that representation to guide graph generation. The framework is coherent, and the reported improvements are encouraging

Critical strengths
  • Learning task-dependent preferences over communication structures is a reasonable way to guide topology generation. The structural descriptors also make the representation easier to interpret than an entirely unconstrained latent vector.
  • In Table 1, TopoPAG achieves the highest score on all six benchmarks. Its average is 93.03%, compared with 90.79% for ARG-Designer. The improvements on AQuA and HumanEval are particularly encouraging.
  • Table 2 examines preference learning, preference weighting, adaptive updates, and structural grounding. Additional experiments with two other LLM backbones strengthen the evidence that the result is not specific to one executor model.
Critical weaknesses
  • In Section 4.1, I noticed it says that 40 or 60 training queries are sampled "from each dataset," while Appendix E mentions a training–validation split. From my reading, the manuscript doesn't clearly explain how these training examples are kept separate from the final evaluation sets, especially for HumanEval, where all 164 benchmark items are listed as test examples. I want to be clear that I'm not claiming data leakage happened, just that the protocol is currently ambiguous. I would need to see the exact split definitions to properly interpret the reported gains.
  • Since the approach already evaluates topology anchors and learns preferences over them, a crucial baseline to me would be simply predicting the best anchor and executing it directly. Another natural comparison would be training a generator on the exact same preference-weighted data, just without the PCA-based topology state. While I found the existing component ablations useful, they don't quite convince me that continuous preference-guided generation outperforms these simpler setups.
  • The main results are presented without confidence intervals or standard deviations, which stood out to me because Appendix C lists only 153 test examples for MMLU. At that sample size, a 1.31 percentage-point improvement over the strongest baseline comes down to roughly two extra correct answers. While that doesn't mean the gain is statistically insignificant, I think it makes including paired uncertainty estimates and repeated runs really important here.
  • The graph is constructed before the agents execute. Its adaptive state sees the partially generated graph, rather than the agents’ actual messages. Therefore, the favorable prompt-attack result may show that the generated topology is more robust, but it does not establish that the system detects and adapts to corrupted reasoning during execution.
Questions for authors

see weakness

3 comments
AuthorsOpenReview ↗

Author Response to Reviewer KYbt

We sincerely thank the reviewer for the constructive feedback and recognition of our work. We have addressed the concerns through additional baseline comparisons, statistical analyses, and clarifications of the data-splitting protocol and topology adaptation. All corresponding changes have been incorporated into the revised manuscript and highlighted in blue, as detailed below.

W1. Data Split Clarification

We thank the reviewer for highlighting this ambiguity and acknowledge that our description was insufficiently clear. Following the protocol adopted in prior work, we sample 40 or 60 queries per benchmark to form a development set, further divided into training and validation subsets at a 9:1 ratio. For HumanEval, the 40 development queries are split into 36 training and 4 validation queries, strictly disjoint from the test problems. The same separation applies to all benchmarks. We have clarified the data-splitting protocol in Section 4.1.

W2. Comparison with Simpler Alternatives

We thank the reviewer for this valuable suggestion. We added two baselines: TopoPAG-Top1, which executes the anchor with the highest predicted preference, and TopoPAG-NoState, which retains the same preference-weighted training data but removes the PCA-based topology state. As shown below, TopoPAG consistently outperforms both baselines across all six benchmarks, improving macro-average accuracy by 2.36 and 3.14 percentage points, respectively. These results support the benefit of continuous preference-guided generation beyond simpler anchor selection and preference-weighted generation. The additional baseline results are included in Table 1 and discussed in Section 4.2, with implementation details provided in Appendix D.

MethodMMLUGSM8KAQuAMultiArithSVAMPHumanEvalAverage
TopoPAG-NoState85.1191.0284.5696.2992.5089.8789.89
TopoPAG-Top186.3792.2184.4097.0293.8490.1590.67
TopoPAG88.2493.8487.3899.4696.1593.1193.03

W3. Statistical Reliability

We thank the reviewer for emphasizing the importance of statistical reliability. We show the results of 10 independent runs and performed paired bootstrap analysis (10,000 resamples) against the strongest baseline on each benchmark. TopoPAG consistently improves performance across all six benchmarks, with 95% paired confidence intervals excluding zero on five benchmarks. The repeated-run results further demonstrate stable performance, including 88.24 ± 1.31% on MMLU. These findings provide additional statistical support for TopoPAG's effectiveness. We have additionally presented their standard deviations and 95% paired bootstrap confidence intervals in Appendix E.

BenchmarkMean ± Std (%)Paired Gain (pp)95% Paired CI (pp)
MMLU88.24 ± 1.31+1.31[0.00, 3.27]
GSM8K93.84 ± 0.15+0.51[0.08, 0.99]
AQuA87.38 ± 0.81+3.74[1.18, 6.69]
MultiArith99.46 ± 0.19+1.33[0.33, 2.50]
SVAMP96.15 ± 0.12+1.82[0.90, 2.80]
HumanEval93.11 ± 0.65+3.00[0.61, 6.10]

Mean ± Std is computed over 10 runs. Paired gains and 95% confidence intervals are estimated using 10,000 paired bootstrap resamples against the strongest baseline on each benchmark.

W4. Scope of Topology Adaptation

We thank the reviewer for this important clarification. We agree that TopoPAG does not detect corrupted reasoning or dynamically reconfigure the topology during agent execution. Instead, adaptation occurs during pre-execution topology generation, where the topology state is updated based on the task query and partially generated graph. Accordingly, the prompt-attack results demonstrate the robustness of the generated communication topology under adversarial interference, rather than runtime attack detection. We have clarified this distinction in Section 4.3, emphasizing that the observed robustness comes from the generated communication topology rather than runtime attack detection or adaptation.

Thank you for your detailed responses and additional experiments. I would like to keep my positive score. Good luck!

AuthorsOpenReview ↗

Author Response to Reviewer KYbt

Thank you very much for your encouraging response and continued support. We sincerely appreciate the time and effort you have devoted to reviewing our work and providing constructive feedback. Your comments have been valuable in improving our paper. Thank you again, and we wish you all the best!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.