acceptodds
Under review as a conference paper at ICLR 2027

Discovering Efficient and Explainable Communication Topologies for LLM-based Multi-Agent Systems via Causal Inference

Abstract

The performance of large language model (LLM)-based multi-agent systems (MAS) largely depends on effective communication topologies. Existing topology generation methods, however, typically learn communication topologies through black-box optimization driven solely by task-level rewards. While effective, such optimization provides little insight into why particular communication edges are selected, making it difficult to identify the critical communication subgraphs responsible for successful collaboration. To address this limitation, we propose E2-Explainer, a model-agnostic framework for providing interpretable explanations of communication topologies produced by arbitrary topology generators. Specifically, we formulate topology explanation as a causal attribution problem that identifies compact communication subgraphs supported by edge-level evidence of task preservation. We obtain this evidence with a Granger-style objective that measures how masking each communication channel changes the task outcome and the stability of the final response. The resulting budgeted subgraphs are then distilled into an amortized explainer, enabling efficient post-hoc explanation without repeated edge-level evaluations at deployment. Extensive experiments on multiple reasoning and coding benchmarks demonstrate that E2-Explainer identifies critical communication subgraphs that preserve successful collaboration. These subgraphs can also be executed directly to prune redundant communication edges, substantially reducing communication costs while maintaining competitive task performance.

Primary area:
interpretability and explainable AI
Keywords:
LLM-MAS, Multi-Agent Communication, Communication Topology Optimization, Causal Inference, Explainable AI

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Zs4dOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes E2-Explainer, a post-hoc, model-agnostic framework to identify compact, task-preserving communication subgraphs in LLM-based multi-agent systems (MAS). It estimates a Granger-style causal contribution for each edge by masking channels and measuring changes in task score and result-level semantic entropy, then projects high-utility edges into executable, budgeted subgraphs and distills these into an amortized explainer that predicts subgraphs at test time without repeated edge evaluations. Experiments across six benchmarks and four topology generators report 20–25% token reductions with maintained or improved accuracy, plus transfer across unseen topology generators and agent scales.

Critical strengths
  1. Using causal technologies to explain MAS topologies sounds interesting.

  2. The evaluation is across various benchmarks.

  3. Offers a framework that can be attached to multiple existing topology designers with applicability.

Critical weaknesses

1. Limited evaluation. Firstly, the baseline comparisons are limited. Although the paper discusses AgentPrune [1], it does not include AgentDropout [2]. Moreover, for general-purpose topology designers (e.g., G-Designer [3]), edge-wise normalization can be directly into the optimization objective for sparse topologies. Secondly, the evaluation uses only Qwen3-8B as the LLM backbone, without experiments on other widely used open-source and closed-source LLMs.

2. Less convincing motivation. As illustrated in Figure 1 and discussed in the Introduction, the learned topologies may contain redundant edges and communication patterns that are merely correlated with task success rather than causally contributing to it. However, this issue has already been extensively discussed in AgentPrune [1] and AgentDropout [2]. The paper should therefore clarify why existing MAS pruning methods still retain redundant edges or remove useful ones, and explain how the proposed method addresses these limitations.

3. Unclear presentation. For example, in Table 1, the Method column spans two lines, whereas the Metric column spans three, and the token consumption values are not directly reported. In addition, Tables 2, 3, and 4 are stacked together, making the results less readable.

4. Lack of code availability. The authors do not provide a code repository, making it difficult to reproduce the experiments and assess the reliability of the reported results.

References

[1] CUT THE CRAP: AN ECONOMICAL COMMUNICATION PIPELINE FOR LLM-BASED MULTI-AGENT SYSTEMS (ICLR 2025).

[2] AgentDropout: Dynamic Agent Elimination for Token-Efficient and High-Performance LLM-Based Multi-Agent Collaboration (ACL 2025).

[3] G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networks (ICML 2024).

Questions for authors
  1. Comparison with RL-based pruning methods. Why does E2-Explainer outperform RL-based pruning methods such as AgentPrune and AgentDropout? Could the authors compare RL-based pruning with causal analysis in terms of effectiveness, scalability, and resource consumption? In particular, I would expect RL-based methods to scale better than the edge-wise causal analysis used by E2-Explainer.

  2. Additional baseline. Could the authors include a comparison with AgentDropout?

  3. More LLM backbones. Could the authors evaluate E2-Explainer on additional open-source and closed-source LLM backbones to demonstrate its generalizability?

  4. Code availability. Could the authors release the code through an anonymous GitHub repository to facilitate reproducibility?

17 comments

We thank the reviewer for the constructive feedback. We have addressed the concerns below and made corresponding revisions to the manuscript. All revisions are highlighted in blue in the revised PDF.

W1 & Q2: Limited baselines

Comparison with AgentDropout:First, following the reviewer's suggestion, we have included AgentDropout as an additional baseline in our main comparison. We adopt the official implementation of AgentDropout, with the only modification being the use of two communication rounds instead of the four rounds reported in the original paper for HumanEval, due to time constraints. All other implementation details and hyperparameters remain identical to those in the official implementation.

As shown in the table below, E2-Explainer can further identify critical task-preserving substructures within AgentDropout-optimized topologies, leading to improvements in both performance and efficiency. Specifically, E2-Explainer improves average accuracy by 1.27 percentage points while reducing online token consumption by 21.9%. These results further demonstrate the generality and plug-and-play capability of E2-Explainer, which can complement existing communication topology optimization methods by identifying essential communication subgraphs to further improve task performance and communication efficiency.

DatasetAccuracy: Original → +E2-Explainer (%)Online tokens: Original → +E2-ExplainerToken reduction
MMLU79.31 → 80.392.35M → 1.80M23.5%
GSM8K93.21 → 93.6913.76M → 11.32M17.8%
MultiArith97.67 → 97.725.47M → 4.27M22.0%
SVAMP94.00 → 94.509.65M → 7.34M23.9%
AQuA84.66 → 87.702.65M → 1.71M35.6%
HumanEval87.60 → 90.080.42M → 0.36M14.1%
Avg. / Total89.41 → 90.6834.32M → 26.80M21.9%

Detailed results are also provided in Table 1 and Table 9 of the Appendix in the revised manuscript.

Comparison with edge-wise normalization:Furthermore, we would like to clarify that G-Designer, which is included in our main comparison, already incorporates a regularization mechanism similar to the edge-wise normalization suggested by the reviewer. Specifically, G-Designer employs nuclear-norm regularization on a low-dimensional matrix to encourage low-rank structure and promote sparse communication topologies, referred to as the "regularization objective" in the original paper of G-Designer. However, both our motivation experiments in Figure 1 and the comparison with G-Designer in Table 1 demonstrate that such regularization alone is insufficient to effectively eliminate redundant communication dependencies. In contrast, E2-Explainer can further identify and remove redundant communication structures, leading to additional improvements in both task performance and communication efficiency.

Overall, we have conducted extensive comparisons to demonstrate the effectiveness and plug-and-play capability of E2-Explainer, including comparisons involving AgentDropout and the edge-wise regularization mechanism highlighted by the reviewer. Specifically, as shown in Table 1, we integrate E2-Explainer with five representative state-of-the-art topology optimization methods (ARG-Designer, OFA-MAS, AgentPrune, AgentDropout, and G-Designer). These results consistently demonstrate that E2-Explainer serves as a model-agnostic, post-hoc framework that identifies compact, task-critical causal subgraphs from communication topologies produced by diverse topology designers, further improving both performance and efficiency. The experimental evaluation is comprehensive and thorough.

W1 & Q3: Generalizability across LLM backbones

To evaluate the generalizability of E2-Explainer across different LLM backbones, we conducted additional experiments with two representative models: an open-source model (Llama-3.1-8B) and a closed-source model (GPT-4o-mini), in addition to the original Qwen3-8B backbone. We evaluated all three backbones on MMLU and HumanEval.

As shown in the results below and in Table 6 of the revised manuscript, E2-Explainer consistently improves task accuracy while reducing online token consumption across all three backbones. These results further demonstrate that E2-Explainer is not tied to a specific LLM backbone and can generalize effectively across different model architectures.

BackboneBenchmarkAccuracy: Original → +E2-Explainer (%)Online token reduction
Qwen3-8BMMLU78.65 → 79.7431.5%
Qwen3-8BHumanEval88.40 → 90.6919.3%
Llama-3.1-8BMMLU72.55 → 75.1626.4%
Llama-3.1-8BHumanEval62.50 → 65.0011.0%
GPT-4o-miniMMLU80.39 → 82.3520.7%
GPT-4o-miniHumanEval85.00 → 86.2510.8%

W2: Why does redundancy remain after AgentPrune/AgentDropout?

Communication redundancy is a well-recognized challenge in LLM-based MAS, and existing methods such as AgentPrune and AgentDropout have proposed various pruning strategies to address it. However, our key observation is that substantial communication redundancy can persist even after these efforts.

As demonstrated in our original motivation experiment (Figure 1), G-Designer-generated topologies still contain substantial removable communication despite incorporating sparsity-promoting regularization. Similarly, our original Appendix Figure 4 reveals residual redundancy in OFA-MAS-generated topologies. To further address the reviewer's concern, we have extended the MMLU random edge-masking analysis to AgentPrune and AgentDropout, with the results reported in Appendix Figure 5 of the revised manuscript. Together with the improvements achieved by E2-Explainer on both methods (Table 1), these results further demonstrate that substantial redundancy can remain even in topologies optimized by existing pruning methods.

Regarding why existing MAS pruning methods may still retain redundant edges or remove useful ones, we have clarified this limitation in the Introduction (Lines 78–84) in the original version. Specifically, existing methods primarily optimize communication topologies through aggregate task-level rewards or sparsity objectives, without explicitly attributing task outcomes to individual communication decisions. Consequently, they may fail to distinguish redundant communication from critical task-preserving substructures.

To address this limitation, E2-Explainer explicitly investigate the causal contribution of individual communication edges to the task outcomes achieved by the overall communication topology. Based on these intervention-derived signals, we identify critical communication paths and construct compact, executable subgraphs that preserve task-relevant information flow. We further distill these subgraphs into an amortized explainer, enabling efficient post-hoc refinement of topologies produced by different optimization methods without repeated edge-level interventions at inference time. The aforementioned discussion can also be found in the Introduction (Lines 86–107) and Method sections of the original manuscript.

W3: Unclear presentation.

We thank the reviewer for pointing out these presentation issues. We have carefully revised the manuscript to improve the clarity and readability of the experimental results. Specifically, we have (1) corrected the inconsistent row spans between the Method and Metric columns in Table 1, (2) reorganized Tables 2–4 to improve their layout and readability, and (3) explicitly reported the previously missing token consumption values in Appendix Table 9 of the revised manuscript.

For the reviewer's convenience, we also provide the token consumption results in the table below, with all values reported in thousands of tokens (K):

MethodVariantMMLUGSM8KMultiArithSVAMPAQuAHumanEvalTotal
ARG-DesignerOriginal6262,6991,6784,3871,06746310,919
+ E2-Explainer5502,2811,2873,4617493968,724
OFA-MASOriginal4957,6371,4452,3026902,20714,777
+ E2-Explainer4215,5601,0761,7545671,61610,994
AgentPruneOriginal1,4306,7822,6605,5041,29870918,384
+ E2-Explainer1,2204,9711,9344,24480463413,806
G-DesignerOriginal9851,5493,06911,3781,34072619,047
+ E2-Explainer6758672,2089,09283858614,266
AgentDropoutOriginal2,35413,7645,4749,6502,65242334,317
+ E2-Explainer1,80011,3164,2697,3411,70936326,800

All values represent the total number of prompt and completion tokens consumed over the corresponding evaluation dataset.

W4 & Q4: Code availability

We have provided the anonymized source code and evaluation scripts in the supplementary material. These materials include the core implementation and the main training and evaluation pipelines, enabling reviewers to examine the implementation details and reproduce the reported experimental results.

Reviewer_4qaiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes E2-Explainer, a post-hoc framework for identifying compact and interpretable communication subgraphs in LLM-based multi-agent systems. The method estimates the contribution of individual communication edges through masking-based interventions, combining task performance changes with semantic entropy to construct subgraph supervision. An amortized explainer is then trained to predict efficient communication structures without repeated interventions at inference time. Experiments across six benchmarks and four topology optimization methods show that E2-Explainer can reduce token consumption while generally maintaining or improving task performance.

Critical strengths
  1. The paper addresses a relevant and practical problem. Communication redundancy is an important source of inefficiency in LLM-based multi-agent systems. Investigating which communication links are actually necessary is meaningful, especially as multi-agent workflows become increasingly complex.

  2. The proposed framework is reasonably well motivated and structured. The combination of edge-level intervention, semantic uncertainty estimation, and amortized subgraph prediction provides a coherent approach to topology refinement. The separation between expensive offline supervision construction and efficient online prediction is particularly useful.

Critical weaknesses
  1. The methodological novelty is somewhat limited. The core approach largely follows an established perturbation-based attribution and knowledge distillation paradigm: individual edges are removed to estimate their importance, important components are selected under a budget, and a lightweight model learns to reproduce these selections. Although applying this pipeline to LLM-based multi-agent communication is useful, the paper does not sufficiently establish what is fundamentally new compared with existing graph explanation and topology pruning methods.

  2. The causal interpretation is not entirely convincing. The proposed Granger-style attribution is essentially based on the performance difference before and after removing a single communication edge. Such interventions provide useful importance estimates, but they do not necessarily identify causal communication substructures in the stronger sense suggested by the paper. In particular, the method does not explicitly account for interactions or dependencies among multiple edges. Two individually redundant edges may become essential when removed together, making the resulting edge ranking potentially unreliable for joint subgraph selection.

  3. The experimental analysis does not sufficiently isolate the contribution of the proposed attribution mechanism. The main comparisons focus on original topology generators versus their refined versions. However, a more informative comparison would include simple pruning strategies, such as random pruning, degree-based selection, and other lightweight importance estimators, under identical edge and node budgets. This is particularly important because the appendix shows that edge-only pruning provides relatively small token savings, while most of the efficiency improvement comes from removing agents. It is therefore unclear how much of the reported benefit can be specifically attributed to the proposed causal scoring strategy.

Questions for authors
  1. What is the main methodological distinction between E2-Explainer and existing perturbation-based graph explainers, particularly those that learn an amortized explanation model from offline supervision?

  2. How does the proposed method handle interactions between communication edges? Have the authors investigated cases where multiple edges are individually unimportant but jointly necessary for successful collaboration?

  3. Could the authors provide comparisons against simpler edge/node selection strategies under the same pruning budgets? In particular, how much of the efficiency improvement comes from the proposed causal attribution rather than node pruning itself?

2 comments

We thank the reviewer for the constructive feedback. We have carefully addressed the concerns and made corresponding revisions to the manuscript. All revisions are highlighted in blue in the revised PDF.

W1 & Q1: Methodological novelty beyond perturbation-based graph explanation

We appreciate the reviewer's question regarding methodological novelty. Our central contribution is the insight that substantial communication redundancy can persist even in topologies produced by existing pruning methods. This motivates a distinct problem: rather than developing another topology optimizer, we aim to identify task-preserving communication subgraphs within already optimized topologies through a general-purpose post-hoc refiner.

The key methodological distinction from existing amortized graph explainers lies in the attribution target, supervision construction, and learning objective. Methods such as GEM and PGExplainer primarily seek subgraphs that explain or preserve the predictions of a target GNN. In contrast, E2-Explainer attributes communication edges based on their contributions to the actual execution of an LLM-MAS. Removing an edge changes the messages received by downstream agents and potentially the entire collaborative reasoning process. We therefore construct supervision from changes in both final task performance and result-level semantic uncertainty, rather than relying solely on prediction fidelity.

Furthermore, instead of learning explanatory subgraphs alone, we construct budget-conditioned, executable communication subgraphs that preserve task performance while satisfying the structural constraints of multi-agent workflows. By learning this graph-to-subgraph mapping, E2-Explainer can directly refine topologies produced by different optimization methods without modifying their original optimization procedures.

Together, these distinctions establish E2-Explainer as an intervention-supervised framework for post-hoc communication refinement, rather than simply applying conventional graph explanation to a new domain.

W2 & Q2: Causal attribution and multi-edge interactions

To investigate the reviewer's concern that "two individually redundant edges may become essential when removed together", we conducted a pairwise edge-intervention analysis on 40 MMLU calibration queries, evaluating 360 edge pairs. Among them, 279 pairs consist of edges with low individual task-impact scores. Our results show that interactions among communication edges do exist: jointly removing a pair of these edges leads to incorrect answers in 4 cases (1.4%), while the remaining 275 pairs (98.6%) preserve correct answers. These results indicate that although such interactions exist, their observed impact is limited in the evaluated setting.

Importantly, these interactions do not undermine our central contribution of using Granger-style intervention signals to guide the learning of task-preserving communication subgraphs. The reviewer's observation also highlights that interactions among multiple edges can be explicitly examined by treating two edges or a larger substructure as an intervention unit and evaluating the effect of their joint removal. However, this would substantially increase the number of required LLM-MAS executions, introducing a trade-off between attribution accuracy and computational cost. Given that only 1.4% of the evaluated edge pairs exhibit harmful interactions, our current approach provides a practical balance: it uses individual-edge intervention signals to guide explainer learning while keeping the computational cost manageable.

The statistical analysis is also reported in Section 5.2 (Pairwise edge-intervention analysis) of the revised manuscript, with detailed cases and execution traces provided in Appendix B.3 (Tables 16 and 17).

W3 & Q3: Separating attribution quality from the benefits of node/edge pruning

To further isolate the contribution of our attribution mechanism from the benefits of sparsification itself, we have added matched-budget comparisons against random pruning and a degree-based structural heuristic. Specifically, all methods use the same pruning setting of removing 25% of communication edges and 20% of agents on the same G-Designer candidate graphs. Random pruning is averaged over three seeds, while the degree-based heuristic prioritizes the removal of low-degree agents and low-scoring edges, where edge scores are determined by the sum of their endpoint degrees.

As shown below, both random and degree-based pruning reduce task accuracy on MMLU and HumanEval, illustrating that identifying critical communication substructures is not simply a matter of reducing graph size. In contrast, E2-Explainer improves accuracy while achieving greater online token reductions, providing direct evidence for the effectiveness of the proposed attribution mechanism.

MethodMMLU Acc. (%)MMLU token reductionHumanEval Acc. (%)HumanEval token reduction
Original G-Designer78.65—88.40—
Random pruning (3 seeds)75.7521.14%85.5413.28%
Degree-based heuristic76.4321.87%86.009.36%
E2-Explainer79.7431.50%90.6919.30%

Regarding the contribution of node pruning, we acknowledge that removing agents contributes substantially to reducing token consumption. However, both edge and node pruning in E2-Explainer are guided by our causal attribution mechanism. Specifically, we first evaluate the causal contribution of each communication edge through interventions. We then compute the importance score of each optional agent by summing the attribution scores of its incident communication edges, and prioritize removing agents with lower scores. These attribution-guided selections are used to construct task-preserving subgraphs, which provide joint supervision for learning edge and node retention. Therefore, the benefits of both edge and node pruning are driven by our proposed causal attribution strategy, rather than an independent node-pruning mechanism.

Furthermore, our original component ablation provides complementary evidence that the proposed attribution mechanism is essential:

E2-Explainer variantMMLU Acc. (%)HumanEval Acc. (%)
Without causal/task-intervention signal75.8286.88
Without semantic-entropy signal77.7887.50
Full E2-Explainer79.7490.69

Removing the task-intervention signal leads to substantial accuracy degradation, while the full model consistently achieves the best performance. Together, the matched-budget comparisons and component ablation demonstrate that the effectiveness of E2-Explainer cannot be attributed to node/edge removal alone, but also depends on the intervention-derived supervision that guides structural selection.

The additional comparisons and corresponding analysis are provided in Table 5 and Section 5.2 of the revised manuscript, with the component ablation reported in Table 3.

Reviewer_a79yOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies communication topology in LLM-based multi-agent systems. The main idea is to identify which communication edges are actually useful and remove redundant ones. The proposed E2-Explainer first removes individual edges from a generated communication graph and measures how the task result and the semantic stability of the final answer change. These measurements are used to rank the importance of communication edges and construct smaller task-preserving subgraphs. Since running these edge-removal tests at inference time would be expensive, the authors further train an explainer to predict the compact subgraph directly for a new communication graph. The method is evaluated on six reasoning and coding benchmarks and several existing topology generators. The experiments show that the proposed method can reduce online token usage while generally maintaining, and sometimes improving, task accuracy. The paper also evaluates transfer across topology generators and from 5-agent to 6-agent settings.

Critical strengths
  1. The most useful aspect of E2-Explainer, to me, is that its explanations can actually be executed. The authors do not stop at assigning scores to communication links; they use those scores to construct a smaller graph and then test whether the multi-agent system still solves the task. That gives the explanation a reasonably concrete meaning. Distilling the expensive edge-removal procedure into a predictor also makes sense, since running those interventions for every test query would largely defeat the efficiency goal.
  2. I also liked the transfer experiments. The explainer is calibrated mainly on G-Designer graphs, but the paper applies it to ARG-Designer, OFA-MAS, and AgentPrune, and includes a 5-agent to 6-agent test. These are more informative than another in-domain pruning result: at least the method is not simply recovering the adjacency patterns it saw during calibration.
Critical weaknesses
  1. I have reservations about calling the resulting scores 'causal' without further qualification. Removing an edge tells us what happens when that edge is absent from a particular graph, with all the other channels still available. It does not tell us whether that edge would matter in a different subgraph. Two paths could substitute for each other, for example, so both receive low individual scores even though removing both would be harmful. The authors acknowledge that they do not enumerate multi-edge coalitions. I think the intervention-based score is useful, but the causal-inference framing in the title is stronger than what is established here.
  2. The evidence that the learned ranking is important is less direct than the evidence that the resulting graphs are smaller. Figure 1 and the additional OFA-MAS random-masking analysis in Appendix B.4 show that indiscriminate deletion often damages accuracy. That is helpful motivation, but it is not a matched-budget pruning baseline. I would have liked to see E2-Explainer next to at least random selection and a simple structural heuristic at the same retained-edge count or online token budget. Otherwise it is difficult to separate the quality of the attribution from the benefit of pruning in general.
Questions for authors
  1. Have you checked a small number of pairs of edges that are individually scored as unimportant? I am particularly interested in cases where the two deletions interact. Even a small diagnostic would help establish how the scores should be read.
  2. What happens in a same-budget comparison with random pruning or a degree-based heuristic? The random-mask plots are informative, but they vary the masking ratio and do not directly answer this question.
  3. Can you give a rough accounting of the offline work (edge interventions, semantic-entropy sampling, and explainer training)? At approximately what deployment workload would that cost be recovered by the reported online savings?
2 comments

We thank the reviewer for the constructive feedback. We have carefully addressed the concerns and made corresponding revisions to the manuscript. All revisions are highlighted in blue in the revised PDF.

W1 & Q1: Causal interpretation and multi-edge interactions

We thank the reviewer for pointing out that "the causal-inference framing in the title is stronger than what is established here." As stated in our Abstract and Introduction, "Our formulation is inspired by the notion of Granger causality." Our method estimates the effects of interventions on individual communication channels within a given graph, rather than identifying a complete causal structure. To better reflect this methodological scope, we have replaced "Causal Inference" with "Granger Causality" in the title of the revised version.

To investigate the reviewer's concern that "Two paths could substitute for each other", we conducted a pairwise edge-intervention analysis on 40 MMLU calibration queries, evaluating 360 edge pairs. Among them, 279 pairs consist of edges with low individual scores. Our results show that interactions among communication edges do exist: jointly removing a pair of these edges leads to incorrect answers in 4 cases (1.4%), while the remaining 275 pairs (98.6%) preserve correct answers. These results indicate that although such interactions exist, their observed impact is limited in the evaluated setting.

Importantly, these interactions do not undermine our central contribution of using Granger-style intervention signals to guide the learning of task-preserving communication subgraphs. The reviewer's observation also highlights that interactions among multiple edges can be explicitly examined by treating two edges or a larger substructure as an intervention unit and evaluating the effect of their joint removal. However, this would substantially increase the number of required LLM-MAS executions, introducing a trade-off between attribution accuracy and computational cost. Given that only 1.4% of the evaluated edge pairs exhibit harmful interactions, our current approach provides a practical balance: it uses individual-edge intervention signals to guide explainer learning while keeping the computational cost manageable.

The statistical analysis is reported in Section 5.2 (Pairwise edge-intervention analysis) of the revised manuscript, with concrete cases and execution traces provided in Appendix B.3 (Tables 16 and 17).

W2 & Q2: Attribution quality under matched pruning budgets

To further evaluate the effectiveness of our Granger-style attribution, we have added matched-budget pruning comparisons against random pruning and a degree-based structural heuristic. Specifically, all methods use the same pruning setting of removing 25% of communication edges and 20% of agents. Random pruning is averaged over three seeds, while the degree-based heuristic prioritizes the removal of low-degree agents and low-scoring edges, where edge scores are determined by the sum of their endpoint degrees.

As shown below, both random and degree-based pruning reduce task accuracy on MMLU and HumanEval, illustrating that identifying critical communication substructures is not simply a matter of reducing graph size. In contrast, E2-Explainer improves accuracy while achieving greater online token reductions, providing direct evidence for the effectiveness of our intervention-guided subgraph selection.

MethodMMLU Acc. (%)MMLU token reductionHumanEval Acc. (%)HumanEval token reduction
Original G-Designer78.65—88.40—
Random pruning (3 seeds)75.7521.14%85.5413.28%
Degree-based heuristic76.4321.87%86.009.36%
E2-Explainer79.7431.50%90.6919.30%

These additional results and the implementation details of the pruning baselines are provided in Table 5 and Section 5.2 of the revised manuscript.

Q3: Offline computational cost and amortization

Regarding the offline computational cost, our supervision construction involves single-edge interventions and semantic-entropy estimation, followed by supervised explainer training. For each calibration graph, we evaluate individual edge removals and use five stochastic executions to estimate the semantic uncertainty of the original and intervened graphs. These evaluations account for the offline LLM-token consumption. The resulting graph-to-subgraph supervision is then used to train the explainer through edge- and node-retention prediction losses, without additional LLM queries during training.

The following table reports the total offline LLM-token consumption for each benchmark, aggregating intervention evaluations and semantic-entropy sampling.

BenchmarkOffline tokens
MMLU4.46M
GSM8K6.74M
MultiArith5.88M
SVAMP5.84M
AQuA4.95M
HumanEval51.22M
Total79.08M

Regarding cost amortization, E2-Explainer exhibits strong scalability and reusability across topology designers. Once trained on the offline dataset constructed for a given benchmark, the resulting explainer can be directly applied to different topology designers without retraining or additional edge-level interventions. As demonstrated in Table 1, an explainer trained solely on G-Designer-generated graphs can effectively refine topologies produced by ARG-Designer, OFA-MAS, AgentPrune, and AgentDropout.

For a rough token-based estimate, applying E2-Explainer to G-Designer reduces online token consumption from 19.05M to 14.27M across the six evaluation sets, saving approximately 4.78M tokens per evaluation-suite equivalent. Relative to the total offline cost of 79.08M tokens, this corresponds to approximately 17 evaluation-suite equivalents with a comparable task distribution. This estimate considers LLM-token consumption rather than total GPU compute time. Importantly, the ability to reuse the trained explainer across different topology designers allows the upfront cost to be amortized over a broader range of deployments.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their careful assessment, constructive feedback, and time devoted to the discussion. For clarity, we refer to Reviewers a79y, 4qai, and Zs4d as R1, R2, and R3, respectively.

We have provided detailed responses to all reviewers and substantially revised our manuscript through additional experiments, methodological clarifications, and code releases. All changes in the revised manuscript are highlighted in blue. Below, we summarize the core contributions of our work and the principal updates made during the rebuttal.

Core Contributions

We propose E2-Explainer, a generator-agnostic post-hoc framework for identifying compact, task-preserving communication subgraphs in LLM-based multi-agent systems. Specifically, we use Granger-style interventions to estimate edge contributions based on task performance and semantic uncertainty, and construct executable subgraphs under budget and structural constraints. We then distill the resulting graph-to-subgraph mapping into an amortized explainer, enabling direct subgraph prediction without repeated causal interventions at test time. Extensive experiments demonstrate its effectiveness and transferability across different topology optimizers and agent scales.

Key Updates and Clarifications During Rebuttal

  • Methodological novelty and motivation (R2, R3). We clarified the distinctions between E2-Explainer, conventional graph explainers, and existing MAS pruning methods. We emphasized that our contribution lies in identifying task-preserving communication subgraphs through intervention-based attribution, rather than simply introducing another topology optimizer.
  • Causal interpretation and multi-edge interactions — Section 5.2 and Appendix B.3 (R1, R2). We refined our causal terminology, updating the title from "Causal Inference" to "Granger Causality." We also evaluated 360 edge pairs, finding harmful joint-removal interactions in only 4 of 279 low-importance pairs (1.4%), and provided concrete cases and execution traces.
  • Matched-budget pruning comparisons — Table 5 and Section 5.2 (R1, R2). We added random pruning and degree-based heuristic baselines under identical pruning budgets. E2-Explainer achieves higher accuracy and greater token reductions. Together with the component ablations in Table 3, these results demonstrate the importance of intervention-guided attribution beyond sparsification alone.
  • Additional topology baseline and redundancy analysis — Table 1 and Appendix Figure 5 (R3). We incorporated AgentDropout into our main comparisons and extended the random edge-masking analysis to AgentPrune and AgentDropout. E2-Explainer further improves their average accuracy by 0.58 and 1.27 percentage points, respectively, while reducing online token consumption by 24.9% and 21.9%, demonstrating its effectiveness on already optimized topologies.
  • Additional LLM backbones — Table 6 (R3). We extended our evaluation to Llama-3.1-8B and GPT-4o-mini, demonstrating consistent accuracy improvements and token reductions across different model architectures.
  • Offline computational cost and amortization (R1). We reported a total offline cost of 79.08M LLM tokens and estimated that it can be recovered after approximately 17 evaluation-suite equivalents under the G-Designer setting. We also clarified that the trained explainer can be reused across different topology optimizers.
  • Code availability and reproducibility (R3). We released our source code, evaluation scripts, concrete baseline integration adapters, and setup instructions. We further clarified the consistency between our methodology and implementation, including the roles of causal discovery, distillation, and baseline execution.
  • Experimental reporting and presentation (R3). We reorganized the experimental tables, corrected formatting issues, and provided detailed token consumption results. We also clarified that our conclusions are based on comparisons of each baseline with and without E2-Explainer under the same experimental conditions.

We sincerely appreciate the reviewers' valuable suggestions, which have helped us substantially strengthen the experimental evidence, clarify our methodological contributions, and improve the reproducibility and presentation of our work.

We hope our detailed responses and revised manuscript provide a clearer understanding of the novelty, effectiveness, and general applicability of E2-Explainer. We would greatly appreciate any further feedback from the reviewers, particularly on whether our additional experiments and clarifications have addressed their original concerns. Thank you again for your time, thoughtful feedback, and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.