acceptodds
Withdrawn from ICLR 2027

ReLoC: Recursive Local Causal Modeling for Explaining and Predicting LLM-Based Multi-Agent Behavior

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language model (LLM)-based multi-agent systems (MAS) are increasingly deployed to solve complex tasks; thus learning how a local intervention on one agent causally affects the overall outcome is necessary. As the MAS scales, however, the space of joint interventions grows combinatorially, making a global causal model prohibitively expensive and preventing measured effects from extending to unseen intervention queries. We introduce Recursive Local Causal Modeling (ReLoC), which progressively builds an approximate global causal model by learning and composing local causal models of individual agents, each built from a few interventions. Specifically, ReLoC learns how each agent responds to interventions on task-defined concepts. Starting from the outcome agent, ReLoC recursively learns how upstream agents produce these concepts, following the communication graph back to the task inputs. ReLoC eventually composes these local models into the global model. Across three benchmarks, three topologies, and four LLMs, ReLoC accurately predicts how each agent responds to concept interventions, how these changes propagate through the system, and how the final answer changes under unseen interventions, consistently outperforming an LLM that directly reasons over the full MAS under the same intervention budget.

Primary area:
causal reasoning
Keywords:
Large Language Models, Multi-Agent Systems, Causal Reasoning, Causal Modeling, Interventional Analysis

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_kdxYOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes ReLoC, which learns agent-local response models through interventions and composes them along a known communication DAG to predict intermediate states and final answers under unseen interventions. Each message consists of a task label and three Boolean concepts. Observed inputs use empirical response frequencies, while unobserved inputs use a Laplace-smoothed factorized count estimator, augmented with joint features when matched controls reveal interactions. Acquisition proceeds recursively upstream from the outcome agent and allocates probes using coverage gaps and prediction errors. Experiments cover three tasks, three topologies, and four LLMs, evaluating local response, propagation, trajectory, and final-answer fidelity under a budget of thirty-two agent calls per case.

Critical strengths
  1. The objective is meaningful. Learning reusable local response models could reduce the execution cost of answering multiple joint-intervention queries, beyond measuring isolated intervention effects. The known communication graph and direct local interventions provide a clear modeling interface.

  2. The evaluation distinguishes outcome prediction from process reproduction. Reporting local response, propagation, trajectory, and final-answer fidelity, together with changed-versus-unchanged outcomes, exposes mechanism errors that aggregate answer fidelity would otherwise conceal. This is a valuable aspect of the experimental design.

Critical weaknesses
  1. High aggregate answer fidelity does not establish reliable causal-mechanism explanations, and the incremental contribution of recursive acquisition is limited against the most relevant baseline. On Qwen3-4B, Invariant already achieves 93.3% versus ReLoC's 93.7%; on Llama, the corresponding figures are 84.4% and 85.6%. Fidelity on genuinely changed outcomes is only 44.8% pooled and 31.5% on Qwen. Much of the large advantage over Global LLM therefore reflects avoiding false changes rather than accurately capturing intervention propagation. The explanatory evidence is weaker still: trajectory exact match is only 19.6--50.8%, and 41.1--77.5% of correctly predicted final answers contain incorrect intermediate trajectories. These results do not establish reliable mechanism explanations. Full-local, which shares the estimator and composition rule, is the more direct test of the acquisition contribution. ReLoC is slightly worse on Hy3 and 1.2 points worse on Llama, leaving a consistent advantage of recursive error-guided acquisition unestablished.

  2. Compositional generalization relies on strong restrictions and insufficiently validated extrapolation, while the proposed future directions correspond to central current gaps. Communication is restricted to a label and three Boolean fields, with a fixed known DAG, predefined intervention catalogs, and case-specific fitting. For unseen inputs, the factorized count estimator uses a conditional-independence approximation; interactions are added only after relevant errors and matched controls are observed. No guarantee establishes that the finite acquisition budget discovers important interactions or controls composition error. Passing the local mode at each step also differs from marginalizing stochastic mechanisms. Empirically, 73.4% of queries require statistical extrapolation, whose fidelity on CEBaB is only 48.2%. Targeting changed intermediate states, free-form messages, and dynamic communication are deferred to future work. However, coverage of changed states already drives current missed effects and directly limits the existing generalization claims. The evidence currently supports an empirical surrogate on a restricted discrete interface, rather than a broadly reliable global causal explanation.

Questions for authors
  1. Which improvements over Full-local have statistical support, and which do not? Please report paired differences separately for changed and unchanged outcomes to establish the contribution of recursive acquisition independently of the large gap to Global LLM.

  2. When the final answer is correct but the intermediate trajectory is wrong, does the model still identify the agents or concepts responsible for the outcome? What independent evidence supports mechanism explanation beyond endpoint prediction?

  3. Under what structural conditions is extrapolation to unseen input combinations valid? How are unreliable predictions identified when matched controls are unavailable or higher-order interactions remain unobserved? For stochastic responses, how much error does local-mode composition introduce relative to probabilistic marginalization?

  4. Acquisition targeting changed intermediate states is proposed as future work, but this corresponds directly to the current dominant failure mode. Under the same budget, how does ReLoC compare with simple minimum-coverage or changed-state-oriented acquisition policies? If free-form messages and dynamic communication remain unsupported, how should the current conclusions be explicitly restricted to four-field messages and fixed DAGs?

Reviewer_kXEWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to understand and predict the effects of interventions in LLM-based multi-agent systems, the authors proposed ReLoC that learns local causal models for individual agents and combines them along the communication graph to predict how changes affect the final outcome. Experiments show that ReLoC can predict unseen intervention outcomes more accurately than several baselines.

Critical strengths

This paper considers an interesting problem, which is about to understand how one agent's behavior can affect other agents and the final outcome is important for explaining and debugging LLM-based multi-agent systems.

Critical weaknesses
  • The most important problem is that this paper claims to explain and predict multi-agent behavior, but the experiments more is to support predicting final outcomes than explaining accurately about how changes propagate through the agents.

  • And I think the findings are too limited to the predefined concepts and agent structures. In the method, the authors assumes that agent messages can be represented using a small set of predefined concepts and that the communication graph is known. It is unclear how well this would work for more realistic multi-LLM-agent systems with open-ended messages and changing interactions (which should be the reason why we want to use LLM-enabled agents to communicate rather than fixed, rule-based communication protocols or just MARL-based multi-agent systems).

  • The intermediate behavior part is not predicted very precisely, I can see in the results, even though the final-answer accuracy is high, the full agent trajectories are correctly predicted in only 19.6–50.8% of cases. This raises my concerns about how well the method could actually explain the interactions between agents like they stated in the introduction, it seem like the method mainly about predicting the final answer.

Questions for authors
  1. How the method work with more realistic multi-agent systems where agents communicate freely and when the communication structure changes?

  2. The final-answer prediction is okay, but the intermediate trajectory accuracy is much lower. How reliable are the learned causal models for explaining agent behavior when the intermediate predictions are incorrect?

  3. How sensitive is the method to the choice of predefined concepts (how strong this assumption is)? Would the results change significantly if different concepts were used?

Reviewer_MpKLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper focuses on learning how a local intervention on one agent causally affects the overall output of an LLM-based multi-agent system.

It proposes ReLoC, which builds an approximate global causal model and learns how each agent responds to interventions on task-defined concepts.

In the experiments, ReLoC is evaluated across three benchmarks, three topologies, and four LLMs, comparing predicted agent responses, concept propagation, and final answers with actual system executions.

Critical strengths
  1. This paper studies an interesting research problem in LLM-based multi-agent systems.

  2. The paper provides extensive experiments across three benchmarks, three communication topologies, and four LLMs.

Critical weaknesses
  1. The performance is not strong.

  2. I would like the authors to explain the importance and practical significance of the research problem and whether the proposed method can be used to improve MAS performance.

  3. The writing need to be improved, and the methodology is not easy to understand.

Questions for authors

Motivation

  1. As mentioned in line 105, intervention design depends on what the learned model must answer. Since LLMs can be very robust, how can you ensure that your interventions influence the LLM's responses as expected?

  2. From my understanding, this paper learns how each agent responds locally and then combines these local models to predict global behavior. Could you further explain the importance of this research question? What are the practical applications of the proposed method in real-world multi-agent systems?

Method

  1. In the preliminaries, LLM responses are represented by a set of task-defined concepts. I doubt whether a small number of concepts can fully capture the information contained in LLM responses.

  2. Could you provide an example of an "intervention query"? How can you ensure that the LLM agent produces the expected output under an intervention?

  3. For the local causal models, how can you ensure that (\hat{p}_a(y\mid x)) is estimated accurately?

  4. For recursive acquisition, why do you start from the outcome agent?

  5. Another question concerns composition. For example, if agent (v) sends a message to agent (a), what happens if (v)'s predicted message contains errors? How reliable would the output of (a) be?

Experiments

  1. The paper mentions that "Each method receives 32 agent calls per case." I wonder whether ReLoC is more computationally efficient than directly executing the MAS.

  2. In Table 1, we can see that although final-answer fidelity is high, trajectory exact match is low, and the proportion of correct final answers with incorrect trajectories is high. How reliable is ReLoC as a model for explaining causal propagation through the MAS?

  3. In Table 2, I find that ReLoC performs very similarly to Full-local in the "All" column. Among the four LLMs, ReLoC outperforms Full-local on only two, with improvements of less than one percentage point. I have concerns about the performance improvements achieved by ReLoC.

  4. In Table 3, when the final answer remains unchanged, ReLoC does not outperform Full-local. When the final answer changes, ReLoC achieves only 44.8% fidelity, although it is the best-performing method. I do not think these results demonstrate sufficiently strong performance for ReLoC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.