acceptodds
Under review as a conference paper at ICLR 2027

Semantic Cooperative Games for Contribution Attribution in LLM-Based Multi-Agent Systems

Abstract

LLM-based multi-agent systems produce outputs through ordered interactions, yet contribution attribution lacks a unified semantic account of how agents generate, preserve, and transform task-relevant information. Existing methods typically rely on counterfactual valuation by removing agents or comparing system scores across agent subsets. These approaches require costly workflow reruns that may alter downstream context. They also leave much of the intermediate semantic information unused, limiting their ability to explain specific contributions within a realized trajectory. We propose Semantic Cooperative Games (SCG), a framework that represents a realized language interaction trajectory as a semantic generation hypergraph and derives an agent-level value function from its semantic support relations. We define the Semantic Shapley Value (SSV) and introduce SLIC, a single-trajectory algorithm that constructs the hypergraph, recovers minimal semantic supports, and applies Boolean absorption to compute SSV without rerunning agent subsets. Our theory provides an axiomatic foundation for SSV and guarantees that it reduces to the classical Shapley value whenever the latter is well-defined within our framework. In a three-agent medical question-answering workflow where the classical Shapley value is well-defined, SLIC reduces API calls by 85.7% relative to an exhaustive Shapley baseline, achieving a Kendall rank correlation of \(\tau_b = 0.814\) with the baseline attribution. In more general workflows involving multiple roles, SSV correlates strongly with score drops under moderate to strong perturbations, while revealing cases in which semantic contribution and failure impact diverge. SCG provides an interpretable paradigm for contribution attribution and lays the groundwork for systematic annotation of semantic actions and their contributions in Group-of-Thought workflows.

Primary area:
interpretability and explainable AI
Keywords:
Large language models, multi-agent systems, contribution attribution, cooperative game theory, Shapley value, semantic hypergraphs

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rd8tOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an new approach to attributing individual agent contributions in LLM-based multi-agent systems. The proposed method formulates attribution as a Semantic Cooperative Game, which does not rely on counterfactual executions. Instead, it constructs a semantic hypergraph from an observed execution trajectory. This hypergraph captures how different agents generate, preserve, and transform task-relevant information throughout the interaction. To quantify individual contributions, the method introduces the Semantic Shapley Value (SSV) and an algorithm that identifies minimal semantic support sets, applies Boolean absorption, and computes agent contributions using multilinear coefficients. The paper also establishes the equivalence between SSV and the classical Shapley value under specified conditions. The proposed method is evaluated through two main experiments. The first compares SSV against baseline Shapley attribution, leave-one-out attribution, and holistic LLM as a judge evaluation on HealthBench cases. The second investigates the alignment between SSV-based attributions and perturbation-induced performance degradation across four multi-agent task settings.

Critical strengths
  1. The authors provide a well-motivated discussion of the limitations of counterfactual-based contribution attribution methods in LLM-based multi-agent systems, highlighting how counterfactual interventions can alter downstream interactions, introduce instability, and incur substantial computational costs.

  2. The proposed approach estimates agent contributions by leveraging the semantic structure of an existing execution trajectory. This enables contribution attribution without requiring counterfactual reruns, offering an efficient alternative that relies on the interactions between agents.

Critical weaknesses
  1. The proposed SSV appears to distribute the Möbius coefficients of the induced semantic game using the established Shapley dividend formula in [1]. This may suggest that the main novelty is in constructing the semantic value function, not in the allocation rule itself.

  2. In addition, the theoretical justification for the claimed extension remains unclear. Specifically, while Eq. (6) follows from the standard Shapley allocation identity once Eq. (5) holds, the equivalence between the induced semantic value function and the classical coalition value function in Eq. (5) does not appear to be independently established.

  3. While the paper provides several experiments evaluating reduction consistency, structural diagnosis, and analyzer sensitivity, I think the scalability of the proposed method is still not sufficiently evaluated. In particular, the experiments are limited to workflows with at most five agents. The authors should evaluate how SLIC scales to larger and more complex MAS. It also remains unclear how the computational cost of constructing the semantic graph and computing Boolean supports grows with the number of agents and the complexity of their interactions.

  4. The conditions of "set-based value reachability," "full observability," and "no masking or order dependence" are not formally defined, making it difficult to assess when the claimed equivalence holds.

[1] Harsanyi, J. C. (1963). A simplified bargaining model for the n-person cooperative game. International economic review (Philadelphia), 4(2), 194–220.

Questions for authors
  1. Could the authors clarify the main theoretical novelty of the proposed SSV relative to the existing Harsanyi-Shapley framework [1]?

  2. Could the authors clarify how the conditions of “set-based value reachability”, “full observability”, and “no masking or order dependence” ensure that the semantic value function captures the values of alternative coalitions whose behavior is not observed, given that it constructed from a single realized trajectory?

3.Could the authors provide an analysis of how the computational cost of SLIC grows with the number of agents and the complexity of their interactions?

  1. Could the authors provide an ablation study to evaluate the contribution of Boolean support computation? For example, how does SLIC compare with a simpler approach that directly assigns contributions based on the reconstructed semantic graph, without the Boolean support logic?
Reviewer_MznnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Semantic Cooperative Games (SCG), a framework that (1) converts a multi-agent trajectory into a graph using an LLM, (2) assigns importance weights to output nodes based on a given rubric, (3) aggregates the Shapley attribution at the agent-level using boolean function analysis (i.e. Mobius coefficients). The main technical contribution is seeing a trajectory as a graph and then applying known boolean function analysis to derive closed-form Shapley values for agents. The faithfulness of the SCG-based Shapley values is evaluated with respect to rerun-based Shapley values in a 3- and 4-agent system with independent agents. The similarity between SCG-based Shapley is also compared to rerun-based leave-one-out methods.

Critical strengths
  • The approach is theoretically well justified. If the goal is to efficiently compute Shapley values, it makes theoretical sense that one would rely on boolean function analysis, which can motivate the construction of a dependency graph. I think the authors can highlight this motivation more in the paper.
  • The approach is computationally efficient compare to removal-based approaches that require rerunning.
Critical weaknesses
  • The practical utility of the SCG is not evaluated. Insights from agent attribution are typically used to inform use cases such as (1) removing unnecessary agents to improve performance and/or reduce computational cost; and (2) identifying agent failures to suggest fixes through prompt or tool optimization. Currently, this paper only shows that SCG can approximate removal-based attributions, but it's not demonstrated whether the approximation error is small enough for practical utility.
  • Experiment 2 evaluates how well SCG-based Shapley values approximate leave-one-out (LOO) attributions. This setup makes it unclear whether the approximation performance comes from (1) the SCG framework faithfully approximating actual removal/hallucination injection, (2) the Shapley values being similar to LOO in the experimental setting, or (3) both. I think the paper would be strengthened if Experiment 2 focuses on how well SCG-based LOO approximates removal-based LOO.
Questions for authors
  • Does the SCG framework provide insights useful for optimizing the performance of multi-agent systems (e.g. agent pruning)?
  • Is the SCG framework specific to the Shapley value? If not, how well does SCG-based LOO approximate removal-based LOO?
  • Can you formalize the assumptions in Theorem 1 and comment on when these assumptions are broken in real-world multi-agent systems?
  • How does Algorithm 1 work without a provided rubric? Does your method depend on a provided rubric?
Reviewer_raFpOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper propose an interesting proxy of Shapley value based on constructed graph for computing the contributions of individual agents in a multi-agent working pipeline. The authors offer both theoretical results and empirical study to validate this new proxy and also show it can be computed with low cost than baselines.

Critical strengths
  • The idea is interesting and the setup of the paper is promising for better quantifying the single agents' contribution in multi-agent pipeline.
Critical weaknesses

I have two big concerns that lead me to this reject decision, but I'm happy to discuss with the authors about better framing of the question and contribution.

  • First I think that saying SSV coincides with classic Shapley value is overclaimed. The calculation of SSV depends on the construction of the hypergraph, and the latter totally relies on an LLM to conjecture. This says the assumptions in Theorem 1 are likely to be violated. Also I'm having troubles following the section 3.1 (which I think is overcomplicated) and the assumptions in section 3.2 lack formal math definitions, so I'm not able to evaluate this part. I think this paper should be written more carefully. A lot of notations don't have a definition before the first usage (such as y_u and x_u in assumption 1 and \tilde s on line 166).
  • The empirical experiment lacks an important baseline method. In Table 1, the authors only use weak baselines such as LOO and textual LLM judge, without other methods for estimating Shapley value (like simple sample-based method).

Overall, I'm not convinced on the main contribution of this paper that SSV is a good proxy of Shapley value in contribution attribution problem.

Questions for authors
  • Could the authors give some intuitive interpretations on the assumptions of Theorem 1?
  • Could the authors provide any evidence that the hypergraph constructed by LLM can fulfill the assumptions of Theorem 1? Or under what circumstances?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.