acceptodds
Under review as a conference paper at ICLR 2027

RICK: Certifying Transmission in Multi-Agent Credit Assignment via Worst-Regime Selection

Abstract

A cooperative team trained from one shared reward learns exactly what its credit assignment tells it, so the kernel that decides which agent earned a delayed outcome is itself the training signal. However, that kernel is fitted to behaviour logs whose releases and arrivals share one exogenous driver, and every quantity that certifies it, the fit residual, the perturbation schedule and the conservation percentage, is read off those same logs, so the evidence certifies itself: a kernel can leave only of the recorded variance unexplained while its coefficients sit from the truth, and no reading inside the log separates the two. We propose the Regime-Invariant Credit Kernel (), which selects the kernel on intervention regimes the team manufactures and pays for. Regime-invariant kernel selection pins the kernel by its worst per-regime discrepancy instead of its aggregate fit. A regime scheduler commits the perturbation family under a bound on the return it costs, and a stop-gradient keeps team return out of the selection. Delayed backtracking and share accounting states the arrived mass a bounded window cannot reach as an unassigned share. Together, the three modules replace each self-certifying readout with one that can fail: a kernel propped up by the shared driver leaves a manufactured regime poorly explained, and the mass a window misses stays on the books. Scored against exact ground truth over ten draws per cell on three delayed environments, recovers the generating coefficients on the fan-in environments to a mean of , against for the pooled aggregate fit at the same recording budget. On a structurally distinct held-out family it keeps that ordering, at against for the pooled aggregate fit.

Primary area:
reinforcement learning
Keywords:
multi-agent credit assignment, delayed credit assignment, worst-regime selection, interventional identification, exogenous confounding, cooperative reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_2LU8OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies delayed credit assignment in cooperative multi-agent systems. A "credit kernel" (per-edge transmission coefficients plus Gamma-shaped lag profiles over a directed agent graph) is fitted to behaviour logs. The paper argues that fit residuals, perturbation schedules, and conservation percentages computed from the same logs can't verify themselves: a shared external driver makes parent agents' outputs collinear, so a good fit can coexist with wrong coefficients. The method, RICK, has three parts:

  • it selects the kernel by its worst per-regime discrepancy over a family of self-applied perturbation regimes (Eq. 3);
  • it chooses that family under a bound on return cost, maximising the smallest eigenvalue of the pooled parent design (Eqs. 7–9);
  • it reports the mass that a bounded backtracking window cannot reach as an explicit "unassigned share" (Eq. 14).

A population-limit bound for the linear one-step case is given in App. A.3. The experiments use three synthetic delayed-graph environments and one held-out synthetic family, with exact counterfactual-replay ground truth and 10 draws per main cell. On the two higher fan-in environments, RICK's mean coefficient error is 0.0935 versus 0.2186 for an aggregate (pooled) fit. On the held-out family, it is 0.0539 versus 0.1151.

Critical strengths
  1. Evaluation against exact ground truth, with negative results reported: The known generating coefficients and exact counterfactual-replay credit let coefficient recovery be measured directly. The appendix openly reports results against the method: the objective-selected schedule buys less identification than round-robin (Table 13), random masks beat the scheduler at 3 of 4 strengths (Table 15), and a time-decayed heuristic beats RICK at short windows (App. A.19).
  2. The unassigned-share convention is a sensible reporting fix: Using the full expansion as the denominator (Eq. 14) exposes truncation that a reached-set percentage hides. Tables 7 and 9 show a reached-set ratio of 1.0000 while 0.82–0.85 of the mass is unreached at window B=1.
Critical weaknesses
  1. The "certifying" and training claims are not tested, and the scheduler is contradicted by the paper's own tables.
  • No policy is trained: The abstract and Algorithm 1 frame RICK as a training signal for a cooperative team. Yet App. A.24 states "the bilevel problem is evaluated at its inner level", and that the return leak is modelled as a "rescaling of the selected coefficients rather than training without the stop-gradient". No team return is reported, and the stop-gradient result (0.9043 to 0.3486, Table 3) holds by construction.
  • The scheduler is not used, and loses when it is: The main tables use index round-robin masks instead of the proposed scheduler (Eq. 9). The scheduler yields lower identification (λ_min) than round-robin at every strength (Table 13), leaves an emitter unexcited (Table 14), and loses to cost-matched random masks at 3 of 4 strengths (Table 15).
  • The certificate is never measured: The property that would make selection a certificate is never reported (App. A.0): per-regime discrepancies should coincide at the true kernel and separate a kernel propped up by the shared driver.
  1. The coefficient-recovery gain is confounded, and the theory does not explain it.
  • The baseline downweights the interventions: The aggregate-fit baseline puts weight 0.90 on the unperturbed regime and 0.10 on the perturbed ones (App. A.9, A.11), even though regimes are sampled equally. This deliberately ignores most of the interventional data that breaks the collinearity.
  • The theory doesn't favour RICK: The App. A.3 lower bound holds equally for the weighted pooled objective, so it gives no advantage to worst-regime selection. Its assumption that per-regime residuals are orthogonal to the parents cannot hold in the confounded observational regime that motivates the paper.
  • The test is weak: Every true coefficient equals 1 (App. A.5), in a correctly specified linear class.
  • The fidelity margins are small: Credit-fidelity margins are mostly within one across-draw SD (Table 8). Most ablations are single-draw.
  • Obvious alternatives are missing: There is no equal-weight pooled fit, no perturbed-only fit, and no comparison or citation to invariance or robust-optimization methods. The worst-regime criterion is essentially group DRO or V-REx over interventional environments, and the scheduler is an E-optimal design.
Questions for authors
  1. What are the Table 1 coefficient errors for a pooled fit with equal regime weights, and for a perturbed-only fit, at the same 16 episodes? A clear margin for RICK over both, with paired standard errors, would substantially change my view of Weakness 2.
  2. Does any experiment train a policy using RICK credit and report team return? If not, would you rescope the title, abstract, and Algorithm 1 to kernel identification?
  3. Does the App. A.3 orthogonality assumption hold in the confounded regime of Challenge #1? If yes, why is the pooled fit inconsistent there? If not, what guarantee covers the setting actually evaluated? A4) re the kernel's functional forms learned or fixed to the generator's? How do results change with heterogeneous true coefficients, including zero-weight edges?
  4. Can you report the per-regime discrepancies for the true kernel versus the driver-propped kernel, showing a case where RICK flags a kernel that the pooled fit accepts?
Reviewer_TR2jOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Summary: This paper proposes a method for creating a credit attribution kernal for multi-agent systems operating on a graph in which they can pass signals to each other. They aim to address the flaw in prior approaches which use the same dataset to train both the credit attribution mechanisms and evaluate them.

Critical strengths

As multi-agent systems are increasingly being used both in automated scientific discovery and broadly interacting AI systems throughout society, understanding how to accurately and reliably train these systems is increasingly important. Moreover, the flaw of training and validating on the same data is serious and we would not want the validity of these systems to be vulnerable to such a fundamental flaw.

Critical weaknesses

However, I worry that this work lacks the clarity that would be necessary to effectively address these issues. I believe the writing assumes a large amount of existing understanding in the reader of the specific niche, and of the approach in the paper, without which it is quite difficult to understand the work well enough to evaluate or reimplement its technical content.

Just for a concrete instance of this, in the abstract I do not know what the behavior logs are in this context, or the releases and arrivals. I don't know what it means for a quantity to "certify the fit". I don't know what the perturbation schedule or the conservation percentage is. The sentence starting with "Delayed" was very difficult to parse because of the use of passive voice. The structure of the sentences in the abstract was not possible to understand and seemed disconnected until I got to the sentence starting in "Together, the three modules....", which only then it was possible to tell that the three things mentioned before where supposed to be component modules of some system that was being introduced. I only point on these in the abstract because it is at the start, where it is clear that the reader does not have additional context. However, these concerns are most serious when it comes to the technical definitions and experiments, where i was unable to determine what exactly was being claimed or run.

As far as I can tell, there is a setup which is assumed that is not entirely explained. My best assessment filling in the gaps is that it seems like a centralised training decentralised execution style credit assignment for agents on a graph-based network, which is being trained with some sort of offline dataset of "behavior logs" which is being used both for credit assignment and evaluation. It also seems that there is some pre-existing notion of "regime" and "perturbation" both of which are assumed to be pre-defined and are being used to make the credit assignment kernel less likely to overfit to the behavior logs. Please let me know if any of these details are incorrect, and where the correct details are in the paper. As far as I can tell the graph or it's interpretation is not formally defined or directly introduced, I can only make inferences from the mention of "edges" and graph networks. A single simple example of an experiment where all of these elements are made concrete would substantially clarify the work.

As another example, the paper repeats the 0.0261 vs 0.4087 numbers in a lot of places but I have not been able to determine where these numbers come from. They are introduced early in the introduction before they have been precisely explained but I have not been able to find the precise definition of the experiment even afterwards.

Questions for authors

Could you please let me know if my understanding of the work is incorrect or incomplete and point me to where in the paper this is made precise?

Reviewer_rTGKOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies multi-agent credit assignment under a shared exogenous driver and delayed propagation. RICK selects a credit kernel by minimizing the worst discrepancy across intervention regimes and explicitly reports the share outside the bounded backtracking reach. Experiments evaluate coefficient recovery and credit fidelity in controlled propagation environments.

Critical strengths

The distinction between fitting observed propagation and recovering the underlying transmission is relevant to credit assignment.

Critical weaknesses
  1. The connection between propagation shares and agent credit is not sufficiently established. Recovering propagation relationships between nodes does not automatically recover the contribution of an agent’s actions to team return. The credit-fidelity experiments provide empirical evidence against a counterfactual-replay reference. However, the transmission weights and baseline in Eqs. (11) and (12) depend on state, so changing an action may change both subsequent states and the propagation process. The conditions under which path shares computed along recorded trajectories represent these counterfactual contributions remain unclear.

  2. The certification result assumes correct model specification, orthogonal residuals, a common noise floor, and sufficient excitation. At finite sample sizes, separating an incorrect kernel from the true one also requires the discrepancy gap to exceed estimation and solver errors. The paper does not quantify these errors or check this condition experimentally. The unassigned-share identity provides a different guarantee. It ensures consistent accounting within the fitted model, but does not establish that the model or attribution is correct.

  3. The proposed framework includes policy improvement, but the experiments evaluate only the inner level of the bilevel problem. No policy-training results or downstream team returns are reported. The evaluation of delayed attribution is also limited. The main credit-fidelity metric sums credit over time before comparing agent rankings, which can conceal errors in when credit is assigned. Peak error provides some additional information, but does not assess the full temporal credit distribution. The lag profile is also constant over time in all experiments, leaving time-varying delay distributions untested.

  4. The backtracking window limits reported credit rather than computation. The algorithm computes the full ancestor expansion before truncating the output by window and depth, and its computational cost does not decrease with a smaller window. Computing the unassigned share still requires this full expansion. The experiments do not examine how to obtain that share when the expansion itself is computationally infeasible.

Questions for authors
  1. Under what assumptions do the path shares correspond to counterfactual contributions to team return? How are these assumptions satisfied in the state-dependent setting?

  2. How should certification be interpreted at finite sample sizes? Can the estimation and solver errors be quantified and the required separation checked experimentally?

  3. If the full ancestor expansion is computationally infeasible, can the unassigned share be estimated without computing it, and how would its accuracy be assessed?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.