- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies delayed credit assignment in cooperative multi-agent systems. A "credit kernel" (per-edge transmission coefficients plus Gamma-shaped lag profiles over a directed agent graph) is fitted to behaviour logs. The paper argues that fit residuals, perturbation schedules, and conservation percentages computed from the same logs can't verify themselves: a shared external driver makes parent agents' outputs collinear, so a good fit can coexist with wrong coefficients. The method, RICK, has three parts:
- it selects the kernel by its worst per-regime discrepancy over a family of self-applied perturbation regimes (Eq. 3);
- it chooses that family under a bound on return cost, maximising the smallest eigenvalue of the pooled parent design (Eqs. 7–9);
- it reports the mass that a bounded backtracking window cannot reach as an explicit "unassigned share" (Eq. 14).
A population-limit bound for the linear one-step case is given in App. A.3. The experiments use three synthetic delayed-graph environments and one held-out synthetic family, with exact counterfactual-replay ground truth and 10 draws per main cell. On the two higher fan-in environments, RICK's mean coefficient error is 0.0935 versus 0.2186 for an aggregate (pooled) fit. On the held-out family, it is 0.0539 versus 0.1151.
Critical strengths
- Evaluation against exact ground truth, with negative results reported: The known generating coefficients and exact counterfactual-replay credit let coefficient recovery be measured directly. The appendix openly reports results against the method: the objective-selected schedule buys less identification than round-robin (Table 13), random masks beat the scheduler at 3 of 4 strengths (Table 15), and a time-decayed heuristic beats RICK at short windows (App. A.19).
- The unassigned-share convention is a sensible reporting fix: Using the full expansion as the denominator (Eq. 14) exposes truncation that a reached-set percentage hides. Tables 7 and 9 show a reached-set ratio of 1.0000 while 0.82–0.85 of the mass is unreached at window B=1.
Critical weaknesses
- The "certifying" and training claims are not tested, and the scheduler is contradicted by the paper's own tables.
- No policy is trained: The abstract and Algorithm 1 frame RICK as a training signal for a cooperative team. Yet App. A.24 states "the bilevel problem is evaluated at its inner level", and that the return leak is modelled as a "rescaling of the selected coefficients rather than training without the stop-gradient". No team return is reported, and the stop-gradient result (0.9043 to 0.3486, Table 3) holds by construction.
- The scheduler is not used, and loses when it is: The main tables use index round-robin masks instead of the proposed scheduler (Eq. 9). The scheduler yields lower identification (λ_min) than round-robin at every strength (Table 13), leaves an emitter unexcited (Table 14), and loses to cost-matched random masks at 3 of 4 strengths (Table 15).
- The certificate is never measured: The property that would make selection a certificate is never reported (App. A.0): per-regime discrepancies should coincide at the true kernel and separate a kernel propped up by the shared driver.
- The coefficient-recovery gain is confounded, and the theory does not explain it.
- The baseline downweights the interventions: The aggregate-fit baseline puts weight 0.90 on the unperturbed regime and 0.10 on the perturbed ones (App. A.9, A.11), even though regimes are sampled equally. This deliberately ignores most of the interventional data that breaks the collinearity.
- The theory doesn't favour RICK: The App. A.3 lower bound holds equally for the weighted pooled objective, so it gives no advantage to worst-regime selection. Its assumption that per-regime residuals are orthogonal to the parents cannot hold in the confounded observational regime that motivates the paper.
- The test is weak: Every true coefficient equals 1 (App. A.5), in a correctly specified linear class.
- The fidelity margins are small: Credit-fidelity margins are mostly within one across-draw SD (Table 8). Most ablations are single-draw.
- Obvious alternatives are missing: There is no equal-weight pooled fit, no perturbed-only fit, and no comparison or citation to invariance or robust-optimization methods. The worst-regime criterion is essentially group DRO or V-REx over interventional environments, and the scheduler is an E-optimal design.
Questions for authors
- What are the Table 1 coefficient errors for a pooled fit with equal regime weights, and for a perturbed-only fit, at the same 16 episodes? A clear margin for RICK over both, with paired standard errors, would substantially change my view of Weakness 2.
- Does any experiment train a policy using RICK credit and report team return? If not, would you rescope the title, abstract, and Algorithm 1 to kernel identification?
- Does the App. A.3 orthogonality assumption hold in the confounded regime of Challenge #1? If yes, why is the pooled fit inconsistent there? If not, what guarantee covers the setting actually evaluated? A4) re the kernel's functional forms learned or fixed to the generator's? How do results change with heterogeneous true coefficients, including zero-weight edges?
- Can you report the per-regime discrepancies for the true kernel versus the driver-propped kernel, showing a case where RICK flags a kernel that the pooled fit accepts?