- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SAW-v2 is a sample-weighting module used when training multi-agent actor-critic methods. A small network looks at the observation and predicts a positive weight for each sample. The weights are normalized over the batch, detached, and used to scale the actor loss. The weight network is trained separately to match two targets: the size of the actor loss and, when twin critics exist, the disagreement between the two critics. The two targets are combined by taking the larger one. The experiments add the module to MAPPO, HAPPO and HASAC on 18 tasks from MAMuJoCo, DexHands, MPE and SMACv2, with three seeds each. The mean score improves in 48 of 54 comparisons, with an average relative gain of 12.3%.
Critical strengths
S1. Broad evidence that the full module helps: Table 1 covers three backbones and four benchmark families, with three seeds and full learning curves. I recomputed the printed scores and confirmed 48/54 positive differences and a mean relative gain of about 12.33%. Several gains in continuous control and manipulation are large, and the six negative cases are reported openly. This makes the method worth considering in practice.
S2. Easy to plug in, with a clean split of gradients: Eqs. (2)-(9) and Algorithms 1-2 keep the supervised weight learning separate from the actor update. Detaching stops the actor loss from sending gradients into the weight network, and the masked normalization keeps the average weight fixed. The network is only used during training (§3.5), so it adds no cost when choosing actions. These are useful properties, even without a convergence guarantee.
Critical weaknesses
W1. The experiments show that the full package helps, but not which part of it is new and useful: The paper motivates SAW-v2 as a fix for the collapse of jointly learned weights (introduction, §3.1), and highlights the dual-stream supervision (§3.4). But all experiments only compare each backbone with and without the full module. Three comparisons are missing:
- the earlier SAW, or a matched jointly trained weighting;
- policy-only versus dual-stream supervision on the same twin-critic backbone;
- a simple control that uses the detached targets directly as weights, to show that the learned predictor is needed.
Algorithm 2 falls back to policy-only supervision when there are no twin critics, so gains across different backbones cannot show what the critic stream adds. A small study on a few tasks, focused on the first two comparisons, would address this. It is not necessary to show that every part is essential.
W2. Some details of the tested variants are missing:
- Eq. (7) uses the per-sample actor loss as its target, but §3.5 does not say what this loss is for each backbone, for example whether entropy terms are included. This matters because the loss size sets the target weights.
- Algorithm 2 mentions optional variance and temporal regularizers, but gives no formulas or coefficients, and does not say which experiments use them.
- The appendix says the common settings in Table 4 apply to only 14 of the 18 HASAC variants (p. 13), but does not say what the other four use. It also promises task-specific choices from the ranges in Table 3, but the task sections only give task descriptions and results.
Because of this, I cannot rebuild exactly what was run, or tell which change produced the gains. This does not mean that the baselines were mismatched or that the experiments were not done.
Questions for authors
Q1. On a representative twin-critic task, how does SAW-v2 compare with a matched SAW or joint-weighting baseline? And what does critic disagreement add over policy-only supervision? Please use the same backbone and budget. Using the targets directly as weights, or a plot of the weight distribution during training, would also help explain the predictor and the collapse story. A small, focused study is enough.
Q2. What exact per-sample loss is used in Eq. (7) for each backbone? Which configurations use both streams? Were the optional regularizers turned on? Please list the four HASAC exceptions and the exact per-task settings, and confirm which backbone settings are kept the same in each baseline vs SAW-v2 pair.