- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes CABLE for multi-reward post-training, which estimates reward interactions from token-logit correlations to allocate weights across rewards, combined with controls for normalization amplitude and, in the CABLE-LE variant, reward-level equalization. The method is evaluated on code generation and mathematical reasoning with 1.5B models against nine multi-reward baselines.
Critical strengths
- Efficient estimation of reward interactions. Estimating reward correlations from token logits requires only one backward pass, making CABLE about 4.2× faster than a parameter-gradient implementation.
Critical weaknesses
- Insufficient ablation of the key components. CABLE combines three controls (geometry, activity, and attainment), but the paper does not clearly show the contribution of each. The main text lacks a systematic ablation, and the only component comparison is deferred to the appendix (Table 5). This table is hard to interpret, as it uses development aliases (e.g., CorrNash, GS-Entropy CorrNash) that are not defined in the main text and does not indicate which components each variant includes. More importantly, although the correlation-aware allocation is claimed as a key contribution, no variant replaces it with uniform weights while keeping the other components. In fact, the geometry-only variant falls below the base model on coding (34.06 vs. 35.44). The activity-related controls (CV gating, amplitude preservation, and scalar standardization) are also changed jointly, so their individual effects cannot be distinguished.
- Incomplete multi-reward evaluation and task-dependent configurations. Although CABLE targets multi-reward optimization, the evaluation focuses almost entirely on task accuracy. Held-out metrics for the other objectives (e.g., execution reliability and length validity) are not systematically reported across methods, and there is no analysis of trade-offs across rewards. As a result, it is unclear whether the accuracy gains reflect better multi-reward optimization or simply more weight on correctness. On coding, the paper attributes the gains mainly to attainment equalization, which upweights correctness as the lowest-scoring reward (Section 5.4), yet neither the other objectives nor a simple baseline with a fixed higher weight on correctness is evaluated. Moreover, no single configuration works across tasks: CABLE-L falls below the base model on coding, while CABLE-LE falls below it on Math.
- Unclear presentation. The paper is difficult to follow: it uses dense notation and many briefly explained custom terms, defers details essential for interpreting the results to the appendix, and is written in a highly compressed style with little interpretation of the results.
Questions for authors
- Most trained methods, including GRPO and GDPO, fall below the base model on coding, and several baselines drop by 4-8 points on Math. Could the authors explain this and clarify how the baselines were tuned?
- Could the authors provide a clear component-wise ablation in the main text that isolates the contribution of geometry, activity, and attainment on both coding and Math?
- Proposition 1 shows that re-standardizing each objective after group normalization amplifies sparse rewards by 1/√f_k, and the introduction presents this as a limitation of separate normalization such as GDPO. However, as we understand it, GDPO applies its second normalization to the aggregated advantage rather than to each objective. Could the authors clarify whether Proposition 1 applies to GDPO? Relatedly, Figure 3 shows GDPO's process reward rising while its correctness collapses, but the paper does not explain the cause. Could the authors clarify what drives this collapse and how CABLE-L avoids it?
- CABLE introduces many hyperparameters (e.g., τ, β, the η schedule, transport ranges, and the number of allocation iterations). Could the authors describe how they were selected and provide a sensitivity analysis for the key ones?
- The best variant differs by task, and the two tasks also use different normalization and transport settings (Appendix B.1). Which configuration would the authors recommend for a new task?