- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses a concrete defect in constrained reinforcement learning. Costs are charged to the step at which they are observed, while consequences often arrive late. This mispairing sends the learning signal to the wrong decision and leaves the responsible action uncorrected. The proposed CCPL is an auditable layer for delayed-cost attribution. It first infers which earlier step a cost came from, and then decides whether to charge it to an action or to keep it at the state level. The first step uses a posterior built from a delay prior and a consequence likelihood, the second uses an entropy gate, and jointly caused events are split by Shapley shares. The formal analysis is only a fixed-kernel contraction, which Section 5 itself acknowledges as equivalent to the standard result. Validation covers the controlled delayed-cost benchmark, a do-operator intervention diagnostic, the Safety-Gymnasium external check, and the GUARD comparison, with a correlation of 0.99 between responsibility shares and measured effects. The paper's core claim is conditional, with improved constraint satisfaction only in the binding delayed regime and the remaining conditions reported as not separating the methods.
My verification is based on a full read of the main text, all figures, tables, and numbered equations, with the appendix checked only at the section level from A to I. Line-by-line checking of the proofs, the appendix numbers, and the reproduction of code and experiments are outside the scope of this review.
Critical strengths
1.The intervention validation is solid. On 1200 samples, the correlation between responsibility shares and measured intervention effects is 0.99, against 0.43 for path-Shapley and 0.00 for visible-step attribution. This provides direct support for the claim that the attribution tracks causal effects. 2.The reporting discipline and the engineering fallback are clear. Unfavorable results are reported as they are, including the uniform baseline's advantage, the null residual control, and the unavailable matched-cost comparison. The 60-line CreditPatch wraps a third-party CPO-FO agent, and the budget gate reduces violation steps from 5.10 to 3.50. 3.The problem is split cleanly. The paper separates two kinds of uncertainty in delayed costs, one about which step a cost came from, and the other about how large the cost is. The unresolved part of the former stays at the state level instead of being forced onto an action, while the latter is described by the scale of the consequence model. This split gives every component of the mechanism a clear motivation.
Critical weaknesses
1.The paper sells attribution, yet it loses the attribution metric to a training-free uniform split. The uniform split's overlap exceeds every learned method's, and the soft-posterior control nearly matches the full Shapley path, so the gain comes mainly from the soft posterior rather than from the split itself, with the numbers in Section 7.4. This directly undercuts the premise of the contribution. 2.The safety benefit is confined to a single regime, external validation shows no advantage, and one anomaly is left unexplained. Gains are limited to the regime with bimodal delay, a tight budget, and separable rewards. On Safety-Gymnasium the reward is lower than CPO-FO and PPO. On GUARD the posterior cost is below the oracle's, which the paper itself cannot explain, see Section 7.4. How far the evidence extends, and whether the protocol is credible, remain open. 3.The structure is clear and each component of the method is described, but the writing is heavily defensive, with qualifiers attached to almost every claim, so readers must assemble the actionable conclusions themselves. A short scope paragraph up front would help. On the figures, the tables themselves are readable, but Figure 1 is a single rollout illustration, and the key numbers on attribution quality are scattered across tables and prose, with no plot of overlap, TAE, or the intervention correlation, which makes the presentation conservative.
Questions for authors
1.Is there a setting in which the learned attribution genuinely outperforms the uniform split on overlap, or on another endpoint? If not, how should the attribution contribution be restated? 2.The soft-posterior control nearly matches the full Shapley path. What does the split contribute on its own, and what would be lost without it? 3.Besides the regime with bimodal delay and a tight budget, is there a second regime where CSR or cost separates? 4.Why is the posterior cost on GUARD below the oracle's? Is it a difference in measurement protocol, or a bias in the oracle condition itself? 5.The pooled conclusion in Table 1 runs opposite to the text, with no-split minus hindsight giving an overlap of minus 0.079 and a TAE of plus 0.209. Why does the text state that the split improves the pooled distribution? 6.Before calibration completes, for example a mean of about 5.26 against a true delay of 10 at 100 episodes, is there an explicit fallback policy such as reverting to the uniform split? The following are suggestions for future work : Direct comparisons against HCA/RUDDER-on-cost, known-delay augmentation controls, and a tuned OmniSafe suite, which Section 6.2 acknowledges are not evaluated.