- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper seems to treat RL post-training as continual RL. It claims a four-way split of capability loss, then a subspace-anchored PPO (SCPO) that keeps about 92% of a capability score while nearly matching unconstrained PPO on math. It also proposes two retention metrics and a scaling law.
Critical strengths
The question looks useful. Target-reward curves do not say which old capabilities moved, and it seems worth separating those losses instead of calling all of them an alignment tax.
Critical weaknesses
The claimed split does not seem to follow from the ablation as written. Several rows move in the opposite direction from the delta printed next to them, and the prose then assigns the same shares to a different set of mechanisms.
Questions for authors
Why anchor a probed subspace inside PPO, rather than a null-space or Fisher penalty fit only on held-out behaviors the scorer never sees?
Why is the main pipeline PPO on verifiable rewards, when the paper’s own motivation is RLHF, GRPO, and agentic RL? It is not clear why those update rules were not the primary test.
2 comments
Concern Regarding Reviewer F75L's Review (Submission 13129)
Dear PC and AC,
We request a human-level review of our paper. The Pangram report (link) flagged 63% as AI-generated, but we do not trust this review. Pangram itself notes false positives (~1 in 10,000), limited confidence on short/formulaic text, and segment-based scoring that mislabels hybrid human-AI editing. We ask that reviewers assess the substantive content and author contributions directly, rather than relying on this automated 63% figure.
Sincerely,
The Authors
Official Rebuttal of Submission 13129 by Authors
Thank you for the review. Your main concern was correct for the submitted version: the ablation table and the prose did not agree. The revision fixes this at the source, and both questions are answered with results already in the paper. Table numbers refer to the revision.
W1. "The split does not follow from the ablation; rows move opposite to their Δ; the prose assigns the shares to different mechanisms."
Cause. Several Δs in the submitted table were computed from a stale SCPO reference (96.3) rather than the reported 92.1. That is why some rows appeared to move against their printed Δ. The prose also attached shares to the wrong removal; for example, the distribution-narrowing loss was described as the entropy-floor removal.
Fix.
Q1. Why a probed subspace inside PPO, rather than a null-space or Fisher penalty fit only on held-out behaviors the scorer never sees?
Q2. Why PPO on verifiable rewards when the motivation is RLHF, GRPO, and agentic RL?
The decomposition now follows from the table, is confirmed by a factorial design, and states how much it leaves unexplained. With that resolved, we would be grateful if you would reconsider your rating.