- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
EAGPO adds a reinforcement-learning stage after an initial knowledge edit to improve generalization beyond the original edit query. Anchor-Generalization Training groups the edit query with subject-alias, reversed-relation, and one-hop queries, using one sampled response per query. The advantage baseline is computed from the generalization queries rather than the full cluster. PGAA then adds a cosine-similarity term between each generalization response’s score gradient and the anchor gradient, measured at a selected layer.
Critical strengths
-
The main comparison shows improvements beyond simply adding post-edit training.
-
The paper includes a useful alignment ablation and evaluations beyond the main benchmark.
Critical weaknesses
-
The generalization guarantee in Theorem 4.2 is not established by the proof. Appendix B.2 defines uniform stability over all neighboring datasets, but its key step assumes that the replaced sample is a “typical conflict sample” and uses the conflict fraction as its probability of occurrence. That is an average-case argument, not the required worst-case bound. If the anchor is replaced, its gradient also changes the alignment coefficients of the other samples; the proof does not bound this shared dependence. The subsequent comparison has another problem: separate upper bounds on the two parameter divergences do not imply the claimed inequality between the divergences themselves. Likewise, a smaller upper bound on generalization error does not establish a strict ordering of the actual errors. These gaps prevent the theorem from supporting the paper’s claim of provably better generalization.
-
The variance guarantee in Theorem 4.5 relies on an unjustified comparison of estimator means and second moments. Appendix B.3 states that the baselines are independent of action sampling for a given query. However, for a generalization query, the peer mean includes that query’s own sampled reward. It is therefore not an action-independent baseline. The added term generally has nonzero expectation as well, so comparable estimator means cannot simply be assumed. The proof then neglects aggregate off-diagonal terms described as , without showing that they are smaller than the retained variance improvement. Under its stated scaling, the retained conflict contribution in Equation 30 can be only . An uncontrolled remainder cannot be discarded when establishing its sign. Figure 2 may show lower measured variance during training, but it does not resolve these analytical issues.
Questions for authors
- Theoretical claims: Can you provide corrected statements and proofs for Theorems 4.2 and 4.5 that address worst-case neighboring datasets, the dependence introduced by the anchor and peer baseline, and the estimator mean and covariance terms? Please also distinguish guarantees for the simplified update from claims about the normalized, clipped training objective.
- Held-out evaluation: Appendix A.1 mentions original train/test splits, while Appendix C constructs training clusters from the anchor and its portability queries. What exactly is held out: anchors, query variants within an anchor, relation templates, or some combination? Please give the training and evaluation counts and explain whether test-anchor portability queries ever enter RL training.
- Implementation: Is the descent sign in Algorithm 1 a typographical error? How is Equation 4 implemented when the generalization rewards have zero or near-zero standard deviation? Please specify the stabilizer, any skipped batches, and which parameters contribute to the alignment calculation.
- Empirical interpretation: How is the policy-gradient variance in Figure 2 estimated, and what do the ± values in Tables 1 and 7 represent? For Table 3, are previous-session facts reevaluated after later sessions, or does each row report current-session performance? These definitions affect the evidence for optimization stability and resistance to forgetting.