- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose ClaimSift, a benchmark built on ProofWriter and FOLIO for evaluating selective revision under mixed correct and incorrect feedback, and PARO, a constrained RL method that combines revision reward with explicit preservation cost and an adaptive Lagrangian penalty weight calibrated to a reference policy's damage rate. ClaimSift decomposes revision evaluation into five dimensions — Proposal Judgment, Target Correction, Proposal Resistance, Dependency Propagation, and Content Preservation — and defines Overall Success as their conjunction on each case. Experiments on Qwen3-8B show PARO achieves 54.47% Overall Success on ProofWriter, exceeding DPO and TD-DPO by ~9.6–9.8 points, with Content Preservation improving 3.42 points over DPO. Additional analysis reveals that the adaptive penalty's advantage over a development-tuned fixed penalty is small (+1.02 on ProofWriter, +2.72 on FOLIO) and not statistically established as an independent effect.
Critical strengths
-
Careful evaluation decomposition. The five-metric framework (TC, PR, DP, CP, OS) with the Section 3.2 six-model diagnostic (Table 15) cleanly demonstrates that no single component predicts Overall Success — Llama-3.1-8B reaches 88.67% TC but only 2.60% OS, Qwen3-14B reaches 97.09% CP but only 3.49% TC — providing genuine diagnostic value that Final Accuracy alone cannot offer.
-
Thorough benchmark construction. The paired-variant design with blocking/restoration dependency validation (Appendix B.1), four feedback scenarios per problem, and explicit admission criteria create a benchmark where every ground-truth label and dependency annotation is semantically verified rather than heuristically assigned.
Critical weaknesses
-
Benchmark domain is too narrow to support general claims about "selective revision in LLMs." Both ProofWriter and FOLIO use binary TRUE/FALSE labels over clean logical structures with perfect ground truth and mechanically verifiable dependencies. Real revision tasks involve natural-language answers with partial correctness, fuzzy dependencies, and no oracle decomposition into target/dependent/unrelated items. The method requires ground-truth labels and dependency annotations during training (Eq. 4–5), making it inapplicable to open-domain settings without substantial additional machinery. The title and abstract frame the contribution as general, but the evidence covers only formal logic with binary labels.
-
The adaptive penalty provides marginal and statistically unconfirmed gains over a tuned fixed penalty. On ProofWriter, PARO achieves 54.47% vs. Tuned Fixed 53.45% (+1.02 points); on FOLIO, 55.66% vs. 52.94% (+2.72 points). The paired analysis (Table 30) shows PARO gains 8.37% of cases over Fixed 1.0 on ProofWriter but also loses 7.34%, with a net of just 1.02 ± 1.89 — the SD exceeds the mean. The paper itself acknowledges this (Section 6.4). Without this, PARO's contribution reduces to "adding a preservation cost to RL improves revision," which is unsurprising given the objective explicitly rewards not damaging correct items.
-
No ablation of reward component weights or controller hyperparameters. The reward weights (αs=0.5, αc=0.25, αj=0.25), reference coefficient (τ=0.01), and controller settings (λ0=1.0, η=0.1, λmax=10) are fixed throughout (Table 6) with no sensitivity analysis. Given that the gap between PARO and Fixed 1.0 is ~1–3 points, these design choices could easily be load-bearing. A different (αs, αc, αj) split or a different η could shift the balance between correction and preservation enough to close or widen the gap, but this is never tested.
Questions for authors
Have you considered evaluating on a non-logical, non-binary revision task to test generalization?
PARO requires ground-truth labels and dependency annotations during training. How would the method generalize to domains without oracle item-level correctness?
2 comments
Thanks for your response, and I will keep my rating.
Response to Reviewer odpg on Task Scope and PARO
Thank you for identifying the value of the component scores and asking how far the evaluation and the adaptive mechanism support our claims. These are distinct questions, and we address each below.
W1/Q1. Non-logical and non-binary revision. ClaimSift's original design covers structured binary answer revision with supplied evidence and roles annotated for scoring. The paired ProofWriter variants keep unrelated labels fixed while changing the target and dependent labels. This lets the evaluation distinguish correction, dependent changes, and retention within one answer. The saved-response diagnostic shows why that matters: Llama-3.1-8B-Instruct reaches 88.67% Target Correction and 2.60% Overall Success, while Qwen3-14B reaches 97.09% Content Preservation and 3.49% Target Correction (original Figure 2 and Table 15). FOLIO uses an in-domain binary protocol. The revised abstract and Section 4 state this task scope. The submission reports no free-form or non-logical revision evaluation. Such an evaluation would need a rubric for partial item correctness and for which dependent edits are required.
W1/Q2. Item-level supervision. Training uses correct values to score repairs and damage, and dependency annotations to score required updates (Eqs. 4–5 and Appendix B). At inference, the model receives evidence, queries, the initial answer, and proposals. The labels and dependency annotations stay with the scorer. In an unannotated domain, a verifier could provide the training signals if it can reliably distinguish new damage from remaining errors and identify required dependent updates. This would need validation against independent annotations before using the verifier's outputs for training.
W2. Preservation cost and adaptive weighting. Consider two revisions that repair the same item, judge both proposals correctly, and miss the same dependent update. With a positive preservation weight and equal revision reward and reference log-ratio, the revision that also damages an initially correct item receives a lower return (original Eqs. 4–6). This distinction applies even when neither revision passes Overall Success.
The original comparison with Development-tuned Fixed 1.0 reports mean Target Correction gains of 7.46 pp on ProofWriter and 7.55 pp on FOLIO (Tables 4–5). Under the reported pairing, the summaries imply gains in at least four of five repetitions per domain (revised Appendix F.8). Reported mean damage across all initially correct items falls from 12.40% to 11.19% and from 15.88% to 14.44%, respectively (original Tables 23–24). The original paired Overall Success differences are +1.02 ± 1.89 pp and +2.72 ± 4.81 pp (mean ± SD, Table 30). These summaries give clearer evidence for Target Correction than for consistent Overall Success gains.
W2. Dense correctness reward. With the same initial answer, two candidates can finish with the same number of correct items. For example, one can repair one more item and damage one more. A dense final correctness reward gives the two candidates the same score. With the other return terms fixed, unequal effective weights for repair and damage can distinguish them. When the effective weights are equal, the combined repair and damage terms and a matched dense correctness reward rank candidates identically (original Appendix H.3). For Eqs. 4–5, matching requires accounting for the normalization denominators (revised Appendix H.3). Comparing trained policies would require selecting the dense coefficient on Development and holding the other reward terms, reference, and policy update fixed.
W3. Sensitivity. Mean Overall Success peaks at 1.0 among the tested fixed weights in both domains.
Table R1. Overall Success under fixed preservation weights (%, original Tables 4–5).
On ProofWriter, increasing the weight from 0.5 to 2.0 raises Content Preservation from 96.64% to 97.01% and lowers Target Correction from 79.33% to 75.20%. This sweep measures sensitivity to the fixed preservation weight. The reward mix, reference coefficient, and adaptive controller settings are held fixed (original Table 6). Sensitivity to those settings remains an open question.