- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies closed-loop revision in LLM applications. To reduce the influence of incomplete or unreliable feedback, the authors propose a fixed-budget revision protocol to evaluate LLMs' revision ability in a controlled setting. They design three types of constraints, exact-length, lexical, and compositional, and evaluate a range of LLMs on these tasks. The results show that model performance still varies substantially even when the feedback is accurate and complete.
Critical strengths
Evaluating the closed-loop revision ability of LLMs appears to be a worthwhile direction for further study. In my view, it is valuable to distinguish the quality of feedback from the model’s ability to revise its output based on that feedback.
Critical weaknesses
- The three constraint families considered in this paper are quite limited and mainly focus on surface-level linguistic properties of the responses. They do not evaluate the quality of the underlying answers, nor do they examine how answer quality changes as models improve constraint satisfaction. It is therefore possible that a revision satisfies the formal constraints while making the answer itself less accurate, relevant, or useful. The paper does not capture this potential trade-off.
- Although these three constraint families allow the authors to provide fully deterministic and correct feedback, the feedback is highly regularized and task-specific. Such feedback also appears relatively easy to optimize for directly during post-training, for example through reinforcement-learning-style objectives. This raises a concern about whether the benchmark is measuring a model’s general revision ability, or mainly its ability to follow and optimize for a small set of predefined constraints.
Questions for authors
- How does task performance change across revision rounds as constraint satisfaction improves? In particular, it would be useful to examine whether better constraint satisfaction is accompanied by improvements, degradation, or trade-offs in the quality of the underlying task response.
- How would targeted reinforcement learning on these tasks affect model performance, both in terms of constraint satisfaction and task quality? It would also be interesting to train a model on one specific constraint family and evaluate whether the resulting behavior transfers to the other two constraint families.
2 comments
Thank you for the thoughtful response and additional experiments, and apologies for my delayed reply. I appreciate the new findings. However, the W1 results also lend some support to my concerns about the trade-off between constraint satisfaction and answer quality, while the generalizability of revision ability remains uncertain. I will therefore maintain my current rating.
Thank you for recognizing the value of studying revision ability separately from feedback quality. Your questions prompted us to examine what happens to the text as it meets the constraints and what transfers when a model trains on just one constraint family.
W1. Answer quality during revision
Q1
On exact-length tasks, all 12 local baselines showed their largest loss of initial-draft entities on the first edit. Across 4,515 model–task pairs with identifiable entities in the initial draft, model-averaged retention fell to 96.0% at that step and was 95.8% at round 8. Over the same rounds, cumulative success across all 5,760 pairs rose from 3.2% to 19.8%. Our Official Comment “A round-by-round audit of content retention and sentence consistency” gives the full trajectory.
To assess the writing itself, GPT-6 Luna and Claude Haiku 5.5 compared drafts against the original writing request with the added constraints hidden. We counted a preference only when both judges chose the same draft in both presentation orders. Across 8,910 exact-length trajectories, they jointly favored the initial draft and the first edit almost equally often. On those same trajectories, they favored the initial draft in 1,031 cases and the final draft in 312. Details are in our official comment "Two-judge comparisons of writing quality across drafts".
The clearest trade-off appeared on the edit that first satisfied every lexical constraint. Across 1,308 such repairs, both judges preferred the preceding draft in 38.8% of cases and the passing draft in 9.4%. For Claude Opus 5, the direction was reversed. Across its 220 repairs, both preferred the passing draft in 28.6% of cases and its predecessor in 16.8%. The lexical and compositional judgments assessed clarity, coherence, and completeness of meaning. The judges had no independent factual reference for those tasks, so the evidence speaks most directly to how the text reads.
W2. Targeted training and cross-family transfer
Q2
Interestingly, we had considered RL as a solution but didn't finish it, because just as the experiment we added proved, closed-loop revision might not the best solution for these constrained tasks considering its cost. Before submission, we ran a 256-task REINFORCE++ pilot with Qwen3-8B on exact-length revision. Following your suggestion, we extended that run to 1,024 training tasks and tested both untrained families. Exact-length final success rose from 9.6% to 26.9%. Targeted training helped, although only about one in four tasks passed by the end of revision. It's far from perfect.
What transferred most consistently was a change in editing behavior. At each successive checkpoint, early revisions less often repeated an earlier failed answer, including on both untrained families. Final success rose modestly on lexical constraints (35.4% to 38.1%) and more on compositional constraints (69.4% to 80.2%). On the 480 compositional tasks, initial successes rose only from 284 to 288, while cases first solved during revision nearly doubled, from 49 to 97. This single run shows transfer across tasks with exact feedback. Our Official Comment “Exact-length post-training and cross-family transfer” reports the training setup and checkpoint results.
We also compared the two checkpoints’ final answers on the same exact-length tasks. Each generated its own initial draft. After training, final answers retained a smaller share of their initial-draft entities (−2.53 percentage points across 363 applicable pairs). Coverage of entities in the shared reference text changed little (+0.07 points across 436 pairs). RL raised success in all three families, yet both judges more often preferred baseline final answers in both orders. On lexical tasks, they favored baseline in 81 of 480 pairs and RL in 36.
Thank you for suggesting these experiments. We would include the round-by-round quality results, the transfer experiment, and their limitations in the revised manuscript.