- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a new training-free self-improvement method for a model-based verifier that co-evolves with the agent's harness state by focusing on improving the weakest criteria family in each iteration. At scheduled checkpoints, new candidate verifiers must outperform the previous ones on response contrasts before being adopted. The experimental results show that the proposed UCoE method outperforms previous feedback methods and the direct generation baseline across six open-ended benchmarks on two sizes of Qwen models (8B and 27B), as well as on three verifiable benchmarks with the 8B model.
Critical strengths
- The proposed method outperforms previous methods and the direct generation baseline by a large margin on diverse open-ended tasks across two model sizes (8B and 27B).
- The proposed method also transfers to other domains, such as the codex agent harness on spreadsheet workflows.
- Good ablation studies verify the usefulness of each component.
Critical weaknesses
- The performance gain seems to stem mainly from standard single-turn revision guided by verifier feedback rather than the co-evolutionary mechanism itself. Freezing the verifier drops preference by only 1.7%, indicating that the complex verifier synthesis and update pipeline contributes relatively little over a fixed verifier baseline.
- Larger models are not evaluated on the verifiable benchmarks, and the performance gains are relatively marginal on these tasks (e.g., Hard-MATH and bigcodebench-hard).
- The method lacks a compute comparison against the baselines in the main paper; leaving it unclear how much of the advantage stems from higher inference compute versus the proposed framework itself.
Questions for authors
Does UCoE generalize to distinct open-weight model families?
3 comments
Response to Reviewer LFTn (2/2)
W3: On inference cost and a compute-matched comparison.
Thank you for asking for a comparison under a common compute budget. UCoE's all-in cost includes relation discovery, initialization, checkpoint updates, and online calls. It averages USD 0.0142 per task versus USD 0.0050 to 0.0063 for the simpler feedback baselines. We will bring these costs into the main paper.
We gave LLM as a Judge and RaR additional revision budgets for best-of-K revision, with budget ceilings matched to UCoE's all-in method cost. With Qwen3-8B, LLM as a Judge generates up to six revision candidates and RaR up to four, allowing each baseline to choose from more candidates. Each baseline's candidates start from the same draft and fixed initial feedback, and each baseline retains its scoring, acceptance, and memory rules from the paper.
The table reports preference percentages against Direct Generation.
| Method | Arena-Hard preference vs Direct | RefineBench preference vs Direct |
|---|---|---|
| LLM as a Judge | 50.60 | 51.98 |
| LLM as a Judge, best-of-6 | 54.00 | 52.88 |
| RaR-style feedback | 51.20 | 51.26 |
| RaR-style feedback, best-of-4 | 53.30 | 59.53 |
| UCoE | 58.30 | 61.15 |
Both baselines improve with more revision candidates, especially RaR on RefineBench, which rises from 51.26% to 59.53%. UCoE retains higher observed preference on both benchmarks at 58.30% and 61.15%. This suggests that the feedback used to guide and select revisions matters alongside the revision budget.
Q1: On distinct open-weight model families.
Thank you for asking whether UCoE generalizes to other open-weight model families. We evaluated NVIDIA Nemotron-3.5-Lightning-30B-A3B as a non-Qwen actor on Arena-Hard and RefineBench. Direct Generation, LLM as a Judge, and UCoE use the same actor and decoding settings. UCoE initializes its verifier on Nemotron calibration responses and starts with empty memory.
The verifier remains Qwen3.6-35B-A3B, and Qwen3.5-397B-A17B evaluates responses using the original blinded, both-order protocol with half credit for ties. Arena-Hard uses mined relations. RefineBench uses the relation-free configuration because its mined relations did not pass calibration. Scores below are preference percentages.
| Benchmark | Tasks | LLM as a Judge vs Direct | UCoE vs Direct | UCoE vs LLM as a Judge |
|---|---|---|---|---|
| Arena-Hard | 500 | 54.20 | 58.70 | 53.10 |
| RefineBench, without relations | 278 | 55.04 | 60.25 | 53.78 |
UCoE reaches 58.70% preference against Direct Generation on Arena-Hard and 60.25% on RefineBench. Its direct comparisons with LLM as a Judge also favor UCoE.
These results show that UCoE's verifier-guided feedback can improve responses from a non-Qwen actor while retaining the same verifier backbone. The RefineBench result further shows that this benefit extends to a setting without mined relations.
Dear Reviewer LFTn,
As the discussion period draws to a close, we would greatly appreciate your feedback on our responses and revised manuscript. We have expanded the fixed-verifier comparisons, added evaluations on another model family, and provided budget-matched comparisons to address your concerns.
Please let us know whether these additions address your questions. If any concerns remain, we would be happy to clarify them before the discussion ends. We would also appreciate it if you could reconsider your assessment after reviewing these updates.
Thank you again for your time and thoughtful feedback!
Sincerely, The Authors
Response to Reviewer LFTn (1/2)
We thank you for the positive assessment of the results across tasks and model sizes, and for recognizing the transfer to an agent harness and the value of the ablation studies. We address each of your points below.
Thank you for asking us to isolate the contribution of continued verifier evolution. We extended the Fixed-Verifier UCoE comparison to all six open-ended and three verifiable benchmarks with Qwen3-8B.
Fixed-Verifier UCoE retains the initial verifier, the same revision procedure, and ongoing memory updates. Both variants use the same initial verifier synthesis and contrast-based selection. Full UCoE can replace its verifier at later checkpoints.
For open-ended tasks, Qwen3.5-397B-A17B evaluates responses against Direct Generation in both presentation orders, with method identities hidden and half credit for ties. Scores below are preference percentages.
Full UCoE scores higher on five of the six benchmarks, with gains of 2.98 percentage points on WildBench and 4.20 on RefineBench. Fixed-Verifier UCoE already improves on Direct Generation on all six benchmarks, so continued evolution adds gains beyond those provided by the initial verifier and ongoing memory updates. WritingPrompts is unchanged because neither candidate met the update criteria, so Full UCoE retained its initial verifier.
The Arena-Hard Full UCoE score includes relations. With matched requirements-only settings, Full UCoE scores 58.10% versus 56.60% for Fixed-Verifier UCoE against Direct Generation. Matched direct comparisons give Full UCoE 51.30% preference on Arena-Hard and 53.12% on WildBench. The WildBench difference is significant with p≈10⁻⁶, while the Arena-Hard difference is not, with p=.23.
For verifiable tasks, we retain the paper's benchmark-specific configurations, including select-only on IFBench. Scores are accuracy percentages for IFBench and Hard-MATH, and Pass@1 percentages for BigCodeBench-Hard.
Evolution adds two correct answers on IFBench. Hard-MATH retains the initial verifier because no candidate met the update criteria. On BigCodeBench-Hard, a verifier update changed the responses while leaving aggregate Pass@1 unchanged. Together, these results show that continued evolution can improve on the initial verifier and memory loop, with larger observed benefits on the open-ended benchmarks.
Thank you for suggesting evaluation with a larger actor on verifiable tasks. We evaluated Qwen3.8-27B on all three benchmarks using the paper's task sets, verifier backbone, output limits, and benchmark-specific configurations, including select-only on IFBench. The LLM as a Judge baseline uses binary judgments to update memory without a separate revision. Scores below are official checker results as percentages.
UCoE scores above Direct Generation on all three benchmarks, with gains of 1.33, 8.64, and 0.68 percentage points. It has the highest accuracy on IFBench and Hard-MATH, while LLM as a Judge scores higher on BigCodeBench-Hard. The largest gain is on Hard-MATH under the original 4,096-token per-response limit, where UCoE solves 33 additional tasks while retaining all of Direct Generation's correct answers.
These results show that a larger actor can still benefit from verifier-guided feedback, particularly on mathematical reasoning. The smaller gains on IFBench and BigCodeBench-Hard indicate that the benefit varies by task. Reference answers and checker results are used only for final evaluation.