acceptodds
Withdrawn from ICLR 2027

ContractRL: Shielded Group-Relative Policy Optimization for Auditable Tool-Call Repair

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Structured tool calls often fail after only a small number of fields violate a schema or an execution contract. Regenerating the complete object enlarges the action surface and makes repeated repair difficult to audit. We introduce ContractRL, a contract-constrained sequential repair protocol that models verifier-guided JSON repair as a bounded decision process. At each step the policy observes the candidate, typed verifier feedback, JSON Pointer, immutable repair history, and remaining budget; a contract-derived action mask filters malformed or prohibited RFC-6902 operations before a deterministic validator performs the transition. We specify a contract-constrained group-relative objective for patch, retry, and abstention decisions while keeping canonical targets and semantic labels outside the online state until trace freeze. On the recorded five-seed held-out sweep, the 2B/r8 ContractRL row reaches 0.9331 semantic success at 34.8 generated tokens and 1.32 reported p95 latency units; the accompanying Patch-SFT, full-regeneration, and constrained-decoding rows reach 0.8924, 0.9041, and 0.8671. Under the completed same-information control, ContractRL reaches 0.9362 semantic success with 34.4 generated tokens, 1.30 p95 latency units, and 0.0169 collateral edits, compared with 0.9076/44.9/1.49/0.0488 for Patch-SFT and 0.9148/137.2/2.54/0.1876 for full regeneration. With the same public state and budgets, policy optimization reaches 0.9375 semantic success versus 0.9186 for ContractRL-SFT and 0.8927 for Patch-SFT. A separate three-seed paired ledger against Patch-SFT reports a semantic difference of (95% CI , ). Scale, feedback, budget, schema-shift, noisy-verifier, transfer, selective-risk, uncertainty, and blinded audit results identify where bounded repair helps and where it should abstain.

Primary area:
reinforcement learning
Keywords:
Reinforcement Learning, Verifier-Guided Learning, Program Repair, Structured Output Generation, Constraint Satisfaction, Auditable AI Systems, Feedback-Guided Optimization, Posthoc Verification

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_j2sJOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ContractRL, a method for repairing incorrect tool calls by editing selected fields rather than generating the entire call again. The model receives error feedback, can make a limited number of repair attempts, and can decline a repair. Proposed changes are checked before the tool is allowed to run. The training reward encourages successful repairs while penalizing unnecessary edits, invalid actions, retries, and generated text. The authors compare ContractRL with supervised patch generation, full-call regeneration, and generation with formatting constraints, reporting higher task success and lower generation costs.

Critical strengths

The paper addresses a useful problem. When only a small part of a tool call is incorrect, repairing that part rather than rewriting everything is a sensible approach. But I am not sure if this is a novel approach or a significant advancement.

Critical weaknesses

The main success measure does not show how many tasks are successfully completed overall. Table 24 calculates semantic success only among calls that are actually executed. Because ContractRL can decline repairs or have them rejected, it can improve this success rate by avoiding difficult cases, rather than by successfully completing more tasks. This is particularly important when comparing the full method with the version that cannot decline repairs, as in Table 29.

The reinforcement-learning procedure is not sufficiently explained. Section 3 gives a reward function, but does not specify how alternative repair sequences are sampled and compared, or how their rewards are used to update the model. Since reinforcement learning is central to the claimed contribution, the manuscript needs the actual update procedure and a clear explanation of how the repair checks interact with training. It is also unclear whether prohibited edits are prevented during generation or rejected after generation.

Questions for authors

Please see above.

Reviewer_25Q9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies LLMs' generation of structured tool calls. It proposes ContractRL, which formalizes the tool-call repair task and then develops a GRPO-based training recipe for it. The results on limited datasets show preliminary benefits over SFT or full regeneration.

Critical strengths
  1. It studies a concrete problem concerning structured output and explores an RL method for improvement.
Critical weaknesses
  1. The test-set scale is as small as 64, which is insufficient to support the superiority of the method.
  2. The writing is poor and far below the bar of a top-tier conference like ICLR, with evidence of AIGC. The content is one page shorter than the page limit, with uninformative content, and the layout is awful; for example, Section 3 has only one subsection. While the topic is related to LLMs, there's no related discussion in the paper.
Questions for authors

I have no specific questions for this paper.

Reviewer_JDhyOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Structured tool calls (I guess, by LLMs or agents) can fail. Regenerating the entire object(?, I guess it means some command) can make the problem worse. (To be honest, the authors say "enlarges the action surface", which is not understandable.) To address it, this work proposes ContractRL, which seems to be a schema-based or grammar-based repair technique. Empirical studies show higher performance than Patch-SFT. No explanation of why, however.

Critical strengths

N/A

Critical weaknesses

W1. Even from the first paragraph of the introduction, I can see that this text is heavily condensed using AI. Near-zero context. Sentences don't logically connect.

W2. The abstract mentions Patch-SFT as a baseline method. I was expecting why the introduction would describe (fundamental) limitations of the method. I don't see any discussion.

W3. The following two paragraphs (second and third) are mere descriptions of the proposed approach. I don't understand why those approaches contain intellectual novelty.

W4. The first paragraph of Related Work is a random collection of unrelated papers. The first sentence notes "structured_output repair". Then, the following sentences should be closely related to it. How is ReAct related to structured output repair? Likewise, how is SWE-bench related to structured output repair? They are not related. Either the authors didn't verify the AI-generated literature review, or they didn't find logically related papers. Either way, this is poor writing. I'm jumping to Section 3 directly, skipping Section 2.

W5. I tried to read Section 3.1 now. Again, no proper description of the problem. I feel like the text is something heavily condensed by AI. The sentences within the same paragraph don't connect one after another.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.