acceptodds
Under review as a conference paper at ICLR 2027

COPA: COmpiling Plans and Answers for Efficient Tool Agents

Abstract

Tool agents often solve complex tasks by interleaving model calls with tool use, but repeatedly processing intermediate results and growing interaction histories incurs substantial token costs. This raises an intriguing question: *Can we avoid this repeated interaction by generating everything needed to complete the task in a single model call?* Simply requesting a complete plan or answer, however, is not enough: interface mismatches can prevent plan execution, while useful candidates generated along the way may be omitted from the final answer. To close these gaps, we introduce **COPA (COmpiling Plans and Answers)**, which guides models to produce tool plans or complete candidate answers for execution and validation without further model calls, with optional model feedback for task review or revision. At the core of COPA are a compiler and an answer processor: the former repairs resolvable interface mismatches, enabling the runtime to execute plans from one planning call; the latter recovers omitted answers by validating and selecting complete candidates already in model responses. On DeepPlanning's original Shopping tasks, shared-tool comparisons across Qwen and Gemma show competitive task success with 94.3%–97.2% fewer total model tokens than ReAct and CodeAct. On AppWorld public train, COPA completes 78/90 tasks with 81.6% fewer tokens than Simplified ReAct. In constructed shopping stress tests, COPA's answer processor increases Qwen's correct returns from 4/96 to 31/96 without additional model calls. Code will be made publicly available.

Primary area:
foundation or frontier models, including LLMs
Keywords:
tool-using agents, large language models, plan compilation, inference efficiency

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Rm9WOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces COPA, a framework for reducing repeated model calls in tool-using agents. It has two paths. The first compiles a one-shot plan into an executable workflow, using tool contracts to repair reference and typing mismatches before deterministic execution. The second extracts complete candidate answers already present in a model response, checks their feasibility, and selects an answer without further generation. Experiments on DeepPlanning Shopping, Travel and AppWorld report competitive or better success with substantially fewer model tokens. The frozen-response experiments examine the contributions of plan repair and candidate processing while keeping the generated responses fixed. The paper’s main finding is that some apparent planning failures can be resolved by improving execution and answer return without asking the model to generate more.

Critical strengths

The frozen-response experiments are the strongest part of the paper for me. Applying different repairs and answer readers to the same saved generations makes the contribution of these components much easier to assess. The reported increase of 40 executable plans, and the improvement from 4/96 to 31/96 correct answers in the Qwen setting, provide concrete evidence that useful solutions can already be present in a response but fail to reach execution or the final answer. These experiments support a specific explanation for the gains, which I find more convincing than an end-to-end comparison alone.

The approach addresses a practical source of overhead in tool agents. Contract-aware repair and candidate validation are explicit operations, and neither requires another model call. The shared-tool comparisons also gives useful evidence that large token savings are possible without a corresponding loss in task success or a large increase in tool calls. I particularly like the distinction between a response that contains no solution and one that contains a feasible solution but fails to return it. That distinction has implications for how agent failures should be evaluated.

Critical weaknesses

My main reservation concerns the scope of the efficiency comparisons. The shared-tool Shopping results are useful, but Travel lacks a direct comparison with an interactive baseline using the same scheduler and tools. Some comparisons with LLMCompiler and PIVOT also use known token lower bounds rather than complete token accounting. This does not by itself invalidate the savings, but it makes the different comparisons harder to interpret together. Token usage also leaves out the computational cost of domain operations such as the coupon-aware search in choose. I would like a clearer account of which efficiency claims are supported by matched measurements, and whether the savings carry through to end-to-end runtime.

The method’s applicability depends heavily on well-specified interfaces and the availability of useful domain computation. The compiler repairs representation-level problems, but a plan with valid types and references can still use a parameter incorrectly. The answer processor also requires a complete candidate to already exist in the response. These limitations are acknowledged, so I do not view them as a fundamental flaw. However, they leave the boundary of the contribution somewhat narrow: the evidence is strongest for structured environments where contracts and feasibility checks are already reliable. More detail on failures that remain after compilation and candidate processing would help establish where an interactive agent is still needed.

Questions for authors

Could you provide a compact comparison table showing which tools, backbones and generation settings are shared across methods? Please distinguish measured token totals from lower bounds. For Travel, is there a matched ReAct or CodeAct result using the same scheduler and tools? This would help clarify which conclusions can be drawn from each comparison

How often does the candidate reader encounter contradictory or misleading candidates in the reasoning text? Could you report cases where validation or selection produces an incorrect result, as well as cases where no complete candidate is available? I would like to understand the failure rate of this component alongside the recovery gains.

Could you report end-to-end latency and the non-LLM computation costs, seperately from model tokens? In particular, how much time is spent in domain operations such as choose, and do the main efficiency conclusions remain when this work is included?

4 comments
AuthorsOpenReview ↗

Follow-Up to Your Review

Dear Reviewer Rm9W,

Thank you again for the time and effort you put into reviewing our work!

We understand that this might be a super busy period for you. We wanted to briefly let you know that we have posted point-by-point responses to your questions and concerns and uploaded a revised manuscript with the changes highlighted.

Whenever you have a chance, we would be grateful for your feedback on whether the responses address your concerns. We would be happy to clarify anything that remains open.

Thank you!

Best,

The Authors of Submission 5204

AuthorsOpenReview ↗

Follow-Up to Your Review (Second Reminder)

Dear Reviewer Rm9W,

Thank you again for the time and care you put into reviewing our work!

We know this is a busy period, so we wanted to follow up briefly on our point-by-point responses and the revised manuscript, where we highlighted the changes for easy reference.

If you have a chance to look over our responses, we would appreciate hearing whether they address your main concerns. If anything remains unclear, we would be happy to clarify it here.

Thanks again for your support!

Sincerely yours,

The Authors of Submission 5204

AuthorsOpenReview ↗

Follow-Up to Your Review (Third Reminder)

Dear Reviewer Rm9W,

We are following up on our point-by-point responses and the revised manuscript, which we posted several days ago. We believe these materials further address your concerns and highlight the changes we made in response to your feedback.

As the end of the Author-Reviewer Discussion phase is fast approaching, please let us know whether our responses address your main concerns or if any points still need clarification. We are available to discuss them here.

Thank you again for your time and effort devoted in this review!

Best,

The Authors of Submission 5204

AuthorsOpenReview ↗

Follow-Up to Your Review (Fourth Reminder)

Dear Reviewer Rm9W,

As the discussion period draws to a close, we would like to briefly follow up on a point you raised in your review.

We appreciate your suggestion to compare COPA with interactive baselines on Travel using the same scheduler and tools. In the matched comparison, COPA and ReAct each solved 83 of 120 tasks, while CodeAct solved 82. COPA used 54.0% fewer tokens than ReAct and 61.1% fewer than CodeAct. The rebuttal reports the full token totals for all three methods under these shared settings.

If any part of our response would benefit from clarification, we would be happy to discuss it here. Thank you again for your time and feedback.

Best,

The Authors of Submission 5204

Reviewer_JzcWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

COPA compiles one-shot tool plans for execution and validates complete answer candidates already present in model responses. The paper reports large token savings with roughly comparable success on controlled Shopping tasks, plus zero-call plan repair and answer recovery. It also evaluates Travel and AppWorld.

Critical strengths

The paper addresses a practical source of agent cost: repeated model calls to relay tool results. Shared-tool comparisons show substantial token savings at near-parity Shopping success, and frozen-response studies help isolate the gains from repair and candidate processing.

Critical weaknesses

The evidence leaves the method’s broader value uncertain. The strongest results use fixed, domain-specific workflows with deterministic tools that do much of the search and scheduling; transfer to less structured tool use is unclear. Candidate recovery is demonstrated mainly on constructed Shopping stress tasks, while the paper does not establish how often this failure occurs in ordinary benchmark traces. On AppWorld, COPA uses fewer tokens but solves 78 tasks versus 85 for Simplified ReAct. The contribution is promising, but its generality and novelty beyond integrating useful components are not yet convincing.

Questions for authors

Can you report how often plan repair and candidate recovery help on unmodified benchmark trajectories, and whether the gains persist with less domain-specific tools?

4 comments
AuthorsOpenReview ↗

Follow-Up to Your Review

Dear Reviewer JzcW,

Thank you again for the time and effort you put into reviewing our work!

We understand that this might be a super busy period for you. We wanted to briefly let you know that we have posted point-by-point responses to your questions and concerns and uploaded a revised manuscript with the changes highlighted.

Whenever you have a chance, we would be grateful for your feedback on whether the responses address your concerns. We would be happy to clarify anything that remains open.

Thank you!

Best,

The Authors of Submission 5204

AuthorsOpenReview ↗

Follow-Up to Your Review (Second Reminder)

Dear Reviewer JzcW,

Thank you again for the time and care you put into reviewing our work!

We know this is a busy period, so we wanted to follow up briefly on our point-by-point responses and the revised manuscript, where we highlighted the changes for easy reference.

If you have a chance to look over our responses, we would appreciate hearing whether they address your main concerns. If anything remains unclear, we would be happy to clarify it here.

Thanks again for your support!

Sincerely yours,

The Authors of Submission 5204

AuthorsOpenReview ↗

Follow-Up to Your Review (Third Reminder)

Dear Reviewer JzcW,

We are following up on our point-by-point responses and the revised manuscript, which we posted several days ago. We believe these materials further address your concerns and highlight the changes we made in response to your feedback.

As the end of the Author-Reviewer Discussion phase is fast approaching, please let us know whether our responses address your main concerns or if any points still need clarification. We are available to discuss them here.

Thank you again for your time and effort devoted in this review!

Best,

The Authors of Submission 5204

AuthorsOpenReview ↗

Follow-Up to Your Review (Fourth Reminder)

Dear Reviewer JzcW,

As the discussion period draws to a close, we would like to briefly follow up on a point you raised in your review.

We appreciate your suggestion to measure how often answer recovery helps on ordinary benchmark tasks. On the 120 unmodified Shopping tasks, processing the saved responses increased the number of correct answers from 78 to 81. We then applied the same processing rules to a fresh batch of responses on those tasks and increased the count from 73 to 78. In both batches, the processor recovered answers without additional generation and preserved all previously correct answers.

If any part of our response would benefit from clarification, we would be happy to discuss it here. Thank you again for your time and feedback.

Best,

The Authors of Submission 5204

Reviewer_ZzkTOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a planning technique called COPA that allows models to produce tool plans or complete candidate answers that are validated but without further model calls. The main motivation is to reduce model calls and tokens (even if the tools calls increase as results show). The approach was run on Shopping, Travel, and AppWorld datasets, with different models and show that the approach saves on the model calls.

Critical strengths

The idea to reduce model calls is relevant and valuable to work on. They have also evaluated it with 3 domains/datasets and a few models showing the promise it holds.

Critical weaknesses
  • The evaluation seems specialized to a specific type of domains (web-related tasks) that have a proper schema which seems to have been leveraged in the approach.
  • If that is the case, it is unclear how well it generalizes to other domains and how much upfront work is needed to ensure we build/understand the domain before implementing or adopting the approach for a new domain, especially in an enterprise setting.
Questions for authors
  • While we see some gains in the reduction of model calls, what are the tradeoff? is the tradeoff the upfront effort to customize it (if needed) for a new domain/schema? There is some tradeoff seen from the results on reduced task accuracy in some cases.. Is that the tradeoff?
  • We see different results for Original Shopping vs. Constructed Shopping. In the original, COPA had the best solved rate, but that's not the case with Constructed Shopping? Do we know why? And could that be due to a poor schema as alluded to in the sec 5.1 paragraph on "Token savings extend to constructed instances, with varying task success" where "with clarified public schemas and execution guidance", the solved got better.
4 comments
AuthorsOpenReview ↗

Follow-Up to Your Review

Dear Reviewer ZzkT,

Thank you again for the time and effort you put into reviewing our work!

We understand that this might be a super busy period for you. We wanted to briefly let you know that we have posted point-by-point responses to your questions and concerns and uploaded a revised manuscript with the changes highlighted.

Whenever you have a chance, we would be grateful for your feedback on whether the responses address your concerns. We would be happy to clarify anything that remains open.

Thank you!

Best,

The Authors of Submission 5204

AuthorsOpenReview ↗

Follow-Up to Your Review (Second Reminder)

Dear Reviewer ZzkT,

Thank you again for the time and care you put into reviewing our work!

We know this is a busy period, so we wanted to follow up briefly on our point-by-point responses and the revised manuscript, where we highlighted the changes for easy reference.

If you have a chance to look over our responses, we would appreciate hearing whether they address your main concerns. If anything remains unclear, we would be happy to clarify it here.

Thanks again for your support!

Sincerely yours,

The Authors of Submission 5204

AuthorsOpenReview ↗

Follow-Up to Your Review (Third Reminder)

Dear Reviewer ZzkT,

We are following up on our point-by-point responses and the revised manuscript, which we posted several days ago. We believe these materials further address your concerns and highlight the changes we made in response to your feedback.

As the end of the Author-Reviewer Discussion phase is fast approaching, please let us know whether our responses address your main concerns or if any points still need clarification. We are available to discuss them here.

Thank you again for your time and effort devoted in this review!

Best,

The Authors of Submission 5204

AuthorsOpenReview ↗

Follow-Up to Your Review (Fourth Reminder)

Dear Reviewer ZzkT,

As the discussion period draws to a close, we would like to briefly follow up on a point you raised in your review.

We appreciate your question about why the results differ on Original and Constructed Shopping. We examined the four Qwen failures on Constructed Shopping and found that repairing JSON delimiters in the saved responses recovered all four, raising the success count from 116 to 120. The repair kept the proposed operations fixed and required no additional model calls. We report this analysis in Section 5.1 and Appendix C.6 of the revised manuscript.

If any part of our response would benefit from clarification, we would be happy to discuss it here. Thank you again for your time and feedback.

Best,

The Authors of Submission 5204

Comments on the paper

5 comments
AuthorsOpenReview ↗

Manuscript Updates

Thank you again for your thoughtful feedback. We have incorporated the following additions into the revised manuscript. New and revised material is highlighted in blue.

  • Applicability beyond Shopping-style domains: The Introduction and Section 5.1 now describe our m&m’s evaluation, covering text retrieval and question answering, audio transcription, segmentation, and image editing. Table 3 reports comparisons using shared official tools, and Appendix B.7 provides the experimental settings and scoring protocol.

  • Understanding the accuracy–efficiency trade-off: Section 5.1 and Appendix B.5.1 now include the AppWorld fixed-plan experiment. Correcting two execution issues increases success from 78 to 84 out of 90 tasks without additional planning calls, narrowing the gap to ReAct’s 85 from seven tasks to one.

  • Performance on constructed Shopping tasks: Section 5.1 and Appendix C.6 explain the four Qwen failures and report a fixed-response experiment that recovers them through JSON delimiter repair. Success increases from 116 to 120 without changing the proposed operations or making additional model calls. The existing Gemma analysis of schema clarification and execution guidance remains in Appendix B.3.

These additions clarify the method’s applicability across domains and provide concrete explanations for the observed differences in task success.

AuthorsOpenReview ↗

Manuscript Updates

Thank you again for your constructive feedback. We have incorporated additional evidence addressing your concerns into the revised manuscript. New and revised material is highlighted in blue.

  • Transfer to native multimodal tools: Section 5.1 and Table 3 now report results on m&m’s, covering text, image, and audio tasks using shared official tools. COPA solves 73 of 105 tasks, compared with 65 for ReAct and 51 for CodeAct, while using 47.87% and 73.94% fewer planner tokens, respectively.

  • Recovery on ordinary benchmark responses: Section 5.3 and Appendix D.3 report candidate processing on ordinary Shopping tasks. Correct returns increase from 78 to 81 in the initial batch and from 73 to 78 in a fresh response batch under fixed processing rules, without additional generation or losses of previously correct answers. The appendix includes candidate-selection controls and an analysis of the recovery mechanisms.

  • Understanding the AppWorld gap: Section 5.1 and Appendix B.5.1 report the fixed-plan execution corrections that increase AppWorld success from 78 to 84 out of 90 tasks without additional planning calls, narrowing the gap to ReAct’s 85 from seven tasks to one.

Together with the original frozen-response experiments, these additions strengthen the evidence for cross-domain transfer and recovery on ordinary tasks, and clarify the sources of the AppWorld performance gap.

AuthorsOpenReview ↗

Manuscript Updates

Thank you again for your detailed and helpful feedback. We have incorporated the following analyses into the revised manuscript. New and revised material is highlighted in blue.

  • Matched Travel comparisons: Section 5.1 and Appendix B.6.1 now include ReAct and CodeAct comparisons using the same tasks, scheduler, and tools. COPA, ReAct, and CodeAct solve 83, 83, and 82 tasks, respectively; COPA uses 54.0% and 61.1% fewer tokens than the two baselines. Table B15 reports the complete token totals, and Appendix B.6.1 specifies the model settings and interaction limits.

  • Non-LLM computation and end-to-end efficiency: Section 5.1 and Appendix B.2.1 report timing measurements for 443 choose calls, with cumulative function time of approximately 0.09–0.13 seconds per method. With tool execution included, COPA reduces mean task time by 34.6%, 43.5%, and 67.8% relative to ReAct, CodeAct, and LLMCompiler, respectively.

  • Candidate-processing reliability: Section 5.3 and Appendix D.3 add ordinary Shopping experiments, including candidate-selection controls, validation disagreements, and cases with no complete candidate. Table D26 shows that the final-answer-first processor improves correct returns from 78 to 81 and from 73 to 78 across two response batches, with no losses of previously correct answers.

  • Judgments that depend on tool observations: Appendix B.7.1 presents a controlled execution-feedback ablation on m&m’s, increasing success from 52 to 64 out of 105 tasks. The analysis describes recovery through interpretation of existing evidence, artifact selection, and additional retrieval.

These additions make the efficiency comparisons and the respective roles of compilation, candidate processing, and execution feedback more explicit.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.