acceptodds
Under review as a conference paper at ICLR 2027

ProgAgent: Learning Tool Orchestration through Programmatic Training

Abstract

Large language models (LLMs) are increasingly expected to orchestrate diverse tools over long-horizon tasks. Existing agents typically use Direct Tool Calling (DTC), where the model invokes tools sequentially, and tool responses accumulate in the model context, which can lead to context rot and orchestration breakdown. In contrast, Programmatic Tool Calling (PTC) invokes tools through executable code and stores intermediate results in runtime variables rather than appending them to the context. Prior work has explored PTC mainly as an inference-time optimization, but its benefits for agentic training remain unclear, and suitable verifiable training tasks are scarce. In this work, we introduce ProgAgent, a full PTC training stack, comprising a pipeline for synthesizing verifiable long-horizon tasks, a lightweight PTC runtime, and a two-stage training recipe. Using only 215 tasks instantiated at four environment scales, ProgAgent yields higher pass rates than the evaluated baselines of comparable size across four benchmarks. On LOCA-bench, ProgAgent-14B achieves a 30.7% pass rate at a 64k-token environment state, where these baselines score zero. Cross-harness experiments show that PTC-trained models outperform DTC-trained models under both evaluation interfaces, suggesting that the benefits of PTC training extend beyond the interface. We will release the model checkpoints and task synthesis pipeline.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agentic training, tool calling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kzaiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies Programmatic Tool Calling, where the agent calls tools by writing code inside a persistent sandbox, and proposes it as a training paradigm rather than only an inference-time trick. The authors build a synthesis pipeline that produces verifiable long-horizon tasks. Each task is instantiated at several environment sizes while the instruction, tools, and verifier stay fixed. They pair this pipeline with a lightweight IPython-based runtime and train Qwen3-8B/14B with cold-start SFT followed by GSPO. With a small task pool, the trained models beat comparable-size baselines on MCPMark, LOCA-bench, Toolathlon-Verified, and MCP-Atlas, and the gains are largest under heavy context growth. The paper also compares DTC, PTC, and Hybrid training under a shared setup. It reports that PTC-trained models outperform DTC-trained ones even when both are evaluated with direct tool calls.

Critical strengths
  1. The environment-scaling idea is simple and well executed. Because only the data size changes while the task logic stays fixed, it gives a clean way to control difficulty and isolate context rot, and the v0-only ablation shows it actually matters for training.
  2. Training DTC, Hybrid, and PTC on the same task pool and hyperparameters is the right experiment to run. The finding that base models do worst under PTC but best after training is a useful corrective to the assumption that a code interface helps on its own.
  3. The gains on long-context settings are large, not marginal, and the limitations section is unusually candid. The analysis of DTC-like behavior on Playwright and of silent MCP server errors will be useful to people building similar harnesses.
Critical weaknesses
  1. I am not convinced the DTC vs. PTC training comparison is fair. Tasks were filtered by the teacher’s pass rate under PTC and are designed around relational aggregation, so the pool favors PTC by construction. DTC’s RL reward stays around 0.1 throughout training, which with 16 rollouts per task means most groups likely produce no gradient. The current results show that DTC fails to learn on these tasks, but that is not the same as showing PTC is the better paradigm. A rerun with tasks filtered under DTC, or with the criterion-level reward used for DTC, would make the claim much more convincing.
  2. The positioning against prior work undersells how much has already been done. CodeAct trained models on code actions and also reported that the gains carried over to text-format actions. Recursive Language Models trained a Qwen3-8B with SFT and RL inside a persistent REPL. The contribution here is a careful controlled study in long-horizon MCP settings, which is valuable, but the paper reads as if PTC training itself were new.
  3. Some important reference points are missing. There is no number for the teacher model under PTC on the four benchmarks, so it is hard to tell how much of the improvement comes from learning the paradigm and how much from distilling a much stronger model. The “baselines under PTC” in Table 7 are small models that never saw a code interface, so their near-zero scores are expected and say little. Strong untrained models evaluated under PTC would be a more informative comparison.
  4. The cross-harness transfer result, which is one of the headline claims, rests on thin evidence. On LOCA-bench under DTC evaluation, the gaps between PTC-trained and DTC-trained models are a few points, which corresponds to a handful of tasks, and no variance or confidence intervals are reported. The MCP-Atlas gap is more convincing, but one benchmark is not enough to support a general claim about transfer.
Questions for authors
  1. What actually changes in the behavior of PTC-trained models when they are run under DTC? For example, do they make fewer tool calls, issue fewer redundant calls, or recover from errors differently? Some trajectory-level analysis would help explain why the transfer happens at all.
  2. In the Hybrid setting, how often does the trained model choose code over direct calls, and does that ratio change during RL? I wonder whether Hybrid underperforms because the model never learns when to switch, rather than because a unified interface is inherently easier to learn.
  3. Figure 4 counts only tool-response tokens. How do total context length and wall-clock time compare across the three paradigms once generated code and reasoning tokens are included?
6 comments
AuthorsOpenReview ↗

Response to Reviewer kzai (Part 1)

Thank you for the thorough and detailed review. We address each point below.

W1: I am not convinced the DTC vs. PTC training comparison is fair ... A rerun with tasks filtered under DTC, or with the criterion-level reward used for DTC, would make the claim much more convincing.

You are right that the task pool is calibrated under PTC. Our internal comparison shows that the interface changes what a model learns from the same pool. On its own, it cannot settle which action space is better in general. The paradigm claim therefore rests on three lines of evidence, and the first does not depend on our pool at all.

(i) Comparison with models trained by other groups. The baselines in Table 2 were trained by other groups on substantially larger corpora built for direct tool calling, and never saw our task pool, so the calibration concern does not apply to them. Each is evaluated under the interface it was trained for. At 8B:

8BMCPMarkLOCA-benchToolathlonMCP-AtlasTraining data
TOUCAN-7B1.20.00.07.01.53M
Simulator-8B2.00.00.00.8135K
EnvScaler-8B5.90.32.813.011.5k
AWM-8B3.00.01.98.03.3k
EnvFactory-8B5.41.01.911.82.5k
Agent-World-8B10.0–––2k+
ProgAgent-8B18.324.39.338.6860

The same ordering holds at 14B (Table 2). This comparison is not a statement about our DTC training. It is a statement about PTC training against the current direct-tool-calling training recipes at comparable model size, while training on 215 tasks, 860 task instances, compared with 2k+ to 1.53M tasks or trajectories for the trained baselines.

(ii) Transfer under a fixed interface. Under the same DTC harness PTC-trained models beat DTC-trained ones on MCP-Atlas, 30.6 vs. 22.2 at 8B and 41.6 vs 33.2 at 14B. The DTC-trained model here is the retrained one from (iii): it was trained on a pool refiltered under DTC with a dense reward and is evaluated under its own harness, while the PTC-trained model is evaluated off the interface it was trained on. Neither pool calibration nor the evaluation interface can explain a gap in that direction.

(iii) The control you ask for. We retrained the 8B and 14B DTC setting by applying our existing protocol to its own harness rather than to PTC. Concretely, the difficulty filter was recomputed with the teacher under DTC, retaining tasks whose pass rate lies in (0, 0.8) at every scale, exactly the criterion used in §3.1.3; and the binary RL reward was replaced with the criterion-level score so that partially correct trajectories also produce gradient. This is the symmetric counterpart of the PTC setting. All other settings follow C.3. We present DTC RL training dynamics at 8B in Figure 4 of the revised paper. As shown, the DTC criterion-level reward rises steadily from 0.16 to 0.45 at 8B (0.16 to 0.54 at 14B, Figure 10), so DTC training receives ample learning signal.

8BMCPMarkLOCA-benchToolathlonMCP-Atlas
DTC (original)6.04.02.819.0
DTC (retrained)6.65.02.322.2
PTC18.324.39.338.6
14BMCPMarkLOCA-benchToolathlonMCP-Atlas
DTC (original)8.09.05.629.7
DTC (retrained)10.110.74.433.2
PTC29.738.317.145.6

However, removing both sources of the asymmetry you identify moves the DTC training very little. Refiltering under DTC and switching to a dense reward changes MCPMark by +0.6(+2.1), LOCA-bench by +1.0(+1.7) and MCP-Atlas by +3.2(+3.5), with Toolathlon marginally lower; the ordering against PTC is unchanged on every benchmark. We therefore do not think the gap is an artifact of DTC being unable to learn on a pool calibrated for PTC.

AuthorsOpenReview ↗

Response to Reviewer kzai (Part 2)

W2: The positioning against prior work undersells how much has already been done. CodeAct trained models on code actions and also reported that the gains carried over to text-format actions. Recursive Language Models trained a Qwen3-8B with SFT and RL inside a persistent REPL... the paper reads as if PTC training itself were new.

Thank you for this point. We discuss both CodeAct and Recursive Language Models in Appendix A, but you are right that the paragraph there does not say that both works train models; the revised Appendix A states this explicitly. Our claim is not that PTC training itself is new. Our contribution is not the introduction of executable code as an action format in isolation; it is establishing, through controlled experiments, how training with this representation affects policy learning in long-horizon environments with live MCP tools. We clarify the distinction below.

Versus CodeAct [1]. CodeAct shows that executable Python is an effective action format and that code-action fine-tuning improves generalization to text-action tasks. Our claim is different in three respects.

First, the comparison differs. CodeAct compares its fine-tuned agents with their backbones and other open models.We train DTC, PTC, and Hybrid models from the same teacher with the same recipe and evaluate each under both interfaces. Crucially, PTC-trained models outperform DTC-trained models even under the same DTC evaluation harness (Table 6), isolating the effect of training representation.

Second, the learning regime differs. CodeAct uses SFT on 7k trajectories without RL, evaluated on relatively short tasks. Our finding concerns policy learning under RL in long-horizon environments, where tool outputs accumulate across turns: at 14B, DTC carries about 39k tool-output tokens per episode at the end of RL, while PTC keeps them below 20k (Figure 10).

Third, the action spaces differ. CodeAct builds capabilities from Python packages, whereas we expose existing MCP services unchanged. The latter lets agents operate on real services without having to implement their functionality correctly in generated code.

Versus Recursive Language Models (RLM) [2]. RLM treats a fixed user prompt as an environment, storing it in a REPL variable. Our agents instead interact with live MCP services, where information is retrieved through actions and the environment can be modified. Retrieving information from a fixed input and acting in a live environment are fundamentally different settings.

Our contributions are threefold. (1) Training action representation matters and generalizes across interfaces: PTC-trained models outperform DTC-trained models even under the DTC harness (Table 6). (2) Training, not the interface alone, drives the gains: untrained models perform worst under PTC in seven of eight cases but best after training (Table 3). (3) Accuracy with data efficiency: ProgAgent outperforms comparable-size baselines while training on only 215 tasks, versus 2k+ to 1.53M examples for trained baselines (Tables 1 and 2).

Using established components does not invalidate an empirical contribution. The key question is whether findings (1)–(3) were already established by prior work. We are not aware of any such work.

AuthorsOpenReview ↗

Response to Reviewer kzai (Part 3)

W3: There is no number for the teacher model under PTC on the four benchmarks... The "baselines under PTC" in Table 7 are small models that never saw a code interface, so their near-zero scores are expected and say little. Strong untrained models evaluated under PTC would be a more informative comparison.

On the teacher model under PTC across the four benchmarks. We thank the reviewer for pointing out this omission. We have added the missing results to the revised Table 2, reporting our teacher model, DeepSeek-V4 Pro, under both interfaces (mean pass rate):

DeepSeek-V4 ProMCPMarkLOCA-benchToolathlonMCP-Atlas
DTC74.786.869.074.2
PTC75.376.751.964.6

Without PTC training, the teacher is worse under PTC than under DTC on three of the four benchmarks: by 10.1 points on LOCA-bench, 17.1 on Toolathlon-Verified and 9.6 on MCP-Atlas, within 0.6 points on MCPMark. Our trained models reverse the ordering on these three benchmarks and lead under PTC on all four (Table 3). The code interface is not a free gain even for a frontier model; it becomes one through training, which is our finding (2).

On how much comes from distillation. The decomposition is reported: Figure 3 breaks every bar on LOCA-bench into the untrained backbone's score, the gain from SFT and the gain from RL, with exact values in Table 9. The teacher numbers above answer the question more directly. The same teacher, which is stronger under DTC than under PTC on LOCA-bench (86.8 vs 76.7), yields a DTC-trained 8B student at 3.0 after SFT and a PTC-trained one at 21.0 (Table 9): relative to the teacher's own score under each interface, the student recovers 27% under PTC and 3% under DTC. If the gain came from distilling a stronger model, the DTC-trained student would inherit more, not less; what the paradigm determines is how much of the teacher transfers. Cold-start SFT on teacher trajectories before RL is standard practice, as in the specialist distillation that precedes mixed RL in DeepSeek-V3.2 [4]; both settings receive it, and what we measure is what the paradigm adds on top of the same teacher and training recipe.

Strong untrained models under PTC. We agree that the near-zero scores of small trained baselines are not informative, and we accept this point. The teacher is exactly the strong model untrained under PTC that you ask for, and it shows that even a frontier model does not benefit from the interface on its own.

W4: The cross-harness transfer result... On LOCA-bench under DTC evaluation, the gaps are a few points, which correspond to a handful of tasks, and no variance or confidence intervals are reported. The MCP-Atlas gap is more convincing, but one benchmark is not enough.

On variance: Table 6 of the revised PDF now reports the standard deviation over four runs for every cell. Under DTC evaluation:

DTC evaluation (mean ± std, 4 runs)PTC-trainedDTC-trainedGap
MCP-Atlas, 8B30.6 ± 2.122.2 ± 1.88.4
MCP-Atlas, 14B41.6 ± 2.433.2 ± 2.08.4
LOCA-bench, 8B6.0 ± 0.45.0 ± 0.51.0
LOCA-bench, 14B13.9 ± 0.610.7 ± 0.53.2

In every cell the gap exceeds twice the run-to-run standard deviation of either model: about 2–2.5× on LOCA-bench at 8B (1.0 point), 3.5–4.7× on MCP-Atlas, and 5–6× on LOCA-bench at 14B. The 8B and 14B DTC-trained models are the retrained ones from W1 (iii).

On one benchmark not being enough: PTC-trained models are ahead of DTC-trained ones under DTC evaluation on both benchmarks at both model sizes. MCP-Atlas [3] is also a broad test rather than a single point: 36 production MCP servers and 220 tools across five application domains.

Reviewer_eJcXOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ProgAgent, a framework for training LLM agents to orchestrate tools through programmatic tool calling (PTC), where agents generate executable Python code to invoke predefined tools, process intermediate results, and manage execution state, rather than relying on direct tool calling (DTC). The framework consists of an automated pipeline for synthesizing verifiable agent tasks (215 tasks, each with 4 variants, across 23 domains), a persistent Python execution runtime, and a two-stage training recipe combining supervised fine-tuning (SFT) and reinforcement learning (RL) using GSPO. Experiments fine-tune Qwen3-8B and Qwen3-14B models and evaluate them on four agentic benchmarks: MCPMark, MCP-Atlas, Toolathlon-Verified, and LOCA-bench, showing that PTC-trained models achieve higher aggregate pass rates than DTC-trained baselines.

Critical strengths
  • The paper gives an interesting empirical observation that PTC-trained models outperform DTC-trained models even when both evaluated under the DTC harness. This suggests that the benefits of PTC training may extend beyond the inference-time advantages of programmatic tool calling, though the underlying mechanism is not sufficiently explained.
Critical weaknesses
  • The proposed framework is largely built upon established components, including synthetic verifiable environments, code-based tool orchestration, and standard multi-turn SFT/RL training. Programmatic tool calling itself is not new, and the paper does not introduce a substantially new training algorithm or agentic orchestration mechanism. The main contribution appears to be an empirical comparison of PTC and DTC training.

  • The paper primarily compares PTC against a basic ReAct-style DTC harness, which is not representative of modern general-purpose agent harnesses. If the main contribution is the empirical effectiveness of programmatic tool orchestration, comparisons with established coding agent harnesses that support flexible tool use, code execution, and persistent workspaces would be important to establish its practical advantages (particularly given the limited methodological novelty of the proposed framework).

Questions for authors
  • Could the authors clarify whether the observed advantages of PTC would hold against more sophisticated, general-purpose coding agent harnesses, rather than the basic ReAct-style (DTC) baseline used in this work?
  • Does the DTC baseline support parallel tool calling? Have the authors evaluated whether PTC provides better e2e execution efficiency (e.g., latency and tool-call overhead) compared to an optimized DTC harness with parallel tool calling?
2 comments
AuthorsOpenReview ↗

Response to Reviewer eJcX (Part 1)

Thank you for the review. We are glad you find the cross-harness result interesting, and we agree with your observation that its mechanism is under-explained; we address that below together with your two concerns.

W1: The proposed framework is largely built upon established components... Programmatic tool calling itself is not new, and the paper does not introduce a substantially new training algorithm or agentic orchestration mechanism. The main contribution appears to be an empirical comparison of PTC and DTC training.

This weakness is stated without reference to any specific prior work, so we answer it by naming the two closest candidates ourselves and stating precisely what remains after each is accounted for. We propose no new training algorithm and do not claim one; the contribution is a finding about training paradigms together with the instrument required to obtain it.

Versus CodeAct [1]. CodeAct shows that executable Python is an effective action format and that code-action fine-tuning improves generalization to text-action tasks. Our claim is different in three respects.

First, the comparison differs. CodeAct compares its fine-tuned agents with their backbones and other open models.We train DTC, PTC, and Hybrid models from the same teacher with the same recipe and evaluate each under both interfaces. Crucially, PTC-trained models outperform DTC-trained models even under the same DTC evaluation harness (Table 6), isolating the effect of training representation.

Second, the learning regime differs. CodeAct uses SFT on 7k trajectories without RL, evaluated on relatively short tasks. Our finding concerns policy learning under RL in long-horizon environments, where tool outputs accumulate across turns: tool outputs entering the context reach 39–44k tokens per episode under DTC, against 19–26k under PTC (§5.1).

Third, the action spaces differ. CodeAct builds capabilities from Python packages, whereas we expose existing MCP services unchanged. The latter lets agents operate on real services without having to implement their functionality correctly in generated code.

Versus Recursive Language Models (RLM) [2]. RLM treats a fixed user prompt as an environment, storing it in a REPL variable. Our agents instead interact with live MCP services, where information is retrieved through actions and the environment can be modified. Retrieving information from a fixed input and acting in a live environment are fundamentally different settings.

Our contributions are threefold. (1) Training action representation matters and generalizes across interfaces: PTC-trained models outperform DTC-trained models even under the DTC harness (Table 6). (2) Training, not the interface alone, drives the gains: untrained models perform worst under PTC in seven of eight cases but best after training (Table 3). (3) Accuracy with data efficiency: ProgAgent outperforms comparable-size baselines while training on only 215 tasks, versus 2k+ to 1.53M examples for trained baselines (Tables 1 and 2).

Using established components does not invalidate an empirical contribution. The key question is whether findings (1)–(3) were already established by prior work. We are not aware of any such work.

W2 : The paper primarily compares PTC against a basic ReAct-style DTC harness, which is not representative of modern general-purpose agent harnesses... important to establish its practical advantages.

On the facts: the DTC setting is not a ReAct-style harness, and the paper does not describe it as one. Appendix C.1 specifies that all three harnesses provide a terminal tool, alongside the task tools; PTC and Hybrid additionally provide a persistent sandboxed IPython kernel with variables preserved across turns (§3.2). Thus, code execution and persistent workspaces are already supported. The sole inference difference between DTC and PTC is whether environment tools are accessed directly or through code. This is precisely the variable we aim to isolate.

On the methodology, full-featured coding harnesses are not the right instrument for this comparison. They bundle additional mechanisms — memory, sub-agent delegation, retrieval, and context compaction — that independently affect training dynamics and would confound attribution. Controlled evaluation therefore requires holding these factors fixed. This is consistent with practice: DeepSeek Harness, for example, provides a minimal mode to isolate model capabilities [3]. Our findings are orthogonal to these additional capabilities: we study how the action space shapes policy learning, not the benefits of an entire harness stack.

AuthorsOpenReview ↗

Response to Reviewer eJcX (Part 2)

Q1: Could the authors clarify whether the observed advantages of PTC would hold against more sophisticated, general-purpose coding agent harnesses, rather than the basic ReAct-style (DTC) baseline used in this work?

Beyond our response to W2, we make one point concretely, because the paper already contains the relevant experiment. Memory mechanisms manage context accumulation; PTC reduces it at the source. The two are complementary, not interchangeable.

LOCA-bench provides a memory tool, and Table 9 evaluates its effect:

8B, LOCA-bench (pass rate)without memory toolwith memory tool
DTC-trained, DTC harness5.05.7
PTC-trained, DTC harness6.09.3
PTC-trained, PTC harness24.3—

Memory improves the 8B model under direct calling, but does not approach the PTC harness result (24.3). Memory compresses accumulated context; it does not eliminate the tool calls needed to gather information.

This distinction should persist across harnesses. For example, paginating through 1,000 pages still requires 1,000 direct calls, regardless of memory or sub-agent support. PTC can execute the same task within a single program. Consistent with this, PTC uses roughly a third as many tool calls per episode and fewer generated tokens (24.9k vs. 28.5k; §5.1).

We therefore expect PTC's advantages to persist within more sophisticated harnesses, although directly evaluating this combination remains valuable future work.

Q2: Does the DTC baseline support parallel tool calling? Have the authors evaluated whether PTC provides better e2e execution efficiency....

The DTC harness supports parallel tool calling without limiting the number of calls per assistant turn; whether the policy issues multiple calls together is a property of the policy, not the harness. Section 2.2 formalises a DTC action as a single ata_t for simplicity, which we now clarify in §2.2 and Appendix C.1. In practice, DTC policies already issue approximately four parallel calls per turn throughout RL training:

DTC RL, parallel tool calls per turnMeanRange over training
8B3.923.55–4.45
14B3.913.51–4.36

We additionally measure end-to-end task completion time on LOCA-bench, with concurrency and deployment held fixed. The table reports mean and median wall-clock time per task (seconds).

8B, time per task, mean / median (s)8k16k32k64k
DTC358 / 158624 / 273572 / 375785 / 512
Hybrid242 / 149381 / 223434 / 279445 / 275
PTC294 / 147403 / 245477 / 317602 / 355

PTC is consistently faster than DTC across all four environment-state sizes, in both mean and median completion time. At 64k, PTC reduces mean latency by 23.3% (602s vs. 785s) and median latency by 30.7% (355s vs. 512s), despite DTC already supporting substantial parallel tool calling.

[1] Executable code actions elicit better llm agents. Arxiv.

[2] Recursive language models. Arxiv.

[3] DeepSeek Harness: Everything is a Plugin. Github.


Reviewer_p6DSOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper compares two action spaces: Direct Tool Calling (DTC), where each invocation is a turn and its output enters the conversation, and Programmatic Tool Calling (PTC), where a program invokes tools and only printed output returns. It asks whether PTC is not just a cheaper interface but a better training paradigm, and supplies a synthesis pipeline with four environment scales, a persistent-kernel runtime, and a cold-start SFT plus GSPO recipe on 215 tasks, training Qwen3-8B/14B under DTC, PTC and Hybrid.

Critical strengths

The cross-harness design in Table 7 is the right instrument: training under one interface and evaluating under another is the only way to separate an interface effect from a training effect, and few papers in this area do it. Environment-state scaling is likewise clean, since instruction, tool schemas and verifier are held fixed while only the data generator's scale block changes (App. B.2), yielding the monotone DTC pass@8 drop from 0.95 to 0.42.

Reporting is above average: §5.4 discloses behaviour against interest, reproducibility material is complete, and the paper does not lean on a SOTA claim.

Critical weaknesses

The premise is that PTC's advantage needs demonstrating. It does not need much: holding tool returns in runtime variables and printing a summary is cheaper than appending them. DTC is also a bounded case of PTC — any sequence of direct calls can be written as a program with no control flow, which is what 40.7% of the paper's own PTC calls turn out to be (§5.4). So the three settings differ in how observations enter the context, a harness configuration rather than a property of the model or the task, and the result can only take the form "under this presentation, this configuration wins".

Practice does not look like any of the three settings. Deployed agents expose both interfaces side by side, and the mixture is forced by what the tools offer — some integrations ship only an MCP server, others are easier to drive from code — rather than by a judgement about which action space a model should be trained on. A paper arguing for the pure PTC setting is arguing against a default that exists for reasons its experiment never varies.

The tasks are built so that the conclusion follows from the construction. App. B.2 requires each tool to return bounded data "for example through pagination, so retrieving large amounts of data requires multiple calls", and §3.1.1 selects tasks as "tool orchestration patterns that can be naturally expressed through programs". With paginated tools and bulk aggregation, a loop beats forty turns of reading JSON with no training involved. Task shapes where DTC has the advantage — exploratory debugging, UI interaction, decisions that depend on the last observation — are absent.

The comparison is also not trainable on equal terms. Difficulty is filtered to a pass rate in (0, 0.8) under the PTC harness with a much stronger model (§3.1.3), so the pool is calibrated for PTC. The DTC setting's reward runs 0.08→0.15 at 14B (Fig. 4) and 0.05→0.08 at 8B, leaving many rollout groups without gradient at 16 rollouts per task. On LOCA-bench both settings start on the floor — 0.3% for DTC, 0.0% for PTC (Table 7) — so the 24.3 against 4.0 gap measures which setting can learn at all rather than which action space is better. What gain there is comes mostly from distillation: 21.0 of 24.3 at 8B is already present after SFT. App. C.3 equalizes the number of trajectories, not their quality, and reports no token counts.

Two reporting issues remain. Table 2 imports baseline scores from other papers while ProgAgent's come from the authors' harness (App. C.4), and no variance is reported although several load-bearing margins are two to five points; RL even reduces 8B PTC at 64k (20.0→17.3). Error handling appears in only 8-10% of code blocks (Table 5) and costs are unmeasured.

Questions for authors

See Weakness

5 comments
AuthorsOpenReview ↗

Response to Reviewer p6DS (Part 1)

Thank you for the thorough and detailed review. We address each point below.

W1: DTC is also a bounded case of PTC... the three settings differ in how observations enter the context, a harness configuration rather than a property of the model or the task, and the result can only take the form "under this presentation, this configuration wins".

We respectfully disagree that the result reduces to a presentation effect, and the instrument you endorse in the strengths is what rules it out. If the three settings differed only in how observations enter the context, fixing the evaluation interface should erase the difference between models. It does not. Under the same DTC harness, PTC-trained models outperform DTC-trained models (Table 6): on MCP-Atlas, 30.6 vs 22.2 at 8B. (The DTC number is the retrained DTC setting described in our response to W4, i.e. the strongest DTC configuration we were able to train.) What the PTC-trained model acquired survives the removal of the programmatic interface, which makes it a property of the model rather than of the presentation.

We agree that DTC is formally a subset of PTC. That is what makes the comparison informative: the question is whether training with the additional expressiveness changes what the model learns, not whether the expressiveness exists. During RL, 82% of generated code blocks use loops and 65% use conditionals (Table 5), so the model does learn to use it. The 40.7% figure is specific to the Playwright suite (Table 10); on the other three MCPMark suites the DTC-like fraction is 10–19% at 14B (6–18% at 8B). We return to Playwright under W3.

On the scope of the conclusion: our evaluation benchmarks are publicly released and were not constructed for this study or around any interface. Toolathlon spans 32 applications and 604 tools over seven domains, from Kubernetes deployment to Canvas and GitHub workflows [1]. It has been adopted by DeepSeek [2] and OpenAI [3] as a standard general-agent evaluation suite. MCP-Atlas spans 36 production MCP servers and 220 tools, with prompts that name neither servers nor tools [4], and has also been adopted by Claude [5]. On these benchmarks ProgAgent outperforms every evaluated model of comparable size while training on 215 tasks, against 2k+ to 1.5M examples for those baselines (Tables 1 and 2). A presentation artifact would not survive that comparison.

W2: Practice does not look like any of the three settings... the mixture is forced by what the tools offer... A paper arguing for the pure PTC setting is arguing against a default that exists for reasons its experiment never varies.

We believe this premise is factually inaccurate. Whether an agent calls tools directly or through code is determined by the harness, not by what the tools offer. Several production systems already ship a pure-PTC configuration in which every tool call must go through code. For example, OpenAI's Codex CLI provides a Code Mode in which MCP tool calls can only be issued from model-generated code scripts [6]; and DeepSeek Harness ships a dedicated PTC preset mode in which the model writes a program against a generated SDK rather than issuing native tool calls [7]. In all of these, MCP servers are driven from code with no change on the server side, exactly as in our runtime (§3.2), where a tool proxy exposes every environment tool as a Python callable. The pure PTC setting is therefore a deployed configuration rather than a hypothetical one, and the variable our experiment manipulates, the harness, is the variable that determines the interface in practice.

AuthorsOpenReview ↗

Response to Reviewer p6DS (Part 2)

W3: The tasks are built so that the conclusion follows from the construction... Task shapes where DTC has the advantage — exploratory debugging, UI interaction, decisions that depend on the last observation — are absent.

We agree that our training tasks are built around relational aggregation, as stated in §3.1. But the relevant question is whether the observed advantage is an artifact of this task construction. The evidence says it is not: the advantage persists on independent benchmarks outside our task synthesis pipeline. The cleanest evidence is the comparison across training paradigms on Toolathlon-Verified and MCP-Atlas (Table 3), where the three settings share training hyperparameters and evaluation protocol. The DTC column reports the retrained DTC model described in our response to W4, trained on a pool refiltered under DTC with a dense reward:

8BDTCPTC
Toolathlon-Verified2.39.3
MCP-Atlas22.238.6

The results demonstrate that the PTC advantage extends well beyond paginated aggregation tasks. At 8B, PTC achieves 9.3 vs. 2.3 for DTC on Toolathlon (+7.0), and 38.6 vs. 22.2 on MCP-Atlas (+16.4). At 14B, PTC leads by +12.7 on Toolathlon and +12.4 on MCP-Atlas. These results are particularly significant given the diversity of these benchmarks as stated in the response of W1. These are not simply tasks involving paginated data retrieval and bulk aggregation. The same performance ordering also holds on MCPMark and LOCA-bench (Table 3).

Your point about UI interaction is well taken, and we answer it directly. Across the four benchmarks there are 14 benchmark-by-size cells; PTC is ahead of DTC in 12, tied in 2, and behind in none. We never claimed PTC wins everywhere, and we do not need that claim. A method that is better on most of the distribution and never worse on the rest is advantageous, and establishing that is sufficient. §5.4 of the submitted paper already analyses this tie: Playwright returns unstructured page snapshots, so the next call depends on reading the last observation and PTC calls degenerate into DTC-like single calls, 34.7% at 8B and 40.7% at 14B against 6–19% on the other three suites (Table 10).

AuthorsOpenReview ↗

Response to Reviewer p6DS (Part 3-1)

W4: The comparison is also not trainable on equal terms. Difficulty is filtered... under the PTC harness with a much stronger model, so the pool is calibrated for PTC... the 24.3 against 4.0 gap measures which setting can learn at all rather than which action space is better. What gain there is comes mostly from distillation... App. C.3 equalizes the number of trajectories, not their quality.

You are right that the task pool is calibrated under PTC. Our internal comparison shows that the interface changes what a model learns from the same pool. On its own, it cannot settle which action space is better in general. The paradigm claim therefore rests on three lines of evidence, and the first does not depend on our pool at all.

(i) Comparison with models trained by other groups. The baselines in Table 2 were trained by other groups on substantially larger corpora built for direct tool calling, and never saw our task pool, so the calibration concern does not apply to them. Each is evaluated under the interface it was trained for. At 8B:

8BMCPMarkLOCA-benchToolathlonMCP-AtlasTraining data
TOUCAN-7B1.20.00.07.01.53M
Simulator-8B2.00.00.00.8135K
EnvScaler-8B5.90.32.813.011.5k
AWM-8B3.00.01.98.03.3k
EnvFactory-8B5.41.01.911.82.5k
Agent-World-8B10.0–––2k+
ProgAgent-8B18.324.39.338.6860

The same ordering holds at 14B (Table 2). This comparison is not a statement about our DTC training. It is a statement about PTC training against the current direct-tool-calling training recipes at comparable model size, while training on 215 tasks, 860 task instances, compared with 2k+ to 1.53M tasks or trajectories for the trained baselines.

(ii) Transfer under a fixed interface. As in W1, under the same DTC harness PTC-trained models beat DTC-trained ones on MCP-Atlas, 30.6 vs. 22.2 at 8B and 41.6 vs 33.2 at 14B. The DTC-trained model here is the retrained one from (iii): it was trained on a pool refiltered under DTC with a dense reward and is evaluated under its own harness, while the PTC-trained model is evaluated off the interface it was trained on. Neither pool calibration nor the evaluation interface can explain a gap in that direction.

(iii) The control you ask for. We retrained the 8B and 14B DTC setting by applying our existing protocol to its own harness rather than to PTC. Concretely, the difficulty filter was recomputed with the teacher under DTC, retaining tasks whose pass rate lies in (0, 0.8) at every scale, exactly the criterion used in §3.1.3; and the binary RL reward was replaced with the criterion-level score so that partially correct trajectories also produce gradient. This is the symmetric counterpart of the PTC setting. All other settings follow C.3. We present DTC RL training dynamics at 8B in Figure 4 of the revised paper. As shown, the DTC criterion-level reward rises steadily from 0.16 to 0.45 at 8B (0.16 to 0.54 at 14B, Figure 10), so DTC training receives ample learning signal.

8BMCPMarkLOCA-benchToolathlonMCP-Atlas
DTC (original)6.04.02.819.0
DTC (retrained)6.65.02.322.2
PTC18.324.39.338.6
14BMCPMarkLOCA-benchToolathlonMCP-Atlas
DTC (original)8.09.05.629.7
DTC (retrained)10.110.74.433.2
PTC29.738.317.145.6

However, removing both sources of the asymmetry you identify moves the DTC training very little. Refiltering under DTC and switching to a dense reward changes MCPMark by +0.6(+2.1), LOCA-bench by +1.0(+1.7) and MCP-Atlas by +3.2(+3.5), with Toolathlon marginally lower; the ordering against PTC is unchanged on every benchmark. We therefore do not think the gap is an artifact of DTC being unable to learn on a pool calibrated for PTC.

Comments on the paper

1 comment

Dear authors and reviewers,

The author-reviewer discussion period closes on November 18 (AoE). Please engage in response periods so that we have our issues discussed thoroughly.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.