- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies Programmatic Tool Calling, where the agent calls tools by writing code inside a persistent sandbox, and proposes it as a training paradigm rather than only an inference-time trick. The authors build a synthesis pipeline that produces verifiable long-horizon tasks. Each task is instantiated at several environment sizes while the instruction, tools, and verifier stay fixed. They pair this pipeline with a lightweight IPython-based runtime and train Qwen3-8B/14B with cold-start SFT followed by GSPO. With a small task pool, the trained models beat comparable-size baselines on MCPMark, LOCA-bench, Toolathlon-Verified, and MCP-Atlas, and the gains are largest under heavy context growth. The paper also compares DTC, PTC, and Hybrid training under a shared setup. It reports that PTC-trained models outperform DTC-trained ones even when both are evaluated with direct tool calls.
Critical strengths
- The environment-scaling idea is simple and well executed. Because only the data size changes while the task logic stays fixed, it gives a clean way to control difficulty and isolate context rot, and the v0-only ablation shows it actually matters for training.
- Training DTC, Hybrid, and PTC on the same task pool and hyperparameters is the right experiment to run. The finding that base models do worst under PTC but best after training is a useful corrective to the assumption that a code interface helps on its own.
- The gains on long-context settings are large, not marginal, and the limitations section is unusually candid. The analysis of DTC-like behavior on Playwright and of silent MCP server errors will be useful to people building similar harnesses.
Critical weaknesses
- I am not convinced the DTC vs. PTC training comparison is fair. Tasks were filtered by the teacher’s pass rate under PTC and are designed around relational aggregation, so the pool favors PTC by construction. DTC’s RL reward stays around 0.1 throughout training, which with 16 rollouts per task means most groups likely produce no gradient. The current results show that DTC fails to learn on these tasks, but that is not the same as showing PTC is the better paradigm. A rerun with tasks filtered under DTC, or with the criterion-level reward used for DTC, would make the claim much more convincing.
- The positioning against prior work undersells how much has already been done. CodeAct trained models on code actions and also reported that the gains carried over to text-format actions. Recursive Language Models trained a Qwen3-8B with SFT and RL inside a persistent REPL. The contribution here is a careful controlled study in long-horizon MCP settings, which is valuable, but the paper reads as if PTC training itself were new.
- Some important reference points are missing. There is no number for the teacher model under PTC on the four benchmarks, so it is hard to tell how much of the improvement comes from learning the paradigm and how much from distilling a much stronger model. The “baselines under PTC” in Table 7 are small models that never saw a code interface, so their near-zero scores are expected and say little. Strong untrained models evaluated under PTC would be a more informative comparison.
- The cross-harness transfer result, which is one of the headline claims, rests on thin evidence. On LOCA-bench under DTC evaluation, the gaps between PTC-trained and DTC-trained models are a few points, which corresponds to a handful of tasks, and no variance or confidence intervals are reported. The MCP-Atlas gap is more convincing, but one benchmark is not enough to support a general claim about transfer.
Questions for authors
- What actually changes in the behavior of PTC-trained models when they are run under DTC? For example, do they make fewer tool calls, issue fewer redundant calls, or recover from errors differently? Some trajectory-level analysis would help explain why the transfer happens at all.
- In the Hybrid setting, how often does the trained model choose code over direct calls, and does that ratio change during RL? I wonder whether Hybrid underperforms because the model never learns when to switch, rather than because a unified interface is inherently easier to learn.
- Figure 4 counts only tool-response tokens. How do total context length and wall-clock time compare across the three paradigms once generated code and reasoning tokens are included?
6 comments
Response to Reviewer kzai (Part 2)
W2: The positioning against prior work undersells how much has already been done. CodeAct trained models on code actions and also reported that the gains carried over to text-format actions. Recursive Language Models trained a Qwen3-8B with SFT and RL inside a persistent REPL... the paper reads as if PTC training itself were new.
Thank you for this point. We discuss both CodeAct and Recursive Language Models in Appendix A, but you are right that the paragraph there does not say that both works train models; the revised Appendix A states this explicitly. Our claim is not that PTC training itself is new. Our contribution is not the introduction of executable code as an action format in isolation; it is establishing, through controlled experiments, how training with this representation affects policy learning in long-horizon environments with live MCP tools. We clarify the distinction below.
Versus CodeAct [1]. CodeAct shows that executable Python is an effective action format and that code-action fine-tuning improves generalization to text-action tasks. Our claim is different in three respects.
First, the comparison differs. CodeAct compares its fine-tuned agents with their backbones and other open models.We train DTC, PTC, and Hybrid models from the same teacher with the same recipe and evaluate each under both interfaces. Crucially, PTC-trained models outperform DTC-trained models even under the same DTC evaluation harness (Table 6), isolating the effect of training representation.
Second, the learning regime differs. CodeAct uses SFT on 7k trajectories without RL, evaluated on relatively short tasks. Our finding concerns policy learning under RL in long-horizon environments, where tool outputs accumulate across turns: at 14B, DTC carries about 39k tool-output tokens per episode at the end of RL, while PTC keeps them below 20k (Figure 10).
Third, the action spaces differ. CodeAct builds capabilities from Python packages, whereas we expose existing MCP services unchanged. The latter lets agents operate on real services without having to implement their functionality correctly in generated code.
Versus Recursive Language Models (RLM) [2]. RLM treats a fixed user prompt as an environment, storing it in a REPL variable. Our agents instead interact with live MCP services, where information is retrieved through actions and the environment can be modified. Retrieving information from a fixed input and acting in a live environment are fundamentally different settings.
Our contributions are threefold. (1) Training action representation matters and generalizes across interfaces: PTC-trained models outperform DTC-trained models even under the DTC harness (Table 6). (2) Training, not the interface alone, drives the gains: untrained models perform worst under PTC in seven of eight cases but best after training (Table 3). (3) Accuracy with data efficiency: ProgAgent outperforms comparable-size baselines while training on only 215 tasks, versus 2k+ to 1.53M examples for trained baselines (Tables 1 and 2).
Using established components does not invalidate an empirical contribution. The key question is whether findings (1)–(3) were already established by prior work. We are not aware of any such work.
Response to Reviewer kzai (Part 3)
W3: There is no number for the teacher model under PTC on the four benchmarks... The "baselines under PTC" in Table 7 are small models that never saw a code interface, so their near-zero scores are expected and say little. Strong untrained models evaluated under PTC would be a more informative comparison.
On the teacher model under PTC across the four benchmarks. We thank the reviewer for pointing out this omission. We have added the missing results to the revised Table 2, reporting our teacher model, DeepSeek-V4 Pro, under both interfaces (mean pass rate):
| DeepSeek-V4 Pro | MCPMark | LOCA-bench | Toolathlon | MCP-Atlas |
|---|---|---|---|---|
| DTC | 74.7 | 86.8 | 69.0 | 74.2 |
| PTC | 75.3 | 76.7 | 51.9 | 64.6 |
Without PTC training, the teacher is worse under PTC than under DTC on three of the four benchmarks: by 10.1 points on LOCA-bench, 17.1 on Toolathlon-Verified and 9.6 on MCP-Atlas, within 0.6 points on MCPMark. Our trained models reverse the ordering on these three benchmarks and lead under PTC on all four (Table 3). The code interface is not a free gain even for a frontier model; it becomes one through training, which is our finding (2).
On how much comes from distillation. The decomposition is reported: Figure 3 breaks every bar on LOCA-bench into the untrained backbone's score, the gain from SFT and the gain from RL, with exact values in Table 9. The teacher numbers above answer the question more directly. The same teacher, which is stronger under DTC than under PTC on LOCA-bench (86.8 vs 76.7), yields a DTC-trained 8B student at 3.0 after SFT and a PTC-trained one at 21.0 (Table 9): relative to the teacher's own score under each interface, the student recovers 27% under PTC and 3% under DTC. If the gain came from distilling a stronger model, the DTC-trained student would inherit more, not less; what the paradigm determines is how much of the teacher transfers. Cold-start SFT on teacher trajectories before RL is standard practice, as in the specialist distillation that precedes mixed RL in DeepSeek-V3.2 [4]; both settings receive it, and what we measure is what the paradigm adds on top of the same teacher and training recipe.
Strong untrained models under PTC. We agree that the near-zero scores of small trained baselines are not informative, and we accept this point. The teacher is exactly the strong model untrained under PTC that you ask for, and it shows that even a frontier model does not benefit from the interface on its own.
W4: The cross-harness transfer result... On LOCA-bench under DTC evaluation, the gaps are a few points, which correspond to a handful of tasks, and no variance or confidence intervals are reported. The MCP-Atlas gap is more convincing, but one benchmark is not enough.
On variance: Table 6 of the revised PDF now reports the standard deviation over four runs for every cell. Under DTC evaluation:
| DTC evaluation (mean ± std, 4 runs) | PTC-trained | DTC-trained | Gap |
|---|---|---|---|
| MCP-Atlas, 8B | 30.6 ± 2.1 | 22.2 ± 1.8 | 8.4 |
| MCP-Atlas, 14B | 41.6 ± 2.4 | 33.2 ± 2.0 | 8.4 |
| LOCA-bench, 8B | 6.0 ± 0.4 | 5.0 ± 0.5 | 1.0 |
| LOCA-bench, 14B | 13.9 ± 0.6 | 10.7 ± 0.5 | 3.2 |
In every cell the gap exceeds twice the run-to-run standard deviation of either model: about 2–2.5× on LOCA-bench at 8B (1.0 point), 3.5–4.7× on MCP-Atlas, and 5–6× on LOCA-bench at 14B. The 8B and 14B DTC-trained models are the retrained ones from W1 (iii).
On one benchmark not being enough: PTC-trained models are ahead of DTC-trained ones under DTC evaluation on both benchmarks at both model sizes. MCP-Atlas [3] is also a broad test rather than a single point: 36 production MCP servers and 220 tools across five application domains.
Response to Reviewer kzai (Part 1)
Thank you for the thorough and detailed review. We address each point below.
You are right that the task pool is calibrated under PTC. Our internal comparison shows that the interface changes what a model learns from the same pool. On its own, it cannot settle which action space is better in general. The paradigm claim therefore rests on three lines of evidence, and the first does not depend on our pool at all.
(i) Comparison with models trained by other groups. The baselines in Table 2 were trained by other groups on substantially larger corpora built for direct tool calling, and never saw our task pool, so the calibration concern does not apply to them. Each is evaluated under the interface it was trained for. At 8B:
The same ordering holds at 14B (Table 2). This comparison is not a statement about our DTC training. It is a statement about PTC training against the current direct-tool-calling training recipes at comparable model size, while training on 215 tasks, 860 task instances, compared with 2k+ to 1.53M tasks or trajectories for the trained baselines.
(ii) Transfer under a fixed interface. Under the same DTC harness PTC-trained models beat DTC-trained ones on MCP-Atlas, 30.6 vs. 22.2 at 8B and 41.6 vs 33.2 at 14B. The DTC-trained model here is the retrained one from (iii): it was trained on a pool refiltered under DTC with a dense reward and is evaluated under its own harness, while the PTC-trained model is evaluated off the interface it was trained on. Neither pool calibration nor the evaluation interface can explain a gap in that direction.
(iii) The control you ask for. We retrained the 8B and 14B DTC setting by applying our existing protocol to its own harness rather than to PTC. Concretely, the difficulty filter was recomputed with the teacher under DTC, retaining tasks whose pass rate lies in (0, 0.8) at every scale, exactly the criterion used in §3.1.3; and the binary RL reward was replaced with the criterion-level score so that partially correct trajectories also produce gradient. This is the symmetric counterpart of the PTC setting. All other settings follow C.3. We present DTC RL training dynamics at 8B in Figure 4 of the revised paper. As shown, the DTC criterion-level reward rises steadily from 0.16 to 0.45 at 8B (0.16 to 0.54 at 14B, Figure 10), so DTC training receives ample learning signal.
However, removing both sources of the asymmetry you identify moves the DTC training very little. Refiltering under DTC and switching to a dense reward changes MCPMark by +0.6(+2.1), LOCA-bench by +1.0(+1.7) and MCP-Atlas by +3.2(+3.5), with Toolathlon marginally lower; the ordering against PTC is unchanged on every benchmark. We therefore do not think the gap is an artifact of DTC being unable to learn on a pool calibrated for PTC.