- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper aims to improve research on agentic midtraining for multimodal trajectories. To do this, the authors extend (while maintaining compatibility) the Agent Data Protocol (ADP), which currently only covers text, developing Multimodal ADP (MMADP). This protocol allows them to standardize and combine 16 existing datasets, building a single corpus of 14.7M trajectories (366B tokens). They then perform SFT on a subset of this dataset at two different model sizes (8B and 26B), showing improvements on various task categories like coding, web and computer use. The authors commit to releasing both the corpus and the infrastructure needed to extend this dataset.
Critical strengths
- S1: Valuable open resource for midtraining research, also for lower-resourced labs.
The authors commit to release schema, converters and the whole dataset. This could allow lower-resourced research labs to benefit for such a contribution. - S2: Extensible to new datasets. The approach is highly extensible to new datasets by building a converter for each.
Critical weaknesses
- W1: Experiments test SFT on a post-trained model, the paper is about midtraining.
The main paper motivation is around improving multimodal agent midtraining. However, all the evaluations are presented against a post-trained model using SFT. While this is not necessarily an issue for the dataset evaluation, for mixed-domain test, it's not clear the same conclusions would hold. - W2: The large-corpus claim rests on WildGUI
WildGUI represents 12.70M of the 14.71M trajectories (340.06B out of 366.35B tokens) . During training the authors subsample in different ratios (section 4.1) to avoid "one dataset from dominating". This means that effectively, in order to use this dataset in the future it'll be necessary to either (1) upsampling the other splits by several orders of magnutude or (2) allow for unbalanced corpus. - W3: Conversion cost is not reported. The cost of the conversion process is not discussed in the manuscript, no code is available so it's not entirely clear how computationally demanding this stage could be.
Questions for authors
- Q1: can the authors clarify and extend their discussion about the mixed-domain advantages? Can they also extend and clarify why they think their evaluation setup can be representative for the actual goal this dataset was built for (why is their evaluation a good proxy for it?)
- Q2: can the authors clarify how the imbalance raised by W2 could be managed?
- Q3: what is the conversion cost for "building" this dataset? Would potential users incur in this cost in order to adopt it?
3 comments
W3: Conversion cost is not reported.
W3: ...no code is available so it's not entirely clear how computationally demanding this stage could be.
Q3: what is the conversion cost for "building" this dataset? Would potential users incur in this cost in order to adopt it?
We agree that this cost should be reported, and provide the measurements in Table R4. All 16 sources convert in 16 h 46 min of total wall time, on machines with at most 8 cores and 64GB RAM, and no GPU. Users of the hosted training subset (106,674 trajectories, Section 3.6) incur none of this cost, and we will also host a converted WildGUI slice of the same size as the other sources (12,000 trajectories). For the full conversion of any other source, a user runs its converter on the source release. The converters, the QA checks, and the selection code are part of the release (Reproducibility Statement). We are unable to share a link at this time due to anonymity.
Sources
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (Chen et al., 2024)
- GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models (GLM-4.5 Team, 2025)
- Midtraining Bridges Pretraining and Posttraining Distributions (Liu et al., ICML 2026b)
- A Survey on LLM Mid-Training (Tu et al., 2025)
- GUI-CIDER: Mid-training GUI Agents via Causal Internalization and Density-aware Exemplar Reselection (Wu et al., 2026)
- Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe (Singhal et al., 2026)
- GLM-5: from Vibe Coding to Agentic Engineering (GLM-5 Team, 2026)
- Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents (Song et al., ICLR 2026)
We thank the reviewer again for the positive assessment. As the discussion period closes today, we would like to remind the reviewer that our response is posted above. We would be glad to know whether our answers on midtraining, the balance of the corpus, and the conversion cost (Table R4) address the reviewer's questions. If they do, we would appreciate it if the reviewer would consider raising the score. If any question remains, please let us know, and we will respond promptly.
We greatly appreciate the reviewer's positive assessment, and the recognition of MMADP as a valuable open resource for midtraining research, in particular for lower-resourced labs, and of its extensibility to new datasets through one converter each. We appreciate the questions regarding our evaluation setup, the balance of the corpus, and the cost of conversion, and we are happy to clarify each below.
We appreciate this point, as it was a decision we spent some time on. We believe our experiment is a reasonable proxy for midtraining for two reasons: the base model has not seen data in a tool-calling or agentic format, and our setup follows published midtraining choices in its starting checkpoint, loss, and scale. InternVL2.5 predates agentic midtraining of many open-source models, such as GLM-4.5 (GLM-4.5 Team, 2025), and its report describes its training data as conversation-format instruction data, with no mention of tool calls (Chen et al., 2024). Accordingly, before training, only 4.0% of the 8B base's SWE-bench Verified tool calls and 4.8% of its AndroidWorld steps are valid (Table 3). This aligns with Liu et al. (2026b), who found that "midtraining benefits are largest for domains distant from general pretraining data, such as code and math."
Secondly, midtraining has no standard recipe. The concept remains "theoretically nebulous", with different papers using the term for different setups (Tu et al., 2025). Published midtraining runs vary widely, and our configuration matches a published choice in each respect:
Our runs follow a published choice on each of these, which is why we consider them a reasonable small-scale proxy.
Please see Section 5.2: the mixed-domain model beats each single-domain model on that model's own benchmark while seeing about a third of that domain's tokens. We now also show that each of these gaps holds under a paired bootstrap test (p = 0.041, 0.026, and 0.007; Table R1).
One concrete behavior that mixed-domain training improves is premature termination, where the agent ends an episode before making progress on the task. Outside its domain, the coding-only model ends 83 of the 99 scored MiniWoB++ episodes at step 1 with a
finishcall (Section 5.2). Inside its domain, it ends 281 and 129 of the 500 SWE-bench Verified instances with an empty patch in its two runs, against 2 and 1 for the mixed-domain model, respectively.Thank you for raising this point. There is a third option, which the paper itself uses: downsampling the large sources (Section 4.1). Our 14 training sources already span 114 to 1,045,845 trajectories (Table R2), and sampling them in proportion to their size would give AITW alone 53% of the tokens (Section 4.1). We instead draw each source to a hand-set share, and the resulting mixture improves all three benchmarks over the 8B base (p < 0.01 each, Table R1). WildGUI can enter a mixture in the same way, at any share that the user sets.
We release the full conversion so that this choice (along with the two provided by the reviewer) stays with the user because midtraining mixtures remain an area of open inquiry. For example, GLM-5 up-samples agent trajectories within its midtraining data (GLM-5 Team, 2026). We also expect the set of sources to grow, as it has for ADP: ADP v1 was published with 13 datasets (Song et al., 2026), and its public repository now holds 56.