- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
JAM is a framework for pretraining world-action models across embodiments. Its main ingredients are reciprocal computation between separate world and action streams and joint prediction of native robot actions and explicit image-space motion tracks alongside future video. The model uses established MM-DiT-style joint attention, while language conditions both streams through cross-attention. Motion tracks represent displacement from current-frame anchors together with observability, providing a shared visual-motion target for heterogeneous robot data and human videos without executable action labels. Independently corrupted targets and masked objectives accommodate partial supervision. A shared pretrained checkpoint is adapted to seven simulation benchmark families and five real-world Franka tasks. The paper also studies coupling designs, track-label coverage and spatial support, scaling, and future-video K/V dropout for action-only inference.
Critical strengths
-
Joint prediction of native actions and explicit visual motion is a useful contribution. Motion tracks describe displacement from current-frame anchors together with observability, while native actions retain each robot’s executable control coordinates. Predicting these targets jointly with video provides an explicit supervision interface between controls and their visible effects, and allows human hand-motion data to contribute without inventing robot-action labels. Figures 4B–C show that track-label coverage and spatial support affect downstream success, with end-effector points plus a moderate scene grid giving the strongest reported geometric-control result. This is substantive evidence for the usefulness of motion-track supervision.
-
Broad practical validation of a shared checkpoint. The model is adapted to single-arm, bimanual, mobile, and humanoid settings and is competitive on several benchmarks. The real-world experiment uses the same 50 demonstrations per task and 20,000 LoRA updates for all compared models, with JAM achieving 71/100 successes versus 52/100 for π0.5 and 49/100 for LingBot-VA (Table 9; Appendix A.5). This supports the practical value of the complete recipe, even though its robustness is not uniformly superior.
Critical weaknesses
-
The specific cross-embodiment contribution of motion tracks remains insufficiently isolated. Figures 4B–C support the utility of track supervision overall and show that coverage and spatial support matter. However, they do not isolate the additional benefit of human track labels at fixed human-video exposure. Figure 6’s video-prior comparison changes the complete world–action pretraining recipe, so its transfer gains cannot be attributed specifically to the shared motion-track interface. Thus, the evidence establishes the usefulness of geometric supervision more clearly than the claimed role of human tracks in learning transferable world–action correspondence. This matters because bridging native robot controls and action-free human motion is a central contribution of the paper.
-
The experimental results and protocols are insufficiently auditable. Section 4.4 defines the scaling score as the mean over six benchmarks. The corresponding main-table scores average to 66.22%, whereas Figure 5’s full-run/6.02B endpoint appears around 70–71%. Different checkpoints or adaptation settings could explain this, but the distinction is not documented. More broadly, the manuscript does not specify the exact inherited video checkpoint, complete pretraining budget, per-benchmark adaptation recipes, provenance of most simulation baseline results, or the meaning of the error bars in Figures 4–6. The “held-out embodiment” exclusions are also unclear given RoboCasa365 in the pretraining mixture. These gaps limit assessment of the comparisons, scaling, and transfer claims.
Questions for authors
-
Motion-track definition: How are robot actuator/scene points and human hand points selected and organized across sources, and how are multiple points packed into the 16 track tokens? Please specify the normalization, displacement gain, observability construction, and coordinate-mask rules, as well as which data sources provide each type of track supervision.
-
Motion-track attribution: At fixed human-video exposure, what additional cross-embodiment benefit comes from human track supervision? Does jointly generating tracks improve control more than using the same tracks as an auxiliary prediction target under matched data and model budgets? These comparisons would materially clarify the distinct contribution of the proposed interface beyond the existing coverage/form ablations.
-
Results and protocols: Please reconcile Figure 5’s full-run/6.02B point with the 66.22% mean from Tables 2–5, 7, and 16. Which main simulation baselines were rerun, under what adaptation and rollout settings, and what do the error bars in Figures 4–6 represent? What precisely was excluded from pretraining for Figure 6’s held-out transfer?