- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper explores masked diffusion model for generative trajectory modeling in behavioral cloning. The method jointly models observations and actions by treating trajectory prediction as an inpainting problem, where random masking is used during training to learn conditional trajectory completion. Experiments on Push-T, Hunter-Prey, and Reacher show improvement in control performance, robustness, and generalization.
Critical strengths
-
The paper presents a simple and unified framework for trajectory modeling, with promising improvements in control performance and robustness.
-
The strong performance at a very high masking ratio (98%) is an interesting finding, which shows the potential benefits of random masking for trajectory prediction.
Critical weaknesses
1. Distinction from Existing Trajectory Diffusion Methods. The proposed method is similar to Diffuser [1], which also supports joint trajectory modeling and conditioning through inpainting. The main difference appears to be introducing random masking during diffusion training. The paper could better clarify underlying design differences and motivations compared to existing trajectory diffusion methods.
2. Non-Diffusion Baselines. The experiments mainly focus on comparisons with Diffusion Policy. Prior work shows that non-diffusion masking approaches are effective for trajectory modelling, such as Masked Trajectory Models (MTM) [2] and Uni[MASK] [3]. Comparisons with these methods would help better understand the benefits of the proposed method.
3. Analysis of Inference Efficiency. The paper evaluates efficiency mainly based on episode length, while computational efficiency is less evaluated. Since the proposed method jointly generates observations and actions, reporting inference latency or computational cost would help better understand its trade-offs compared to Diffusion Policy.
References
[1] Janner, M., Du, Y., Tenenbaum, J., & Levine, S. Planning with Diffusion for Flexible Behavior Synthesis. ICML, 2022.
[2] Wu, P., et al. Masked Trajectory Models for Prediction, Representation, and Control. ICML, 2023.
[3] Carroll, M., et al. Uni[MASK]: Unified Inference in Sequential Decision Problems. NeurIPS, 2022.
Questions for authors
-
In Hunter-Prey, both Diffusion Policy (obs-to-all) and the proposed method achieve 100% catch rate, but the former requires fewer steps (40 vs. 53). Could the authors comment on this result and explain the reason?
-
The fixed-mask baseline outperforms random masking at the same 75% masking ratio, while higher random masking ratios achieve better results. Could the authors provide additional fixed-mask experiment to better understand the benefits of random masking?
-
There is a growing use of flow matching in generative world models. Have the authors considered to extend their approach to flow matching? It would be interesting to see whether similar improvements can be achieved with fewer sampling steps.
Thank you for the constructive review. The general response covers novelty, the masking ratio and the new baselines; here we answer your points in order.
W1. Distinction from Diffuser. You are right that the main difference is random masking during training. Both methods model joint state–action trajectories and express conditioning as known entries of the trajectory. They differ in three ways.
The motivation for random masks is that the model must solve many conditional problems, not one. Our fixed-mask ablations (general response, part 2) are the comparison closest to Diffuser's regime, and they support this. With a fixed mask our model reaches 0.93–0.96 success and 207–208 steps on vanilla Push-T, against 0.98 and 190 with random masks, and under noisy actions it falls to 0.83–0.84 against 0.93 (Table 1). Models trained with random masks at high ratios also keep their performance when the history length is changed at inference (Section 4.4, Fig. 6).
W2. Non-diffusion baselines. We added Diffusion-Free: the same transformer and random masking, trained with an L2 loss as in MTM, Uni[MASK] and MaskDP. It scores 70% to 83% on Push-T against 90% to 98% for our model, and 20% on Reacher's held-out quadrants against 53%. Two limits: it is a controlled ablation in our architecture, not the released MTM or Uni[MASK] code, and we ran it only at a ratio of 0.98.
W3. Inference efficiency. All models share depth, width and the number of denoising steps. Observations and actions occupy the same tokens (Appendix A), so predicting observations adds no tokens: both models attend over 16 tokens, and Diffusion Policy also cross-attends to 2 observation tokens. The only additional parameters are the observation projections: the totals are 35.765M versus 35.756M (a difference under 0.03%). We also measured wall-clock time per action chunk for the trained Push-T checkpoints on one RTX 2080 SUPER (100 denoising steps per chunk; 50 chunks timed per model in each of 5 rounds, alternating the two models; we report the mean, with a standard deviation across rounds of at most 3.4 ms). At batch size 1 our model takes 864 ms against 995 ms for Diffusion Policy (108 against 124 ms per executed action); at batch size 50, 1,010 against 1,204 ms. Predicting observations adds no measurable inference cost: a Diffusion Policy that also predicts observations has the same latency as the action-only one. Any cost is in capacity instead: at a matched parameter count, the same network must also model the observations. Our model is faster for an implementation reason: Diffusion Policy uses attention masks, which keep PyTorch off its fused attention path; with them removed (a timing-only check) it comes within 2% of ours. Optimising inference for robot hardware is left to future work.
Q1. Hunter-Prey, 40 versus 53 steps. All three reviewers raised this, so we answer it in full in the general response (part 2, "Hunter-Prey"). In short, the two models in that comparison differ in more than masking: ours also conditions on past actions, which slows it by 9 steps and slows Diffusion Policy by 76. Without it, our model takes 44 steps against 40 for Diffusion Policy. Observation prediction accounts for the catch rate, and action conditioning for most of the gap.
Q2. Fixed mask at 75%. The 75% label was an error. With two of 16 steps visible, the inference mask hides 88%. In the sweep (Fig. 4, 4,500 epochs) the fixed mask is slightly worse than random masking at that ratio: 0.94 versus 0.98 success at the best checkpoint (0.91 versus 0.94 over the last ten), and 243 versus 202 steps. After 8,000 epochs (Table 1) it reaches 0.96 and 207 steps, against 0.98 and 190 for our main model. As you suggested, we added further fixed-mask experiments: a fixed mask at 98% (0.93 and 208 steps on vanilla Push-T), and an evaluation of each model under different history–future splits (Section 4.4, Fig. 6).
Q3. Flow matching. We have not tried it and agree it is a promising direction. The masking scheme only determines which elements are noised, so we expect it to carry over, and fewer sampling steps would also reduce the inference cost raised in W3.
We would be glad to know whether this addresses your concerns.