acceptodds
Under review as a conference paper at ICLR 2027

Generative Trajectory Modelling through Masked Diffusion

Abstract

Behavioural cloning aims to imitate expert behaviour through supervision: a model is trained to predict the expert's actions from the observations it encountered. However, this framing discards the generative structure of trajectories, which arise from an interplay between the expert's policy and the environment's latent dynamics. Generative modelling is therefore a more natural fit. We motivate this perspective through four benefits: richer auxiliary learning signals, improved generalisation, temporal coherence, and robustness to unseen dynamics. Building on this, we propose a method for generative trajectory modelling that combines diffusion models with masked modelling, framing future trajectory prediction as a trajectory “inpainting” problem. We find this approach performs best under extremely high masking ratios of over 95%, reflecting the temporal redundancy in trajectories. Across a range of tasks, our method outperforms existing alternatives in efficiency (steps to task completion), accuracy (lower deviation from expert actions, both in-distribution and out-of-distribution), and generalisation to novel dynamics. To our knowledge, this is also the first demonstration that masked modelling can be effectively combined with denoising diffusion models.

Primary area:
reinforcement learning
Keywords:
Behavioural cloning, batch RL, self-supervised learning, diffusion modelling, generative modelling, masked modelling, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_rdxqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores masked diffusion model for generative trajectory modeling in behavioral cloning. The method jointly models observations and actions by treating trajectory prediction as an inpainting problem, where random masking is used during training to learn conditional trajectory completion. Experiments on Push-T, Hunter-Prey, and Reacher show improvement in control performance, robustness, and generalization.

Critical strengths
  1. The paper presents a simple and unified framework for trajectory modeling, with promising improvements in control performance and robustness.

  2. The strong performance at a very high masking ratio (98%) is an interesting finding, which shows the potential benefits of random masking for trajectory prediction.

Critical weaknesses

1. Distinction from Existing Trajectory Diffusion Methods. The proposed method is similar to Diffuser [1], which also supports joint trajectory modeling and conditioning through inpainting. The main difference appears to be introducing random masking during diffusion training. The paper could better clarify underlying design differences and motivations compared to existing trajectory diffusion methods.

2. Non-Diffusion Baselines. The experiments mainly focus on comparisons with Diffusion Policy. Prior work shows that non-diffusion masking approaches are effective for trajectory modelling, such as Masked Trajectory Models (MTM) [2] and Uni[MASK] [3]. Comparisons with these methods would help better understand the benefits of the proposed method.

3. Analysis of Inference Efficiency. The paper evaluates efficiency mainly based on episode length, while computational efficiency is less evaluated. Since the proposed method jointly generates observations and actions, reporting inference latency or computational cost would help better understand its trade-offs compared to Diffusion Policy.

References

[1] Janner, M., Du, Y., Tenenbaum, J., & Levine, S. Planning with Diffusion for Flexible Behavior Synthesis. ICML, 2022.

[2] Wu, P., et al. Masked Trajectory Models for Prediction, Representation, and Control. ICML, 2023.

[3] Carroll, M., et al. Uni[MASK]: Unified Inference in Sequential Decision Problems. NeurIPS, 2022.

Questions for authors
  1. In Hunter-Prey, both Diffusion Policy (obs-to-all) and the proposed method achieve 100% catch rate, but the former requires fewer steps (40 vs. 53). Could the authors comment on this result and explain the reason?

  2. The fixed-mask baseline outperforms random masking at the same 75% masking ratio, while higher random masking ratios achieve better results. Could the authors provide additional fixed-mask experiment to better understand the benefits of random masking?

  3. There is a growing use of flow matching in generative world models. Have the authors considered to extend their approach to flow matching? It would be interesting to see whether similar improvements can be achieved with fewer sampling steps.

1 comment

Thank you for the constructive review. The general response covers novelty, the masking ratio and the new baselines; here we answer your points in order.

W1. Distinction from Diffuser. You are right that the main difference is random masking during training. Both methods model joint state–action trajectories and express conditioning as known entries of the trajectory. They differ in three ways.

  • Conditioning set. In Diffuser it is fixed: the current state, plus the goal state in Maze2D, is held at its observed value. Ours is random, so one model conditions on any visible subset and the history length can be chosen at inference.
  • What is conditioned on. Diffuser conditions on states only, never on actions or on a history of several steps. We condition on observations and actions over several steps, which the random-dynamics task requires: the dynamics can only be inferred from past actions and their effects.
  • Purpose. Diffuser is a planner that steers samples with reward guidance or a goal state. Behaviour cloning has no reward, so guidance has no counterpart here.

The motivation for random masks is that the model must solve many conditional problems, not one. Our fixed-mask ablations (general response, part 2) are the comparison closest to Diffuser's regime, and they support this. With a fixed mask our model reaches 0.93–0.96 success and 207–208 steps on vanilla Push-T, against 0.98 and 190 with random masks, and under noisy actions it falls to 0.83–0.84 against 0.93 (Table 1). Models trained with random masks at high ratios also keep their performance when the history length is changed at inference (Section 4.4, Fig. 6).

W2. Non-diffusion baselines. We added Diffusion-Free: the same transformer and random masking, trained with an L2 loss as in MTM, Uni[MASK] and MaskDP. It scores 70% to 83% on Push-T against 90% to 98% for our model, and 20% on Reacher's held-out quadrants against 53%. Two limits: it is a controlled ablation in our architecture, not the released MTM or Uni[MASK] code, and we ran it only at a ratio of 0.98.

W3. Inference efficiency. All models share depth, width and the number of denoising steps. Observations and actions occupy the same tokens (Appendix A), so predicting observations adds no tokens: both models attend over 16 tokens, and Diffusion Policy also cross-attends to 2 observation tokens. The only additional parameters are the observation projections: the totals are 35.765M versus 35.756M (a difference under 0.03%). We also measured wall-clock time per action chunk for the trained Push-T checkpoints on one RTX 2080 SUPER (100 denoising steps per chunk; 50 chunks timed per model in each of 5 rounds, alternating the two models; we report the mean, with a standard deviation across rounds of at most 3.4 ms). At batch size 1 our model takes 864 ms against 995 ms for Diffusion Policy (108 against 124 ms per executed action); at batch size 50, 1,010 against 1,204 ms. Predicting observations adds no measurable inference cost: a Diffusion Policy that also predicts observations has the same latency as the action-only one. Any cost is in capacity instead: at a matched parameter count, the same network must also model the observations. Our model is faster for an implementation reason: Diffusion Policy uses attention masks, which keep PyTorch off its fused attention path; with them removed (a timing-only check) it comes within 2% of ours. Optimising inference for robot hardware is left to future work.

Q1. Hunter-Prey, 40 versus 53 steps. All three reviewers raised this, so we answer it in full in the general response (part 2, "Hunter-Prey"). In short, the two models in that comparison differ in more than masking: ours also conditions on past actions, which slows it by 9 steps and slows Diffusion Policy by 76. Without it, our model takes 44 steps against 40 for Diffusion Policy. Observation prediction accounts for the catch rate, and action conditioning for most of the gap.

Q2. Fixed mask at 75%. The 75% label was an error. With two of 16 steps visible, the inference mask hides 88%. In the sweep (Fig. 4, 4,500 epochs) the fixed mask is slightly worse than random masking at that ratio: 0.94 versus 0.98 success at the best checkpoint (0.91 versus 0.94 over the last ten), and 243 versus 202 steps. After 8,000 epochs (Table 1) it reaches 0.96 and 207 steps, against 0.98 and 190 for our main model. As you suggested, we added further fixed-mask experiments: a fixed mask at 98% (0.93 and 208 steps on vanilla Push-T), and an evaluation of each model under different history–future splits (Section 4.4, Fig. 6).

Q3. Flow matching. We have not tried it and agree it is a promising direction. The masking scheme only determines which elements are noised, so we expect it to carry over, and fewer sampling steps would also reduce the inference cost raised in W3.

We would be glad to know whether this addresses your concerns.

Reviewer_wxTgOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper frames behavior cloning as generative modeling of whole trajectories. They train a diffusion transformer to inpaint randomly masked observations and actions (best at very high mask ratios), keep the history visible at inference and generate the future in a receding-horizon way. Experiments are on Push-T (with noise and random-dynamics variants), a new Hunter-Prey task and Reacher, compared against Diffusion Policy and its variants.

Critical strengths
  • The comparisons against Diffusion Policy are controlled: the codebase, architecture and hyperparameters are largely shared, and the variants (obs→obs+action, obs+action→action, all-to-all) separate what comes from predicting observations, from conditioning on actions and from the masking.

  • The Reacher result is interesting: trained on one quadrant, the inpainting agent gets 0.53 success on the held-out quadrants vs 0.29 for Diffusion Policy, with lower error to the ground-truth policy.

  • The best credit I can give them is that they might show that training with an inpainting objective is better (Fig. 4, p = 1.0 gets 0.38 vs 0.97 at p = 0.98). But if that is true, it should be true for other diffusion models as well, be it only video, LLaDA, actions etc. And this masked modeling with diffusion has already been done for videos, where frames on both sides can be visible and the model generates what is in the middle. MCVD (Voleti et al., NeurIPS 2022) randomly masks past and future frames during training, so one model does prediction, generation and interpolation. RaMViD (Höppe et al., TMLR 2022) trains with randomly chosen conditioning frames and does both prediction and infilling. FDM (Harvey et al., NeurIPS 2022) trains on random sets of observed and generated frames, so any frames can be given and the rest filled in. None of these are cited, so I don't think this part is new either, and the claim to be "the first demonstration that masked modelling can be effectively combined with denoising diffusion models" does not hold.

Critical weaknesses
  1. Section 3.1, predicting future observations and actions together, reminds me of many papers in the 2024–2026 era that predicted video and actions together, like PAD (Prediction with Action, NeurIPS 2024), which jointly denoises future images and actions with a diffusion transformer, the Unified Video Action model (UVA, RSS 2025), VideoVLA (NeurIPS 2025), Unified Diffusion VLA (ICLR 2026), Cosmos Policy (ICLR 2026), GR-1 (ICLR 2024) and WorldVLA (arXiv), and there are more papers like that. From this line of work the paper only cites Unified World Models (Zhu et al., 2025) and Diffusion Forcing (which it dismisses, see below). So this idea is definitely not novel.

  2. Even the masked modelling method (even though I can see why it seems novel) is kind of like diffusion forcing, and Diffusion Forcing itself (NeurIPS 2024) already denoises observations and actions together. Also, like I covered before, the same idea was also covered in video diffusion space by MCVD. Multimodal Diffusion Forcing (arXiv 2511.04812) also gives each timestep and each modality (observations, actions, force) its own noise level and uses one model as a policy, forward dynamics model and state estimator. It is arXiv only, so I am not holding the missing comparison against the paper, but the authors should be aware of it.

  3. Most video models these days use a latent generation model in which a single frame (and action) can generate four actual frames. This paper works on low-dimensional states rather than video, but the point is the same: how finely a token is defined is a design choice, so the per-step granularity does not seem that novel (it's like just noising a certain segment?), and the paper does not show that it matters.

  4. And during inference, when we do not see the entire thing, or only have the start frame and goal frame and infill what is in between, that is also something very similar to diffusion forcing. The paper also only ever tests the standard "history visible, predict the future" setting, so these other cases are only motivation.

  5. Their primary argument is that wind can cause the sail to behave differently. In the experiments they represent this with random noise (randomised controller gain and damping in Push-T, random heading noise on the prey in Hunter-Prey). But if during generation I don't see this randomness, generate actions, take a few steps and things change, a model that generates both actions and video together could also handle that. My contention is that they don't have enough evidence for this claim: they should have shown that other models, specifically the models that can generate both actions and video together, are not able to handle this randomness. Their own Table 1 already shows part of this: Diffusion Policy (all-to-all), which conditions on past actions, gets 0.89 on random dynamics vs 0.93 for the inpainting agent.

  6. There are no experiments and no comparison with Diffusion Forcing / DFoT or UVA (which even reports Push-T against Diffusion Policy), and DFoT and UVA are not even mentioned. Diffuser and Diffusion Forcing are cited, but Diffuser is not compared against (the p = 1.0 row is close to a Diffuser-style ablation but is not presented as one), and Diffusion Forcing is mentioned only to dismiss it. This seems bad for a paper whose main claim is novelty.

Questions for authors
  1. How is this different from video continuation / interpolation results with masked or diffusion-forcing style video models?

  2. What happens during inference when we do not see the entire thing, or only have the start and goal and infill what is in between? Does the model work for these cases without retraining?

  3. If training with an inpainting objective is what helps, why should this be specific to trajectories and not hold for other diffusion models too (video only, LLaDA, actions)? How do you position this against the inpainting papers above?

  4. Can models that generate both actions and video together handle the same randomness? If so, what does the inpainting agent add?

  5. Fig. 4: with the average of the last 10 checkpoints, random masking at p = 0.98 and the fixed inference mask are tied at 0.91. Is the gain from random masking robust, or does it depend on picking the best checkpoint?

  6. On Hunter-Prey, Diffusion Policy (obs-to-all) catches the prey in 40 steps vs 53 for the inpainting agent. How does this fit with the claim of better efficiency?

2 comments
AuthorsOpenReview ↗

Response to Reviewer wxTg (1/2)

Thank you for the detailed review. We follow your numbering and refer to the general response (GR) where it covers a point. Part 2 answers your questions.

Strength 3: the inpainting objective, and prior work in video. Your reading of Fig. 4 is right: at p = 1.00 the model never sees clean context in training and is conditioned only at sampling time, and it reaches 0.38 against 0.97 at p = 0.98. We agree that MCVD, RaMViD and FDM already combine random masking with diffusion, and have withdrawn the claim to be first. Section 2 now cites them and presents the method as an adaptation of RaMViD and FDM (GR part 1, "Novelty"). Our question is what the scheme offers in closed-loop control, where the measure is task success, not sample quality. On other diffusion models, see Q3.

W1. Joint prediction of observations and actions is not new. We agree, and Section 2 now cites the models you list. We do not claim joint prediction as a contribution; our claims concern the masking scheme. Those models take image observations and build on pretraining. We use low-dimensional states, no pretraining and one architecture for every method, which gives a controlled test of what observation prediction contributes. On Hunter-Prey it raises the catch rate from 0.34–0.42 to 1.00 in all four paired comparisons.

W2 and W4. Diffusion Forcing. We agree that Diffusion Forcing is the closest method in control: it denoises observations and actions jointly and offers the same flexible conditioning at inference. Section 2 no longer describes it as limited to causal models, and now cites DFoT and Multimodal Diffusion Forcing. It is also a baseline now (GR part 2). We implemented the causal version, as in the original paper; it scores 0.72–0.82 across the Push-T variants against 0.90–0.98 for our model. The two differ in the noise scheme and in attention (causal against bidirectional), so the comparison does not separate them. Diffusion Policy is also causal and scores higher (0.81–0.93), so causal attention alone is unlikely to explain the gap. One possible reason is that Diffusion Forcing sees more visible context during training (GR part 1, "Relation to prior conditional diffusion"); we have not tested this.

W3. Token granularity. We agree that how finely a token is defined is a design choice. One token per time step was a reasonable default (Appendix A), not a contribution, and exploring other granularities was beyond the scope of this work.

W5. Random dynamics. We agree that a model which generates observations and actions together can handle this randomness, and our all-to-all ablation shows it: Diffusion Policy (all-to-all) reaches 0.90 against 0.92 for our model. Our argument concerns the history, not inpainting. In Push-T the gain and damping are drawn once per episode, so the uncertainty about them is epistemic, not step-wise noise. The standard Diffusion Policy sees observations only and cannot infer them, so for it this uncertainty acts as control noise (0.81). A model that also conditions on past actions can infer the dynamics and so reduce the uncertainty, which is our explanation for the all-to-all scores.

W6. Missing comparisons. GR part 2 lists the additions: Diffusion Forcing, Diffusion-Free (masked regression) and two fixed-mask ablations. Section 2 now cites DFoT and UVA.

  • DFoT. Not evaluated. Our Diffusion Forcing baseline is causal, as in the original paper. A non-causal version, as in DFoT, would separate the noise scheme from the attention pattern, but we could not run it within the response period.
  • UVA. Not evaluated: its Push-T result is image-based and uses pretrained visual components, so it is not comparable with our state-based numbers.

Novelty is no longer the paper's main claim. Our claims are those in GR part 1, "What we add": in control the masking ratio decides whether the method works, and one model trained at a high ratio tolerates different inference-time masks.

AuthorsOpenReview ↗

Response to Reviewer wxTg (2/2)

Q1. Difference from video continuation and interpolation. In mechanism, little: RaMViD and FDM use the scheme we adapt, and diffusion-forcing-style models replace the binary mask with per-token noise levels. The setting and the measure differ: in control the inference mask is always history-to-future, the output is executed and replanned, and the measure is task success, not sample quality. There the masking ratio decides whether the method works at all, and high ratios make one model insensitive to the history length at inference (GR part 1, "What we add").

Q2. Partial histories and start–goal infilling without retraining. For partial histories, yes: without retraining, models trained at high ratios keep roughly the same performance for history lengths from 1 to 9 steps on Push-T and 1 to 13 on Reacher (Section 4.4, Fig. 6). These masks are all visible prefixes; histories with gaps are untested. Start–goal infilling is goal conditioning as inpainting, which MaskDP and Uni[MASK] study. Like the other special inference-time masks of masked trajectory models (MTM), we have left it to future work. It should need no retraining, since it is one more mask and Fig. 6 shows that these models tolerate masks they were not trained on, but we have not tested it, and with a 16-step window only goals inside the window could be inpainted. For distant goals we use global conditioning, as in Reacher, where the model reaches held-out goals more often than Diffusion Policy (0.53 against 0.29).

Q3. Why should this be specific to trajectories? It should not be, and we do not claim it is. In video, training with clean conditioning frames is what MCVD, RaMViD and FDM do, and we now position our method as an adaptation of that scheme to observation–action trajectories (GR part 1, "Novelty"). In LLaDA, masking is itself the corruption process, so its objective is already of this kind. What we add is closed-loop evidence, and it shows that the regime does not carry over automatically. FDM and MCVD train with about half of the frames or conditioning blocks visible, whereas in our tasks a ratio of 50% fails (0.24 on Push-T, 0.12 on Reacher) and only high ratios work (Figs. 4 and 5). We have not tested action-only or image-based models.

Q4. Can models that generate actions and video together handle the same randomness? In our low-dimensional setting the counterpart is a model that generates observations and actions together, and the answer is yes: the all-to-all Diffusion Policy reaches 0.90 under random dynamics against 0.92 for ours (W5), so the inpainting agent adds little there. (Diffusion Forcing and Diffusion-Free score 0.70–0.73, but they are also lower on the vanilla task.) What inpainting adds elsewhere is robustness to noisy actions (0.93 against 0.80 for the all-to-all Diffusion Policy), shorter episodes on vanilla Push-T (190 steps against 206) and the choice of split at inference. Image-based video–action models are untested and are important future work. We expect the same of them: what control needs from predicted observations is task-relevant structure, not pixel detail, the argument behind joint-embedding predictive architectures (LeCun, 2022; Assran et al., 2025).

Q5. Is the gain from random masking robust? The trend with the masking ratio holds in two tasks that differ in kind: Push-T uses human teleoperated demonstrations, and Reacher uses noise-free demonstrations from a scripted expert. Episodes are shorter on the last-ten average too (213 against 255 steps; Fig. 4). And models trained at high ratios are insensitive to the history-future split of the context window (Fig. 6).

Q6. Hunter-Prey, 40 versus 53 steps. Hunter-Prey was not meant to show efficiency. We designed it to test whether predicting observations matters in behavioural cloning, which Diffusion Policy deliberately avoids. The variant you cite is part of that test: Diffusion Policy catches the prey in 40 steps only once it also predicts observations, and every variant that does not stays at a catch rate of 0.34–0.42 (GR part 2, "Hunter-Prey"; W1). Most of the gap to our model comes from conditioning on past actions: without it, ours takes 44 steps. Our efficiency results concern Push-T, where our model has the shortest episodes in three of the four variants (Table 1).

We would be glad to know whether this changes your assessment.

Reviewer_wHpCOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The work proposes a framework for behavioral cloning using Gaussian diffusion to reconstruct masked portions of the trajectories. During training, the model receives masked trajectories and at inference, it predicts future observations and/or actions. The framework is evaluated in 3 environments.

Critical strengths

o. The idea of modeling observations and actions together is sound, particularly for problems where understanding environmental dynamics is important.

o. The authors include several variants of Diffusion Policy to investigate the effect of conditioning on actions and predicting observations.

Critical weaknesses

Novelty

o. Despite the title, the paper does not introduce a masked diffusion process. It uses standard Gaussian diffusion and applies noise only to randomly masked elements of a trajectory. This is essentially a diffusion inpainting formulation. The terminology is misleading, particularly given the established use of masked diffusion for discrete-state diffusion models.

o. The claim that this is the first demonstration of combining masked modeling with denoising diffusion is too broad. The authors acknowledge related approaches, including Diffusion Forcing and Unified World Models, but do not convincingly establish how their contribution differs from existing conditional diffusion and trajectory-inpainting methods.

o. The experimental comparisons are almost entirely restricted to Diffusion Policy and modifications of it. Several closely related approaches, including Diffuser, Diffusion Forcing, Unified World Models, Masked Trajectory Models, and UniMASK, are discussed but never evaluated. Comparisons with at least some of these methods are necessary to establish the contribution relative to existing work.

o. Randomly mask observations and trajectories is trivial, which is not inherently a bad thing. Nevertheless, the paper does not provide sufficient evidence to justify its existence.

Experiments

o. The evaluation is restricted to three relatively simple control environments.

o. The masking ratio is selected using Push-T and then fixed at 98% for all other experiments. The paper provides little explanation for why such an extreme ratio should be appropriate across different environments.

o. Hunter-prey: the diffusion-policy variant also reaches 100%, this suggests that predicting observations accounts for most of the improvement, rather than the masking.

Presentation

o. The writing is full of subjective statements without justification. For example, the authors claim that diffusion models are "the method of choice" and that behavioral cloning is "more naturally framed" as a generative modeling problem. This makes reading the paper very frustrating.

o. The description of the method is often unnecessarily elaborate. For example, the authors refer to a "modification of the diffusion process" when the main change is just to add a boolean mask.

o. The figures provide little additional information beyond the text. Figures 1 and 2 largely restate the method, while Figure 5 makes it difficult to visually distinguish the reported differences in generalization performance.

o. The bibliography contains duplicated entries. Some citations seem out of place (i.e. Amodei)

Overall

I am not convinced that the paper establishes a novel methodological contribution. The proposed approach is straightforward application of Gaussian diffusion inpainting to trajectories, and the experimental comparisons do not establish its advantages over existing methods.

Questions for authors

o. What is the actual contribution of masked inpainting beyond predicting future observations?

o. What evidence supports the claim that the model learns a better representation of environmental dynamics?

1 comment

Thank you for the detailed review. Points covered in the general response (GR) are referenced below; the rest we answer in the order you raised them.

Novelty

  1. "Masked diffusion" is misleading. We agree; see GR part 1, "Terminology". The paper will be retitled around trajectory inpainting, the term you suggest.
  2. "First demonstration" is too broad; differences from Diffusion Forcing and Unified World Models. We agree and have withdrawn the claim (GR part 1, "Novelty" and "Relation to prior conditional diffusion"). In brief: Diffusion Forcing assigns each token an independent noise level, where we use binary masks; Unified World Models assigns one noise level to each modality as a whole, so it cannot mask positions within a trajectory; Diffuser conditions on a fixed set of states.
  3. Comparisons restricted to Diffusion Policy. The revision adds Diffusion Forcing, a regression-based masked model (the objective of MTM and Uni[MASK]) and two fixed-mask ablations, the regime closest to Diffuser (GR part 2). We did not evaluate Unified World Models, which is built for image-based video–action pretraining.
  4. Random masking is trivial and lacks evidence. We have added a masking-ratio study on a second task, fixed versus random masks, and robustness across inference-time splits (GR part 1, "What we add").

Experiments

  1. Three relatively simple environments. This is a fair limitation, and we could not add larger benchmarks within the response period. We chose low-dimensional tasks because they allow controlled comparisons: every method shares one architecture and one set of hyperparameters, which makes the ablation grid possible. Each task isolates one property: Push-T, a standard benchmark with multimodal human demonstrations, with perturbed dynamics and noise; Hunter-Prey for anticipating dynamics; Reacher, whose expert is known in closed form, for policy error on held-out goals. We present the results as a proof of principle for three things: the masking ratio decides whether the method works, one model tolerates different history–prediction splits at inference, and the diffusion objective matters (Diffusion-Free). Image-based and larger manipulation benchmarks we leave to future work.
  2. Ratio chosen on Push-T and fixed at 98%. See GR part 1, "Why a high masking ratio" and "What we add": the redundancy argument, and a new Reacher sweep that shows the same pattern (Fig. 5).
  3. Hunter-Prey: observation prediction accounts for the improvement. We agree; see GR part 2, "Hunter-Prey".

Presentation

  1. "Modification of the diffusion process". We agree. The change is a binary mask that determines which elements are noised, which is the standard conditional-diffusion formulation; the text no longer presents it as ours.
  2. Figures. The numbers behind the Reacher error maps (now Figure 7) are in Table 3 (mean held-out error 0.0205 versus 0.0430). The key point to note is that the green-yellow region, indicating higher error, is more diffuse in the bottom left quadrant with diffusion policy.
  3. Bibliography. Fixed: the duplicate entries are removed and the Amodei citation is dropped.

Questions

  1. Contribution of masked inpainting beyond predicting observations. Two comparisons separate them. Diffusion Policy (all-to-all) predicts the same targets from the same inputs without masking: on Push-T it scores 0.80 to 0.93 across variants against 0.90 to 0.98 for our model. The fixed-mask ablations keep our architecture and remove only the randomness of the mask: they score 0.83 to 0.96 and take 207–208 steps on the vanilla task against 190. On Hunter-Prey, as you note, there is no gain in catch rate beyond observation prediction.
  2. Evidence for a better representation of dynamics. The evidence is indirect and comes from two tasks. Under random dynamics on Push-T, where the gains can only be inferred from past actions and their effects, our model scores 0.92 against 0.81 for Diffusion Policy and 0.90 for its all-to-all variant, so the gain there comes from conditioning on past actions. On Hunter-Prey, the agent must anticipate the prey's motion on orbits never seen in training: every variant that predicts observations reaches a 100% catch rate, while every action-only variant stays at 0.34–0.42. This indicates that predicting observations yields a usable model of the prey's dynamics. It holds for Diffusion Policy (obs-to-all) as well, so it supports observation prediction, not masking. We have no direct measurement, such as forward-prediction error or a probe for the latent gains, and have reworded the claim accordingly.

We would be glad to know whether this changes your assessment.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response (1/2): terminology, novelty and masking ratio

We thank the reviewers for their time and their detailed feedback. This response covers the points raised by more than one reviewer.

Updated Push-T results. We retrained the Push-T models in Table 1 for 8,000 epochs instead of 4,500, because the fixed-mask ablations had not converged at 4,500. All methods received the same budget. The numbers in Table 1 and in this response therefore differ from the original submission, but the conclusions are unchanged: the ordering of methods and the benefit of high masking ratios are the same. The masking-ratio sweep in Fig. 4 is the one from the submission, at 4,500 epochs.

Terminology. We agree that "masked diffusion" names a different method, discrete absorbing-state diffusion, which we do not use. We will retitle the paper "Behavioural Cloning as Trajectory Inpainting: Training Diffusion Models with Random Masks". The revised text no longer uses the term for our method.

Novelty. We agree that our claim of algorithmic novelty was overstated and have removed it from the abstract, introduction and conclusion. Reviewer wxTg points to MCVD (Voleti et al., 2022), RaMViD (Höppe et al., 2022) and FDM (Harvey et al., 2022). RaMViD and FDM are closest: both keep a random subset of frames clean as conditioning and add noise only to the rest, which is the mechanism we use; MCVD masks whole blocks. We now cite all three and present our method as an adaptation of this scheme (Section 2). These methods were developed and evaluated for video generation. Our question is what the scheme offers in closed-loop control, where the measure is task performance and robustness, not sample quality.

Why a high masking ratio. The masked-modelling literature ties the masking ratio to the redundancy of the data: BERT masks 15% of tokens, MAE 75% of image patches and VideoMAE 90–95% of video (Devlin et al., 2019; He et al., 2022; Tong et al., 2022). Liu et al. (2022) argue that trajectories are redundant in the same way and need a high ratio (95%), although with their regression objective a fixed 95% did not work well and they mix ratios from 15% to 95%. With a diffusion objective we find that very high ratios work best on their own (Figs. 4 and 5). Our reported 98% is nominal: with a 16-step window it masks 15 of the 16 steps.

Relation to prior conditional diffusion. Training with little clean context is not new in itself. RaMViD keeps one to four frames clean out of 16 to 20. FDM marks each group of frames as observed with probability 1/2, and MCVD drops each conditioning block with probability 1/2. Diffusion Forcing has no masking ratio as such, but it trains with more visible context than we do. Each token's noise level is drawn independently and uniformly, so a heavily noised token is usually surrounded by less noised ones: in a 16-token window, about four tokens on average lie in the least-noised quarter of the range. In our scheme all masked steps share one noise level, so at high noise the only reliable context is the few clean steps.

What we add. First, in control the ratio decides whether the method works at all: on Push-T, success is 0.16 and 0.24 at ratios of 25% and 50%, and 0.91 to 0.98 at ratios from 75% to 98% (Fig. 4). A new sweep on Reacher shows the same pattern: the score is 0.12 and 0.29 at ratios of 50% and 75%, and 0.53 and 0.54 at 88% and 98% (Fig. 5). This addresses Reviewer wHpC's concern that the ratio was chosen on Push-T alone. Second, models trained at high ratios are robust to a mismatch between the training masks and the inference mask. Unlike Diffusion Policy, where the architecture fixes the split between history and prediction, our model sets the split at inference time. We now evaluate each model under different history lengths and find that performance stays roughly constant only for models trained at high ratios (Section 4.4, Fig. 6). The value we claim is this combination: one model, trained once, that tolerates different inference-time masks in closed-loop control. We have not tested whether training with little clean context would help the methods Reviewer wxTg lists, such as Diffusion Forcing, and leave that to future work.

AuthorsOpenReview ↗

General response (2/2): baselines and ablations

We agree that comparisons beyond Diffusion Policy were needed. The revision adds one method from prior work and three controlled ablations. All use the same transformer architecture as our model and Diffusion Policy.

  1. Diffusion Forcing (Chen et al., 2024), reimplemented in that architecture with causal attention, as in the original paper, and sampled under two schedules. The model is trained once with independent per-token noise levels; a schedule then sets the noise level of each future token at each denoising iteration, with the observed history held at zero noise. Under the full-sequence schedule all future tokens share one noise level, as in our own sampler. Under the pyramid schedule, each token starts denoising with a delay proportional to its distance into the future, so the near future is resolved first.
  2. Diffusion-Free (masked regression). To isolate the diffusion objective, we train the same transformer, with the same random masking at a ratio of 0.98, to predict the masked observations and actions in one forward pass with an L2 loss. MaskDP (Liu et al., 2022), MTM (Wu et al., 2023) and Uni[MASK] (Carroll et al., 2022) use the same L2 objective for continuous states and actions. It is a controlled ablation, not a reimplementation of those methods.
  3. Fixed inference mask (88%). The training mask is identical to the inference mask: the first two steps are visible and the remaining 14 are masked.
  4. Fixed mask at 98%. The ratio matches the random masking ratio of our main model, but the positions are fixed: only the first step is visible and the remaining 15 are masked. This separates the effect of random mask positions from the effect of the ratio itself.

Results (Table 1). Diffusion Forcing scores 72% to 82% across the Push-T variants, below Diffusion Policy (81% to 93%) and our model (90% to 98%). It differs from our model in the training scheme (independent per-token noise levels versus binary masks) and in its causal attention, so this comparison does not separate the two. Diffusion Policy is also causal, so causal attention alone is unlikely to explain the gap. One possible reason is the greater visible context described in general response 1, but we have not tested this. Diffusion-Free scores 70% to 83% on Push-T. On Reacher it reaches 93% on the training quadrant and 20% on the held-out quadrants, below Diffusion Policy (100% and 29%). Push-T demonstrations are multimodal, since the block can be approached from either side, and diffusion was introduced to behaviour cloning to model such distributions (Pearce et al., 2023; Chi et al., 2024); the drop is consistent with this. We ran this ablation only at 0.98, and Liu et al. report that a fixed 95% ratio works poorly with a regression loss, so it shows that regression fails in our high-ratio regime, not at every ratio. The fixed masks score 84% to 96% (88% mask) and 83% to 93% (98% mask), against 90% to 98% with random masks. The gap is largest under noisy actions (83–84% against 93%) and in episode length on the vanilla task (207–208 steps against 190).

Diffusion Forcing on Reacher. We did not run this comparison. Reacher is goal-conditioned, and Diffusion Forcing as published specifies goals through guidance at sampling time, so it has no established goal input. In our architecture the goal could enter through cross-attention, as in our model, or be appended to the observation, as for Diffusion Policy, and the result may depend on that choice. For both Inpainting and Diffusion Policy we compared two such options and kept the better one; we could not do the same here within the response period, and will add the comparison with both options.

Hunter-Prey (Table 2). All three reviewers note that Diffusion Policy (obs-to-all) also reaches a 100% catch rate and is faster (40 versus 53 steps). We agree: observation prediction, not masking, accounts for the result. The task was designed to test this. Diffusion Policy models actions alone to avoid "the cost of inferring future states" (Chi et al., 2024); here that choice fails. In all four pairs of variants, for Diffusion Policy and for our model alike, predicting observations raises the catch rate from 0.34–0.42 to 1.00, which agrees with PAD and UVA in image-based settings. The two new baselines, which also predict observations, reach 1.00 as well. Conditioning on past actions then slows both methods: by 76 steps for Diffusion Policy (40 to 116) and by 9 for our model (44 to 53). One possible explanation is causal confusion (de Haan et al., 2019), in its copycat form (Wen et al., 2020): a policy that sees its past actions can learn to continue them instead of tracking the prey. Random masking hides past actions for most of training, which may be why our model is affected less; we have not tested this.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.