acceptodds
Under review as a conference paper at ICLR 2027

Decoding and Steering Contact Structure in Trajectory Diffusion

Abstract

When can contact structure read from a partially denoised trajectory support useful intervention? We study trajectory diffusion in three simulated multi-body systems and separate three quantities that are easily conflated: the error of a probe that reads intermediate states, how concentrated the final outcomes of a state are under repeated stochastic continuation, and the realized utility of acting on the readout. Repeated continuations from the same state show that outcomes concentrate during denoising and that, from the middle of the schedule onward, readout error rather than residual randomness dominates the probe's prediction error. The readout is nevertheless directionally useful: one-shot probe ascent transfers from a distance-based proxy target to an orientation-aware geometric diagnostic in all six fixed models tested, and selected multi-step guidance raises held-out joint target-and-quality success by 12.7–20.1 percentage points in the two three-dimensional systems and by 25–30 points in the two-dimensional system under a scene-consistent quality envelope. The same readout is not usable for selection: probability-gated early rejection establishes no gain in yield per computation, although a sixteen-branch empirical probability of the same event, if it were available at no cost, would. A short invariance argument shows what each use requires. Neither requires calibration, so the probe's overconfidence after optimization is beside the point; guidance needs the probe's gradient to align locally with the true success probability, whereas selection needs its scores to rank states like that probability, and the contact readout ranks poorly. The limitation is ranking, not calibration and not a structural limit of selection. We provide the branching, paired-intervention and held-out policy protocols that separate readout error, outcome concentration and intervention utility, with geometric correspondence, trajectory quality and computation reported explicitly.

Primary area:
generative models
Keywords:
diffusion models, trajectory generation, contact structure, probing, controllability, sampling dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8mLMOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether the contact outcome of a trajectory diffusion model can be judged midway through denoising. It trains probes on intermediate denoising states to read out contact information, and compares two uses of these probes: guiding trajectories and discarding samples early. Experiments in simulated scenes show that guidance helps while early discarding does not pay off. However, the conclusions are validated only in simple simulated scenes, so their generality is unclear.

Critical strengths
  1. The paper gives an interesting experimental finding: mid-denoising judgments about contact can be used to guide trajectories, but not to discard samples early.

  2. The criterion for when early discarding pays off, and the analysis method of copying the same intermediate state and finishing it with different noise, are simple and practical. They are useful references for "mid-way pruning" designs in diffusion models and for understanding the denoising process.

Critical weaknesses
  1. The experiments are rigorous but conducted only in simulated scenes. It is unclear whether the conclusions extend to larger scenes, pretrained models, or real planning tasks.

  2. The writing is hard to read. There are many self-coined terms ("mothers", "frozen/secondary envelope", "families", "uncharged oracle", etc.), key settings are scattered across the main text and multiple appendices, and the main experiments and conclusions are hard to grasp from the abstract alone.

  3. The main text has too few figures and tables, while the evidence supporting the core conclusions is mostly in the appendix. I suggest condensing the prose and moving the most important figures and tables into the main text.

Questions for authors

pls see weaknesses.

Note: If the authors can add validation in real application scenarios, or convince me that the current experiments are sufficient, I will raise my score.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 8mLM (1/2): a released pretrained policy on a planning task (W1)

We thank the reviewer for the clear summary and for stating what would change the assessment.

W1. "The experiments are rigorous but conducted only in simulated scenes. It is unclear whether the conclusions extend to larger scenes, pretrained models, or real planning tasks."

Q "If the authors can add validation in real application scenarios, or convince me that the current experiments are sufficient, I will raise my score."

We added this validation: a public pretrained model on a manipulation planning task (new Section 4.6, Table 2, Appendix L). We applied the same measurements to the released Diffusion Policy for Push-T (Chi et al., RSS 2023), with the authors' released checkpoint: a 100-update DDPM that plans a 16-step trajectory of actions and observations, conditions on the latest observations and executes eight actions per chunk in closed loop. It is a model we did not train, with its own horizon, noise schedule and conditioning. The contact events are whether the pusher touches the bar or the stem of the T during the executed chunk, and every outcome is measured by executing the actions. All four findings of the paper reappear (readout and steering on 1,024 held-out decision states from 256 episodes):

Finding in our scenesReleased Diffusion Policy on Push-T
Contact structure becomes readable during denoisingProbe AUROC rises from 0.74 (bar) and 0.66 (stem) with all 100 updates remaining to 0.97 and 0.94
Outcomes concentrate while readout error dominatesAt 75 remaining updates two continuations of a state share their contact graph 87% of the time (31% for different states); readout error is 84–88% of the probe's squared error
Early rejection on the probe does not pay; the limit is rankingProbe-scored gate 1.00–1.03 times the no-gate yield; empirical probability from 16 continuations, its cost set aside, 1.43 and 1.18 at 75 and 50 remaining updates
Guidance steers the contactsRequested contact 48.9% to 65.2% (+16.3 points [13.3, 19.4]); requested contact with the task solved 23.6% to 32.6% (+9.0 [5.8, 12.1]); task success 48.7% and 50.2%

The probe's direction is what holds the pusher off a requested part of the T: requested absence rises by +13.7 (bar) and +10.9 (stem) points, which is +12.5 [8.2, 17.0] and +10.2 [6.5, 14.3] points more than random steps of the same per-update norm (Table 2 lists every arm).

Why the evidence is sufficient for the paper's claims.

  1. The division between steering and selection holds beyond our models. It appears in two cube scenes with three independently trained denoisers each, and in a released policy for a closed-loop manipulation task.
  2. Steering is confirmed by physics, not by a proxy. In the cube scenes every generated initial state is executed in the simulator and scored with the simulator's own force-based contact rule. Guidance raises the rate at which the requested contacts actually occur by +12.7 to +31.2 points in all twelve held-out cells (random steps: at most 1.2 points), and by +5.0 to +25.1 points for three to five simultaneous contacts (Section 4.4, Figure 3).
  3. The explanation is not only empirical. Proposition 1 states what each use requires, alignment for guidance and ranking for selection, and the new fixed-target comparison confirms the ranking limit in every model (Section 4.5, Figure 4).
  4. The protocol transfers, and simulation makes it exact. It needs only repeated continuation and an outcome label, which is why the same measurements apply directly to a released policy. Simulation is what permits exact outcome labels for repeated continuations of the same intermediate state, paired interventions that share their future noise, and the execution of every sample.

The reviewer noted that our continuation analysis is a useful reference for "mid-way pruning" designs. To make it easy to reuse, the supplementary material contains the code to rerun every experiment, including the pretrained-policy study, and our trained denoisers and probes are available at an anonymous link: https://osf.io/xbjwq/overview?view_only=417b35cca1f145a9a438cdea02d040ce (also in Appendix J).

We hope the validation on a released pretrained policy and the four points above convince the reviewer that the current experiments are sufficient, and we would be grateful if the reviewer would reconsider the score.

AuthorsOpenReview ↗

Response to Reviewer 8mLM (2/2): terms, settings, figures and tables (W2, W3)

W2. "The writing is hard to read. There are many self-coined terms ("mothers", "frozen/secondary envelope", "families", "uncharged oracle", etc.), key settings are scattered across the main text and multiple appendices, and the main experiments and conclusions are hard to grasp from the abstract alone."

We revised all three points.

Terms. Every coined term is replaced throughout the paper:

SubmissionRevision
motherintermediate state
branchcontinuation
family (of comparisons)"Bonferroni-adjusted over k comparisons"
frozen / secondary enveloperemoved: success is verified by executing the generated initial state in the simulator
uncharged oracleempirical probability from sixteen continuations, its cost reported separately
NFEdenoiser updates

Settings in one place. The new Table 1 lists every main study with its models, inference units, comparison and outcome.

Abstract. The abstract now states the main experiments with their numbers: guidance raises simulator-verified success by 12.7–31.2 points where random steps change it by at most 1.2; within fixed targets a probe-scored gate yields 0.98–1.02 times the no-gate yield where an empirical probability, its own cost set aside, raises it by 6–9.5%; and the released Push-T policy shows the same division.

W3. "The main text has too few figures and tables, while the evidence supporting the core conclusions is mostly in the appendix. I suggest condensing the prose and moving the most important figures and tables into the main text."

We followed this suggestion. Each core conclusion now has its evidence in the main text:

Core conclusionMain-text evidence
Readout error, not residual randomness, dominates from mid-scheduleFigure 1
Probe directions transfer to a geometric diagnosticFigure 2
Guidance raises simulator-verified successFigure 3 (new): all twelve held-out cells and the three- to five-contact requests, each with its random control
Early rejection is limited by rankingFigure 4 (new): gate yield within fixed targets, probe against empirical probability
The findings hold on a released policyTable 2 (new), Section 4.6
Studies, units and outcomesTable 1 (new)

The prose was condensed to make room: the main text now carries four figures and two tables within ten pages. Sections 4.1–4.5, Proposition 1 and Figures 1–2 keep their numbers.

Thanks for the rebuttal and new experiments. The paper is improved, but I still have concerns about the generality and reliability of the results.

The Push-T experiment is helpful, but it's still a simple simulated task. More importantly, the code inconsistencies pointed out by RF47 are quite concerning. Although the authors have corrected them, I'm still not fully confident about the reproducibility of the results.

Without these code issues, I would be happy to raise my score to 3.

I was considering a score between 2 and 3, but since that option isn't available, I'll keep my original score for now.

Reviewer_RF47OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies whether contact predictions from partially denoised trajectories can guide generation or support early rejection. It separates probe error, uncertainty across repeated continuations, and intervention utility, and gives a short argument about recalibration. Experiments use two cube-drop scenes and a 2D disc scene, with paired interventions and separate policy-selection and evaluation cohorts. Guidance improves joint target-and-quality success by 12.7–20.1 percentage points in the cube scenes, while early rejection does not establish a gain in success per computation. The larger 2D gains depend on a post-hoc relaxation of the quality threshold.

A reproduction attempt checked several implementation components successfully, but missing datasets, data-generation scripts, and required experiment configurations prevented reproduction of the reported results.

Critical strengths
  1. The separation of probe error, sampling uncertainty, and intervention utility is useful. Repeated continuations show that increasingly consistent outcomes can still be poorly predicted. This gives a clearer account of what intermediate probe accuracy does and does not tell us.

  2. The steering experiments include meaningful controls: paired future noise, matched random perturbations, and separate policy-selection and evaluation cohorts. Positive effects on the geometric diagnostic across six cube-scene models strengthen the evidence that the probe gradients carry a useful steering signal.

Critical weaknesses
  1. The main conclusions need stronger validation. The cube quality checks allow substantial penetration, while the 2D checks cannot detect overlap and the large 2D gains depend on a post-hoc relaxation of the threshold. This leaves the physical significance of the steering gains unclear. The explanation of gating failure also relies on retrospective cohorts with a different target mixture and threshold protocol, using an oracle whose branching cost is excluded. That comparison shows a ranking gap on those cohorts, but does not establish why the prospective gate failed.

  2. The submitted package is insufficient to reproduce the reported results, and the code disagrees with the experimental description. Component checks passed, but missing datasets, data-generation scripts, and required configurations prevented reproduction of the main results. The code uses Adam rather than the stated AdamW, a probe learning rate of 5e-4 rather than 1e-3, and fixed sinusoidal rather than learnable timestep embeddings. These discrepancies leave it unclear which configuration produced the results and weaken confidence in the experimental record.

Questions for authors
  1. Do the steering gains persist under quality checks that reject substantial penetration and disc overlap? This would help establish whether the reported improvements extend beyond satisfying the contact proxy and current empirical quality criteria.

  2. What evidence attributes the prospective gating failure to poor ranking under that same task distribution? The retrospective comparison uses a different target mixture and threshold protocol. Can probe and empirical-probability scores be compared on states matching the prospective evaluation, with the estimation costs reported separately?

  3. Which training settings produced the reported results? The code and paper disagree on Adam versus AdamW, the probe learning rate, and fixed versus learnable timestep embeddings. Can you resolve these differences and provide the missing datasets or generation scripts and experiment configurations needed to reproduce the main results?

4 comments
AuthorsOpenReview ↗

Response to Reviewer RF47 (1/3): physical validity of the steering gains (W1, Q1)

We thank the reviewer for the reproduction attempt and for pressing on physical validity. Both led to the main additions of the revision.

W1. "The cube quality checks allow substantial penetration, while the 2D checks cannot detect overlap and the large 2D gains depend on a post-hoc relaxation of the threshold. This leaves the physical significance of the steering gains unclear."

Q1. "Do the steering gains persist under quality checks that reject substantial penetration and disc overlap?"

In the cube scenes they persist under a stricter standard than a quality check: physics itself. A generated trajectory predicts what the simulator would do, so we now score it by running the simulator (Sections 3.3, 4.1 and 4.4, Appendix C). The generated initial state (positions, orientations and velocities of all cubes) is executed in the unchanged scene generator, and a success requires that the requested contacts occur in the simulated rollout under the generator's own rule, a MuJoCo contact with a normal force of at least 0.01 N, the rule that labels the training data. No trajectory-quality threshold enters. Cube masses, which are not part of a state, are drawn from the generator's range, identically in paired arms, with four draws per proposal.

Gain of selected guidance over the unmodified sampler in simulator-verified success (percentage points; 512 paired held-out proposals per cell; 95% intervals Bonferroni-adjusted over 8):

SceneDenoiser1 requested contact2 requested contactsRandom control (1 / 2)
Bounded drop1+16.7 [12.1, 21.3]+14.3 [9.6, 19.2]0.0 / +0.5
Bounded drop2+14.7 [10.3, 19.4]+13.2 [8.7, 17.8]-0.9 / 0.0
Bounded drop3+19.1 [14.4, 23.9]+12.7 [8.5, 16.9]-0.9 / -0.3
Cluster drop1+28.0 [22.4, 34.0]+27.7 [21.9, 33.4]0.0 / +0.1
Cluster drop2+31.2 [25.7, 37.0]+26.5 [21.1, 31.9]+1.2 / -0.1
Cluster drop3+26.8 [21.2, 32.4]+31.2 [25.4, 37.0]+0.9 / +0.6
  • Every cell. Success rises from 26.9–44.1% (unmodified) to 44.3–71.3% (guided); all twelve adjusted intervals exclude zero, with the smallest lower bound at +8.5 points.
  • Direction, not perturbation. Guidance replaced by a random direction of the same norm changes success by -0.9 to +1.2 points.
  • Larger requests. Three, four and five simultaneous contacts gain +9.3, +6.2 and +5.0 points in bounded drop and +25.1, +23.0 and +10.5 points in cluster drop (all six adjusted intervals above zero).
  • One step. A single probe-ascent step at 600 remaining updates beats a random step of the same size by +12.7 to +13.8 points (bounded drop) and +18.1 to +18.7 points (cluster drop) in all six models.
  • Penetration. The simulated rollouts have the penetration statistics of real data: median 0.4–0.6 mm after the initial transient, against 0.6 mm (Appendix C).
  • No dependence on the earlier criterion. Ranking the candidate configurations by simulator-verified success selects the same configuration in all twelve selection tasks. On the submission's proxy outcome the same cohorts gave +10.7 to +23.6 points, so the physical standard does not shrink the effect.

Disc scene. We agree that its threshold was set post hoc. The revision removes the disc-scene steering analysis; the steering claims of the paper rest on the simulator-verified results above and on the released Diffusion Policy for Push-T, where every outcome is measured by executing the actions (Section 4.6).

AuthorsOpenReview ↗

Response to Reviewer RF47 (2/3): gating on the prospective task distribution (W1, Q2)

W1. "The explanation of gating failure also relies on retrospective cohorts with a different target mixture and threshold protocol, using an oracle whose branching cost is excluded."

Q2. "Can probe and empirical-probability scores be compared on states matching the prospective evaluation, with the estimation costs reported separately?"

Yes. We ran this comparison (Section 4.5, Figure 4, Appendix K), and it removes each difference the reviewer names.

  • State distribution. States are drawn by the unmodified sampler at the gate times of the prospective evaluation (600, 400 and 200 remaining updates): six models, 128 states each, 32 continuations per state.
  • Target mixture. Each of the twenty supported single-edge targets per model and time is evaluated on its own update budget, so a gate cannot gain by preferring easier targets.
  • Threshold protocol. Thresholds are per-target rank thresholds (keep the top 90%, ..., 10% of a target's states), which no recalibration can change; the keep fraction is chosen on one half of the states and evaluated on the other.
  • Outcome. Simulator-verified contact, as in Response 1/3.
Remaining updatesProbe-scored gateEmpirical-probability gate (16 continuations)
6000.979–1.000 (0 of 6 above 1)1.073–1.092 (6 of 6 above 1)
4001.001–1.020 (0 of 6 above 1)1.062–1.095 (6 of 6 above 1)
2001.000–1.013 (0 of 6 above 1)1.015–1.027 (2 of 6 above 1)

Held-out successes per denoiser update relative to no gate, range over the six models; "above 1" counts 95% intervals Bonferroni-adjusted over 18 groups. The empirical probability's own sixteen continuations are not counted here; their cost follows below.

At 600 remaining updates the empirical-probability gate improves the held-out yield of 120 of 120 model/target combinations, the probe-scored gate of 9. On the targets of the first denoiser's prospective budget blocks, the ratios are 1.06–1.15 (empirical probability) and 0.99–1.04 (probe) at 600 and 400 remaining updates. On the same states, the same targets and the same thresholds, a score that ranks well makes early rejection pay and the probe's score does not: the probe's ranking is what limits gating, not the gate design, the threshold protocol or the target mixture. In the earlier mixed-target cohorts, 75–98% of the empirical probability's gain is within-target.

Estimation cost, reported separately. The empirical probability uses sixteen continuations per state. Charged in full as overhead, its yield is 0.08–0.24 of the no-gate yield (Appendix K), near the ceiling 1/(1+15r/T) = 0.10, 0.14 and 0.25 that any sixteen-continuation estimate has at r = 600, 400 and 200. It is the measurement reference of the comparison: it shows that the states of a target can be ranked well enough at these times, and what a readout with that ranking, at the cost of a forward pass, would deliver. The conclusion is worded accordingly in Sections 1, 4.5 and 5: the prospectively selected probe gates establish no benefit, and within fixed targets the limiting factor is the probe's ranking.

The same result on a released policy. On the Diffusion Policy for Push-T, the probe-scored gate yields 1.00–1.03 and the empirical probability 1.43 and 1.18 at 75 and 50 remaining updates, both adjusted intervals above one (Section 4.6, Table 2).

AuthorsOpenReview ↗

Response to Reviewer RF47 (3/3): training settings and reproduction package (W2, Q3)

W2. "The submitted package is insufficient to reproduce the reported results, and the code disagrees with the experimental description."

Q3. "Which training settings produced the reported results? [...] Can you resolve these differences and provide the missing datasets or generation scripts and experiment configurations needed to reproduce the main results?"

Thank you for running the component checks and for listing the three disagreements exactly.

The code is the record, and the appendix now states it (Appendices E and F). Every number reported for the simulated scenes was produced with the settings in the right-hand column.

SettingSubmitted appendixCode that produced the results (now in the appendix)
Denoiser optimizerAdamWAdam (learning rate 2e-4, batch 128, gradient clip 1.0, EMA 0.9999)
Probe optimizer and learning rateAdamW, 1e-3Adam, 5e-4 with cosine annealing (batch 256, 100 epochs, dropout 0.1)
Probe timestep embeddinglearnablefixed sinusoidal, dimension 64

The denoiser uses a fixed sinusoidal timestep embedding of dimension 128 followed by a two-layer MLP. The probes use an 80/20 train/validation split of the cached intermediate states and keep the checkpoint with the best validation score.

The package now reproduces the results from the generators onward.

  1. Datasets. The three scene generators with their configurations and seeds. Each regenerates its training, validation and test arrays bit for bit, and the package lists the SHA-256 of every array.
  2. Configurations. The frozen configuration file and the exact command of every experiment and analysis run, with a guide that maps each main-text figure and result to its script.
  3. Code. Denoiser and probe training, repeated continuation and the Brier decomposition, paired interventions, held-out guidance and gating policies, and the new analyses of this response: the simulator replay, the fixed-target gates and the pretrained-policy study.
  4. Checkpoints. The trained denoisers and probes, at an anonymous link: https://osf.io/xbjwq/overview?view_only=417b35cca1f145a9a438cdea02d040ce (one archive with the 14 checkpoints in the folder layout the configurations expect, a README and SHA-256 sums).

The Push-T checkpoint and demonstrations are the public release of Chi et al. (2023), identified in the package by their SHA-256.

Each question now has a direct answer.

  • Q1 (Response 1/3). In the cube scenes, the steering gains hold under physics itself, with the controls the reviewer noted: paired future noise, matched random perturbations, and separate policy-selection and evaluation cohorts. Guidance raises simulator-verified success by +12.7 to +31.2 points in all twelve held-out cells; random steps change it by at most 1.2 points.
  • Q2 (Response 2/3). On states matching the prospective evaluation, within fixed targets and with per-target rank thresholds for both scores, an empirical probability makes early rejection pay in every model at the two earlier gate times and the probe's score does not, with the estimation cost reported separately: the probe's ranking is what limits gating.
  • Q3 (above). The appendix states the training settings that produced the results; the package regenerates every dataset bit for bit and reruns every experiment, and the trained denoisers and probes are at the anonymous link.

We hope these results resolve the concerns behind the soundness assessment, and we would be grateful if the reviewer would reconsider the score.

AuthorsOpenReview ↗

Response to Reviewer RF47: A short summary

Thank you again for the reproduction attempt and for the three precise questions, which led to the main additions of the revision. Before the discussion period closes, we summarize the answers:

  • Q1, physical validity. Steering is now scored by physics itself: every generated initial state is executed in the scene's simulator, and a success requires the requested contacts under the simulator's own force-based contact rule, with no trajectory-quality threshold. Guidance raises verified success by +12.7 to +31.2 points in all twelve held-out cells, random steps of the same norm change it by at most 1.2 points, and the simulated rollouts have the penetration statistics of real data (Section 4.4, Figure 3, Appendix C).
  • Q2, gating on the prospective distribution. On states matching the prospective evaluation, within fixed targets and with per-target rank thresholds for both scores, the probe-scored gate stays at the no-gate yield (0.98–1.02), while an empirical probability of the same event raises it by 6.2–9.5% in all twelve model/time groups at the two earlier gate times. Its estimation cost is reported separately (Section 4.5, Figure 4, Appendix K).
  • Q3, settings and reproduction. Appendices E and F state the settings of the code that produced the results, and no reported number changed. The supplementary material regenerates every dataset bit for bit and gives the configuration and exact command of every run, and the 14 trained checkpoints (denoisers and probes) are at the anonymous link.

If these results resolve the concerns behind your soundness assessment, we would be grateful if you would reconsider your rating.

Reviewer_FNDzOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper analyzes whether contact predictions from intermediate diffusion states can be used to guide trajectory generation and reject samples early. It distinguishes three quantities: probe readout error, variation in final results across stochastic continuations from the same intermediate state, and the effectiveness of probe-based interventions. The authors argue that guidance needs gradients that point toward higher success probability, while early rejection needs scores that rank states well. In the 3D experiments, probe-based guidance consistently increases the fraction of samples that meet both the contact target and the empirical quality criteria. While the tested prospective early-rejection policies show no clear gain per unit of computation, a retrospective analysis show a gain when the cost of estimating these probabilities is excluded.

Critical strengths
  • The authors separate three quantities that are easy to confuse and evaluate them against the actual final results.

    • Repeated continuations separate the remaining stochastic uncertainty from probe readout error, while interventions on the same state can measure actual changes in target success. The finite branch correction, intermediate-level resampling, and independent policy evaluation provide clearer evidence than probe accuracy curves alone. This measurement protocol is a useful methodological contribution.
  • The positive guidance shows consistent improvement across several models and controls

    • Both one-shot and multi-step guidance improve success rates, with positive results on additional denoiser seeds and larger edge conjunctions.
Critical weaknesses
  • The explanation of selection failure is somewhat stronger than the evidence directly supports

    • Prospective gate selection only searches thresholds of 0.25, 0.5, 0.75 and 0.9. However, authors report only 0.05, 0.1 retrospective thresholds whenever it chooses a nontrivial gate in Appendix B.2. I believe that the prospective results alone do not fully separate poor ranking from a failure to search useful thresholds. The monotone-transformation invariance in Proposition 1 concerns the full family of thresholds and does not guarantee equal performance on a fixed numerical grid. The retrospective comparison also mixes different targets, while each prospective budget block keeps the same target through respawns. This makes it difficult to determine how much of the retrospective gain comes from distinguishing states within a fixed target. I think the conclusion should be more specific: the tested prospective gate shows no clear benefit, while the retrospective analysis suggests that limited discrimination contributes to this result.
Questions for authors
  • Could the authors clarify how directly the retrospective oracle comparison explains the prospective results?
    • In the retrospective analysis, how much of the empirical-probability reference's gain can be attributed to distinguishing states within a fixed target, rather than differences in difficulty between targets? any evidence from the existing analysis that separates these effects would help to assess the explanation of selection failure.
3 comments
AuthorsOpenReview ↗

Response to Reviewer FNDz (1/2): how much of the gain is within a fixed target

We thank the reviewer for the careful reading and for a precise question. We answer it with a new analysis (Section 4.5, Figure 4, Appendix K).

"The retrospective comparison also mixes different targets, while each prospective budget block keeps the same target through respawns. This makes it difficult to determine how much of the retrospective gain comes from distinguishing states within a fixed target."

"In the retrospective analysis, how much of the empirical-probability reference's gain can be attributed to distinguishing states within a fixed target, rather than differences in difficulty between targets?"

Most of it: 75–98%. We split the success gain among the states that each retrospective gate keeps into a between-target part (the kept states have easier targets) and a within-target remainder. For the empirical-probability gate, 75–98% of the gain is within-target in all eighteen model/time groups (Appendix K; the analysis you cite remains Appendix B.2).

A direct test within fixed targets. We then removed target mixing altogether. The unmodified continuations do not depend on a target, so every supported single-edge target (twenty per model and time) can be scored on all 128 states of a cohort: six models, 32 continuations per state, simulator-verified outcomes. The states are drawn by the same unmodified sampler, at the same gate times, as in the prospective evaluation. Each target is evaluated on its own update budget, so a gate cannot gain by preferring easier targets, and the keep fraction is chosen on one half of the states and evaluated on the other.

Remaining updatesProbe-scored gateEmpirical-probability gate (16 continuations)
6000.979–1.000 (0 of 6 above 1)1.073–1.092 (6 of 6 above 1)
4001.001–1.020 (0 of 6 above 1)1.062–1.095 (6 of 6 above 1)
2001.000–1.013 (0 of 6 above 1)1.015–1.027 (2 of 6 above 1)

Held-out successes per denoiser update relative to no gate, range over the six models; "above 1" counts 95% intervals Bonferroni-adjusted over 18 groups. The empirical probability's own sixteen continuations are not counted here; they are reported separately (Response 2/2).

At 600 remaining updates the empirical-probability gate improves the held-out yield of 120 of 120 model/target combinations, the probe-scored gate of 9. On the targets of the first denoiser's prospective budget blocks, the ratios are 1.06–1.15 for the empirical probability and 0.99–1.04 for the probe at 600 and 400 remaining updates.

The retrospective comparison therefore explains the prospective result directly: on the prospective state distribution and within fixed targets, the probe does not rank the states of a target well enough to repay the rejected prefixes, and an empirical probability of the same event does. The same comparison on the released Diffusion Policy for Push-T gives 1.00–1.03 for the probe and 1.43 and 1.18 for the empirical probability at 75 and 50 remaining updates (Section 4.6).

AuthorsOpenReview ↗

Response to Reviewer FNDz (2/2): threshold grid and the wording of the conclusion

"Prospective gate selection only searches thresholds of 0.25, 0.5, 0.75 and 0.9. However, authors report only 0.05, 0.1 retrospective thresholds whenever it chooses a nontrivial gate in Appendix B.2. [...] The monotone-transformation invariance in Proposition 1 concerns the full family of thresholds and does not guarantee equal performance on a fixed numerical grid."

We agree, and the new comparison no longer depends on a numerical grid. It uses rank thresholds: for each target the gate keeps the top 90%, 80%, ..., 10% of the states by score. A rank threshold is unchanged by any increasing recalibration of the score (Proposition 1(b)), so the comparison covers the family of keep sets itself, and no choice of grid values can favor one score. Figure 4 shows the full curves over keep fractions for all six models.

A finer numerical grid does not rescue the probe either. With thresholds from 0.05 to 0.9, the probe's best threshold is always the lowest one, 0.05, the gate that rejects least, with in-sample yield 0.92–1.03 times the no-gate yield (Appendix K).

"I think the conclusion should be more specific: the tested prospective gate shows no clear benefit, while the retrospective analysis suggests that limited discrimination contributes to this result."

We adopt this formulation, and the fixed-target comparison lets us state its second half as a measured result. The revised text (Sections 1, 4.5 and 5) says:

  • the prospectively selected probe gates establish no benefit;
  • within fixed targets, on the same states, the probe-scored gate stays at the no-gate yield while an empirical probability of the same event exceeds it in every model at the two earlier gate times, so the limiting factor is the probe's discrimination among the states of a target, not the decision rule.

Cost of the reference. The empirical probability uses sixteen continuations per state, which we report separately: charged in full, its yield is 0.08–0.24 of the no-gate yield (Appendix K). Its role is to measure what a readout with that ranking would deliver; the constructive implication, stated in Section 5, is to train readouts for ranking.

The other half of the asymmetry is also stronger. The reviewer listed the consistent guidance gains as a strength. They are now verified in the simulator: with every generated initial state executed and scored by the simulator's own contact rule, guidance raises success by +12.7 to +31.2 points in all twelve held-out cells, and by +5.0 to +25.1 points for requests of three to five simultaneous contacts (Section 4.4, Figure 3).

The concern now has a direct answer.

  • The question (Response 1/2). 75–98% of the empirical probability's gain is within-target. Within fixed targets, on the prospective state distribution and with rank thresholds that no recalibration changes, the probe-scored gate stays at the no-gate yield while the empirical probability, its own cost set aside, exceeds it in every model at the two earlier gate times.
  • The conclusion (above). It now uses the reviewer's formulation, with its second half a measured result.
  • The protocol (Section 4.6). The measurement protocol that the reviewer called a useful methodological contribution now also applies to a released pretrained policy, the Diffusion Policy for Push-T, which shows the same division between steering and selection.

We hope this resolves the concern, and we would be grateful if the reviewer would consider raising the score.

AuthorsOpenReview ↗

Response to Reviewer FNDz: A short summary

Thank you again for your review and for the precise question about fixed targets. Before the discussion period closes, we summarize how the revision answers it:

  • Within a fixed target. 75–98% of the empirical-probability gate's retrospective gain is within-target, in all eighteen model/time groups (Appendix K). Scored directly within fixed targets, on the state distribution of the prospective evaluation, the probe-scored gate stays at the no-gate yield (0.98–1.02), while the empirical probability, its own cost set aside, raises the yield by 6.2–9.5% in all twelve model/time groups at the two earlier gate times (Section 4.5, Figure 4).
  • No dependence on a threshold grid. The comparison uses per-target rank thresholds, which no increasing recalibration of a score changes (Proposition 1(b)).
  • The conclusion in your formulation. Sections 1, 4.5 and 5 now state that the prospectively selected probe gates establish no benefit and that, within fixed targets, the limiting factor is the probe's discrimination among the states of a target. The second half of your formulation is now a measured result.
  • The strengths you named are stronger. The guidance gains are now verified in the simulator, +12.7 to +31.2 points in all twelve held-out cells (Section 4.4, Figure 3), and the measurement protocol also applies to the released Diffusion Policy for Push-T, which shows the same division between steering and selection (Section 4.6).

If these results resolve your concerns, we would be grateful if you would consider raising your rating.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of the revision (to all reviewers and the ACs)

We thank the reviewers for careful and constructive reviews. All three recognized the paper's central finding, that a contact readout of a partially denoised trajectory supports steering but not early rejection, and its measurement protocol. The revision keeps the submission's central finding and structure, and adds the evidence the reviews asked for:

  1. Steering is verified by physics (Sections 3.3 and 4.4, Figure 3). Every generated initial state is executed in the scene's simulator and scored with the simulator's own force-based contact rule. Guidance raises verified success by +12.7 to +31.2 percentage points in all twelve held-out cells; random steps of the same norm change it by at most 1.2 points.
  2. Gating is compared within fixed targets, on the state distribution of the prospective evaluation (Section 4.5, Figure 4). The probe-scored gate stays at the no-gate yield (0.98–1.02); an empirical probability of the same event, its cost reported separately, raises the yield by 6.2–9.5% in all twelve model/time groups at the two earlier gate times.
  3. A released pretrained policy (new Section 4.6, Table 2). The findings reappear on the released Diffusion Policy for Push-T, a model we did not train: its contact structure becomes readable during denoising, a probe-scored gate does not pay, and guidance holds the pusher off a requested part of the block in 11–14 points more of the executed action chunks, where random steps of the same norm add 1 point, without costing task success.
  4. Reproducibility (Appendices E, F and J, supplementary material). The package regenerates every dataset bit for bit and contains the configuration and command of every run; the appendix states the training settings of the code.
  5. Presentation. Coined terms are replaced by plain ones, Table 1 lists the main studies, and the evidence for each core conclusion is a figure or table in the main text.

Sections 4.1–4.5, Proposition 1 and Figures 1–2 keep their numbers, Appendices A, B and D–J keep their letters, and Appendix B.2 keeps its number. Detailed responses follow under each review.

The central finding now rests on physics-verified outcomes. In all six cube-scene models and on a released pretrained policy, the same contact readout steers the executed contacts but does not rank states well enough for early rejection to pay, and the protocol that shows this needs only repeated continuation and an outcome label. We hope the revision resolves the reviewers' concerns, and we would be grateful if they would reconsider their assessments.

AuthorsOpenReview ↗

Summary of the rebuttal for the Area Chairs: where each review point is addressed

We thank the Area Chairs for overseeing the review of our submission. As the discussion period closes, we summarize what the reviewers value, how the revision answers each review point, and where the discussion stands; the detailed responses are under each review.

What the reviewers value. All three reviewers recognized the central finding, that a contact readout of a partially denoised trajectory supports steering but not early rejection, and the protocol that measures it. FNDz called the protocol "a useful methodological contribution" and noted that guidance "shows consistent improvement across several models and controls"; RF47 found the separation of probe error, sampling uncertainty and intervention utility "useful" and the steering controls "meaningful"; 8mLM called the finding "interesting" and the continuation analysis "simple and practical".

Each review point and its answer.

  1. Physical validity of the steering gains (RF47 W1, Q1). Every generated initial state is now executed in the scene's simulator, and a success requires the requested contacts under the simulator's own force-based contact rule, with no trajectory-quality threshold. Guidance raises verified success by +12.7 to +31.2 points in all twelve held-out cells, random steps of the same norm change it by at most 1.2 points, and the simulated rollouts have the penetration statistics of real data (Section 4.4, Figure 3, Appendix C). The paper's steering claims rest on these executed outcomes and on those of item 3.
  2. Why early rejection does not pay (FNDz; RF47 W1, Q2). On states matching the prospective evaluation, within fixed targets and with per-target rank thresholds that no recalibration changes, the probe-scored gate stays at the no-gate yield (0.98–1.02), while an empirical probability of the same event, its estimation cost reported separately, raises it by 6.2–9.5% in all twelve model/time groups at the two earlier gate times; 75–98% of its retrospective gain is within-target (Section 4.5, Figure 4, Appendix K). This answers both reviewers' questions, and the conclusion now uses FNDz's formulation, with its second half a measured result (Sections 1, 4.5 and 5).
  3. A released pretrained policy (8mLM W1). All four findings reappear on the released Diffusion Policy for Push-T, a model we did not train, with every outcome measured by executing the actions: guidance raises the rate of the requested contact by +16.3 points, and the probe-scored gate stays at 1.00–1.03 times the no-gate yield while the empirical probability, its cost set aside, reaches 1.43 and 1.18 (Section 4.6, Table 2).
  4. Settings and reproduction (RF47 W2, Q3; 8mLM's comment). Appendices E and F state the settings of the code that produced the results, and no reported number changed. The supplementary material regenerates every dataset bit for bit and gives the configuration and exact command of every run, and the 14 trained checkpoints (denoisers and probes) are at an anonymous link.
  5. Presentation (8mLM W2, W3). Coined terms are replaced by plain ones, the abstract states the main experiments with their numbers, Table 1 puts the settings of every main study in one place, and the evidence for each core conclusion is a figure or table in the main text.

Where the discussion stands. In the discussion, 8mLM found the paper improved and the Push-T experiment helpful, and named the code issues raised by RF47 as the main remaining concern and what keeps the score at 2: "Without these code issues, I would be happy to raise my score to 3." Our reply answers these issues point by point (items 3 and 4). Each of RF47's three questions now has a direct answer (items 1, 2 and 4), and FNDz's question is answered by the decomposition the reviewer asked for (item 2).

In sum, the central finding now rests on physics-verified outcomes in six cube-scene models and on a released pretrained policy, its explanation is a measured ranking limit within fixed targets, and every result can be checked from the supplementary material. We thank the Area Chairs and the reviewers for their time and care.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.