acceptodds
Under review as a conference paper at ICLR 2027

EventCone-WM: Counterfactual Event Cones for Reliable Action-Conditioned Latent Planning

Abstract

Action-conditioned world models can predict plausible futures while propagating an action's effects to entities it cannot physically reach. We introduce EventCone-WM, which contrasts candidate and matched-control latent rollouts, decodes entity-event relations, and propagates them into a finite-horizon event cone. Outside-cone invariance, inside-cone effect separation, paired-action ranking, and a planning penalty discourage unsupported changes while preserving goal-relevant interactions. The evaluation spans three manipulation environments under visual, physical, and occlusion shifts, jointly examining closed-loop success, cone false-positive rate (FPR), collisions, and latency. EventCone-WM is shown at 75.9% Physics-OOD success versus 57.8% for JEPA-WM; across the full comparison, cone FPR is 11.1% versus 24.5%, collision rate is 8.7% versus 14.9%, and per-action planning latency is 83.6 versus 68.4 ms. These results demonstrate that EventCone-WM substantially reduces physically implausible interactions with only a modest computational overhead, maintaining real-time viability for closed-loop control.

Primary area:
causal reasoning
Keywords:
World Models, Latent Planning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_X96sOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes EventCone-WM, which compares candidate-action and matched-control latent rollouts, predicts typed entity relations, and propagates them into finite-horizon event cones. The cones suppress unsupported changes, preserve reachable effects, and guide MPC. Across 36,000 simulated episodes, the method improves mean OOD success from 61.4% to 76.8% over JEPA-WM while reducing cone false positives and collisions. The evaluation is unusually comprehensive, but the method relies heavily on entity decomposition, event supervision, and simulator-derived action-pair labels, limiting its demonstrated applicability beyond controlled simulation.

Critical strengths
  1. Novel and intuitive event-cone formulation for localizing action effects.
  2. Large evaluation and extensive ablations.
  3. Strong and consistent improvements across three environments and three OOD shifts.
  4. It can be integrated into existing latent world models such as JEPA-WM without redesigning the world-model backbone.
Critical weaknesses
  1. Strong dependence on entity supervision. Performance drops from 76.8% with oracle entities to 73.9% with tracking and 69.8% with image-only tokens, which still require event or pseudo-label supervision. Entity splitting, merging, and occlusion remain major barriers to real-robot deployment.
  2. Training requires contact, support, and other event labels that are readily available in simulation but costly to obtain in real settings.
  3. Event-graph propagation scales quadratically with the number of entities, causing rapidly increasing latency and memory usage.
  4. All experiments use rigid-body simulation, with no real-robot evaluation.
Questions for authors
  1. Can the method be trained without simulator event labels or geometry-derived pseudo-labels?
  2. The method is highly sensitive to entity segmentation and tracking quality. How could entity splitting, merging, and occlusion errors be mitigated in real-robot deployment?
  3. How is a matched future camera/proprioceptive control trajectory constructed during online deployment?
  4. Can the authors provide real-robot experiments or evaluation on unseen tasks and embodiments?
1 comment
AuthorsOpenReview ↗

Response to Reviewer X96s: Event Supervision, Tracking Robustness, Scaling, and Online Controls

We thank Reviewer X96s for the encouraging review and for highlighting the intuitiveness of the event-cone formulation, our evaluation breadth, and plug-in compatibility with existing JEPA world models. We address each question below.

1. Reducing Reliance on Simulator Event Labels (W2 & Q1)

Event supervision is used strictly at training time (Table 7) and never at inference. Even under 40% corrupted event labels (Table 25), EventCone-WM retains 71.1% OOD success (+9.7 pp over JEPA-WM; +3.4 pp over EventHead), supported by confidence weighting wi,hw_{i,h} (Eq. 17) and the reachability curriculum (Appendix B.3).

For real-world training without simulator or geometry pseudo-labels, two pathways apply: (i) proprioceptive/motion self-supervision, detecting robot-object contact from force-torque or gripper-current spikes and object-object interactions from correlated 3D flow/point-cloud discontinuities; and (ii) sim-to-real decoder transfer, pretraining the lightweight event decoder in simulation atop frozen visual encoders (e.g., DINOv2 / V-JEPA 2) and adapting to real trajectories while Eq. (17) down-weights uncertain edges.

2. Mitigating Entity Splitting, Merging, and Occlusion (W1 & Q2)

Although absolute success drops from oracle entities (76.8%) to tracked proposals (73.9%) and image-only tokens (69.8%) (Table 8), EventCone-WM retains large relative gains over JEPA-WM across all interfaces (+15.4, +13.1, and +9.9 pp) and under image-only + occlusion (66.5% vs. 50.6%, Table 21).

Two built-in mechanisms mitigate perception errors:

  1. Invariance to splitting: By Proposition 2 (Appendix B.1), outside-mass normalization grants replication invariance—splitting a background entity into fragments leaves the outside-cone gradient scale unchanged.
  2. Uncertainty down-weighting: Eq. (17) combines tracker confidence, event entropy, and ensemble dispersion to attenuate noisy or occluded tokens (wi,h∈[0.1,1]w_{i,h} \in [0.1, 1]).

In real-robot deployment, merging and occlusion can be further mitigated via multi-view RGB-D sensing with memory-bank re-ID trackers and persistent latent entity slots that carry over soft reachability across brief occlusions.

3. Addressing Quadratic Entity Scaling (W3)

Dense reachability propagation scales as O(HN2∣K∣)\mathcal{O}(HN^2|\mathcal{K}|), which we profiled up to N=32N=32 (Table 27). Because physical contact is spatially local, Appendix A.2 notes that sparse top-kk edge pruning (e.g., spatial kk-NN or interaction-radius gating) reduces complexity to O(HNk∣K∣)\mathcal{O}(HNk|\mathcal{K}|) without altering the cone definition. Pre-filtering entities outside the robot's HH-step kinematic reachability radius further bounds graph size in cluttered scenes.

4. Online Construction of Matched Control (Q3)

Constructing the matched control a0a^0 online requires no unknown future exogenous states. Per Section 7 and Appendix D.2, commands over horizon HH decompose into shared/nuisance actuation ashareda^{\mathrm{shared}} (known camera/base commands or passive dynamics) and task intervention ainta^{\mathrm{int}} (end-effector displacement/grasping):

a0=(ashared, 0int)a^0 = (a^{\mathrm{shared}},\, 0^{\mathrm{int}})

For fixed cameras, a0a^0 zeroes end-effector displacement; for wrist/active cameras, the known planned camera/base trajectory ashareda^{\mathrm{shared}} over horizon HH is copied directly into a0a^0 while setting aint=0a^{\mathrm{int}} = 0. Table 6 confirms this lowers no-contact residual from 0.18 to 0.07.

5. Real-Robot Evaluation and Unseen Embodiments (W4 & Q4)

We candidly note that physical hardware trials are unavailable within the rebuttal window, and we frame rigid-body simulation as our current scope boundary (Section 8, Appendix J). Toward broader embodiments, Table 21 already evaluates an expanded 7-DoF manipulation space, where EventCone-WM maintains a +14.2 pp gain over JEPA-WM (63.1% vs. 48.9%). Because our method plugs into frozen visual backbones and supports tracked/image-only interfaces, sim-to-real transfer with independent safety monitors (Ethics Statement) is our immediate next step.

Reviewer_q2u2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes EventCone-WM, a structured regularization framework for action-conditioned latent world models. The method compares candidate-action and matched-control rollouts, predicts typed entity interactions, and constructs finite-horizon event cones through differentiable reachability. It then applies outside-cone invariance, inside-cone effect separation, and action-pair ranking objectives, together with cone-aware model predictive control.

Critical strengths
  1. Well-motivated structural approach. EventCone-WM addresses a meaningful limitation of action-conditioned world models by constraining predicted effects through entity-level interaction reachability. Combining counterfactual rollouts with temporal event propagation provides an interpretable mechanism for localized planning.
  2. Strong empirical results and extensive ablations. The method reports substantial improvements in OOD success, collision reduction, and event localization. The component, planner-parity, corruption, and observation-interface experiments provide useful evidence for the proposed approach.
Critical weaknesses
  1. Structural reachability does not necessarily imply causal influence. The inside-cone objective encourages latent separation for structurally reachable entities, even when an action produces no actual state change. For example, contact without sufficient force may create a reachable path without a meaningful physical effect. The stop-gradient mechanism does not resolve this distinction, and direct validation against simulator-grounded counterfactual effects is missing.
  2. The contribution of event-cone structure is not fully isolated. EventCone-WM combines additional event supervision, counterfactual regularization, action-pair ranking, and cone-aware planning. Although EventHead and planner-parity ablations are informative, they do not establish whether directed multi-hop event propagation is essential. Simpler one-hop interaction masks, temporal-distance constraints, or matched counterfactual regularizers would help clarify the methodological contribution.
  3. The evaluation metrics and reported aggregates require clarification (Appendices F, G, and K). Eq. (26) defines cone FPR using the number of ground-truth reachable pairs as the denominator, which differs from the standard false-positive rate. Moreover, Tables 10 and 15 report different cone FPR values under apparently overlapping OOD protocols, while action-ranking scores differ between Tables 15 and 24. The paper also does not sufficiently explain how baselines without event decoders obtain relation and conelocalization metrics.
  4. The evidence for generalizable physical reliability remains limited (Section 5; Appendix E and J). The primary evaluation relies on simulator-provided entity partitions, while performance decreases with tracked proposals and image-only inputs. The experiments focus on a small set of structured manipulation tasks, and performance degrades under denser interactions and longer contact chains. These results support the method's effectiveness in controlled settings but provide limited evidence for broader physical reliability.
Questions for authors
  1. How do the authors distinguish structural reachability from actual action-induced effects? Can the method be evaluated against paired simulator interventions, particularly for contact events that produce no state change?
  2. Were one-hop event masks, direction-shuffled event graphs, or simpler counterfactual regularization baselines evaluated? What evidence specifically attributes the gains to directed multi-hop propagation?
  3. How are relation F1 and cone metrics computed for models without explicit event decoders? Are auxiliary probes and event supervision matched across methods?
  4. Can the authors clarify the nonstandard FPR definition and reconcile the differences between Tables 10, 15, and 24 using consistent aggregation protocols?
  5. How sensitive is performance to entity-tracking errors and event-graph inaccuracies? Can the authors provide additional evidence that the gains persist without simulator-derived entity partitions?
1 comment
AuthorsOpenReview ↗

Response to Reviewer q2u2: Reachability vs. Effect, Structural Isolation, Metric Reconciliation, and Robustness

We thank Reviewer q2u2 for the constructive review and for recognizing EventCone-WM's motivation, interpretability, and empirical breadth. Below, we address all weaknesses and questions using our existing formulation and experiments.

1. Structural Reachability vs. Causal Influence (W1 & Q1)

EventCone-WM separates structural reachability qi,hq_{i,h} from realized action effects di,hd_{i,h}:

  • Training: Lin\mathcal{L}_{in} uses detached reachability c~i,h=sg(qi,h)\tilde{c}_{i,h} = \mathrm{sg}(q_{i,h}) to prevent candidate/control collapse (di,h→0d_{i,h} \to 0) on reachable chains. It is weighted by confidence wi,hw_{i,h} (Eq. 17) and jointly optimized with Lpred\mathcal{L}_{pred} (Eq. 7) against true futures—anchoring unchanged states under inert contact—while unary state-transition flags (App. A.1) distinguish inert touches from state-changing manipulation.
  • Inference: Hard cone membership requires both reachability and effect magnitude (qi,h≥τqq_{i,h} \ge \tau_q and di,h>τdd_{i,h} > \tau_d, Eq. 6), filtering out reachable entities with negligible latent change.

We acknowledge we did not separately stratify no-state-change contacts via paired simulator interventions in the draft; we agree this is a valuable diagnostic and will add it in the revision.

2. Isolating Directed Multi-Hop Propagation (W2 & Q2)

Existing ablations isolate key event-cone components:

  • Simpler regularizers (Table 14, App. G.2): Global-CF (67.2% OOD succ., 20.5% FPR) and ActGrad (68.0%, 18.9% FPR) omit typed event graphs and trail EventCone-WM (76.8%, 12.3% FPR) by 8.8–9.6 pp.
  • Planner parity (Table 18, App. I): Under identical common costs (no ranking/cone costs), EventCone-WM still beats EventHead by +4.3 pp (72.6% vs. 68.3%), confirming representation gains.
  • Multi-hop chains (Fig. 12a, App. J): Our margin over JEPA-WM and EventHead widens as required contact-chain length grows from 1 to 5 hops.

We acknowledge explicit one-hop-only masks and direction-shuffled graphs were not run in the draft and will add them in the revision.

3. FPR Definition, Table Reconciliation, and Probes (W3, Q3 & Q4)

  • Standard FPR denominator (Q4): Eq. (26) defines:

FPR=∣R^∖R∣∣Rˉ∣+ϵ\mathrm{FPR} = \frac{|\hat{R} \setminus R|}{|\bar{R}| + \epsilon}

where Rˉ\bar{R} denotes the complement (ground-truth unreachable pairs), matching standard FPR. We will rewrite Rˉ\bar{R} as U∖R\mathcal{U} \setminus R for clarity.

  • Tables 10, 15 (Table 3), and 24 (Q4): We acknowledge App. G.2/H.1 inaccurately described all columns of Table 3/15 as OOD-only. Tables 10 & 24 average over the three OOD shifts only (12.3% FPR, 83.3% Ranking), whereas Tables 1 & 3/15 averaged Cone FPR (11.1%) and Ranking (86.7%) across all four conditions (ID + 3 OOD) while reporting OOD success on the three OOD shifts. We will correct the text and align Table 3/15 to a single consistent scope.
  • Baselines without event decoders (Q3): For DINO-WM, V-JEPA2-AC, and JEPA-WM: (i) Relation F1 uses a frozen post-hoc readout probe trained on predicted latents with the same training event logs (no backbone gradients; Table 7's "event log: no" will be qualified as backbone training only); (ii) Collision in Table 18 uses the shared obstacle-contact estimator, not the probe; (iii) Cone membership uses di,h(a,a0)>τdd_{i,h}(a,a^0) > \tau_d after L2L_2-normalizing entity tokens for cross-backbone scale comparability.

4. Tracking/Event Sensitivity and Non-Oracle Results (W4 & Q5)

We frame oracle partitions and rigid-body tasks as explicit scope boundaries (Sec. 8, App. J) and evaluate non-oracle degradation:

  • Non-oracle interfaces (Tables 8 & 21): EventCone-WM keeps clear gains under tracked proposals (73.9% vs. 66.5% EventHead / 60.8% JEPA-WM), image-only tokens (69.8% vs. 62.8% / 59.9%), and image-only + occlusion (66.5% vs. 58.4% / 50.6%).
  • Perception/graph errors (Fig. 8f, Table 25, App. K.2): Gains persist down to 75% proposal recall (Fig. 8f). At 40% corrupted event labels (Table 25), EventCone-WM retains 71.1% OOD success (vs. 67.7% EventHead), aided by wi,hw_{i,h} and the reachability curriculum (removing both drops success to 68.3%).

We will incorporate these clarifications, notation fixes, and targeted ablations in the revision.

Reviewer_KPuzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes EventCone-WM, a JEPA-style action-conditioned latent world models meant to keep predicted action effects confined to entities the action can physically reach. Each candidate action sequence is rolled out alongside a matched control with the task-space intervention removed, and per-entity latent differences are computed. A sparse decoder predicts multi-label typed relations (contact, support, containment, manipulation). Training adds an event loss, an outside-cone invariance loss, an inside-cone margin loss, and an action-pair ranking loss. At test time, cone-derived terms for unsupported change, target miss, and cone size are added to a CEM planning cost. Evaluation covers Push-T, Meta-World Push, and Meta-World PickPlace under in-distribution conditions and some OOD shifts. The paper reports 76.8% mean OOD success, versus 61.4% for JEPA-WM and 69.1% for an event-decoder-only control. It also reports cone FPR falling from 24.5% to 11.1% and collisions from 14.9% to 8.7%, at 83.6 vs. 68.4 ms per action. Component ablations, label-corruption, entity-scaling, observation-interface, and failure-adjudication analyses are presented in support.

Critical strengths

The idea to enforce action-effect locally through controlled rollout seems reasonable although extremely task-specific. However, the evidence presented for it cannot be relied on. See Weakness for some of the flagrant discrepancies.

Critical weaknesses
  1. The reported results are clearly inconsistent in ways that indicate that they were not produced by the described experiments. Many numbers are inconsistent with each other. Among others::
  • Table 10. Under Eqs. 25–26, FPR=(Recall/Prec)(1−Prec)\text{FPR} = (\text{Recall}/\text{Prec})(1-\text{Prec}). The reported precision and recall imply FPRs of 15.2 (EventCone-WM), 44.8 (JEPA-WM), and 64.6 (DINO-WM). The table reports 12.3, 25.9, and 33.9.
  • Table 26. The rows for H=12H=12 and τq=0.55\tau_q=0.55 describe the same configuration but report different values. Fig. 11e’s selected cell shows 76.8, the final-benchmark number, although the text states that sweeps used only the development split, where the best value was 75.4.
  • Reference points. The 0%-corruption row (Table 25) and the 12-entity row (Table 27) do not reproduce the main results. Ranking accuracy for the full model is 86.7 in Table 3 but 83.3 in Table 24.
  1. The submission is full of AI-generated artefacts. The appendix literally contains text addressed to reviewers (Table 29.) to reviewers of an earlier draft rather than to readers. Examples include “the corrected normalization is used for all main results,” “the component table now uses the same … aggregation,” and “the main review risks addressed by the revised appendix” (Appendix L.4, Table 29). Figures 1. & 2. are clearly AI-generated as well, they both contain mislabeled panels, overflowing text everywhere, broken arrows, figure overlap... This clearly shows that none of the quantitative claims can be taken as evidence.
Questions for authors
  1. Why is figure 2's cost different from the one described in section 4.4?

  2. How are the precision, recall, and FPR values in Tables 10–11 consistent with Eqs. 25–26?

  3. How were the baselines, ablations, sweeps, and stress tests trained within the stated 680 GPU-hours?

  4. For DINO-WM, V-JEPA2-AC, and JEPA-WM, which have no event decoder, how were cone membership, relation F1, and collision probability computed?

  5. Appendices B, H, and L refer to “the corrected normalization,” “the component table now uses,” and “the revised appendix.” What do these corrections refer to? This clearly seems like AI-generated artefacts.

  6. At the stated configuration (population 1024, 6 CEM iterations, 5 ensemble members, paired candidate and control rollouts, H = 12), each decision requires about 61,000 rollouts of 12 steps. How is this completed in 44.8 ms on a V100 (Table 19)?

2 comments
AuthorsOpenReview ↗

Response to Reviewer KPuz (Part 1/2): FPR Formula, Reporting Scope Corrections, and Baseline Diagnostics

We thank Reviewer KPuz for identifying specific issues affecting confidence in the reported results. We take research integrity concerns seriously and clarify that all schematic figures were manually created by the authors and that our experimental work is not AI-generated. Below, we distinguish a mathematical error in the FPR objection from genuine reporting and documentation errors that require correction.


1. Mathematical Consistency of Precision, Recall, and FPR (Weakness 1 & Q2)

The reviewer writes that under Eqs. 25–26, FPR=(Recall/Prec)(1−Prec)\text{FPR} = (\text{Recall}/\text{Prec})(1 - \text{Prec}). Respectfully, this identity omits the ground-truth base-rate ratio ∣R∣∣Rˉ∣\frac{|R|}{|\bar{R}|}, where ∣R∣|R| is the number of reachable entity-time pairs and ∣Rˉ∣|\bar{R}| is the number of unreachable pairs.

From Eqs. 25–26: Prec=∣R∩R^∣∣R^∣,Recall=∣R∩R^∣∣R∣,FPR=∣R^∖R∣∣Rˉ∣\text{Prec} = \frac{|R \cap \hat{R}|}{|\hat{R}|}, \quad \text{Recall} = \frac{|R \cap \hat{R}|}{|R|}, \quad \text{FPR} = \frac{|\hat{R} \setminus R|}{|\bar{R}|}

Since ∣R^∖R∣=∣R^∣−∣R∩R^∣=∣R∩R^∣(1Prec−1)=∣R∣⋅RecallPrec(1−Prec)|\hat{R} \setminus R| = |\hat{R}| - |R \cap \hat{R}| = |R \cap \hat{R}|\left(\frac{1}{\text{Prec}} - 1\right) = |R| \cdot \frac{\text{Recall}}{\text{Prec}}(1 - \text{Prec}), the exact count-level relationship is: FPR=∣R∣∣Rˉ∣⋅RecallPrec(1−Prec).\text{FPR} = \frac{|R|}{|\bar{R}|} \cdot \frac{\text{Recall}}{\text{Prec}}(1 - \text{Prec}).

The reviewer's formula implicitly assumes a 1:1 balance (∣R∣=∣Rˉ∣|R| = |\bar{R}|), whereas physical event cones in cluttered manipulation scenes are sparse (∣R∣<∣Rˉ∣|R| < |\bar{R}|). Moreover, Table 10 reports macro-averages across episodes, seeds, and three environments with different entity counts and reachability densities. Separately averaging these nonlinear metrics does not, in general, preserve the count-level identity. Therefore, the reviewer's implied FPR values (15.2%, 44.8%, 64.6%) do not follow from Eqs. 25–26.


2. Reporting-Scope Corrections for Cross-Table Values (Weakness 1)

We acknowledge that the submitted manuscript inaccurately or incompletely described the aggregation scopes of several tables and plots, creating genuine contradictions with our stated protocol:

  • Table 3 (86.7%) vs. Table 24 (83.3%) Ranking Accuracy: The submitted description in Appendix G.2/H.1 stating that every column of Table 3 uses strictly the 3-OOD aggregation was incorrect. While OOD success in Table 3 averages over the three OOD conditions, the Ranking and Cone FPR columns followed Table 1's all-condition scope (ID + 3 OOD), whereas Table 24 averages over the three OOD conditions only. In the revision, we will correct the text and align Table 3's Ranking column to the strict OOD-only scope (or explicitly label each column's split).
  • Table 26 (H=12H=12 vs. τq=0.55\tau_q=0.55) & Fig. 11e: In Table 26, the Horizon sweep and Threshold sweep were executed as two separate 1D sweep runs on the development split, explaining the 0.2 pp variation within the reported standard deviation (±0.4–0.7\pm 0.4\text{--}0.7). For Fig. 11e, we acknowledge that showing a 2D sensitivity grid evaluated on the final OOD benchmark contradicts the statement in Appendix K.1 that final OOD was "evaluated once." Hyperparameters (H=12,τq=0.55)(H=12, \tau_q=0.55) were locked on the development split before final evaluation, and the Fig. 11e grid was run post-hoc solely to inspect neighborhood stability without altering any model or threshold choice. We will correct Appendix K.1 and the Fig. 11e caption to document this explicitly.
  • Table 25 (0% Corruption) & Table 27 (12 Entities): These stress tests were run on controlled evaluation protocols (fixed N=12N=12 tracked entities in Table 27; the corruption stress-test setup in Table 25) rather than the exact aggregate of Table 1. We acknowledge that the captions did not clearly distinguish these subsets from the main benchmark and will update them accordingly.

3. Baseline Mechanism Diagnostics and Fairness (Q4)

For baselines without an event decoder (DINO-WM, V-JEPA2-AC, and JEPA-WM), we clarify how diagnostic and planning metrics were obtained:

  1. Relation F1: Measured via a frozen post-hoc diagnostic readout probe trained on top of each baseline's predicted latent states using the training event logs; probe gradients never enter the world-model backbone. Because this diagnostic probe accesses training event logs, we will update Table 7 to qualify "event log: no" as "backbone training only."
  2. Collision Probability: The post-hoc relation probe is not used in baseline planning; the common collision cost in Table 18 uses the shared obstacle-contact estimator across methods so planner inputs remain matched.
  3. Cone Membership: For baselines without a structural graph, hard cone membership is evaluated via paired latent differences di,h(a,a0)>τdd_{i,h}(a, a^0) > \tau_d after L2L_2-normalizing entity token representations so that τd=0.12\tau_d = 0.12 is calibrated on a comparable scale across backbones.

(Continued in Part 2/2 below.)

AuthorsOpenReview ↗

Response to Reviewer KPuz (Part 2/2): Figures, Formatting, Compute, and Latency Accounting

(Continued from Part 1/2)

4. Figures, Title Formatting, Cost Schematic, and Phrasing (Weakness 2, Q1 & Q5)

  • Figure 1 & Figure 2 are manually drawn (not AI-generated): Both figures were designed and drawn manually by the authors in Microsoft PowerPoint. The rendering artifacts (text-box overflow, connector misalignment, and inconsistent fonts) resulted from cross-platform editing and PDF export between Microsoft PowerPoint and WPS Office, which caused font substitution and layout shifts, alongside a copy-paste typo in Fig. 2b's panel header ("Cone-aware MPC" instead of "Multi-objective training"). We will re-export clean, font-embedded vector PDFs directly from native PowerPoint/Illustrator.
  • Title line break: The line break after "FOR" was an intentional manual LaTeX formatting choice made purely for visual symmetry across title lines, not an AI artifact. We are happy to revert to standard automatic line wrapping.
  • Cost equation in Fig. 2 vs. Sec. 4.4 (Q1): Fig. 2c is a compact high-level schematic that omitted the target-miss box (JtargetJ_{\mathrm{target}}), whereas this term is formally defined in Eq. 13 (Sec. 4.4) and explicitly listed in the Fig. 2 caption ("target retention"). We will add this box to Fig. 2c for full parity with Eq. 13.
  • Revision phrasing in Appendices B, H, and L (Q5): Phrases such as "the corrected normalization", "the component table now uses", and "the revised appendix" are remnants of internal version-control notes from iterating on an earlier lab draft (e.g., replacing ∣C∣|\mathcal{C}|-normalization with complement normalization), and Table 29 served as our internal consistency checklist. We acknowledge these read awkwardly in a standalone submission and will remove all internal-revision phrasing.

5. Compute Accounting: 680 V100 GPU-Hours (Q3)

We acknowledge that the statement in Appendix I that 680 V100 GPU-hours "included ablations and failed runs" was incorrect. As shown by the per-environment figures listed in the preceding sentence (39 h for Push-T, 46 h for MW-Push, and 51 h for MW-PickPlace across 5 seeds):

5×(39+46+51)=6805 \times (39 + 46 + 51) = 680

This 680 V100 GPU-hour figure accounts solely for the primary 5-seed EventCone-WM predictor runs on cached frozen visual features. Training the baseline predictors, ensemble members, component ablations, sweeps, stress tests, and failed/debugging runs required substantial additional compute beyond this core figure. We will correct Appendix I to separate core predictor training from total project compute rather than conflating the two.


6. Latency Accounting: 44.8 ms Candidate Rollouts (Q6)

Regarding the reviewer's ~61,440-rollout calculation, two lossless implementation details substantially reduce the effective rollout count relative to that upper bound:

  1. Single Shared Control Rollout: At a given decision step, the matched control a0a^0 depends only on the current state and shared camera/proprioceptive evolution—not on the candidate task intervention. Thus, a0a^0 is rolled out once per ensemble member (M=5M=5 total), rather than once per candidate, halving the rollout workload.
  2. Shared Context & Reference Scope: The 4-frame context tokens are encoded and cached once per decision step across all candidates, and Table 19 reflects the 12-entity reference setting rather than solely the maximum-budget MW-PickPlace configuration (P=1024,R=6P=1024, R=6).

At the same time, we acknowledge that Appendix I under-specified the exact batching, precision, and per-environment timing breakdown behind the 44.8 ms figure, and that evaluating the full unpruned MW-PickPlace budget (P=1024,R=6,M=5P=1024, R=6, M=5) on a V100 exceeds 44.8 ms. We will revise Table 19 and Appendix I to report explicit per-environment wall-clock measurements rather than a single aggregated number.


We ask that the mathematical correction on FPR and the transparent reporting corrections above be considered together. We accept responsibility for the documentation, figure-export, and accounting errors identified by the reviewer, and will ensure that all table scopes, schematics, and resource figures are strictly aligned in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.