acceptodds
Under review as a conference paper at ICLR 2027

CoView-WAM: A Geometry-Aware World-Action Model with Cross-View Consistent Imagination

10 views

Abstract

Recent advances in World-Action Models (WAMs) have shown that future latent representations can effectively guide action prediction without explicitly decoding future videos or target-state images during inference. However, existing multi-view WAMs typically fuse image observations through implicit cross-view feature interaction without explicitly modeling geometric correspondences between views. As a result, complementary cues may be exchanged ineffectively across current observations, while future representations from different views may encode inconsistent scene geometry, ultimately weakening world-to-action guidance. In this paper, we introduce CoView-WAM, a geometry-aware cross-view WAM that incorporates geometric correspondence into both cross-view interaction and future representation learning. First, we develop a geometry-guided cross-view feature interaction mechanism that establishes spatial correspondences across camera views and enables feature exchange between corresponding regions. A gated interaction strategy is further introduced to suppress irrelevant cross-view information while preserving complementary cues. Second, we jointly model future image and depth representations and use camera extrinsics to encourage cross-view geometric consistency in 3D space, guiding future representations from different views to capture a shared underlying scene structure beyond appearance similarity. Future geometric interaction and depth supervision are used only during training. At inference time, CoView-WAM requires no explicit future image or depth decoding. Experiments on both simulated (LIBERO and CALVIN) and real-world multi-view manipulation tasks demonstrate the effectiveness of CoView-WAM. CoView-WAM achieves an overall success rate of 99.20% on LIBERO, 73.8% success rate on real-world tasks, and an average completed sequence length of 3.987 on CALVIN, with strong gains on tasks involving partial occlusion, precise placement, and cross-view spatial coordination.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, Robotic Manipulation, Multi-View Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vj77OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CoView-WAM adds epipolar-restricted retrieval to ImageWAM’s action-conditioning cache and uses training-only bidirectional future-view interaction and depth supervision. The authors argue that these components improve cross-view information transfer and encourage 3D-consistent future representations. Matched comparisons show better manipulation performance on LIBERO, CALVIN, and five real-robot tasks; the evidence for the proposed geometric mechanisms is less direct.

Critical strengths

The current-view module places a valid geometric prior directly in the action-conditioning path. Equations 1–3 restrict retrieval using camera calibration without requiring correspondence labels, and the current-only ablation in Table 4 improves LIBERO success from 97.55% to 98.65% over matched ImageWAM. This is evidence that structured cross-view retrieval can be useful for control.

Critical weaknesses
  1. The methodological novelty of the current-view operator appears limited. Epipolar candidate selection followed by similarity-weighted fusion has precedents [1, 2]. CoView-WAM’s use in an action-conditioning cache is distinct, but Eqs. 2–5 always retrieve candidates and use one gate across target patches; the gate cannot explicitly reject a bad local match while retaining good ones. Moreover, the detached residual in Equation 4 gives the matching scores no direct gradient through this retrieval path. Table 5 isolates a 0.45-point advantage for geometric over global candidate selection on near-saturated LIBERO, without variation across training runs.
  2. The future branch does not yet establish its proposed cross-view geometric effect. Training-only future-depth supervision has a close precedent in [3]. CoView-WAM’s depth readout also has a local feature path (Appendix A.3), and Eq. 9 applies depth and spatial-difference losses within each view. Accurate per-view depth could therefore be learned without a cross-view-aligned representation, although the bidirectional interaction may encourage one. The joint future-interaction-plus-depth gain over the current-only model is 0.55 points on LIBERO (Table 4), with no direct cross-view consistency measurement or variation across training runs. [1] He, Y., Yan, R., Fragkiadaki, K., & Yu, S.-I. “Epipolar Transformers.” arXiv:2005.04551, 2020. [2] Wang, J., et al. “MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulation.” arXiv:2602.09878, 2026. [3] Li, Y., et al. “WAM4D: Fast 4D World Action Model via Spatial Register Tokens.” arXiv:2606.14048, 2026.
Questions for authors
  1. For occluded target patches or regions without view overlap, how often does the Top-32 set contain a valid visible correspondence? Under matched capacity, does a per-patch gate or null-match option improve the current shared-gate design on those cases?
  2. Relative to a capacity-matched local-only depth head and the current-plus-depth model, does the full future branch improve held-out cross-view depth agreement after calibrated reprojection? For example, using the known camera calibration, please report cross-view reprojection error for predicted future depths on mutually visible regions (such as colmap or worldscore[4] ), and compare the full model with Current+Depth and a local-only depth head.

[1] Duan, H., et al. “WorldScore: A Unified Evaluation Benchmark for World Generation.” ICCV, 2025.

3 comments
AuthorsOpenReview ↗

Response to Reviewer vj77 (Part1)

Thank you for your careful review and detailed questions. We have added two evaluations: a candidate-selection comparison on CALVIN, and a cross-view depth-agreement measurement against the matched-depth baselines. We respond to the four points below.

W1. Differences from prior work, the shared gate

Our key differences from prior epipolar-based methods lie in the retrieval strategy and interaction target. Epipolar Transformers focuses on multi-view pose estimation, while MVISTA-4D uses learnable offsets for epipolar sampling in RGB-D generation. In contrast:

  • Retrieval: We directly obtain candidate features using prior camera calibration parameters, without using learnable sampling offsets, correspondence labels, or additional matching parameters.
  • Interaction target: Our method applies the interaction to both the current and future branches during training, while MVISTA-4D only uses it in future generation. The prior therefore affects the representation the action branch reads rather than the generated output, so its benefit does not depend on decoding future images.

The shared gate modulates global feature injection strength, while cosine weights provide local match modulation. Although the gate cannot explicitly reject individual mismatches, cosine weighting reduces the contribution of less relevant candidates.

Stop-gradient prevents retrieval scores from drifting away from appearance-based correspondence. It keeps the retrieval criterion independent of the optimization objective while preserving gradients through the direct feature path and learnable gates. The stop-gradient is needed because the gate's injection strength is estimated from the current features: letting the matching scores receive gradients would make that estimate track the optimization objective instead of the appearance correspondence, whereas the gradient on the original path is passed back through the subsequent interaction branch.

We further validate our geometric retrieval's (Geometry Top-32) advantage on CALVIN. The 0.45-point LIBERO improvement alone may be insufficient to validate our method, we further add the experiments on CALVIN. Geometry Top-32 improves CALVIN's average completed sequence length from 3.779 to 3.875 over Global Semantic Top-32.

Configuration≥1 task≥2 tasks≥3 tasks≥4 tasksAll 5 tasksAvg. length
ImageWAM91.7%82.2%73.9%66.6%56.6%3.710
ImageWAM + Depth92.6%85.6%74.9%69.5%60.5%3.831
Current-only (Global Semantic Top-32)91.1%83.3%74.9%68.2%60.4%3.779
Current-only (Geometry Top-32)91.5%84.3%76.7%71.4%63.6%3.875
Current + Future92.2%84.3%77.8%72.0%64.3%3.906
Current + Depth92.7%84.1%77.9%72.4%64.9%3.920
Full model93.0%86.6%79.9%72.8%66.4%3.987

W2. Cross-view effect of the future branch

We added the evaluation for cross-view consistency, and the results support that future geometric interaction gives better cross-view generation consistency. Adding the bidirectional interaction to the future branch of Current + Depth reduces the reprojection error in both reprojection directions, and per-view depth MAE also reduces from 9.39 to 8.96 mm.

The gain from future geometric interaction also appears on CALVIN. Since LIBERO is saturated with 0.55 points' improvement, we perform the corresponding ablation on CALVIN. The future branch's bidirectional interaction also shows a gain from 3.920 to 3.987.

Our approach to cross-view interaction differs from that of WAM4D. Unlike WAM4D, which aggregates cross-view context into spatial register tokens, we perform per-location feature updates at each Transformer block under epipolar constraints, with shared depth queries preserving geometric correspondence in depth prediction. This enables fine-grained, geometry-aware cross-view interaction. WAM4D is belongs to the concurrent work and will be sited in the revision.

AuthorsOpenReview ↗

Response to Reviewer vj77 (Part2)

Q1. Occlusion and regions without view overlap

Occluded regions provide limited reliable cross-view correspondence, but the gate and the cosine weights mitigate the effect. When the retrieval statistics indicate unreliable matches, the gate lowers the overall injection into the wrist representation, and the cosine weights lower the contribution of the candidates that agree poorly with the query. Out-of-view regions are of less practical concern, since manipulation occurs within the external camera's field of view.

A per-patch gate or an explicit null match would be hard to learn and would add inference cost. Which candidate is correct is defined geometrically, so a learned gate has no supervision and tends to degenerate. Instead, our method uses the extrinsics to narrow the search and the cosine weights to rank the candidates; as Figure 2 shows, this reduces irrelevant cross-view matches.

Q2. Cross-view agreement of the future predictions

The two views' predicted future depths agree more closely once the future interaction is added. We measure agreement by reprojecting one view's predicted depth into the other view and comparing it with the depth predicted there; Full gives the lowest error in both directions, roughly halved relative to the two baselines, whereas the baselines differ from each other by only about 3%, so the gap is attributable to the future branch rather than to depth supervision or to the current interaction.

E: external view, W: wrist view

ConfigurationE→W (mm)W→E (mm)
ImageWAM + Depth136.65125.82
Current + Depth132.64125.62
Full model65.2177.05

Both views' ground-truth depths are rendered from the same 3D scene, so lower depth error in both views is indirect evidence of closer agreement between the two views. Relative to Current + Depth, the patch-level MAE falls from 9.39 to 8.96 mm (external 9.96 → 9.54, wrist 8.82 → 8.38), so both predictions move closer to the same underlying geometry. The agreement measure above tests this directly and also places Full lowest.

ConfigurationOverall (mm)External (mm)Wrist (mm)
ImageWAM + Depth9.4210.048.80
Current + Depth9.399.968.82
Full model8.969.548.38
AuthorsOpenReview ↗

Follow-up to Reviewer vj77

We thank the reviewer again for the careful review. We now measure cross-view agreement directly by reprojecting predicted future depth: relative to the depth-matched baseline the error falls from 132.64 to 65.21 mm (E-to-W) and from 125.62 to 77.05 mm (W-to-E), while per-view depth MAE moves only from 9.39 to 8.96 mm, so the future interaction improves agreement rather than per-view accuracy. We also add the CALVIN candidate-selection comparison (3.779 to 3.875), and we now state how our retrieval differs from prior epipolar-based approaches: candidates are constructed analytically from calibration rather than learned as sampling offsets, and the same operator updates the action-conditioning cache read by the action expert. As the discussion period closes soon, we would be grateful to hear whether this addresses the reviewer's concerns, and we would be glad to clarify any remaining point.

Reviewer_74ANOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CoView-WAM, a geometry-aware world-action model for multi-view robotic manipulation. It uses calibrated epipolar geometry to restrict cross-view feature retrieval, then applies a reliability-aware gate to control the amount of retrieved information injected into current representations. During training, bidirectional geometric interaction between future views and auxiliary depth supervision are used to encourage spatially consistent predictive representations. These future pathways are removed at inference. Experiments on LIBERO, CALVIN, and five real-world manipulation tasks show improvements over a matched ImageWAM baseline, including an overall LIBERO success rate of 99.20%, an average CALVIN completed sequence length of 3.987, and a real-world success rate of 73.8%.

Critical strengths
  1. The method incorporates calibrated camera geometry directly into cross-view retrieval instead of relying only on implicit feature interaction. The gated residual update also addresses the fact that geometrically plausible matches can still be unreliable under occlusion or limited view overlap.

  2. The evaluation includes both simulated benchmarks and real-world robot manipulation. The matched comparison with ImageWAM shows improvements on LIBERO, CALVIN, and the five real-world tasks. The ablations also indicate that current cross-view interaction provides a substantial part of the gain, while combining future geometric interaction with depth supervision adds further improvement.

Critical weaknesses
  1. The approach depends on accurate camera calibration and explicitly constructed epipolar candidate sets, but the paper does not evaluate sensitivity to calibration errors or changes in camera configuration. This is important for real robotic deployment, where calibration can be imperfect and camera poses may change. The authors should clarify how robust the method is to calibration perturbations and whether the gains persist when geometric correspondences are less reliable.

  2. The real-world evaluation covers five tasks with 100 trials per task, while the LIBERO result is close to saturation. These results are encouraging, but they provide limited evidence about generalization to unseen tasks, objects, camera arrangements, or environments. The current-view interaction is also asymmetric: external-view features are retrieved to update wrist-view representations for action conditioning. The paper should clarify the implications of this choice and provide stronger evidence that the method generalizes beyond the tested setup.

Questions for authors
  1. How does CoView-WAM perform when camera intrinsics or extrinsics contain small calibration errors? Have the authors tested perturbations to camera poses or changes in camera configuration?

  2. Can the authors report results on unseen task or camera configurations, or explain what evidence supports generalization beyond the five real-world tasks and the evaluated simulation benchmarks?

  3. Why is current action conditioning updated only through external-to-wrist interaction, while future representations use bidirectional interaction? Was the reverse current-view direction evaluated?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 74AN (Part1)

Thank you for your careful review and detailed questions. We have added three parts: how the method behaves under camera calibration errors, how it generalizes on LIBERO-Plus, and why the current interaction is unidirectional, with a bidirectional comparison in support.

W1 and Q1: Sensitivity to Camera Calibration

When introducing single-axis extrinsic errors of 1–5 cm / 1–5°, the success rate of CoView-WAM drops by 0.49–0.85 percentage points relative to the unperturbed setting, and remains higher than ImageWAM's 97.55%.

We focus on extrinsics because the intrinsics of a fixed rig generally do not change; intrinsic errors are not evaluated separately here. At inference, we apply a fixed-magnitude single-axis translation or rotation to the extrinsics of one camera at a time, keeping the other camera's extrinsics, both image streams and the intrinsics unchanged, which simulates inaccurate calibration. For each axis we add perturbations of different magnitudes in both the positive and negative directions, and all changes are measured against the unperturbed score of 99.20% reported in the paper.

We add the extrinsic perturbations on wrist camera and external camera respectively, the success rate change is compared to the unperturbed setting with 99.20% success rate.

PerturbationMagnitudeWrist cameraExternal cameraWrist changeExternal change
Translation1 cm98.71%98.56%−0.49 pp−0.64 pp
Translation3 cm98.42%98.43%−0.78 pp−0.77 pp
Translation5 cm98.52%98.35%−0.68 pp−0.85 pp
Rotation1°98.46%98.46%−0.74 pp−0.74 pp
Rotation3°98.51%98.46%−0.69 pp−0.74 pp
Rotation5°98.45%98.38%−0.75 pp−0.83 pp

The method is affected by extrinsic perturbations but stays above ImageWAM within the tested range. One likely reason is that the perturbation shifts the whole candidate region, while the true correspondence still remains in the Top-32 set with high probability and is then given a larger weight by cosine similarity.

AuthorsOpenReview ↗

Response to Reviewer 74AN (Part2)

W2 and Q2: Generalization to Unseen Conditions

On LIBERO-Plus, CoView-WAM achieves an average success rate of 82.5% against 81.2% for ImageWAM. LIBERO-Plus changes the camera, object layout and other conditions on existing tasks. We obtain higher success under changes in object placement, robot initial state, lighting, background and language, and lower success under camera viewpoint and sensor noise.

Two properties explain the gains: geometry keeps the retrieval anchored to the corresponding region, and depth supervision reduces the reliance on surface appearance. The former helps when objects or the robot move, and the latter under lighting and background changes. The two categories where we fall below ImageWAM are camera viewpoint and sensor noise. Both alter the overlap region, which affects what the retrieval returns rather than the geometry itself — unlike W1, where the images are fixed and only the supplied extrinsics are perturbed.

LIBERO-Plus success rates (%)

MethodP.T.CameraRobotLanguageLightBackgroundNoiseLayoutAvg
UniVLA✓1.846.269.669.081.021.231.942.9
OpenVLA-OFT✓56.431.979.588.793.375.874.269.6
π0✓13.86.058.885.081.479.068.953.6
π0-Fast✓65.121.661.073.273.274.468.861.6
WorldVLA✓0.127.941.643.717.110.938.025.0
FastWAM✗16.444.568.978.253.737.760.751.5
ImageWAM✗78.447.791.495.984.593.077.681.2
Ours✗74.953.794.098.786.987.082.082.5

Q3: Unidirectionality of the Current Interaction

We supplied the experiments of applying different information directions to the two branches: Current interaction is unidirectional because the wrist view makes more contribution to action generation. The external view provides global scene context, while the wrist view captures fine-grained manipulation details. As shown in Table 6, the wrist-only input (97.80%) outperforms the external-only (95.95%) and concatenated views (97.55%). This motivates injecting geometrically aligned external-view features into the wrist representation to enhance action prediction.

Future interaction is bidirectional because both are used for predicting the same future 3D scene. We therefore perform bidirectional cross-view retrieval with future camera calibration parameters and auxiliary depth supervision to enforce geometric consistency.

The ablation validates the effectiveness of unidirectional current interaction. The ablation study shows that adding wrist-to-external injection reduces success from 99.20% to 98.45%, probably because the representation from the rapidly changing wrist view influences the external view's stable scene representation.

The comparison experiment results are as follows:

ConfigurationSpatialObjectGoalLongOverall
Baseline97.0100.097.096.297.55
Baseline(External only)96.698.896.292.295.95
Baseline(Wrist only)97.699.698.295.897.80
Current only99.099.898.897.098.65
Full model(bidirectional current)98.699.898.696.898.45
Full model98.899.899.099.299.20
AuthorsOpenReview ↗

Follow-up to Reviewer 74AN

We thank the reviewer again for the careful review. The tests the reviewer asked for show that single-axis extrinsic errors of 1-5 cm / 1-5 degrees cost at most 0.85 points and always stay above the 97.55% baseline, that CoView-WAM reaches 82.5% average success on LIBERO-Plus against 81.2% for the matched baseline, and that making the current interaction bidirectional lowers LIBERO success from 99.20% to 98.45%, which is why we keep it external-to-wrist. As the discussion period closes soon, we would be grateful to hear whether this addresses the reviewer's concerns, and we would be glad to clarify any remaining point.

Reviewer_isghOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

CoView-WAM describes a robot policy learning system based on 3D inductive bias and auxiliary losses for future view and future depth supervision. The basic idea is that for a robot with a wrist and external camera available, the information can be passed between image tokens extracted from those views in a manner comparable to attention but with the mixing parameters determined by epipolar lines given the known extrinisics and intrinsics of the two cameras. The evaluation is with respect to LIBERO, Calvin, and a custom real-world dataset.

Critical strengths

The results of the evaluation are uniformly promising and the modifications atop ImageWAM are well explained. The paper is written well overall.

Critical weaknesses

Several big weaknesses:

  • I don't understand why it's necessary to use epipolar constraints to artificially select tokens for other tokens to exchange information with. Presumably this could just be learned from data (and is in fact the entire point of attention).
  • The ablation on LIBERO is very under-powered because the dataset is so saturated (~98+% accuracies seem the norm). To be honest the benefits of future view and present-view mixing seem very unclear, though future depth makes intuitive sense that it would help.
  • Use of privilged depth information is an unfair advantage over ImageWAM
Questions for authors
  • Can the authors please do ablations on a dataset that is not so saturated so that we can reasonably conclude which components of the system are helpful? CALVIN itself seems like it may also be too saturated though it is marginally better. Perhaps a less saturated synthetic dataset could be selected.

  • Could the cross-view mixing for an arbitrary pattern be implemented approximately by way of attention? If so, what is the advantage of present and future view mixing? Is it mainly a data efficiency technique?

2 comments
AuthorsOpenReview ↗

Response to Reviewer isgh

Thank you for your careful review and detailed questions. We have added three parts: a matched-budget candidate-selection comparison, CALVIN component ablations on a less saturated benchmark, and a depth-matched ImageWAM + Depth baseline.

W1 / Q2. Epipolar constraints for cross-view interaction

Thanks for pointing this out. We think epipolar constraints are necessary for the following reasons:

We illustrate this necessity with the experiments from our paper:

  • Motivation Cross-attention needs enough data for training to learn cross-view correspondences. In fact, ImageWAM uses the cross-attention component, but as shown in Table 6, ImageWAM concatenates the two views and makes them interact based on image features, achieving 97.55%, which is lower than the 97.80% of the wrist-only view. This indicates that such a manner does not fully utilize the cross-view data and correspondence under existing training data. This observation motivated us to introduce explicit epipolar constraints to guide cross-view interaction, which raises LIBERO success to 99.20% (Table R1).
  • The effectiveness of prior epipolar constraints. Under the same training settings, selecting candidates by epipolar deviation instead of global feature similarity raises LIBERO success from 98.20% to 98.65% and CALVIN average completed length from 3.779 to 3.875.
  • Epipolar constraints on cross-view interaction are important for both current-branch and future-branch. Taking cup grasping as an example, current-branch interaction injects global scene information of cup and gripper's relative positions from the external view into the wrist-view representation, helping the action expert predict more accurate actions. Future interaction instead performs bidirectional retrieval in corresponding regions using camera calibration parameters, together with per-view depth supervision, encourages the two predictive representations to capture consistent scene geometry. Both future geometric interaction and depth supervision are used only during training.

Therefore, with same training settings, our method has a higher performance with the epipolar constraints guidance.

W2 / Q1. Component ablations on CALVIN

CALVIN ablations also show that current interaction yields the largest single-component gain. We agree that LIBERO is close to saturation, so we add ablations on the 1,000 official five-task sequences of CALVIN ABC→D. ImageWAM completes all five tasks in 56.6% of the sequences and the full model reaches 66.4%. Measured by average completed sequence length, adding current interaction improves the score from 3.710 to 3.875. On top of Current-only, adding future geometric interaction or depth supervision separately gives 3.906 and 3.920, and using both together reaches 3.987.

Table R1 Ablations on CALVIN

Configuration≥1 task≥2 tasks≥3 tasks≥4 tasksAll 5 tasksAvg. length
ImageWAM91.7%82.2%73.9%66.6%56.6%3.710
ImageWAM + Depth92.6%85.6%74.9%69.5%60.5%3.831
Current-only (Global Semantic Top-32)91.1%83.3%74.9%68.2%60.4%3.779
Current-only (Geometry Top-32)91.5%84.3%76.7%71.4%63.6%3.875
Current + Future92.2%84.3%77.8%72.0%64.3%3.906
Current + Depth92.7%84.1%77.9%72.4%64.9%3.920
Full model93.0%86.6%79.9%72.8%66.4%3.987

W3. Comparison with matched depth supervision

With the same depth supervision, our full model still outperforms ImageWAM w/ Depth on both CALVIN and LIBERO, indicating that the benefit of cross-view interaction is independent of depth supervision. We add to ImageWAM the same depth prediction branch and supervision used by our full model. With depth supervision alone, the CALVIN average completed sequence length increases from 3.710 to 3.831, confirming that depth supervision itself brings gains. However, the configuration with only current interaction and no depth supervision reaches 3.875, and when both models use depth supervision, the full model is 0.156 higher than ImageWAM w/ Depth on CALVIN. Thus, depth supervision does bring gains, but the advantage of the full model over ImageWAM is not entirely due to depth supervision. Note that depth information is used only during training, no configuration requires depth input at inference.

Table R2 The table below reports LIBERO success rates (%). C, F, and D denote current cross-view interaction, future geometric interaction, and auxiliary depth supervision, respectively.

ConfigurationCFDSpatialObjectGoalLongOverall
ImageWAM———97.0100.097.096.297.55
ImageWAM + Depth——✓98.099.899.096.898.40
Current-only✓——99.099.898.897.098.65
Current + Future✓✓—98.4100.099.096.698.50
Current + Depth✓—✓99.099.898.497.298.60
Full model✓✓✓98.899.899.099.299.20
AuthorsOpenReview ↗

Follow-up to Reviewer isgh

We thank the reviewer again for the careful review. In response to the two main concerns, we report component ablations on CALVIN, which is less saturated than LIBERO: the average completed sequence length rises from 3.710 (ImageWAM) to 3.987 (full model), and under identical depth supervision the geometric modules still improve CALVIN from 3.831 to 3.920 and LIBERO from 98.40% to 98.60%, so the gain is not the depth objective in disguise. As the discussion period closes soon, we would be grateful to hear whether this addresses the reviewer's concerns, and we would be glad to clarify any remaining point.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to reviewers

We thank the three reviewers for detailed and consistent feedback. Read together, five concerns recur:

  • whether the geometric prior is necessary given that attention can learn cross-view correspondences;

  • whether the LIBERO ablations are informative on a saturated benchmark;

  • whether training-time depth supervision gives our model a privileged advantage over ImageWAM;

  • whether the method remains reliable under perturbed extrinsics and changing observation conditions;

  • whether the future branch establishes the cross-view geometric effect claimed for it.

We have added experiments for the first four and a direct measurement for the fifth. The evidence is summarised below.

Evidence added. 

(i) A matched-budget candidate-selection comparison: under the same training settings, selecting candidates by epipolar deviation instead of global feature similarity raises LIBERO success from 98.20% to 98.65% and CALVIN average completed length from 3.779 to 3.875.

(ii) Component ablations on CALVIN, where the benchmark is not saturated: ImageWAM completes all five tasks in 56.6% of sequences and the full model in 66.4%, and current interaction gives the largest single-component gain.

(iii) A depth-matched baseline, ImageWAM + Depth, trained with our depth branch and supervision: the full model is 0.156 higher in average completed length on CALVIN and 0.80 percentage points higher on LIBERO.

(iv) Perturbed camera extrinsics: with single-axis errors of 1–5 cm / 1–5° applied to the calibration supplied to the model, success drops by 0.49–0.85 percentage points and stays above ImageWAM.

(v) LIBERO-Plus: 82.5% versus 81.2% on average, higher in five of seven categories , lower under camera viewpoint and sensor noise than ImageWAM.

(vi) A cross-view depth-agreement measurement against the two matched-depth baselines, in which the full model roughly halves the agreement error of both.

We will incorporate these results in the revision, clarify the distinction between per-view depth accuracy and cross-view depth agreement, and specify the calibration conditions covered by our evaluation. If any point remains unclear, we would be glad to continue the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.