acceptodds
Withdrawn from ICLR 2027

GUIDE: Self-Consistent Geometric Deception in Spatial Foundation Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Spatial foundation models jointly predict camera poses, depth, and 3D points from shared visual features, making geometric consistency a useful signal for validating their predictions. However, consistency among these outputs does not necessarily imply correctness, as coordinated errors may remain mutually compatible and pass internal geometric checks. We propose GUIDE (Geometric Untruths In Dependent Estimates), the first attack that systematically constructs self-consistent geometric deception against spatial foundation models. Specifically, GUIDE first specifies a target camera trajectory and constructs shared world anchors under the target geometry. It then projects these anchors across views to derive compatible depth and point targets. Finally, it optimizes bounded image perturbations to jointly steer the model toward these prescribed geometric predictions. We evaluate GUIDE on three models and five datasets. GUIDE achieves 29/36 joint successes, compared with 12/36 for a camera-only attack. We further evaluate its downstream impact on depth-label generation and robot control, where GUIDE leads to 103/144 accepted erroneous depth predictions and reduces task success by 39.5 and 37.4 percentage points for FALCON and R3DP, respectively.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
adversarial attacks, spatial foundation models, multi-view 3D reconstruction, geometric consistency, adversarial robustness, robot learning, camera pose estimation

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_TDxsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposed GUIDE, an adversarial attack framework to attack a spatial foundation model (e.g., VGGT). The core contribution is that the paper constructs a self-consistent attack target for the adversarial optimization, such that the attacked camera trajectory, depth, and point cloud share a consistent 3D geometry. This design can help the attacked results bypass geometric checkers. Experiments are conducted on 3 models (VGGT, VTTG-Ω, and Déjà View) on 16 scenes from 3 datasets (CO3D, Sintel, and Tanks and Temples). Results show that GUIDE achieves a higher attack success rate compared to the camera trajectory-only attack baseline when considering the checker. The paper further experiments on downstream robot manipulation and show GUIDE can lead to lower task success rate.

Critical strengths

1. The experiment is carefully designed to include attacks on both the model and downstream tasks. The ablation also shows that spatial consistency is necessary, rather than an additional auxiliary loss.

I appreciate the authors' effort in designing the experiment to justify the robustness of the proposed GUIDE.

  • First, the downstream robot manipulation evaluation shows the real application of the proposed attack, instead of an academic prototype. The attack can effectively reduce the task success rate, showing the potential for real-world usage.

  • Second, the ablation study in Table 2 (b) shows that shared anchors (which are the core design) are necessary to bypass the consistency checker. This experiment shows that the auxiliary support loss works by being consistent, rather than simply adding more supervision signal.

However, I have some major concerns that need more experiments to address (discussed below).

Critical weaknesses

1. The attack efficiency and effectiveness trade-off is not sufficiently studied, and the real-time attack effectiveness in the downstream evaluation.

  • First, in Table 2 (a), the paper shows that under the same optimization budgets (i.e., optimization steps), GUIDE shows better joint success. However, this experiment only has three step setups (e.g., 200, 600, 2000). From the trends, even though GUIDE still shows better results under step=2k, GUIDE seems to converge faster, while the baseline continues to improve substantially with more optimization steps. (which improves joint success +26, while GUIDE improves +8 from 200 to 2k steps). Therefore, it's unclear whether GUIDE only converges faster or can also lead to better convergence results. To address this, the author should provide the "step-joint_success" curve and show more data points beyond 2000 until plateau. This can justify that GUIDE not only converges faster but also better.

  • Second, this convergence experiment is evaluated using "optimization steps" as a budget. But in practice, it's more realistic to use computation FLOPs or optimization time as the budget since we want to attack in real time. GUIDE introduces additional computational operations, including constructing shared geometric targets and optimizing additional support losses. Therefore, it's unclear whether, if we use FLOPs or E2E optimization time as the budget, GUIDE can still show better results compared to the baseline.

The second concern becomes particularly important in the downstream robot experiments. Table 3 evaluates only GUIDE without comparing against Camera-only or other baseline attacks. Although the authors use asynchronous optimization, the computational latency of attack generation may affect when perturbations can be delivered during closed-loop control. Under the same wall-clock attack budget, GUIDE may complete fewer optimization steps than a simpler baseline due to its additional computational overhead. Therefore, it remains unclear whether GUIDE provides better downstream attack effectiveness under realistic real-time computational constraints. A comparison of GUIDE and Camera-only under matched wall-clock budgets, including target construction and optimization time, would help address this concern.


Given this is the main contribution of the paper, I currently lean toward weak reject due to the concerns above. I'm willing to raise my score to weak accept if the authors could provide additional evidence showing either (1) that GUIDE achieves better convergence outcomes than the baseline when both methods have reached a performance plateau, or (2) that GUIDE maintains superior attack effectiveness under matched wall-clock time budgets, for both geometric joint success and downstream task failure rates.

Questions for authors

Please see the weakness above.

Reviewer_NQELOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a method (GUIDE) to attack spatial foundation models which jointly predict camera poses, depth, 3D points etc. from one or more images. Given some initial camera trajectory, GUIDE specifies a target camera trajectory and proposes image perturbations that result in steering the model towards these target predictions (given in the target camera trajectory). Experiments are conducted which show success at these task on multiple datasets.

Critical strengths

Unfortunately, the paper has been very difficult to understand because of which it has been difficult to assess its strengths.

Critical weaknesses
  • The paper is very difficult to read. Right from the beginning, the precise problem statement is not explained clearly. For example, what is the input (multiple images of a scene?), what things are being predicted from that input, what is the attacker supposed to do. I think Section 3 is supposed to do this task, but the writing is not clear. For example, the authors mention the phrase "camera trajectory", "coupling of prescribed camera deformation", "world anchors" (lines 126 - 128) without explaining what these things are. These examples are not meant to be exhaustive, but highlight a common pattern throughout the introductory text of the proposed method (Section 3 and 4). Consequently, it is difficult to understand the rest of the work. It is fine to use AI tools for writing, but not in a way that makes things so unclear for a reader who doesn't already know the key idea of the work and its details.

  • Below, I will list some more phrases that I encountered in a small segment of Section 4.1 which are not clear. These could technically belong to the "Questions for authors", but they constitute an overall critical weakness of the paper regarding the lack of clarity.

    • Line 161: who is specifying the "endpoint twist"? It seems that this prescription/specification is part of the problem statement but has not been defined.
    • Line 162-163: what exactly is the global similarity alignment? This term, again, has not been defined.
    • Line 164: "frame-varying command"?
    • Many of the variables of Eq. 2 are not defined (e.g., X*_k, R*_s(k)).
  • Overall, irrespective of the results, the paper is not ready for publication.

Questions for authors

Please look at the weaknesses.

Reviewer_4BdXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies attacks on spatial foundation models, which predict camera poses, depth, and 3D points from images. GUIDE adds small changes to the input images so that the predictions become incorrect but remain consistent with each other. It does this by constructing shared geometric targets across views. An attack is considered successful only when it reaches the target, produces an error against an independent reference, and still passes a geometric checker. The paper evaluates GUIDE on three models and also studies its effects on depth labels and robot tasks.

Critical strengths
  1. The paper addresses a useful problem: camera poses, depth, and 3D points can agree with each other while still describing the scene incorrectly. The joint success criterion is appropriate because it checks target attainment, error against an independent reference, and checker acceptance on the same output. This makes the evaluation directly relevant to the claimed failure mode.

  2. The comparisons provide evidence for the benefit of shared geometric targets. In Table 1, GUIDE and the camera-only attack both reach 41/48 targets, but their joint successes on the reference subset are 29/36 and 12/36. The matched-budget comparisons and checker-aware baselines in Table 2 strengthen this result. The association ablation also helps show why sharing targets across views matters.

Critical weaknesses
  1. The trajectory alignment procedure needs clarification. Appendix A, lines 731–774, states that the attacked trajectory is aligned to the target trajectory before evaluating target attainment. However, the linked code appears to use a different procedure: it aligns the attacked trajectory and the target separately to the clean trajectory, then compares their displacements. Here, the clean trajectory is the model's prediction on the original images, not the ground-truth trajectory.

Aligning both trajectories to the clean prediction can be a reasonable choice. My concern is whether this gives the same metric as directly aligning the attacked trajectory to the target. The two alignments estimate separate transforms, and the scale normalization also needs to be consistent with this choice.

This matters because it can affect whether an instance counts as reaching the attack target, and therefore whether it counts as a joint success. The discrepancy alone does not establish that the reported results are wrong. Please clarify which evaluator version produced the tables and whether the two procedures give the same decisions. If their equivalence cannot be established, rescoring the saved outputs would help show whether the success counts and the gap between methods change.

  1. The protected RGB-D validation needs more detail. The authors report that additional observations hidden from the attacker reduce attack success, but the observation selection, residual calculation, and threshold configuration are not clear enough to reconstruct the acceptance decisions.

The clean acceptance rate for this validator is also important. Rejecting more attacks is less useful if many valid outputs are rejected as well. The paper already reports clean calibration results for the primary geometric checker; comparable information is needed for the protected RGB-D validator.

The robot experiments raise a secondary point. The drop in task success shows downstream impact, but without a matched camera-only control, these results do not isolate how much additional impact comes from shared geometric targets. The authors already distinguish downstream outcomes from geometric joint success, so this is not my main concern. For R3DP, per-task paired counts would help interpret the aggregate result across only three task policies.

Overall, the problem is meaningful and the main controlled comparisons are useful. My main uncertainty is whether the stated success criterion, released evaluator, and reported results are consistent. A clear explanation, together with confirmation that the main results hold under a consistent evaluation procedure, would substantially reduce this concern.

Questions for authors
  1. Could you clarify which alignment and normalization procedure was used for the reported results? Appendix A describes aligning the attacked trajectory to the target, while the released code appears to align the attacked and target trajectories separately to the clean prediction. Are these procedures equivalent for the reported metrics? If not, would using the procedure described in Appendix A change the target attainment (A) and joint success (J) results?

  2. For the protected RGB-D validator, could you explain how the independent observations are selected, which residuals are measured, and how the acceptance thresholds are set? What fraction of clean outputs does this validator accept?

Reviewer_BvQ2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GUIDE, a targeted adversarial attack against spatial foundation models that jointly predict camera poses, depth maps, and 3D points. The main idea is that these geometric predictions may remain internally consistent even when they collectively represent an incorrect scene.

GUIDE constructs a prescribed camera trajectory and uses shared 3D anchors to generate compatible depth and point targets across multiple views. It then optimizes bounded image perturbations to guide the model toward these coordinated geometric predictions. Attack success is evaluated based on target attainment, geometric error relative to an independent reference, and acceptance by a geometric consistency checker.

The authors evaluate GUIDE on three spatial foundation models and multiple datasets, comparing it with several adversarial attack baselines. Additional experiments investigate geometric validation, erroneous depth-label acceptance, and downstream robotic manipulation. The results indicate that GUIDE achieves higher joint attack success rates than the compared baselines and can negatively affect downstream applications.

Critical strengths
  1. The paper investigates an important limitation of geometric validation in spatial foundation models. Rather than simply degrading prediction accuracy, GUIDE aims to generate coordinated geometric errors that remain internally consistent. This provides a useful perspective for evaluating the reliability of spatial foundation models.

  2. The paper evaluates the proposed method across multiple models and datasets and includes comparisons with different attack baselines. The shared-anchor ablation provides evidence for the importance of cross-view geometric associations. The experiments on depth-label generation and robotic manipulation further demonstrate the potential practical impact of the proposed attack.

Critical weaknesses
  1. The methodological contribution relative to existing approaches could be better clarified. The proposed method combines camera trajectory deformation, shared 3D anchors, geometric projection, and adversarial optimization. While the overall framework is meaningful, several of these components are based on established geometric and optimization techniques. A clearer discussion of the specific methodological advantages over existing multi-view adversarial attacks would help establish the contribution.

  2. The downstream evaluation does not fully isolate the benefit of the proposed geometric consistency mechanism. Although the robotic manipulation experiments show substantial reductions in task success, the paper does not report corresponding results for simpler adversarial attacks under the same online evaluation protocol. Therefore, it remains unclear how much of the downstream degradation is specifically attributable to the shared-anchor construction rather than adversarial perturbations in general. In addition, the substantially lower attack success under protected RGB-D validation suggests that the effectiveness of GUIDE depends on the validation mechanism.

Questions for authors

Q1. Could the authors explain more explicitly how the proposed shared-anchor construction differs from directly optimizing camera, depth, and 3D point predictions using a joint adversarial objective? This clarification would help establish the specific contribution of GUIDE beyond existing geometric optimization techniques.

Q2. Could the authors clarify whether camera-only or other direct adversarial attacks were evaluated on the FALCON and R3DP tasks under the same online attack settings? If such results are available, reporting them would help determine whether the proposed geometric consistency mechanism provides additional benefits in downstream robotic manipulation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.