acceptodds
Under review as a conference paper at ICLR 2027

LatentGauge: Planning-Geometry Attacks in a Measure-Preserving Blind Spot

Abstract

Latent world models plan by comparing imagined futures with an encoded goal. Accurate prediction and a regular latent distribution constrain the representation, while planning also depends on the geometry delivered to its scorer. Follow- ing the planning equations, we identify an inference-time degree of freedom: a shared coordinate map inserts a metric into the action-side Gauss–Newton ma- trix while the predictive model stays fixed. We introduce LATENTGAUGE, a planning-geometry attack that searches within Gaussian-measure-preserving radial transformations under a displacement budget. Probability conservation determines the admissible construction; spatially varying rotation changes its geometry; an exact cost-shift identity connects that change to candidate ranking. For CEM, the native elite boundary determines calibration, and a finite-round bound links exact score changes to successive mean and variance updates. A shared scoring operation and one constrained ranking objective make this chain executable with- out modifying the encoder or predictor. A shared kernel extension specifies the radial profile; feasible trust-region optimization fits it under the exact displacement constraint. We derive the closed-form map, its induced metric and spectrum, and the separation between displacement and local distortion. Controlled Gaussian and affine-model calculations exhibit an exact ranking reversal within an independently integrated budget and reproduce the geometric invariances. The resulting con- struction links distribution-preserving geometry to candidate selection and native planning updates

Primary area:
generative models
Keywords:
World Model Robustness

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_oEQ8OpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LatentGauge, an inference-time attack on planning with latent world models. The central observation is that accurate latent prediction and preservation of the latent marginal distribution do not uniquely determine the geometry used by a downstream planner. The authors exploit this degree of freedom by applying a shared, radius-dependent rotation to predicted latent endpoints and goals. The transformation is constructed to preserve the Gaussian reference distribution while changing local distances and therefore the ranking of candidate action sequences.

The proposed attack uses calibration candidate pools from the planner, including predicted endpoints, goals, scores, and the native candidate-selection structure. For CEM, the method specifically targets pairs around the elite-selection boundary. A kernel parameterization controls the radial rotation profile, and the coefficients are optimized under a displacement constraint to reverse candidate margins while preserving the reference latent distribution.

The paper provides a substantial theoretical analysis connecting the transformation to the induced latent metric, the Gauss-Newton geometry of planning, candidate ranking, CEM elite-set changes, and propagation of these changes through later CEM updates. Empirically, the method is evaluated across DINO-WM, temporally straightened representations, PLDM, and LeWorldModel settings. LatentGauge consistently reduces planning success and produces considerably more ranking flips than identity, orthogonal, random-profile, and random-direction controls.

I find the central perspective interesting: prediction correctness and marginal-distribution preservation do not necessarily imply robustness of the geometry used for planning. However, I have concerns regarding the degree of methodological novelty and the strength of the attack-access assumptions.

Critical strengths

1. Interesting identification of a planning-specific blind spot.

The strongest conceptual contribution is the observation that correct world-model predictions and preservation of a regular latent marginal are insufficient to guarantee stable planning decisions. The paper separates prediction, distributional validity, and scoring geometry in a clean way. I think this is a useful perspective for evaluating latent world models.

2. Strong mathematical characterization of the mechanism.

The paper does considerably more than simply demonstrate empirical score changes. It derives the induced metric of the transformation, relates it to the action-side Gauss-Newton matrix, derives an exact candidate-cost shift, and analyzes how boundary crossings affect CEM elite selection and subsequent moment updates. This makes the failure mechanism unusually transparent compared with many empirical adversarial attacks.

3. Careful controlled validation.

The authors include useful controls such as identity transformations, global orthogonal rotations, random kernel profiles, and random-direction optimization. The controlled experiments support the specific claim that spatially varying measure-preserving geometry, rather than arbitrary latent movement, causes the observed ranking changes. The comparison between top-one and elite-boundary calibration is also informative because it directly connects the attack objective to the planner's native update rule.

Critical weaknesses

1. The individual methodological ingredients appear to have limited novelty.

This is my main concern. Measure-preserving transformations and latent non-identifiability are established ideas, Gaussian-preserving spiral/radial transformations have already appeared in related representation-learning work, and attacking rankings or imagined trajectories in world-model planners has also been explored. Kernel interpolation, constrained optimization, and pullback-metric analysis are likewise standard tools.

The paper combines these ingredients in a thoughtful way and adapts them specifically to planner scoring, but I am not convinced that the resulting attack constitutes a sufficiently new methodological contribution for ICLR. In my view, the strongest novelty is the formulation and analysis of the connection between these existing concepts rather than the attack construction itself.

2. The attacker requires unusually strong access to the planning pipeline.

The calibration procedure uses predicted latent endpoints, encoded goals, clean candidate scores, complete candidate pools, native elite counts, and planner-specific candidate boundaries. In CEM, the attack explicitly extracts the candidates at positions kk and k+1k+1 around the elite boundary. The method therefore appears closer to a strong gray-box or planner-interface attack than to a generally applicable deployment-time adversary.

This matters because an attacker able to insert an arbitrary differentiable transformation directly between the world model and its scoring function already has substantial control over the decision pipeline. The paper would benefit from a much clearer justification of realistic scenarios in which this precise access is available while direct modification of the scorer or planner is not.

3. The practical relevance of preserving the Gaussian reference marginal is not fully established across all evaluated representations.

The strongest theoretical guarantee assumes preservation of the reference Gaussian law. However, several evaluated representations, including pretrained DINO features and other learned representations, are not necessarily distributed according to this exact reference law. The paper acknowledges this distinction and separately evaluates learned-feature detectability, but this weakens the interpretation that the transformation is generally "distribution preserving" in realistic victim representations.

The theoretical blind spot is therefore cleanest for explicitly Gaussian-regularized models such as LeWM. For the broader experiments, I would like stronger evidence that preservation of the theoretical reference distribution corresponds to actual stealth or validity under the empirical latent distribution.

4. Experimental scale is somewhat limited for an attack making broad claims about latent world-model planning.

Although the paper covers several model families and tasks, the primary checkpoint evaluation uses only 50 held-out goal pairs and three attack-calibration seeds. Furthermore, some comparisons to other attacks are inherited from different source protocols rather than being fully reproduced under a single matched threat model.

The results demonstrate the proposed mechanism, but stronger evidence across independently trained checkpoints, more environments, and matched implementations of competing attacks would make the generality claim considerably stronger.

Questions for authors

1. Threat-model clarification.
What concrete deployment scenario gives an attacker access to predicted latent endpoints, encoded goals, complete candidate pools, candidate scores, and CEM elite counts, while preventing the attacker from directly manipulating the scoring function or planner? I think clarifying this distinction is important for assessing the practical significance of the attack.

2. Novelty relative to existing Gaussian-preserving transformations.
Can the authors state more precisely what is mathematically new in the transformation itself relative to prior Gaussian-preserving spiral/radial constructions? If the transformation family is largely inherited, I think the paper should more explicitly position the contribution as planner-specific optimization and analysis rather than a new transformation family.

3. Novelty relative to existing world-model planning attacks.
Could the authors provide a matched experimental comparison against TRAP and/or WMAttack under the same victim models, attack information, and computational budget? Literature-reported numbers from different protocols make it difficult to determine how much additional attack capability LatentGauge provides.

4. Empirical latent distribution.
For representations that are not explicitly Gaussian-regularized, how closely does the empirical latent distribution match the reference distribution that the transformation preserves? Could the authors report distributional discrepancy before and after the transformation under the actual learned-feature distribution?

5. Reduced-access ablation.
How does the attack perform if the attacker does not know the native elite boundary or does not observe latent endpoints directly? A reduced-information experiment would help determine whether the attack reveals a broadly exploitable weakness or primarily a vulnerability under privileged planner access.

Reviewer_P7i3OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that latent world-model planners (DINO-WM, LeWM, PLDM, Temporal Straightening) have an "inference-time degree of freedom."

Critical strengths
  1. The analysis identifies that CEM updates depend only on elite-set membership (Prop. 1, Eqs. 42–43), so the pairs that matter are those at the k / k+1 boundary rather than the top-one pair. This is the one theoretical property that the method exploits and that is validated on a real checkpoint.

  2. Matched controls for the optimization component. The random-profile and random-direction controls share the plane, kernel coordinates, budget correction and call allowance (Table 2A). This shows that the fitted objective, not merely the parameterization, drives the effect within this map family.

Critical weaknesses
  1. The text is extremely dense; many sentences chain technical terms without stating what they establish. Key quantities in the main tables (Boundary FlipRate, top-one replay, TPR, calibration bank) are defined only in the appendix.

  2. The threat model is never stated, and the implied one does not support the paper's central "measure-preserving blind spot" claim. Pieced together from Sec. 3 (L137–141), Sec. 4.1 and Appendix G.3–G.4, the implied adversary is: (i) reads internal planner state from deployed planning calls; (ii) can insert arbitrary code between the predictor output and the scorer; (iii) aims at untargeted degradation of task success. An adversary with access (ii) can already perturb or swap the goal embedding, negate the cost, or alter the executed action, each far more damaging than 96 → 62% SR. The γ-preservation constraint is therefore justified only by stealth.

  3. The paper's own detector already flags the attack. In Table 2 the MMD detector's TPR is 15% / 18% for LatentGauge (8% / 10% even for the orthogonal-Q control) at a nominal 1% FPR. This is a per-batch rate, so a defender monitoring repeated batches would detect the attack with near certainty.

  4. Apart from one property, the theory is never tested on real planners. The theory describes how the map should affect planning: it hides from distribution checks, it changes the conditioning of gradient-based planning, and it flips the CEM elite set when boundary gaps are small. However, almost none of this is checked on the actual LeWM or DINO checkpoints. The "undetectable" property is verified only on synthetic Gaussian samples, where it holds by construction, and on real models the paper's own detector flags the attack above chance.

  5. The conditioning argument and the multi-round CEM bound (Thm. 6) are demonstrated only on a 2-D toy model. For example, the paper never tests whether planning calls with small boundary gaps are the ones that fail, which would be its most useful practical prediction. The only theoretical property with real-model support is that targeting the elite boundary beats targeting the top-one candidate (Table 2B). So the theory explains how the attack is built, but not why or when it works on real systems. The real-model results are also weak in themselves: most clean success rates are copied from the original papers rather than re-run on the same goals and seeds, and no confidence intervals are reported.

Questions for authors
  1. Please state the threat model explicitly: who can insert T in a deployed planner, and through what channel (e.g., a compromised third-party adapter)? What logs does the attacker need for calibration? Is the goal targeted or untargeted? Exactly what does the defender observe and check? Why would an adversary with scorer-pipeline access restrict itself to γ-preserving maps?

  2. On the real LeWM/PushT and DINO+TS/PushT checkpoints: Do clean elite-boundary gaps, relative to b_t, predict which planning calls flip and which episodes fail? How tight is the Thm. 6 envelope compared with observed moment differences? How does the positive-spectrum conditioning of JᵀG_TJ change under the fitted map, and does it correlate with GD-MPC failures?

Reviewer_5py7OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LatentGauge, an attack that modifies the scoring geometry of latent world models while preserving the reference Gaussian distribution and keeping the predictive model unchanged. Using a shared radial transformation and budget-constrained optimization, the method reverses candidate rankings and disrupts CEM updates. Experiments show substantial planning performance degradation, but the practical validity of the displacement constraint and the strength of the empirical evidence remain concerns.

Critical strengths

The paper identifies an important vulnerability: preserving prediction accuracy and latent marginal distributions does not guarantee stable planning decisions. The mathematical analysis clearly connects measure-preserving transformations to ranking changes and CEM updates. The method includes useful invariant and random-search baselines. Results across multiple world models suggest that planner-aligned calibration can effectively exploit scoring-geometry vulnerabilities.

Critical weaknesses
  1. The displacement constraint may not reflect real task distributions. The authors should report displacement statistics and tail quantiles under actual endpoint and goal distributions and relate them to detection rates and attack effectiveness.

  2. The threat model needs stronger justification. The authors should clarify how attackers could access the prediction–scoring interface in practice and why the Gaussian displacement budget is realistic.

  3. The novelty needs clearer positioning. The authors should distinguish their contributions from prior work and compare budget-constrained fitting and CEM boundary calibration against simpler parameterizations.

  4. The ablation studies do not isolate individual contributions. Table 2B simultaneously changes calibration pairs, rotation plane, and kernel centers, making it difficult to attribute improvements specifically to elite-boundary calibration.

5. The theoretical and computational evidence is incomplete. The authors should validate Theorem 6 on real models and report bound tightness, kernel center count, fitting time, memory usage, and total computational overhead.

Questions for authors
  1. Can you report endpoint and goal displacement distributions under the actual task distributions, including tail quantiles, and explain their relationship to detection and attack success?

  2. Can you clarify the realistic threat model and isolate the contribution of elite-boundary calibration through controlled ablations?

  3. How tight is Theorem 6 on real models, and what are the actual computational costs of kernel construction and fitting?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.