acceptodds
Under review as a conference paper at ICLR 2027

COORDINATEWISE PRECISION ADAPTATION FOR FROZEN WORLD MODELS

Abstract

Local sensor faults change the reliability of observations used to infer a world model's latent state. We combine a calibrated diagonal base with causal innovation statistics for coordinatewise precision adaptation on frozen world models. We evaluate forty predictive coding world model (PCWM) and Gaussian recurrent state-space model (RSSM) checkpoints in two low-dimensional environments. Under local variance faults, one-step error decreases by 5.56–17.81% relative to independently calibrated static weights. Same-base controls retain 6.29–15.43% improvements on PCWM. A prospective comparison gives static and adaptive refinement the same filtering-time cap and four optimizer families, keeping their precision calibration fixed. Validation-selected adaptive configurations reduce local-fault error by 5.60–24.90% across all four settings relative to validation-selected static configurations. A separate multistep development evaluation yields 5.3–22.1% lower fault-condition error than static refinement at five transitions, with gains persisting at twenty transitions. These findings support coordinatewise state refinement for frozen world models under localized sensor corruption.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
world models, state estimation, test-time adaptation, adaptive filtering, observation precision, sensor noise, robust filtering, recurrent state-space models, predictive coding, prospective evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_UQmjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes coordinatewise precision adaptation for frozen world models under localized sensor corruption. The approach combines calibrated observation-noise scales with exponentially averaged innovation statistics to dynamically adjust the relative importance of observation coordinates during latent-state refinement. The authors evaluate the method on predictive coding world models (PCWM) and Gaussian RSSMs, primarily using Pendulum and Cartpole environments.

Although the paper contains extensive implementation details and statistical comparisons, I have significant concerns about its technical novelty, the limited scope of the experimental evaluation, and the lack of consistent improvements over existing robust filtering techniques

Critical strengths

The problem of adapting frozen world models to changing observation reliability is practically relevant.

The authors provide detailed experimental protocols, including calibration procedures, paired comparisons, and computational-cost analysis.

The proposed approach demonstrates improvements over static precision weighting in the evaluated low-dimensional settings.

Critical weaknesses
  1. Limited methodological novelty.

The core mechanism is closely related to classical adaptive filtering and innovation covariance matching. The method estimates observation uncertainty from accumulated innovation residuals and uses these estimates to adjust coordinatewise precision during latent-state refinement.

In particular, the authors explicitly acknowledge that their coordinatewise adaptation with λ = 1 reduces to a clipped innovation covariance-matching statistic (Section 3.2).

The introduction of a calibrated diagonal base and a shrinkage exponent appears to be a relatively incremental modification of established adaptive filtering principles.

I find it difficult to identify a substantial new methodological or theoretical contribution that would justify the scope of this work at ICLR.

  1. The empirical evaluation is restricted to overly simple environments.

The primary experiments are conducted exclusively on two custom low-dimensional environments, Pendulum and Cartpole, with observation dimensions of three and five.

Although the authors evaluate multiple checkpoints and two world-model architectures, these experiments remain within a very narrow class of dynamical systems.

The paper does not demonstrate effectiveness on established, challenging world-model benchmarks, complex visual environments, or downstream control tasks.

Consequently, it is unclear whether the reported improvements generalize to practical world models with substantially more complex observations and dynamics.

  1. The proposed method does not consistently outperform established robust filtering baselines.

This is a major empirical concern.

According to the primary comparisons in Appendix A.3, the proposed adaptive approaches outperform the selected Huber or inverse-multiquadric (IMQ) baselines in only two out of eight settings. In the remaining six settings, the robust baselines achieve lower prediction errors.

For the coordinatewise approach specifically, only one of the four local-fault settings outperforms the selected robust reference.

Furthermore, the multistep evaluation reports an average improvement of only 0.18% relative to selected robust weighting across the four settings.

These results raise questions about whether the proposed approach provides a meaningful advantage over simpler, established robust estimation methods.

  1. The visual-input experiment fails to establish the effectiveness of coordinatewise adaptation.

The visual pilot in Section 5.5 and Appendix E is particularly concerning.

When extending the method to 32 × 32 image observations, the validation procedure selects λ = 0, effectively disabling coordinatewise adaptation and recovering static refinement.

The manually examined active configuration also fails to outperform the independently selected static baseline on the aggregate evaluation.

Thus, the only higher-dimensional observation experiment does not provide positive evidence for the main proposed mechanism.

This substantially weakens the paper's broader claims regarding adaptation in frozen world models.

  1. Limited robustness to different fault types and unclear practical trade-offs.

The method is primarily developed and validated under synthetic variance faults drawn from predefined corruption families.

Appendix A reports that under bias-drift corruption, coordinatewise adaptation produces higher error than static refinement in all four evaluated settings.

This suggests that the mechanism may be specialized to variance changes rather than general sensor corruption.

Moreover, the proposed adaptation introduces additional computational overhead. The reported filtering latency is approximately 1.5–1.6 times that of static refinement in the multistep evaluation.

Given the limited improvements over robust filtering and the absence of demonstrated downstream control benefits, the practical value of this additional complexity remains unclear.

Questions for authors

What is the fundamental methodological contribution beyond established innovation covariance matching and adaptive Kalman filtering?

Why should the proposed method be preferred over Huber or IMQ filtering when these alternatives outperform it in most primary comparisons?

How do the authors explain the failure of validation to select active coordinatewise adaptation in the image-observation experiment?

Can the approach demonstrate meaningful improvements on more representative world-model benchmarks, particularly in closed-loop planning or control?

Reviewer_5q9YOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The observation precision used for latent state refinement of frozen world models is adapted online by modulating a calibrated diagonal base.

Critical strengths

The method is simple and cheap.

Critical weaknesses

The contribution is an incremental modification of an existing quantization/inference pipeline on existing models and datasets. The evaluation is even not supporting the claims.

Questions for authors

Does the proposed method help on any publicly available world model (for instance DreamerV3 ) or any standard dataset?

Reviewer_vyULOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies observation-precision adaptation for latent-state inference in frozen world models. The method combines a calibrated diagonal residual scale with causal moving averages of squared innovations and decoder-projected prior variance. The resulting coordinatewise weights guide a small number of latent-state refinement steps while the encoder, transition model, and decoder remain frozen. Evaluation covers 40 checkpoints from two model families, PCWM and Gaussian RSSM, on Pendulum and Cartpole. The paper includes comparisons against calibrated static and robust weighting, same-base controls, optimizer selection under a common filtering-time cap, multistep forecasting, and a small visual-input pilot. The strongest evidence supports improved prediction over static weighting under localized variance faults; advantages over robust alternatives are mixed.

Critical strengths
  1. Careful experimental design and transparent reporting. The evaluation separates calibration, validation, and test trajectories, locks selections before generating the main test trajectories, and uses checkpoint-level paired comparisons with shared inputs and prediction randomness. The multiple-comparison corrections and explicit distinction between primary, exploratory, and development results strengthen the credibility of the reported findings. The paper also clearly reports unfavorable results, including losses against robust references and the static fallback selected in the visual pilot.

  2. Convincing evidence of benefits over fixed precision within the evaluated setting. The same-base controls help distinguish online modulation from simply choosing a different fixed scale: coordinatewise adaptation reduces local-fault error by 6.29–15.43% on PCWM when compared with its own base without modulation. The additional optimizer study allows both static and adaptive refinement to select among four optimizer families under a common filtering-time cap, with selected adaptive configurations improving error by 5.60–24.90% across the four settings (Table 20). These controls provide useful evidence that online adaptation contributes beyond the original fixed-weight configuration.

Critical weaknesses
  1. The incremental contribution over established adaptive and robust weighting remains insufficiently established. As acknowledged in the paper, the update is closely related to innovation covariance matching, combined with calibrated shrinkage and latent-state refinement. The resulting method is reasonable, but the evidence for a consequential advantage over robust alternatives is limited. In the locked primary comparisons, the selected robust reference has lower error in six of the eight settings (Table 3), while the mean aggregate multistep improvement over robust weighting is only 0.18% (Table 23). The expanded search improves some adaptive results, particularly RSSM/Pendulum, but also demonstrates sensitivity to the search domain (Table 13). Furthermore, the common-time-cap optimizer study strengthens the comparison against static precision without giving the robust alternatives the same optimizer selection. Overall, the paper establishes an improvement over fixed precision more clearly than an advance over existing robust approaches.

  2. The demonstrated benefit is concentrated in a narrow, calibrated fault regime. The primary positive results come from two low-dimensional systems, with representative corruption families available during calibration and clean targets used for validation. Evidence beyond this setting remains limited: the visual pilot selects the static fallback, and the active coordinatewise configuration does not outperform the independently selected static reference (Section 5.5 and Appendix E). Under bias drift, its mean error increases relative to static weighting in all four settings (Table 8). These limitations are appropriately disclosed and do not invalidate the local variance-fault results. However, together with the modest methodological advance, they leave the broader significance of the contribution insufficiently demonstrated for acceptance.

Questions for authors
  1. Clarification of the contribution and existing ablation evidence. Which results isolate the benefit of calibrated shrinkage and temporal accumulation beyond covariance matching and instantaneous coordinatewise weighting? In particular, are held-out comparisons available against separately selected variants with lambda = 1 and rho = 1 under the same validation objective and nominal constraint? Table 14 provides a useful memory-rate comparison, but reports selected validation results rather than held-out effects.

  2. Additional comparative analysis that could affect my assessment. Does the adaptive method retain a meaningful advantage when Huber or standardized-IMQ weighting is also allowed to select its optimizer under the same filtering-time cap? Please clarify whether such results are available. A focused comparison would help determine whether the proposed controller offers a practical advantage over the strongest alternatives, beyond the demonstrated improvement over static refinement.

  3. Clarification of calibration dependence and transfer evidence. Is there evidence that a single locked controller retains its advantage when corruption severity or temporal structure shifts outside the calibration/validation distribution, without reselection? The reported stuck-sensor and bias-drift experiments are informative, but leave open how much the positive results depend on having representative target corruptions during offline selection. Any existing sensitivity results would help establish the intended operating conditions more precisely.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.