acceptodds
Under review as a conference paper at ICLR 2027

How Representational Priors Shape Generalization in World Action Models

Abstract

World action models (WAMs) are shown to substantially diverge under generalization performance based on their generative backbones. We investigate this gap through the representational geometry that WAMs inherit from their backbones and how policy training reorganizes it. Our central hypothesis is that a useful representational prior would structure geometry that separates nuisance information from the in-distribution manifold in advance of training as a policy. We quantify this with nuisance orthogonality; orthogonality of the feature under visual variation outside the empirical in-distribution manifold. Across various image and video backbones, we show that a higher orthogonality from the representational prior is associated with stronger robustness to visual variations after WAM training, enabling policy generalization measurable prospectively for world models. In addition, analyzing the resulting robust WAMs reveals nuisance suppression; relative suppression of feature displacement caused by such variations and robust action generation of the policy. Together, these findings provide a geometric perspective on how the representational prior of the world model shapes the generalization performance of WAMs.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Model, Robot Learning, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_N5heOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how the representational geometry of pretrained generative backbones relates to the generalization performance of world action models. The authors introduce nuisance orthogonality, which measures the fraction of feature displacement induced by visual perturbations that lies outside the subspace of clean-scene representations, and nuisance suppression, which characterizes how representations change after policy adaptation. Across nine generative backbones, they report a positive correlation between this metric and downstream robustness on the LIBERO-Plus benchmark.

Critical strengths

I like the motivation of this paper. The question is well chosen, the metric is simple and intuitive, and the mean-level correlation is decent. I like the authors' definition of nuisance orthogonality and the way they use it to predict generalization. It is very interesting, and these points are the main reason I see value in this work.

Critical weaknesses

From my perspective, the biggest weakness of this paper is that the analysis remains too shallow. Given that the core measurement tool, quantifying feature displacement relative to a PCA subspace, is standard, and that predicting robustness from pretrained representations has been explored in prior work, the paper’s contribution must lie in the depth of its insights. The current version falls short in this regard. While the paper shows that backbones differ in nuisance orthogonality and that these differences correlate with robustness, it does not explain what drives these differences. Section 3.1.2 identifies tokenizers, architecture, initialization, language injection, and pretraining data as potential factors, yet none is systematically isolated or ablated. The scratch FLUX.2 result raises precisely this question but leaves it unanswered. Section 5 is similarly descriptive. Ultimately, the paper establishes that models differ but offers little insight into the underlying causes or actionable guidance on how practitioners might obtain better priors.

Questions for authors
  1. What exactly is scratch FLUX.2? Which components are randomly initialized (e.g., VAE, text encoder, transformer), and which are kept from pretraining?

  2. What does the metric capture if a random backbone scores high? If the FLUX.2 transformer is randomly initialized, its orthogonality (0.724) exceeds that of pretrained Wan2.2 (0.594). This suggests that a large part of the metric does not come from learned knowledge. Have the authors analyzed where it comes from, and how does this fit the claim that orthogonality reflects a useful representational prior?

  3. Can the contributions of the VAE, architecture, and pretraining be separated? For example, by reporting orthogonality at the VAE latent level and for a randomly initialized Wan2.2.

  4. Can the authors further analyze the controlled comparison already presented in the paper? FlashMotion and Kiwi-Edit are both adapted from Wan2.2-TI2V-5B, yet their orthogonality scores differ substantially (0.645 and 0.786 vs. 0.594). Since they share the same architecture and tokenizer, this comparison could help isolate the effect of the fine-tuning objective. What insights can the authors draw from this comparison, and how do the findings relate to the claim in Section 3.1.2?

2 comments

We thank the reviewer for the thoughtful evaluation of our work and for the constructive comments, which helped us strengthen our evidence and deepen our analysis of what drives the orthogonality isolating each contribution of VAE, architecture, pre-training and post-training.

Edited: Following up on our previous response, we have completed the additional policy experiments, evaluations and VAE output orthogonality, then reorganized the answers based on each question and revised the paper.

W1. Requires deeper insight.

We agree that deeper insight into what drives the observed representational differences beyond establishing a correlation would help evidence of our diagnostics. We thank the reviewer for this constructive comment. We would like to reinforce our claim and provide deeper insights by answering the following questions with additional analysis and WAM training.

Q1. Clarification of Scratch FLUX.2 setting

First, to clarify our Scratch FLUX.2 setting, we only re-initialized the pretrained diffusion transformer of FLUX.2, while keeping the vision and language modules frozen. We added a brief clarification to Section 3.1.2 of the revised paper.

To answer each question, we additionally executed extensive analysis on the orthogonality random initialization and trained parameters of the VAE and the transformer of FLUX.2, Wan2.2, FlashMotion and Kiwi-Edit to present the metrics with isolation under each component, pretraining and post-training. Results over three random seeds are as below.

Table R7

ModelVAETrans.VAE Orth.Trans. Orth.LIBERO-Plus Perf.Libero Perf.
FLUX.2PretrainedPretrained0.8540.79989.0%98.4%
FLUX.2PretrainedRandom0.8540.727 ± 0.00363.9%96.5%
FLUX.2RandomRandom0.796 ± 0.0090.668 ± 0.01010.0%84.1%
Wan2.2PretrainedPretrained0.7760.59446.5%97.6%
Wan2.2PretrainedRandom0.7760.708 ± 0.00839.9%91.8%
Wan2.2RandomRandom0.651 ± 0.0200.677 ± 0.02331.2%87.7%

Q2. Why does random backbone have high orthogonality? Where orthogonality comes from? And how does this fit the claim that orthogonality reflects a useful representational prior?

The orthogonality of the transformer are composed as a combination of nuisance separation from each pretrained component and randomness of high dimensional vectors in randomized components. Comparing pretrained VAE and random VAE under random transformer settings, increased orthogonality of pretrained VAEs lead to higher orthogonality of transformer also with increased policy performance.

Also, an untrained random transformer of Wan2.2 exhibits substantial orthogonality without useful task representations. This high orthogonality of random transformers is due to high dimensional random variations being biased towards orthogonality, even without semantic representation and separation of nuisance information. WAM trained from this Wan2.2 with random transformer reaches 39.9% average LIBERO-Plus success, below the pretrained Wan2.2 (46.5%) despite its higher orthogonality. To our analysis, this policy adaptation from random initialization also shows less nuisance suppression.

While this interpretation makes orthogonality’s association with policy generalization limited to backbones that already provide useful representations, comparing fully pretrained models with the semantic knowledge that serve as world model backbone candidates, orthogonality reliably estimates how displacements from nuisance information are separated. At the same time, we can again use this property to audit pretraining or post-training before policy adaptation to check if training objective is beneficial for further policy generalization performance, evidenced by substantial performance improvement gaps of FLUX.2 and Wan2.2.

Q3. Can the contributions of the VAE, architecture, and pretraining be separated?

The contribution of pretraining is explicitly visible through increased orthogonality of pretrained FLUX.2 and decreased orthogonality of pretrained Wan2.2. The editing objective of FLUX.2 significantly improves the orthogonality through learning to control the representations of visual variations while the video prediction objective of Wan2.2 could not. This supports our claim that the pretraining objective contributes to representational priors and their geometry relative to clean scene variations.

While completely isolating VAE and architecture to their effect on orthogonality and generalization performance since they are not completely interoperable across pretrained models due to their distinct designs, we provide VAE output orthogonality and WAM performances under VAE randomized settings.

One observation is that performance increases as each component uses a pretrained parameter, which supports semantic knowledge from pretraining definitely helps. Considering pretrained VAE as a combination of its architecture and pretraining, we can say this result is a joint effect of all components.

Another observation is that despite FLUX.2’s superior performance with pretrained modules, it has significant performance drop in the fully randomized setting. With the observation that random FLUX.2 VAE has high orthogonality from randomness and low generalization performance, its architectural property when given random initialization significantly differs from that of Wan2.2 in a way that prohibits generalization. Despite this VAE architectural disadvantage, FLUX.2 pretraining with the editing objective has reshaped the representation of VAE.

Q4. Can the authors further analyze the controlled comparison already presented in the paper? Isolating fine-tuning objectives of post-training models (Wan2.2, FlashMotion and Kiwi-Edit). What insights can the authors draw from this comparison, and how do the findings relate to the claim in Section 3.1.2?

Kiwi-Edit post-trained with video editing significantly improved orthogonalization, while the FlashMotion with video prediction has less improved. This is consistent with the pretraining result above and with our motivation in Section 3.1.1 that this observation is not limited to image models. Also, from our claim in Section 3.1.2 and experiments as an answer to Q3 that orthogonality is a joint effect of all design choices, this post-training experiments can be framed as isolated post-training objective experiments with fixed VAE and architecture. The results show that the representational structure of the transformer can be reshaped to favor separation of nuisance displacement via editing objective and provides future direction of representational post-training for policy generalization.

We added these analysis as a Section 4.2.2, 4.2.3 and Appendix B.3 of the revised paper.

Reviewer_mCjnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how generative models' representational priors relate to visual generalization in World Action Models (WAMs). It introduces nuisance orthogonality, the fraction of perturbation-induced feature displacement outside an empirical subspace of clean observations, and relates it to LIBERO-Plus success after policy training. It also examines nuisance suppression: changes in perturbation responses normalized by clean-feature variation during adaptation.

The main contribution is a geometric diagnostic for assessing backbones before policy training. The cross-model results are encouraging, but its added value over simple stability metrics and its sensitivity to reference-scene selection remain unclear.

Critical strengths

S1. The approach addresses a practical problem.

WAM training is expensive, so assessing backbones beforehand could help prioritize candidates. The metric uses pretrained features without training another predictor, and the experiments clearly focus on visual perturbations that preserve action semantics.

S2. Multiple backbones support the cross-model association.

Table 1 covers nine configurations, including image models, video models, and scratch-initialized FLUX.2. Average orthogonality correlates with average success at approximately r = 0.914, consistent with Figure 3's R² = 0.835. The four additional backbones also have matching orthogonality and success rankings.

S3. The analysis makes representation changes interpretable.

Paired observations preserve simulator state while changing appearance. Checkpoint-specific normalization accounts for changes in feature scale, and Page 9 separates changes in clean-feature variance from those in perturbation components.

Critical weaknesses

W1. The added value over simple representation stability is unclear.

The paper does not compare orthogonality with normalized feature displacement or paired-feature cosine similarity. These comparisons would test whether subspace direction adds predictive information beyond general feature stability.

Table 1 also omits clean-environment success rates. Differences in perturbed success may partly reflect basic control ability rather than visual robustness. Reporting clean performance and perturbation-induced drops, alongside simple stability baselines, would clarify the metric's contribution.

W2. Stability across reference-scene selections has not been evaluated.

The metric depends on a subspace constructed from 40 clean observations. Figure 4 varies retained rank but not the selection or number of reference scenes.

The authors should show whether model rankings and their correlation with success remain stable when reference scenes are resampled or their number is increased. Results with uncertainty across reference sets would establish whether the metric supports reliable model comparisons.

Questions for authors

Q1. How does orthogonality compare with normalized displacement and cosine similarity on the same paired observations? What are each model's clean success rate and perturbation-induced performance drop?

Q2. Are model rankings and correlations stable across resampled or larger reference sets?

Q3. Please specify the denoising steps, noise settings, language conditioning, token selection, and handling of randomness across paired inputs used for diagnostic feature extraction. These details are incomplete in Section 4.1 and the appendix.

4 comments

We thank the reviewer for the thoughtful evaluation of our work and for the constructive comments, which helped us strengthen our evidence and clarify concerns about comparison with other metrics, clean-domain performance, the stability of the reference set and the detailed configuration of our diagnostic.

W1/Q1 - (1). Concerns about comparison to other metrics.

Paired-sample metrics such as normalized feature displacement or cosine similarity do not respect the directions of subspace created by the clean observations. For example, rotating the displacement around the clean feature as an axis preserves both its magnitude and the cosine similarity, but can move it from inside the clean subspace to outside changing its orthogonality.

In Table R3, we compute each metric on Pearson correlation with the average LIBERO-Plus success rate. Nuisance orthogonality shows the strongest relationship, while the metrics that ignore the clean reference space are much weaker. This indicates that the direction of the displacement relative to the clean span carries more predictive information than other metrics. We added a this analysis as a Appendix B.2 of the revised paper.

Table R3

Metricr with generalization performance
Nuisance Orthogonality0.914
Cosine Similarity0.540
Normalized Squared Displacement0.216

W1/Q1 - (2). Clarification of clean domain success rate.

Table R4

BackboneWan2.2FLUX.2Scratch FLUX.2OmniGen2Ovis-U1FlashMotionKiwi-EditSANA-VideoMAGI-1
Clean LIBERO SR (%)97.6†98.4†96.5——96.996.597.696.2
LIBERO-Plus SR (%)46.5†89.063.977.3†72.9†64.077.175.476.2
† Published results.

In our experiments and as in Table R4, the success rate on the clean LIBERO benchmark is saturated at 96%+ for all trained WAMs, which is a commonly observed results when they are all given sufficient model scale (5B scale) and training computation budget under proper training configurations. In contrast, the performance in LIBERO-Plus success significantly differs on various backbone models revealing the relationship between representational prior and robustness to visual variations. The induced performance drops are essentially almost the same information as the absolute LIBERO-Plus success rate in this case. But to clarify the results, we added the clean success rates to the Table 1 of the revised paper.

W2/Q2. Concerns about stability of small reference set selection.

Table R5

ObservationsFLUX.2 orthogonalityWan2.2 orthogonality
40 - resampled 10 times0.799 ± 0.0040.594 ± 0.006
400 - resampled 5 times0.805 ± 0.0040.595 ± 0.004
4,0000.8150.576
Full 277,7130.8220.564

Table R6

BackboneWan2.2FLUX.2Scratch FLUX.2OmniGen2Ovis-U1FlashMotionKiwi-EditSANA-VideoMAGI-1
400.5940.7990.7240.7460.7720.6450.7860.7420.750
4000.5950.8050.7430.7460.8070.6570.7840.7340.732

In Table R5, we show the orthogonality of FLUX.2 and Wan2.2 with more reference samples and repeated samplings. While more samples improve the estimate of the full-dataset orthogonality, metrics from 40 reference samples are consistent and do not significantly diverge from that of more samples. Together with the results in Table R1 of answer to the reviewer BMUW and rank-consistent orthogonality, adequate coverage of few principal components are sufficient for estimating orthogonality for the models. Which further lets small reference set ranking being consistent to that for more larger reference sets. Table R6 results also show that they are consistent across full models even though they does not keep strict ordering. We added a this analysis as a Section 4.2.1 and Appendix B.1 of the revised paper.

Also, taking into account that the compute budget is multiplied by the number of various visual variation types and degrees (x30 in our experiments), our diagnostics need to keep an economical budget even if some trade-off exists. We think visual variations produce representational changes distinct from clean-span changes, which allows orthogonality and representation displacement to be reliably estimated even from a small reference set.

Q3. Clarification of detailed configurations of our diagnostic.

Since we analyze the representational properties under visual variations, we input the clean (not noisy) images as a conditional input corresponding to the “current observation”, and analyze the representations of the transformer backbone. Rather than analyzing the denoising capability which is further adapted to world modeling, we focus on the vision processing capability of the backbone itself, as a large vision-language encoder for the action expert. Thus, there is no denoising timestep or noisy-input construction in our feature extraction. We added a brief clarification to Section 4.1 of the revised paper.

This also means the randomness does not exist in the model part, but does exist in data sample selection which we address in W2/Q2. We select the tokens from the current observation tokens where clean image are fed to the transformer backbone. For language conditioning, we use pretrained frozen language modules that respect the design of each backbone and additional models.

Thank you for the additional analyses. The comparison with cosine similarity and normalized displacement addresses my main concern in W1. Across the nine backbones, nuisance orthogonality has the strongest correlation with LIBERO-Plus success. The clean LIBERO results are also helpful, since the seven models with reported scores perform similarly in the clean environment. I also appreciate the clarification in Q3 that features are extracted from the current observation without adding diffusion noise or running denoising steps.

The reference-set comparison helps clarify W2. The overall correlation remains strong at both reference-set sizes. However, FLUX.2 ranks first with 40 reference observations, while Ovis-U1 ranks first with 400. Since one intended use of the diagnostic is to choose a backbone before policy training, this change in which model it would select still matters to me. For that reason, I maintain my original rating of weak rejection.

We thank for the follow-up and are glad that some concerns are resolved.

We agree that the change in the top-ranked backbone between reference sizes is worth noting. However, at 400 references, Ovis-U1 and FLUX.2 differ by 0.002, which is within the variation across resampled reference sets. Also, We view the diagnostic as a way to narrow down candidate backbones before policy training rather than to replace policy evaluation among closely scored candidates.

We appreciate the reviewer's constructive comments throughout the discussion.

Reviewer_BMUWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies why robot policies built from different image and video models respond differently to changes in camera position, lighting, background, and image noise. The main idea is that a useful pretrained model should represent these visual changes separately from the differences between ordinary task scenes. The authors measure this separation before robot-policy training using a score called nuisance orthogonality. They then examine how training changes the model's sensitivity to these visual changes. Experiments with nine backbone configurations on LIBERO-Plus show an association between the proposed score and the resulting policy's success rate. The paper also compares ImageWAM and FastWAM in more detail to explain how their representations change after training. The contribution is mainly a way to analyze and potentially select pretrained models, rather than a new robot-learning algorithm.

Critical strengths
  1. The paper is well-motivated and addresses a useful question. Selecting a large pretrained model for robot learning can be expensive, so a diagnostic that can be measured before policy training would be valuable. I appreciate that the authors go beyond the simple explanation that image models are better than video models and investigate what properties of their representations may matter.

  2. The paired observations make the analysis easier to interpret. The clean and modified images correspond to the same simulator state, which helps separate changes in appearance from changes in the underlying task. The comparison also includes four additional backbones and a randomly initialized FLUX.2 configuration, rather than relying only on the two motivating models. Table 1 and Figure 3 present the overall relationship clearly.

  3. The authors make a useful distinction between what a model provides before policy training and what changes during training. The rank study in Figure 4 and the camera-view experiment in Figure 6 provide additional checks beyond the main success-rate comparison. I also appreciate the explicit acknowledgment in the limitations section that the results show an association, not an isolated cause of better performance.

Critical weaknesses
  1. The main diagnostic depends on a very small reference set. Section 3.2 uses 40 clean observations to define the reference space, and the modified observations are collected from those same states. Section 4.2 retains 39 directions. It is not clear whether this small sample captures the variation between ordinary task scenes well enough to support comparisons across different models. A model could receive a high score partly because the reference set leaves many useful directions unrepresented.

  2. The evidence for using the score to choose a new model is still limited. Figure 3 reports a strong relationship across the evaluated configurations, but this is different from showing that the score reliably predicts the performance of a model that was not part of the analysis. Table 1 already contains cases where the ranking is not preserved: Ovis-U1 has a higher average score than OmniGen2, but a lower success rate.

  3. Some details needed to reproduce the diagnostic are not clear. The paper specifies spatial averaging, layer averaging, and the retained rank, but I could not determine the exact denoising timestep, noisy-input construction, and conditioning setup used when extracting features from each generative model.

Questions for authors

N/A

1 comment

We thank the reviewer for the thoughtful evaluation of our work and for the constructive comments, which helped us strengthen our evidence and clarify the concerns about the number of reference samples, the use of the orthogonality score for model selection and configuration of our diagnostic.

W1. Concerns about a small number of reference data samples and their representation.

In our experiment, even a small reference set was able to capture the majority of the dominant clean geometry and provide useful diagnostic information, showing that an economical reference set can be used for our diagnostic. To show this, we present (1) the coverage of the 40-sample PCA compared with the full-data PCA on FLUX.2 and (2) orthogonality on the full dataset and on reference sets of various sizes for FLUX.2 and Wan2.2.

Table R1

Full-data rank \ 40-sample rankr=8r=16r=39
r=872.6%78.4%81.2%
r=1659.5%72.4%77.8%
r=3960.3%70.2%72.4%

In Table R1, we compute how much of the full-dataset variance in the clean domain is covered by the subspace of the 40-sample reference set. The coverage of a 40-sample reference set covers a meaningful majority of the principal directions of a full dataset.

Table R2

ObservationsFLUX.2 orthogonalityWan2.2 orthogonality
40 - resampled 10 times0.7993 ± 0.00410.5942 ± 0.0058
400 - resampled 5 times0.8052 ± 0.00400.5946 ± 0.0038
4,0000.81530.5763
Full 277,7130.82150.5642

In Table R2, we show the orthogonality of FLUX.2 and Wan2.2 with more reference samples and repeated samplings. While more samples improve the estimate of the full-dataset orthogonality, metrics from 40 reference samples are consistent and do not significantly diverge from that of more samples. Together with the results in Table R1 and rank-consistent orthogonality in Figure 4, it shows that adequate coverage of few leading principal components is sufficient for estimating orthogonality for these models.

Also, taking into account that the compute budget is multiplied by the number of various visual variation types and degrees (x30 in our experiments), our diagnostics need to keep an economical budget even if some trade-off exists. We think visual variations produce representational changes distinct from clean-span changes, which allows orthogonality and representation displacement to be reliably estimated even from a small reference set. We added a this analysis in Section 4.2.1 and Appendix B.1 of the revised paper.

W2. Concerns about using the orthogonality score for a model choice.

Our diagnostics provide useful evidence for prioritizing backbones before expensive policy training. We presented this by showing orthogonality and generalization performance of the baseline models and additional models extended to WAMs. We agree that this relationship of orthogonality and generalization performance is not a guarantee of strict ordering, and the candidate models may require policy evaluation to check the final performance. We claim this as a primary diagnostic of world model backbones to check its representational properties before policy adaptation shown with the additional backbones following the same trend, but not as an exact performance predictor. We modified the Section 4.2.3 of the revised paper to clarify our scope.

W3. Clarification of detailed configurations of our diagnostic.

Since we analyze the representational properties under visual variations, we input the clean (not noisy) images as a conditional input corresponding to the “current observation”, and analyze the representations of the transformer backbone. Rather than analyzing the denoising capability which is further adapted to world modeling, we focus on the vision processing capability of the backbone itself, as a large vision-language encoder for the action expert. Thus, there is no denoising timestep or noisy-input construction in our feature extraction. We added a brief clarification to Section 4.1 of the revised paper.

Comments on the paper

3 comments

We thank the reviewers for their comments. All three reviewers found the paper well motivated that a diagnostic measured before expensive WAM training addresses a practical question (BMUW, mCjn, N5he), appreciated the paired-observation design and the evaluation across nine backbones (BMUW, mCjn), valued the distinction between what a backbone provides before and after policy (BMUW) and found the definition of nuisance orthogonality simple and intuitive (N5he). The response to their main concerns and questions are as below.

  1. Stability and reliability of reference set size. Reviewers BMUW and mCjn asked whether 40 clean references are sufficient and stable. The diagnostic depends on the principal component directions of clean variation, which a few samples already capture, so a small reference set gives the same picture as the full dataset. Scores are stable under resampling, although strict ordering is not preserved among backbones with close scores.

  2. Sources of orthogonality with deeper insight. Reviewer N5he asked what drives the differences between backbones. By comparing each backbone with its random initialization and across training stages, we find that the training objective is the main source that determines their orthogonality. Editing objectives strongly separate displacement directions orthogonally in both image and video backbones unlike video prediction. We also find that a randomly initialized transformer can show high orthogonality without semantic representations, yet this does not lead to nuisance suppression after policy training. Our analysis also distinguishes orthogonality that arises from randomness from orthogonality that arises from the training objective.

  3. Clarifications on scope, configuration and comparisons. In response to the reviewers' concerns and requests for clarification, we state that orthogonality is a way to prioritize backbones and audit a training objective before policy adaptation, not an exact performance predictor. It is also clarified that features are extracted from clean observations, independent of the denoising process. We also included comparison to other metrics showing that a metric considering the displacement inside the clean subspace from the one outside is the distinction that they relates to generalization. Clean task performances are clarified, showing that the backbones sufficiently saturated for clean domain can diverage significantly on generalization.

Dear Reviewers,

Please kindly review the authors' rebuttal and engage in discussions if you have further questions.

Thanks, AC

We thank the reviewers for the discussion. We summarize the contribution of our work, how the reviewers' concerns were addressed, and the scope of our claims.

Contribution. Selecting a generative backbone for a world action model is expensive, since each candidate must be trained as a policy before its generalization can be evaluated. Our work studies what property of a pretrained backbone determines this generalization before policy training. We introduce nuisance orthogonality, which measures whether the feature displacement caused by a visual variation lies outside the subspace that distinguishes clean scenes, and show across nine image and video backbones that higher orthogonality is associated with stronger robustness after WAM training. We further analyze how policy adaptation reorganizes this. The reviewers' main concerns were whether nuisance orthogonality adds information beyond simple feature stability, whether a small reference set gives reliable scores, and what drives the differences in orthogonality between backbones. We addressed each with additional analyses and WAM training, summarized below.

What drives orthogonality (N5he). Comparing each backbone with its random initialization and across training stages shows that the training objective shapes orthogonality. Editing objectives raise it in both image and video backbones, while video prediction lowers it. With fully trained WAMs from each setting, the objective that raises orthogonality also brings a substantial gain in robustness which gives practitioners a way to audit a pretraining or post-training objective before policy training.

Reference set size (BMUW, mCjn). Orthogonality is stable under resampling and up to the full dataset, and its association with generalization remains strong at every reference size. We claim this as a tendency rather than a strict ordering: the change of the top-ranked backbone is a near-tie within resampling variation, and such close candidates remain for policy evaluation.

Comparison with other metrics (mCjn). Nuisance orthogonality correlates far more strongly with generalization than cosine similarity or normalized displacement. Reviewer mCjn confirmed that this resolves their main concern, together with the clean success rates and the extraction details.

We sincerely thank all the reviewers for their constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.